A collector for every source
Each website gets its own collector, written for how that site is built. When one site changes, the rest keep running, and we fix the one that broke without touching the others.
Managed data programmes
Hundreds of sources. One dependable delivery.
Some jobs outgrow a one-off extract. You need the same figures from dozens or hundreds of websites, every week or every month, for as long as the work runs. We build a collector for each source, run them on a schedule, check every delivery before it reaches you, and tell you plainly what we could not collect and why.
What you get
Built for data you rely on every week
A programme is only useful if you can trust each delivery without re-checking it yourself.
Checked before it ships
Every delivery passes an automated quality gate first: row counts against the last run, missing sources, values outside expected ranges, duplicates. If a check fails, the delivery is held until we have fixed it.
A coverage report every time
Each delivery lists which sources were collected, which were not, and why: the site was down, the page was removed, the figure is no longer published. You never have to guess where a gap came from.
Alerts when a source changes
When a site drops rows, removes a listing or changes its layout, the run flags it and we repair the collector, ideally before the next scheduled delivery.
The same format, every run
Spreadsheet, CSV, database or SFTP drop, with the same columns each time. Every row carries its source URL and capture date, so any figure can be traced back.
Yours to use in your own work
You can publish figures derived from the data in reports, research and client advice, and show it in your own dashboards. The licence terms are in the engagement letter.
Typical programmes
Where a programme fits
Programmes suit any question that can only be answered by looking at many websites, repeatedly. Common ones: prices and stock across retailer and distributor sites, property listings and rents across agent and portal sites, decisions and notices across regulators, and product or tariff data across suppliers. If the sources are public and reachable in a browser, we can usually collect them. For a worked example on an open source, see our complete dataset of every UK CMA case.
How a programme runs
From source list to scheduled delivery
You get a written scope and a fixed quote before any work starts, and a baseline to check before the first scheduled run.
01
Scope the sources
You give us the list of websites, or the market you want covered, plus the fields and how often you need them. We check each source and tell you which we can collect and which we cannot.
02
Build and baseline
We build the collectors and run a baseline collection. You check it against what you expect and sign off the format and field definitions.
03
Scheduled runs
The programme runs weekly, monthly or on the cadence you set. Each run goes through the quality gate and arrives with its coverage report.
04
Maintain and grow
We repair collectors when sites change and add new sources as your list grows. Each new source is scoped and quoted before it is built.
How it is priced
Scoped per programme, quoted in writing
Programmes have two parts: a one-off build fee to create and baseline the collectors, then a fee per scheduled run. Both depend on the number of sources and how hard they are to reach, so we quote each programme individually after checking the sources. Need data from a single website? Our starter extract and maintained scraper tiers are priced on the web scraping page.
Questions
Before you send a source list
How many sources can a programme cover?
There is no fixed limit. Programmes run from around twenty sources to several hundred. Each source is checked during scoping, so you know which ones we can collect before you commit.
What happens when a website changes or goes down?
The quality gate flags it on the run that catches it, the coverage report records it, and we repair the collector. If a site is down on the day, the delivery says so.
Can we add sources once the programme is running?
Yes. New sources are scoped and quoted as you add them, and join the next scheduled run once they are built and checked.
Do you use AI to extract the data?
No. Each collector follows fixed rules written for its source, so the same page gives the same output every time and any figure can be traced back to where it came from.
Is the work confidential?
Yes. We work under NDA as standard. We do not name clients or reference their work in our marketing, and we do not reuse your source lists or deliveries for anyone else.
Send us your source list
Tell us which websites you need covered and how often. We will check each one and come back with a written scope and a fixed quote. UK-based, ICO-registered, working under NDA.