Where does our data come from?

Blog Post: Scraping the Barrel

Avatar

An oft received enquiry from our end is “where do you get your data from?” - a reasonable inquiry and, commonly, based on the assumption that most B2B data available is scraped by a flotilla of auto-bots (no, not those kind) picking up names, emails and addresses from every crevice of the world wide web and associated electronic directories and repositories.


This approach - often employed by either tech focused or resource poor businesses has its place (legalities and accuracy notwithstanding) - particularly if the ambition is to cover a lot of data quickly and follow the "broad and shallow" rather than "narrow and deep" philosophy of database coverage. 75% of a picture for little or no internal cost is good, right?


Does this work for compiling a database of every Public Sector organisation and all the key political and management decision makers? - it does not. So, back to the original question; where does Oscar data come from? - the answer my friend is not blowing in the wind (significantly less mellifluous but more straightforward in fact):


We have a bunch of processes backed by experienced humans to ensure we cover every Public Body and our team of 16 in-house researchers working on fixed update cycles liaising directly with every organisation, requesting updates, reviewing our coverage and making updates accordingly, collectively putting in over 30,000 hours of manual research time each year. Curating rather than scraping. The result, a complete view of entities and posts not possible without the hand-balling and direct human approach. Or at least not possible until the much quoted A.I. transforms all our lives and we can all sit on the beach drinking pina-coladas.


Until then, a deep dive into every part of our direct research and update processes can be seen here


Newsletter

Receive our weekly summary spotlight emails highlighting changes across Public Services and our quarterly newsletters.


Sign Up Here