Screen scraping—any automated process to extract content from a website for use in another context—has found countless applications over the past two decades. It is now used for targeted advertising, price aggregation, budgeting apps, website preservation, academic research, journalism, and more. Analytic start-ups draw insights for industries by scraping public data, while Fintech firms purchase data made available by aggregators to develop new products and services. However, screen scraping can be controversial. It can be detrimental to the data host and the consumer. Scraping is parasitic when it undercuts a website’s revenue (ie, by diverting customers away from a scraped website). It can facilitate copyright infringement at scale or even impact the data host’s services by overloading servers. Screen scraping can also raise privacy concerns for consumers if it collects identifiable information or enables new surveillance forms. In the banking context—where login credentials may be shared to allow the scraping of account data—there are additional concerns about cybersecurity, data breach, and liability allocation for unauthorized transactions. These complex applications of screen scraping have led to litigation against scrapers—most notably in the US.
Despite such divergence, one may see a sea change amid the trend of data-sharing under the banner of ‘Open Banking’ in coming years. While these three jurisdictions are divided in the way they treat such technology (ie, screen scraping), Open Banking initiatives’ rise in recent years could moderate concerns and bring a certain level of convergence. I argue that to the extent Open Banking mandates or facilitates data sharing, it could reduce the need for screen scraping. This is especially so in the European context—and even more so if the UK Smart Data initiative expands these data-sharing principles beyond the financial sector. Conversely, the financial data-sharing environment is less evident in the US, which lags behind when it comes to building up Open Banking. Australia lies in the middle of these two extremes: it has a comprehensive Consumer Data Rights (CDR) regime that can theoretically reduce screen scraping needs. However, given that it imposes no ban on screen scraping (unlike the EU/UK model), data miners may continue their data scraping practice even though the CDR regime has been put in place.
In short, with the emerging trend of data sharing, one could witness a sea change in the screen scraping legal landscape. Insofar as data sharing schemes enable information flow between entities, one would expect some level of convergence. Such a convergence, however, is qualified by the institutional design of data sharing schemes—whether or not it explicitly addresses screen scraping (as in the case of Australia and the UK) and whether there is a top-down, government-mandated data-sharing regime (as in the case of the US). These are, of course, just preliminary findings that can serve as a starting point for a larger project exploring the role of screen scraping in the digital economy. Debates on the legality of screen scraping and some issues left out in this paper—such as the linkage between data sharing and antitrust and conflict of laws (eg, scraping taking place in China but the content is scraped from the EU)—continue to rage. It remains to be seen whether (and how) the governments in these three jurisdictions and others will develop a more holistic approach towards this technology by striking the right balance between different stakeholders entering the age of big data.
Han-Wei Liu is a Lecturer at Monash University, Australia.