Policymaker Toolkit
As algorithmic management systems are becoming embedded in workplaces around the world, so too are the harms which span from algorithmic opacity to mental and physical harm. Policymakers around the world have become aware of the need to tackle the challenges that arise from the development and deployment of algorithmic management systems. In Europe, existing and newly established laws including the General Data Protection Regulation (GDPR), EU AI Act, and Platform Workers Directive (PWD) recognise the harms that algorithmic management poses to employees. While some of these harms can be addressed by existing protections, these frameworks can be patchy and still in a nascent stage.
In our toolkit for policymakers, we draw attention to some of the gaps and shortcomings existing in the current laws and regulations which do not adequately address algorithmic management. Following this, we offer potential ways forward that policymakers can begin to tackle these challenges centred around improving the measures that already exist. Where possible, for each of normative proposal, our toolkit offers an example of a similar precedent that can inform what improved algorithmic management protections look like.
Our toolkit summarises and extracts key information from over five years of research conducted by the iManage team, led by Professor Jeremias Adams-Prassl. Our analysis focuses on the UK and EU contexts, but may be applicable to other jurisdictions with similar legal landscapes or be drawn upon for comparative analysis. The toolkit only offers a snapshot of our research, with the full published papers linked where applicable and a further research list at the bottom of the page.
Where are the gaps in existing protections for algorithmic management? How best should they be filled?
“The solution thus lies in strengthening, complementing, and particularising existing protections, through both legislative and non-legislative measures, both domestically and at the Union level.” – Jeremias Adams-Prassl, Halefom Abraha, Aislinn Kelly-Lyth, Sangh Rakshita, and Michael ‘Six’ Silberman
Data protection and algorithmic management
While data protection laws in the EU and UK apply to employment contexts in both the private and public sector, these rules were not designed with unique labour relations in mind. As a result, data protection laws suffer from exceptions, structural deficits, fragmentation, and uncertainties which limit their effectiveness in the algorithmic management context.
Key examples include:
- The right to access (Article 15) is subject to qualifications and limits: employers can oppose to data access if this is necessary to protect the rights and freedoms of others, with explicit carve-outs to protect intellectual property (e.g., trade secrets) which can pose barriers to data access, even when information is only a 'factual secret'.
- In the EU, the safeguards that apply to automated decision-making in Article 22 only apply when a decision is 'solely automated' (see SCHUFA) and the prohibition on these kinds of decisions can be escaped by employers using contractual necessity or consent as a lawful basis (Article 22(2) EU GDPR).
- Data protection law typically ignores collective and societal data protection and privacy harms, neglecting collective workers' rights and interests (Abraha, 2023)
- There is significant fragmentation across the 27 EU Member States, so data protection offers varying safeguards and protections depending on the jurisdiction (Abraha, 2023)
- The enforcement of data protection remains a significant issue, with Data Protection Authorities (DPAs) being under-resourced, understaffed, and with limited labour expertise (Abraha, 2023)
Options include:
Member States should leverage opportunities provided by Article 88 EU GDPR (this does not apply to UK though) and introduce their own additional employee data protection laws that meet the specific requirements of processing workplace data and the risks of algorithmic management (Abraha 2022).
Precedent: Germany offers promising developments and political will for implementing freestanding data protection legislation focused on the workplace, which also addresses the risks of algorithmic management. Other Member States should follow suit (read read more here).
Make use of collective agreements provided by Article 88 of EU GDPR which lay the foundation for social partners to be involved in the governance of data protection, including issues relevant to algorithmic management (Abraha 2022).
Precedent: Spain's Riders' Law is an example of how trade unions, employer organisations, and governments can use collective bargaining agreements to govern algorithmic management (read more here).
Updated regulatory guidance from the European Data Protection Board (EDPB) or UK Information Commissioner's Office (ICO): there is a lack of clear, up-to-date guidance on how personal data should be processed in the workplace, with attention to the specific challenges emerging from algorithmic management (Abraha 2022).
Precedent: the Article 28 Working Party Opinion 2/2017 on data processing at work was previously published, but it does not account for the mass data processing harms caused by the rise of algorithmic management systems in workplaces.
Code of conduct and certification schemes: Trade unions and employers' associations can prepare a code of conduct and certification schemes to provide clarity and best practice on the data protection compliance challenges that may arise in the context of algorithmic management (Article 40 EU GDPR; Article 42 EU GDPR)
Discrimination law and algorithmic management
Doctrinal analysis of the equality framework in the UK and EU by Adams-Prassl et al reveals that its rules are capable of adapting to the harms that may arise from discriminatory and biased algorithmic management systems. Indeed, where issues may arise in terms of application, this largely exposes long-standing unresolved questions in equality law, or stem from the judicial interpretations of discrimination as a human (not algorithmic) phenomenon. In the face of this, reform should be focused on securing effective enforcement of discrimination law as opposed to reforming substantive aspects of discrimination laws.
Key examples include:
- Algorithmic decision-making processes are typically opaque, which presents barriers to litigation and enforcement action (Adams-Prassl et al, 2022; Kelly-Lyth, 2021)
- Proving discrimination without access to information is challenging for claimants, particularly those with fewer resources and those facing power imbalances (Adams-Prassl et al, 2022; Kelly-Lyth, 2021)
- Cases on algorithmic management remain rare, preventing opportunities for important questions of law to be clarified by the courts (Adams-Prassl et al, 2022).
Options include:
Algorithmic Management Impact Assessments (AMIA) (Adams-Prassl et al, 2022; Kelly-Lyth and Thomas, 2023): access to information and evidence about algorithmic management is imperative so that litigation can take place on an individual and collective level. While improved rights for potential claimants to access information would go someway to address the informational asymmetries, a proactive sharing of information through ARMIA would be more effective. This would ensure that individuals and representative bodies (e.g., trade unions) can understand and compare information necessary to ground claims of direct or indirect discrimination. The ARMIA could include (see Policy Option 8; Adams-Prassl et al, 2022):
(a) all system level information which is to be provided to individual employees and applicants;
(b) a description and evaluation of the relevant impacts and risks, by reference to quantitative information about the operation of the systems where relevant;
(c) a description and assessment of any retained or new safeguards adopted to mitigate those impacts and risks;
(d) an evaluation of the effectiveness of new and existing safeguards, including an assessment of whether they are appropriate for the impacts and risks identified;
(e) a description of the consultation(s) carried out with workers and their representatives, and of the changes made in response to views expressed.
Precedent: there are multiple ARMIA-adjacent measures, but each of them is limited in scope, availability, or content for the purposes of regulating algorithmic management:
- The EU AI Act includes algorithmic management systems within its 'high-risk AI' category and subjects these technology developers to certain documentation requirements in terms of the functioning and performance of the AI systems. However, job applicants and workers do not have access to these documents (Article 11, EU AI Act; Article 43, EU AI Act)
- The Platform Workers Directive (PWD) offers platform workers (though not job applicants) with access to information about algorithmic management systems (Article 9, PWD) — but the scope of this legislation is limited to digital labour platforms (Article 1, PWD). Furthermore, while platforms will be required to 'monitor and evaluate' the impacts of algorithmic management systems, there is a lack of clarity on the format and publication of these ongoing assessments.
- Data protection law also contains provisions which require the completion of Data Protection Impact Assessments (DPIAs), but DPIAs are not subject to transparency/publishing requirements and do not set any concrete standards or formats for evaluating rights impacts (Halefom, 2023). DPIA obligations will also not be triggered where algorithmic management systems process anonymous (not personal) data (Recital 26, UK GDPR; Recital 26 EU GDPR).
Existing labour policies can, however, provide a useful framework for the contents of AMIA:
- The Good Work Charter offers examples that can be built upon to inform how AMIA should be developed. The ten Charter principles also offer an appropriate content for the assessment (Kelly-Lyth and Thomas, 2023)
Labour law and algorithmic management
The Platform Work Directive (PWD) has been enacted to improve working conditions in the platform economy and regulate algorithmic management across the EU. It provides a critical first response to protect platform workers who were acutely exposed to the harms of automation. Yet, following the enactment of the PWD, a curious paradox has emerged since platform workers, long unprotected by the law, are likely to have more protections against intrusive monitoring and error-prone algorithmic management than 'traditional employees'. Furthermore, the beneficial protections of the PWD depend on faithful and good-intentioned interpretation of the provisions by platforms to meet legislative goals (see Article 1, PWD).
Key examples include:
- The scope of the Directive is limited to 'digital labour platforms', which will only cover organisations which match individual client requests to available workers at a distance and via electronic means (Article 1(a), PWD)
- As a result, not only are traditional employers not covered by the Directive, but even closer online 'platforms' (e.g., content marketplaces) fall outside of the scope (Silberman, 2023)
Options include:
- Expanding the scope of the PWD to all employers, whether platform-based or not
- Include content marketplaces within the definition of digital labour platforms to acknowledge the similar challenges faced by this category of workers and the need for improved protections (Silberman, 2023)
Precedent: the Spanish Rider's Law cover all employers and offers comprehensive protections to workers subjected to algorithmic management.
AI law and algorithmic management
Many algorithmic management systems fall within the scope of the definition of "AI" (Article 3(1), EU AI Act). Indeed, these technologies are also likely to fall within the scope of 'high-risk AI' and thus will be subject to a specified set of requirements which fall on the AI provider (the person/body who develops or markets the AI) (Article 6, EU AI Act; Article 16, EU AI Act; Annex III, EU AI Act). The AI Act is a welcome development which requires algorithmic management providers to take clear risk mitigation measures before deploying high-risk AI systems, however, its effectiveness is diluted by several key structural and practical failures in how it regulates algorithmic management (Kelly-Lyth, 2021).
Key examples include:
- Even though high-risk AI systems are subject to requirements for 'conformity assessments' to ensure that certain standards have been met, these are merely self-assessments. This leaves AI providers to 'mark their own homework' with limited oversight over how ambiguous assessments on data quality are interpreted and applied (Kelly-Lyth, 2021)
- The transparency provisions do not provide employees with access to technical documentation detailing the algorithmic management systems used by their employers, reinforcing information asymmetries and disempowering litigation (Kelly-Lyth, 2021).
- Since the AI Act is a positive harmonisation measure, it prohibits EU Member States from maintaining any laws incompatible with the standards laid down by the Act – even those which offer higher standards on AI deployment (e.g., Member States which limit the use of workplace monitoring tools) (Kelly-Lyth, 2021).
Options include:
- A clearer and binding timeline for external certification of all high-risk AI, including algorithmic management (Kelly-Lyth, 2021)
Precedent: the AI Act already imposes external compliance checks for certain forms of AI, and other cases will be subject to external certification after the 'initial phase of application' (Article 6(1)(b), EU AI Act)
- Ensure that the Act's provisions establish a floor, not a ceiling on Member State rules concerning AI (Kelly-Lyth, 2021)
Precedent: Article 88 of EU GDPR provides that Member States can "provide for more specific rules to ensure the protection of the rights and freedoms in respect of the processing of employees’ personal data" (read more in Abraha, 2022)
- Empower those subject to AI systems by ensuring workers can access information about high-risk AI systems to assert their employment, equality and data protection law rights.
Precedent: the PWD offers more comprehensive transparency obligations on platforms, with access rights provided to individual workers alongside information and consultation rights (Article 9 PWD).
Looking for a set of regulatory options for algorithmic management?
Our 'Regulating AI at Work: Policy Blueprint' sets out eight regulatory options to address the regulatory gaps exposed by algorithmic management. The blueprint goes beyond providing comprehensive normative solutions. It also presents concrete illustrations from jurisdictions across the EU and beyond of how these have been implemented in practice.
Looking for further information?
Find a list of the research the iManage team conducted and drew on to develop the employee toolkit.
Algorithmic management:
The following article by Aislinn Kelly-Lynth and Anna Thomas and provides an overview of the impacts of algorithmic management. This Wired article also provides a shorter, lay summary of algorithmic management and the work of the iManage team.
Data protection law:
In the following article, Regulating algorithmic employment decisions through data protection law, Halefom Abraha examines the extent to which the GDPR offers the necessary tools to protect workers from harm stemming from algorithmic management. In another piece, Halefom Abraha digs into Article 88 and EU Member State Employee Data Protection rules following the CJEU's decision in Hauptpersonalrat der Lehrerinnen concerning the live streaming of video classes in Germany during COVID. An additional piece by Halefom Abraha also focuses on Article 88 in upholding privacy in the workplace which is described as a 'doubled-edged sword'.
Discrimination law:
For a detailed account of the application of UK direct discrimination law to algorithmic decision-making, Directly Discriminatory Algorithms by Jeremias Adams-Prassl, Reuben Binns, and Aislinn Kelly-Lyth examines the case law on direct discrimination in the UK (with brief comparisons to the EU and US), showing its utility in the context of biased algorithmic decision-making. For an detailed account specific to the employment context, see Aislinn Kelly-Lyth's Algorithmic discrimination at work which applies direct and indirect discrimination law to algorithms at work, while advocating for the need for improved transparency for these existing protections to have any weight. Aislinn Kelly-Lyth and Anna Thomas' Algorithmic management: Assessing the impacts of AI at work provides an analysis of what this transparency might look like by proposing Algorithmic Management Impact Assessments (AMIA).
Platform Workers Directive:
For an analysis of the Platform Workers Directive with respect to algorithmic management, see Fortifying the algorithmic management provisions in the proposed Platform Work Directive by Michael Veale, Michael 'Six' Silberman, and Reuben Binns. A summary of the Platform Workers Directive can be found on blogs here and here, alongside blog posts (here and here and here) which criticises the limited scope of the legislation. Note, this research was conducted while the Platform Workers Directive was being debated, so its analysis is focused on the proposal/draft version of the Directive.
EU AI Act:
For an overview of the applicability of the EU AI Act to algorithmic management, see Aislinn Kelly-Lyth's article in the Comparative Labor Law & Policy Journal.
This site is part of a project that has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No. 947806).
If you have any questions or feedback about this site or our research, please get in touch by emailing ai.work@law.ox.ac.uk.