Skip to main content
Hafan

Meaningful human involvement

Contents

In brief: defining meaningful human involvement

A clear understanding and application of meaningful human involvement is crucial to determining whether ADM is occurring. 

The ADM provisions apply to ‘solely automated’ decisions. These are decisions that don’t reflect real human control over the end result. This is clearly reflected in article 22A of the UK GDPR (as introduced by the DUAA), which states that “a decision is based solely on automated processing if there is no meaningful human involvement in the taking of the decision”. 25

Whilst this has been introduced into article 22A since we started this engagement, our guidance has long been clear that human involvement in a decision must be active and not just a token gesture or ‘rubber stamp’. This was the case under the previous article 22, and Parliament has since clarified this in the new article 22A by the explicit inclusion of the wording ‘meaningful human involvement’.

The critical question is whether:

  • a human can exercise real influence over decision before it is applied and has the authority, discretion and competence to alter it; or
  • they are simply directly applying the outcome of an automated system without meaningful consideration of whether the automated decision was appropriate. 

In the first case, there’s meaningful human involvement. In the second, there isn’t.

Further reading

Our findings about meaningful human involvement

We found that most employers thought their use of automated recruitment tools constituted decision support rather than decision-making. They mainly based this on their view that meaningful human involvement was present. However, the evidence we saw indicated that, in practice, employers were using the tools to make solely automated decisions. In other words, there was no meaningful human involvement.

Suitability or ‘fit’ scores are a common output of automated recruitment tools. They are a form of profiling. Algorithms analyse information gained from candidates during, for example, written applications and skills tests. They then assign a score based on that analysis. The profile produced is based on weights set by the recruiter for certain features that suit the role, such as being adaptable or resourceful. 

Generally, employers told us that they required hiring managers to:

  • review the scores (usually via a dashboard); and
  • manually select candidates for the next stage of the recruitment process. 

On this basis, they argued that the process included meaningful human involvement. 
 
However, employers could not consistently demonstrate how they had mitigated the risk of their hiring managers relying disproportionately on the scores when faced with a high volume of applications. For example, we frequently saw evidence suggesting hiring managers were unlikely to review the scores or responses of lower-scoring candidates. In other words, the scores influenced the decisions about both: 

  • the highest-scoring candidates, who they prioritised; and
  • the lower-scoring candidates, for whom they were ‘rubber-stamping’ the score or, in some cases, not intervening at all.

Case study

An employer uses a tool that gives the recruiting manager access to the system’s output, including: 

  • each candidate’s overall score (‘red’, ‘amber’ or ‘green’); and
  • a copy of each candidate’s responses to the questions asked by the tool.

The manager’s training has taught them that they should review all of the scores. However, they prioritise looking at ‘green’ candidates and only take a glance at ‘red’ candidates before rejecting them for the role. Some ‘amber’ candidates may be reviewed. 

The manager has ‘rubber-stamped’ the ‘red’ candidates’ rejections. This constitutes solely ADM within the scope of article 22. 

Our findings suggest an inconsistent approach to meaningful human involvement among employers engaged in automated recruitment. Even within the same application process, some candidate applications may be subject to meaningful human involvement, while others are not. A successful candidate will have their score, profile and responses weighed up by a hiring manager. Another candidate may be rejected, even though by the click of a human hiring manager, based on a low score generated by an automated process. 

Discussion

Our view is that the newly reformed ADM provisions clarify the position on meaningful human involvement that already existed in the ICO’s previous guidance. This clarification in the law supports the use of these tools to: 

  • support decision-making with consistent meaningful human involvement; or
  • make decisions without meaningful human involvement.

The latter use requires the application of the safeguards mentioned in article 22C, which we discuss in the next section. Ultimately, it is for employers to choose between incorporating meaningful human involvement (so the decision-making moves out of scope of the ADM provisions) or carrying out solely automated decision-making and applying the ADM safeguards.

In our AI in recruitment report, we focused on developers of recruitment systems. There, we identified that employers using these systems could use suitability scores to make, rather than support, decisions. 26  At the time, we noted that many tool providers did not design or market their tools for this purpose. However, our findings from engaging with employers deploying the tools suggest that this is happening. Similarly, a recent survey of 1,000 UK-based HR and talent professionals by background-checking platform Zinc reported that 37% automated rejections entirely. 27

Example

An employer uses an AI recruitment tool which:

  • analyses candidates’ responses to a series of interview questions; and
  • ranks the candidates based on their suitability for the role. 

A recruiter then goes through all the scores and profiles, including the responses, to weigh up the tool’s recommendations. They also have access to additional factors in making their decision, such as CVs. They then make a final decision about whether to progress or reject each candidate from the hiring process. 

This is considered meaningful human involvement and and therefore the decision-making isn’t in scope of the ADM provisions.

We recognise that these tools benefit employers by increasing the efficiency and speed of recruitment. Such automated processes necessarily and logically involve decreasing human involvement, since they are explicitly designed to reduce human labour. However, trust cannot be built without fairness and consistency. Meaningful human involvement can work as its own safeguard, but it must be applied to every candidate, not just those who score highly.  

We note that humans (eg hiring managers and technical or psychological specialists) participate in designing and configuring the models that generate fit scores. This alone is not enough to constitute meaningful human involvement in the fit scores or hiring recommendations generated. This is because the design phase happens long before any real-world decisions are made about people, so it cannot directly influence or alter a specific outcome. We acknowledge that in some scenarios, employers may use algorithms to execute filtering based on predetermined and specific criteria. 

Our public attitudes research identified “a central tension between the desire to adopt [ADM] for efficiency and the need to maintain human oversight” 28.  During our roundtable with stakeholders, ensuring and demonstrating that human involvement was not tokenistic emerged as a key challenge. Stakeholders discussed the need for clear guidance on what constitutes meaningful human involvement. We’ve provided this as part of our draft guidance on the revised ADM provisions, including profiling, which (at the time of publication) is open for consultation 29.   

We’ve also provided use cases towards the end of this report to help employers more clearly understand the application of meaningful human involvement. 

Expectations

Employers must either: 

  • apply the safeguards with the ADM provisions (covered below); or
  • adapt their organisational processes to ensure that there is meaningful human involvement in each decision about each candidate. 

We expect employers across the economy deploying automated recruitment tools to review their processing and consider our findings. If employers find that hiring managers are making recruitment decisions relying only on fit scores, they must follow one of the options above. 

We also expect developers of automated recruitment tools to review these tools’ design and marketing regarding meaningful human involvement. This builds on our AI in recruitment report and may involve developers producing new advice and guidance for their customers. 

We provide further clarity on applying meaningful human involvement, consistent with this report, in:

  • our updated guidance on ADM and profiling; and
  • our forthcoming recruitment and selection guidance. 

We expect employers to follow this guidance. 

 

25  Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (United Kingdom General Data Protection Regulation) (Text with EEA relevance)

 26  AI in Recruitment Outcomes Report

27 AI used by 73 per cent of Recruiters | The Global Recruiter

28 Understanding public perceptions towards automated decision-making in recruitment

29 Automated decision-making, including profiling