How do we carry out content moderation lawfully?

In detail

• What does it mean for our content moderation processing to be lawful?
• What do we need to consider if we’re using legal obligation as our lawful basis?
• What do we need to consider if we’re using legitimate interests as our lawful basis?
• Can we use contract as our lawful basis?
• Can we use consent, vital interests or public task?
• What if our content moderation involves special category information?
• What if our content moderation involves criminal offence information?

What does it mean for our content moderation processing to be lawful?

The first data protection principle requires any processing of personal information to be lawful, fair and transparent.

You must identify a lawful basis before you start using personal information in your content moderation system.

There are six available lawful bases for processing set out in Article 6 of the UK GDPR. No one basis is always better or more important than the others.

Your decision depends on the specific purposes you are going to use content moderation for.

In practice, the lawful bases that are most likely to be relevant to your content moderation processing are:

• legal obligation – you can rely on this lawful basis if you need to process someone’s personal information to comply with a common law or statutory obligation. For example, you may be carrying out content moderation to comply with your safety duties under the OSA; and
• legitimate interests – you can rely on this lawful basis if you have a legitimate interest in processing someone’s personal information. For example, you may be using content moderation to enforce your terms of service. This basis involves balancing your interests against the person’s interests, rights and freedoms. 

Although legal obligation and legitimate interests are the most likely lawful bases to apply in practice, we have included guidance if you are considering the remaining lawful bases. 

If your content moderation system uses special category information or criminal offence information, you must have a lawful basis and an additional Article 9 or Article 10 condition for processing. (See sections below on ‘What if our content moderation involves special category information?’ and ‘What if our content moderation involves criminal offence information?’ for more information.)

What do we need to consider if we’re using legal obligation as our lawful basis?

You can use legal obligation as your lawful basis if you need to process personal information to comply with a common law or statutory obligation. The information in this section is about using legal obligation under the OSA. However, there may be other statutory obligations or common law duties that are relevant to you. 

If you process personal information in your content moderation to comply with your obligations under the OSA, you are likely to be able to rely on legal obligation as your basis for this processing. But, the processing must be necessary and proportionate to achieve compliance.

You are also likely to be able to use this lawful basis for personal information processing that you need to do to apply the measures recommended in Ofcom's codes of practice under the OSA. This is because the codes provide measures that enable you to comply with the legal obligations set out in the OSA.

You must directly link your processing to a legal obligation placed on you. Your processing does not need to be essential for you to comply with your obligation, but you must ensure it is a reasonable and proportionate way of achieving compliance. You should document your decision to rely on legal obligation and identify which part of the legislation specifies this.

You must not rely on this lawful basis for content moderation processing that goes beyond what is required for you to meet your duties in the OSA (unless there are other common law duties or legislative obligations that apply).

What do we need to consider if we’re using legitimate interests as our lawful basis?

Legitimate interests is the most flexible of the lawful bases. It isn’t focused on a particular purpose, which means you can potentially rely on it in a range of circumstances. 

Legitimate interests is most likely to apply where you want to use personal information in ways that:

• people would reasonably expect; and
• don’t have an unjustified adverse impact on people’s rights and freedoms.

If there is an impact on people, legitimate interests may still be available, but you must show that there is a compelling benefit to the processing and the impact is justified.

You are likely to have a legitimate interest in detecting and taking action on content in accordance with your content policies. However, you must balance this interest against the interests, rights and freedoms of users. You must also make sure that your personal information processing is necessary to achieve your purpose. If you can reasonably achieve the same result in another less intrusive way, legitimate interests does not apply.

To ensure you’ve considered these issues, you should carry out a three-part test to:

• identify your legitimate interest;
• show that processing is necessary to achieve that interest; and
• consider whether people’s interests, rights and freedoms override the legitimate interest you’ve identified.

It’s likely that your use of personal information in content moderation will involve a level of intrusion into your users’ privacy. This means you must demonstrate that you have a compelling justification for this (eg safeguarding your users).

You can consider legitimate interests for processing children’s personal information, but you must take extra care to protect their interests.

What you tell people in your privacy information is one of the factors that affects whether they can reasonably expect the processing. You should be clear with users about what types of content are prohibited on your service and why, and how you action certain content. If you provide this information, users are more likely to expect the processing that you undertake to detect and remove this type of content.

Can we use contract as our lawful basis?

You can use contract as your lawful basis if using personal information is objectively necessary to a deliver a contractual service to a relevant person, or to users of your service in general. 

You must not rely on contract if:

• there is a less intrusive way of processing personal information to provide the same service; or
• the processing is not objectively necessary for the performance of the contract.

If you are thinking about using contract, you should consider the following questions:

• Are you carrying out the processing to deliver a contractual service? For example, are you processing personal information to fulfil obligations that you have outlined in your terms of service? 

If your answer is yes:

• Is the processing necessary for the performance of that contract? For example, is processing the information a targeted and proportionate step that is integral to delivering the contractual service? 

In most cases, although you may be able to rely on contract for your contract moderation processing, it is likely that legal obligation or legitimate interests are more suitable.

If the contract is with a child under 18, you must consider whether they have the necessary competence to enter into a contract. If you have doubts, you may wish to consider an alternative basis, such as legitimate interests. Using legitimate interests as your lawful basis can help you demonstrate that you properly considered and protected the child’s rights and interests.

You are unlikely to be able to rely on contract for processing personal information for purposes such as ‘service improvement’ of your content moderation systems. In most cases, collecting personal information about how people engage with a service in order to develop new service functions is not objectively necessary to provide a contract. This is because you can deliver the service without this processing. 

Can we use consent, vital interests or public task?

It is unlikely that these lawful bases can apply to your content moderation processing.

Consent is about giving people genuine choice and control over their information. Consent won’t apply as you’re unlikely to be offering people a free choice about whether you process their information for content moderation.

Vital interests generally only applies in specific matters of life and death. It is unlikely to apply to your content moderation processing, particularly at scale.

Public task is unlikely to be relevant to the user-to-user services this guidance applies to.

What if our content moderation involves special category information?

In order to lawfully process special category information, you must identify a condition for processing, as well as a lawful basis.

There are 10 conditions for processing special category information outlined in Article 9 of the UK GDPR. Five of these require you to meet additional conditions and safeguards set out in schedule 1 of the DPA 2018. In many cases you also need an ‘appropriate policy document’ in place in order to meet a schedule 1 condition in the DPA 2018. 

When choosing a condition for processing, being clear about your purpose for content moderation will help you identify the most appropriate condition.

If you plan to use special category information, or if you are moderating content that includes special category information about users, then you must identify a condition for processing. (See the section of this guidance on ‘Do content moderation systems use special category information?’ for more information.)

If you are not sure whether the user-generated content you intend to moderate contains special category information, you should identify a condition for processing to cover that possibility and minimise the privacy risks.

Below, we discuss some of the conditions for processing special category information that may be relevant in content moderation.

Substantial public interest

In order to rely on this condition, you must demonstrate that your processing has substantial public interest benefits.

In order to demonstrate this, you must meet one of 23 specific substantial public interest conditions. For almost all of these conditions, you must have an appropriate policy document in place. The conditions set out in part 2, schedule 1 DPA 2018 that may be relevant include:

• preventing or detecting unlawful acts – this condition is met if your use of personal information is necessary to prevent or detect an unlawful act;
• safeguarding of children and individuals at risk – this condition applies if your use of personal information is necessary to protect a child or at-risk person from neglect or harm, or to protect their wellbeing; and
• regulatory requirements – this condition applies if your use of personal information is necessary to comply with a regulatory requirement that involves establishing whether someone has committed an unlawful act or has been involved in dishonesty, malpractice or other seriously improper conduct. 

All of these conditions also require you to demonstrate that your specific processing of special category information is “necessary for reasons of substantial public interest”.

What if our content moderation involves criminal offence information?

You must ensure that you process any criminal offence information lawfully, fairly and transparently, and that you have an Article 6 lawful basis for processing.

In addition, Article 10 of the UK GDPR states that you must only process criminal offence information if this processing is:

• under the control of official authority; or
• authorised by domestic law. In the UK, this means you need to meet one of the conditions in schedule 1 of the DPA 2018.

You are unlikely to be processing under the control of an official authority when carrying out content moderation (see our guidance on criminal offence information for more details). Therefore, you must identify a specific condition for processing in schedule 1 of the DPA 2018, if your content moderation processing involves criminal offence information.

As with special category information, you may require an appropriate policy document depending on the condition you rely on.

The following schedule 1 conditions may be relevant for processing criminal offence information in content moderation systems:

• preventing or detecting unlawful acts (see above);
• safeguarding of children and individuals at risk (see above); and
• regulatory requirements (see above).