Emotion Analysis in Call Centres: What the AI Act Prohibits

The rule: Article 5(1)(f), in force since 2 February 2025#

The AI Act prohibits placing on the market and using AI systems intended to infer the emotions of natural persons in the workplace and in educational institutions, except where the use is for medical or safety reasons. This prohibition was among the first provisions to apply, on 2 February 2025.

Two definitions in Article 3 mark out the rule:

• an emotion recognition system identifies or infers people's emotions or intentions on the basis of their biometric data (Article 3(39));
• biometric data results from specific technical processing of physical, physiological or behavioural characteristics (Article 3(34)), which includes a voice analysed for that purpose.

Regulation (EU) 2026/1744, known as the Digital Omnibus, left this prohibition untouched.

Agents or customers: the dividing line#

The Commission's guidelines on prohibited practices, adopted on 4 February 2025, give two examples that speak directly to contact centres. A centre that tracks its employees' emotions (anger, for example) by webcam or voice recognition falls under the prohibition. A centre that uses voice recognition to track its customers' emotions is not covered by Article 5(1)(f).

Text or voice: why the modality changes everything#

The concept rests on biometric data. The guidelines draw a practical consequence from this: inferring an emotion from written text is not emotion recognition, whereas inference from the tone, rhythm or intonation of the voice is.

For a Moroccan centre, this means that the same analytics provider may offer one permitted module (sentiment on written verbatims or on transcripts) and one module that is prohibited when applied to agents (emotion inferred from acoustics). Each function must therefore be classified, not the tool as a whole.

The conversation analytics trap#

Voice analytics platforms often process the full recording, with both voices mixed. If the tool produces, from the voice or image, an emotion indicator for the agent ("agent stressed", "agent irritated"), it falls within the prohibition, even if the stated aim is customer service quality. Points to check with the software vendor:

1. Does the tool separate the customer and agent channels?
2. Can agent-side emotional indicators be switched off, and not merely hidden?
3. Do supervision dashboards display emotion scores per agent?
4. Is historical data already produced deleted after deactivation?
5. Is the chosen configuration logged and dated?

Hiding an indicator on screen is not enough if the system keeps calculating it.

Analysing customers' emotions: what is still required#

Not being prohibited does not mean being unregulated. For analysing customers' emotions:

• Since 2 August 2026, the deployer must inform the people exposed, at the latest at first contact, and process the data in accordance with the GDPR (Article 50(3) and 50(5)). The welcome message can say so simply, in French for French customers: « Pour améliorer notre service, cet appel est analysé par un outil automatisé, y compris le ton de la voix. » ("To improve our service, this call is analysed by an automated tool, including tone of voice.") See our example disclosure messages.
• From 2 December 2027, the use is high risk (Annex III, point 1(c)). The deployer will in particular have to follow the provider's instructions, assign oversight to trained and authorised people, monitor operation, report serious incidents, keep logs for at least six months and use the provider's information for its impact assessment (Article 26).
• In Morocco, Law 09-08 on personal data protection applies to recordings and their analysis: informing the people concerned and completing formalities with the CNDP (Morocco's data protection authority). The precise regime depends on the processing; our guide to CNDP compliance describes the approach.

The underlying question remains: does emotion analysis provide information that the transcript and call reasons do not already give? Often, behavioural signals (repeat calls, escalations, cancellations) tell you more, with less risk.

The medical or safety exception#

The exception is narrow. The guidelines cite detecting pain or fatigue, for example in pilots. Productivity, perceived quality or preventing absenteeism are not safety reasons. A centre invoking the exception would need to be able to demonstrate a genuine safety risk, the absence of a less intrusive alternative and use limited to that purpose.

What if the AI Act does not apply directly?#

The AI Act covers third-country providers and deployers where the system's output is used in the Union. An analysis of agents' emotions whose output stays in Morocco may, depending on the facts, fall outside its scope. There are nonetheless three reasons to rule it out:

• European clients often receive quality reports produced by these tools, which brings the use closer to the regulation's scope;
• they increasingly prohibit this practice by contract and ask about it in their audits, as buyer-side third-party risk management shows;
• emotional monitoring of employees raises, independently of the AI Act, questions of proportionality under Law 09-08.

In addition, where the AI Act applies, an employer using a high-risk system in the workplace will have to, from 2 December 2027 for Annex III systems, inform workers' representatives and the employees concerned before putting it into service (Article 26(7)).

Penalties#

• Prohibited practice (Article 5): up to €35 million or 7% of worldwide annual turnover.
• Breach of deployer or transparency obligations (Articles 26 and 50): up to €15 million or 3%.
• The higher amount applies, except for SMEs and start-ups, for which it is the lower.

Audit checklist for your analytics tools#

1. List every tool that processes the voice, image or text of calls and chats.
2. For each, identify the functions that infer an emotion or intention.
3. Distinguish the source: written text, transcript, voice acoustics, image.
4. Switch off any emotion inference targeting agents and purge the history.
5. For customers, decide whether the use is justified; if so, draft the disclosure message.
6. Prepare the high-risk file for 2 December 2027: instructions, oversight, logs.
7. Update CNDP formalities and contracts with the software vendor and clients.
8. Inform and involve workers' representatives.