AI and Complaints: Automated Triage, Generated Replies and Safeguards

What AI can do in a complaints department#

Four uses are spreading, with different levels of risk:

• Triage and routing: detecting that a message is a complaint, assessing its urgency and sending it to the right team.
• Classification: assigning a subject, a journey stage and a presumed cause, according to the company's taxonomy.
• Summarisation: condensing a long history (letters, transcribed calls, attachments) for the adviser or the committee.
• Drafting: proposing a draft reply based on the file and internal rules.

Used well, AI frees up time for analysis and helps spot more quickly the recurring reasons described in our guide to fixing the causes of complaints. Poorly governed, it moves errors to places where nobody sees them.

Risk 1: a misclassified regulatory complaint#

In a Moroccan bank, Bank Al-Maghrib Circular No. 10/W/16 requires immediate acknowledgement, a response within 40 working days and a quarterly report broken down by subject and status (see banking complaints). If the model classifies a disputed direct debit as an "information request", the case leaves the process: no acknowledgement, no deadline tracked, no line in the reporting.

This risk is silent. A model that is highly accurate on average can look excellent while its rare errors fall precisely on sensitive complaints. The prudent rule: when in doubt, the system classifies as a complaint, never the other way round.

Risk 2: the reply that commits the company#

A generative model can invent a rule, a deadline or a goodwill gesture. A customer who receives "we will refund the fees" will treat the promise as settled. Our guide to AI in customer service recalls the 2024 Canadian decision in which an airline was held liable for incorrect information given by its chatbot. In a complaints department, where every reply may be produced before a mediator, this risk is heightened.

Risk 3: bias and languages#

A model trained mainly on standard French may classify messages in Arabic, in Darija or mixing several languages less well, or underestimate the urgency of a simply written message. The result would be unequal treatment depending on the customer's profile. Performance must be measured separately by language and by channel before going live.

If the tool analyses tone or emotion, the distinction matters: analysis of written text is not emotion recognition within the meaning of the AI Act, because it does not rely on biometric data; analysis of the voice is. See emotion analysis in call centres.

Risk 4: personal data#

A complaint almost always contains personal data, often financial, sometimes sensitive. In Morocco, Law 09-08 applies: a specified purpose, notification to or authorisation by the CNDP, information for customers, and proper safeguards for any transfer to an AI provider hosted abroad (Articles 43 and 44). Its Article 11 also governs decisions with legal effects taken solely on the basis of automated processing intended to define the person's profile; as a precaution, any decision unfavourable to the customer should be validated by an agent. See transferring data abroad and data protection in banking.

What the EU AI Act says#

The AI Act (Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744) applies to providers and deployers established outside the Union where the output of the system is used in the Union (Article 2). For a Moroccan institution serving customers living in Europe, two points need checking:

• Article 50, applicable since 2 August 2026: a system that interacts directly with people, such as a conversational assistant for filing complaints, must be designed to inform them that they are interacting with AI, unless this is obvious. The information must be given at the latest at the time of the first interaction. See the duty to inform the customer.
• The company's role: provider or deployer, depending on whether it develops the tool under its own name or uses it. See identifying your role.

A breach of Article 50 can be punished by a fine of up to €15 million or 3% of worldwide turnover (Article 99). The general framework is presented in our guide to the AI Act and Moroccan BPOs.

Designing the human in the loop#

1. The adviser approves every reply before it is sent; the AI proposes, it does not decide.
2. Regulatory categories are protected: any message that may be a complaint follows the regulatory process, even if a human reclassifies it afterwards.
3. Drafts draw on a validated, dated rule base, and the system cannot propose a goodwill gesture outside that base.
4. The adviser sees why the system classified the message as it did (the parts of the message concerned) and can correct it in one click.
5. Corrections feed into improving the model, under the supervision of a designated owner.
6. A sample of cases is reviewed every month by someone independent of the handling team.

Controls table#

Pre-launch checklist#

• Authorised uses of AI defined in writing (triage, summarisation, drafting), together with excluded uses.
• "When in doubt, it is a complaint" rule configured and tested.
• Tests carried out in French, Arabic and Darija.
• Human approval mandatory before anything is sent.
• CNDP formalities and transfers checked.
• Customer information planned, particularly for those living in the European Union.
• Model owner appointed, indicators tracked, logging active.