Terms of (dis)service: Comparing misinformation policies in text-generative AI chatbot
Eu DisinfoLab (2025), 8 pp.
"● In the 11 chatbots considered, references to misinformation vary (and sometimes are missing). Even when the term is mentioned, it is rarely defined. This is not uncommon: what some platforms call ‘deletion’, others call ‘termination’, though potential nuances remain unclear. Additionally, it is unspecified who enforces practices, how long suspensions last, and what distinguished warnings, suspensions, and terminations. On another note, more stringent prohibitions generally focus on hate speech, highlighting the illegal nature of the issue.
● Overall, ToS violation detection and content moderation processes are unclear. Some platforms mention dedicated moderation teams and processes – typically a mix of automated and human reviews – but the details remain vague. There is no transparency regarding the resources invested in content moderation, including the number of personnel, their locations and language coverage, and the timeline for action.
● Given our focus on misinformation, we concentrated on content reporting. Nevertheless, it is evident that platforms prioritise reporting breaches of and vulnerabilities in their architecture over content-related concerns.
● Law compliance is usually referenced in broad terms as adherence to “all applicable legislation”. Platforms primarily emphasise privacy and data protection, the prohibition of illegal activities (with a particular focus on child protection), restrictions on reverse-engineering platform functions, and using the services for biometric identification. Notably, there is a perception that users bear greater responsibility for providing accurate data and avoiding platform abuse than the platforms themselves do in terms of accountability. This impression arises because the ToS are detailed regarding user obligations but more superficial about platform responsibilities. Uncertainty remains regarding platform incentives and their actual capacity to enforce these rules.
● Most platforms specify that fact-checking is the user’s responsibility as if trying to exempt themselves from liability related to providing false and inaccurate answers. However, it is worth noting that the general provisions of the EU’s Digital Services Act (DSA) require online platforms to remove illegal content expeditiously once they have actual knowledge of its illegality.
● If current platform approaches to misinformation already seem inadequate, there are valid concerns that the situation will deteriorate further. The cross-platform trend of dismantling safety policies and moderation teams reinforces these fears. Examples pointing in this direction include OpenAI’s decision to remove some content warnings on ChatGPT, and xAI’s introduction of “Unhinged Mode” in Grok-2, a beta feature described as intentionally “objectionable, inappropriate, and offensive, much like an amateur stand-up comic who is still learning the craft.” (Concluding remarks, page 8)
● Overall, ToS violation detection and content moderation processes are unclear. Some platforms mention dedicated moderation teams and processes – typically a mix of automated and human reviews – but the details remain vague. There is no transparency regarding the resources invested in content moderation, including the number of personnel, their locations and language coverage, and the timeline for action.
● Given our focus on misinformation, we concentrated on content reporting. Nevertheless, it is evident that platforms prioritise reporting breaches of and vulnerabilities in their architecture over content-related concerns.
● Law compliance is usually referenced in broad terms as adherence to “all applicable legislation”. Platforms primarily emphasise privacy and data protection, the prohibition of illegal activities (with a particular focus on child protection), restrictions on reverse-engineering platform functions, and using the services for biometric identification. Notably, there is a perception that users bear greater responsibility for providing accurate data and avoiding platform abuse than the platforms themselves do in terms of accountability. This impression arises because the ToS are detailed regarding user obligations but more superficial about platform responsibilities. Uncertainty remains regarding platform incentives and their actual capacity to enforce these rules.
● Most platforms specify that fact-checking is the user’s responsibility as if trying to exempt themselves from liability related to providing false and inaccurate answers. However, it is worth noting that the general provisions of the EU’s Digital Services Act (DSA) require online platforms to remove illegal content expeditiously once they have actual knowledge of its illegality.
● If current platform approaches to misinformation already seem inadequate, there are valid concerns that the situation will deteriorate further. The cross-platform trend of dismantling safety policies and moderation teams reinforces these fears. Examples pointing in this direction include OpenAI’s decision to remove some content warnings on ChatGPT, and xAI’s introduction of “Unhinged Mode” in Grok-2, a beta feature described as intentionally “objectionable, inappropriate, and offensive, much like an amateur stand-up comic who is still learning the craft.” (Concluding remarks, page 8)
