Paper Number
ICIS2025-1922
Paper Type
Complete
Abstract
Generative AI platforms face unique governance challenges as they lower barriers to creating Not Safe for Work content. Unlike traditional platforms that moderate after publication, AI systems must govern during the creative process, fundamentally altering the user experience by reshaping perceived affordances. This shift in governance model remains critically understudied despite its growing importance. Collaborating with a leading text-to-image AI provider, we examine how implementing stricter NSFW content moderation affects user behavior. Our natural experiment with 88,427 users reveals that stricter governance led to a significant relative decline in subscription rates among users with NSFW intentions, reflecting affordance contraction obstructing user goals, while retained users adapted by complying with policies without reducing prompt diversity. The findings advance our understanding of affordance in responsible AI implementation by demonstrating how governance mechanisms can effectively redirect user behavior while preserving creativity, contributing to both platform safety and sustainable innovation.
Recommended Citation
Cao, Jisu; Zhao, Keran; Liu, Xiaowei; Liu, Che-Wei; and Duan, Jiang, "Balancing Safety, Engagement, and Compliance: Evaluating the Impact of NSFW Governance on AI-Generated Content" (2025). ICIS 2025 Proceedings. 9.
https://aisel.aisnet.org/icis2025/ethical_is/ethical_is/9
Balancing Safety, Engagement, and Compliance: Evaluating the Impact of NSFW Governance on AI-Generated Content
Generative AI platforms face unique governance challenges as they lower barriers to creating Not Safe for Work content. Unlike traditional platforms that moderate after publication, AI systems must govern during the creative process, fundamentally altering the user experience by reshaping perceived affordances. This shift in governance model remains critically understudied despite its growing importance. Collaborating with a leading text-to-image AI provider, we examine how implementing stricter NSFW content moderation affects user behavior. Our natural experiment with 88,427 users reveals that stricter governance led to a significant relative decline in subscription rates among users with NSFW intentions, reflecting affordance contraction obstructing user goals, while retained users adapted by complying with policies without reducing prompt diversity. The findings advance our understanding of affordance in responsible AI implementation by demonstrating how governance mechanisms can effectively redirect user behavior while preserving creativity, contributing to both platform safety and sustainable innovation.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
05-ResponsibleIS