IRAIS 2026 Proceedings
Abstract
AI systems increasingly create content that other AI systems evaluate. Platform governance explains how firms, states, civil society organizations, and human custodians set and enforce content rules (Gorwa, 2019; Gillespie, 2018). Research on algorithmic moderation examines automated decisions inside those institutions (Gorwa, Binns, & Katzenbach, 2020), while IS delegation theory explains transfers of task rights and responsibilities between people and agentic artifacts (Baird & Maruping, 2021). We ask: How does automation across content generation, moderation, and policy setting change accountability for a specific content decision, and where can human review restore it?
The core case is a pipeline in which a generative AI produces an image and a model such as ShieldGemma 2 evaluates it against a safety policy (ShieldGemma Team, 2025). The unit of analysis is one decision within a configuration of generation, moderation, and policy setting. Each stage may be human-run or AI-run, producing eight configurations. The platform or deploying organization remains the ultimate principal, model vendors supply technical capabilities, the models are operational agents, and users, appeals bodies, regulators, or courts may be accountability forums (Eisenhardt, 1989; Grote, Parker, & Crowston, 2026). The problem is not that these organizations disappear. They may be disconnected from the path that makes and executes the particular decision.
We define a governance vacuum as a configuration in which the immediate decision path is not linked to an actor and forum able to reconstruct the decision, explain the rule, hear a contest, and trigger consequences. These functions are traceability, explainability, contestability, and sanctionability. Leonardi (2025) explains how responsibility circulates when humans attribute agency to AI while retaining formal authority. Grote et al. (2026) examine control-accountability misalignment among available stakeholders, and Mayer et al. (2025) show power shifts among human platform actors. Our construct covers the narrower execution-level condition in which those actors still exist but do not perform the immediate governance work for a specific automated decision.
The mechanism is accountability decay. Appraisal, distribution, and coordination can be automated, but they lose governing force when their outputs are not tied to a named role that can review and revise them (Baird & Maruping, 2021). AI autonomy, learning, and inscrutability make that separation harder to manage (Berente et al., 2021). P1 states that accountability decays as automated delegation mechanisms become disconnected from an identifiable review role. P2 states that decay is stage-dependent. Automating generation removes a referent for an input, automating moderation removes one at enforcement, and automating policy setting removes the referent for rules applied downstream. P2 predicts the largest loss when policy setting is automated and the final human review link is removed.
We define a re-humanization point as required justification to an authorized human before an automated decision takes effect. P3 states that placing this point at policy setting restores more accountability than placing it at moderation or generation because an upstream policy governs many enforcement acts. P4 addresses normative mismatch: enforcement becomes incoherent when a generator and moderator learn from different community distributions. ModelCitizens shows that community membership and context change toxicity judgments (Suvarna et al., 2025). Automation does not always weaken accountability. Durable logs and exception routing can strengthen traceability when an identifiable forum can use them. A vacuum emerges when the records are not connected to a forum that can interpret, contest, and sanction the decision.
X-Teaming shows that coordinated AI agents can challenge another model's safety controls, but it is a human-designed red-teaming system, not evidence of autonomous adversarial governance (Rahman et al., 2025). Adversarial agents outside a governed pipeline form a competitive relationship, not a principal-agent one. The theory is bounded to automated content pipelines under an organizational mandate and does not claim that firms, developers, vendors, or regulators lose legal or organizational responsibility.
Two simulations illustrate, but do not test, the theory. The first maps the eight human and AI configurations across the four accountability dimensions. The second varies the community distributions used for a generator and moderator. The configuration count is only a summary. The argument depends on whether a human role can use the record and change an outcome, not on how many people appear in the architecture. A nominal human-in-the-loop does not restore accountability if that person lacks authority. The European Digital Services Act transparency database offers a possible source for later validation. The contribution is a stage-specific account of when an artifact-artifact decision path becomes disconnected from accountable actors and forums. Re-humanization is therefore a design choice about where a person can review and change the pipeline, not a general call to keep someone in the loop.
References
Baird, A., & Maruping, L. M. (2021). The next generation of research on IS use: A theoretical framework of delegation to and from agentic IS artifacts. MIS Quarterly, 45(1), 315–341.
Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021). Managing artificial intelligence. MIS Quarterly, 45(3), 1433–1450.
Eisenhardt, K. M. (1989). Agency theory: An assessment and review. Academy of Management Review, 14(1), 57–74.
Gillespie, T. (2018). Custodians of the internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.
Gorwa, R. (2019). The platform governance triangle: Conceptualising the informal regulation of online content. Internet Policy Review, 8(2).
Gorwa, R., Binns, R., & Katzenbach, C. (2020). Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1), 1–15.
Grote, G., Parker, S. K., & Crowston, K. (2026). Taming artificial intelligence: A theory of control-accountability alignment among AI developers and users. Academy of Management Review, 51(2), 278–299.
Leonardi, P. M. (2025). Homo agenticus in the age of agentic AI: Agency loops, power displacement, and the circulation of responsibility. Information and Organization.
Mayer, A.-S., Kostis, A., Strich, F., & Holmström, J. (2025). Shifting dynamics: How generative AI as a boundary resource reshapes digital platform governance. Journal of Management Information Systems, 42(2), 400–430.
Rahman, S., Jiang, L., Shiffer, J., Liu, G., Issaka, S., Parvez, M. R., Palangi, H., Chang, K.-W., Choi, Y., & Gabriel, S. (2025). X-Teaming: Multi-turn jailbreaks and defenses with adaptive multi-agents. Conference on Language Modeling (COLM) 2025.
ShieldGemma Team. (2025). ShieldGemma 2: Robust and tractable image content moderation. Google Research.
Suvarna, A., Chance, C., Naranjo, K., Palangi, H., Hao, S., Hartvigsen, T., & Gabriel, S. (2025). ModelCitizens: Representing community voices in online safety. arXiv:2507.05455.
Recommended Citation
Safari, Ali and Ku, Chih-Hao (Justin), "When Machines Moderate Machines: Toward a Governance Vacuum Theory" (2026). IRAIS 2026 Proceedings. 5.
https://aisel.aisnet.org/irais2026/5
Abstract Only