The mechanisms of information technology (IT) governance have been widely recognized as practices to sustain alignment of business and IT units. However, the IT governance literature so far has drawn little attention to the possible idiosyncrasies of governance arrangements in the public sector. In this paper we propose a conceptual model to investigate the relationship between IT governance mechanisms and according performance outcomes specifically for public sector organizations. A survey instrument is developed and validated based on in-depth interviews with IT representatives from three different municipalities in Germany. A cross-case analysis particularly provides evidence for the importance of structural and relational mechanisms and demonstrates how different mechanisms can compensate each other. Our findings provide relevant insights for government practitioners and an impetus for further research.