Today in many organisations in the public sector, the use of IT has become pervasive in every facet of the organisations’ endeavours in supporting and evolving public service delivery. This pervasive use of technology has caused a critical dependency on IT, which in this environment involves a complex mix of political, organisational, technical and cultural concerns that call for a specific focus on effective IT governance. Much research has been done on IT governance effectiveness in these organisations. However adequate focus has not been given to such organisations in a developing country environment like Tanzania, which on one hand is characterised by IT resources, knowledge, and culture constraints and on the other hand by the increase of IT investment and applications. In this paper specifically we address such a gap by identifying the critical success factors (CSFs) for effective IT governance in this environment. The study is based mainly on IT governance focus areas in analysing the relevant CSFs from the literature and five organizations in the Tanzanian public sector. Moreover we have taken into account the fact that the IT value to be achieved due to effective IT governance is related to efficient and cost effective IT delivery, innovation and business impact. As a result of this research we have identified eleven CSFs that should be considered for effective IT governance practice in Tanzanian public sector organizations.