Many organizations have initiated knowledge management practices to improve the effectiveness of knowledge creation, transfer, and utilization. However, few studies have been attempted to address what makes some work units within the organization more effective in their use of knowledge than others. In this study we explore the potential contribution of inter-unit network structure as well as intra-unit task environment to the overall KM effectiveness within an organization. The results indicated that both a unit’s boundary-spanning role and its extensive interaction with other units help promote the overall effectiveness of the unit’s KM activities. In addition, we found that a unit’s KM strategy played a mediating role between intra-unit task characteristics and KM effectiveness.