Knowledge management (KM) plays a prominent role in IT and software development companies. It is often introduced as part of larger organizational change processes that aim at improving their software development processes. Process improvement in the IT sector has led to the establishment of the software process improvement (SPI) discipline. We present the conceptual basis for investigating how software companies can ground their improvement activities on knowledge management through a study of the KM and SPI literature. We identify two archetypes of knowledge organizations which we label exemplary and situational and two approaches to process improvement, which we call normative and reflective. Our analysis of the relationship between KM and SPI leads to a proposal for a balanced theory of KM in SPI and provides valuable insights into how meaningful KM can be conducted for process improvements in IT and software organizations.