Driven by the ability to perform sequential decision-making in complex dynamic situations, Reinforcement Learning (RL) has quickly become a promising avenue to solve inventory control (IC) problems. The objective of this paper is to provide a comprehensive overview of the IC problems that have been effectively solved due to the application of RL. Our contributions include providing the first systematic review in this field of interest and application. We also identify potential extensions and come up with four propositions that formulate a theoretical framework that may help develop RL algorithms to solve complex IC problems. We recommend specific future research directions and novel approaches in solving IC problems.