Linear and mixed integer optimisation problems have demonstrated their strength in the field of logistics and supply chain management for years. However, real-world optimisation problems are complex in nature, and various mathematical programming solvers are leveraged to solve these problems today. With several advances in solver technologies in recent years, there has been growing interest in carrying out comparative evaluations of solvers for a range of applications. However, there appears a lack of guidance for decision makers to conduct solver performance assessment and inter-comparison. To address this gap, we aim to derive a framework of parameters deemed most relevant for evaluating and comparing different solvers for a given application. To this end, we perform a systematic literature review. The resulting parameters are classified into three core categories: performance metrics, stopping conditions, and performance enhancing elements of a solver.