Reinforcement Learning and Operational Research: A Review of Methodological Integration and Research Directions

arXiv:2609.24750v2 Announce Type: replace-cross Abstract: The growing demand for real-time, data-driven decision-making in complex and dynamic systems is placing increasing pressure on Operational Research (OR) methodologies. Sequential decision-making under uncertainty has long been studied in OR through Markov decision processes, dynamic programming, and their approximations, and reinforcement learning (RL) provides simulation-based methods for learning decision policies in such problems. Recent research shows an increasing interest in integrating RL with OR to address dynamic decision-making problems and enhance heuristic and exact methods for combinatorial optimization. The overarching goal across these efforts is to improve solution quality, computational efficiency, and robustness. Given the diversity of integration approaches and application settings, there is a clear need for a systematic and technically detailed review of how RL empowers OR methods. To address this gap, this paper presents a structured review of two methodological roles of RL in OR. The first concerns sequential decision problems in dynamic environments, where states evolve and decisions are repeated over time. The second concerns static combinatorial optimization problems, where inputs are fixed before the solution process begins and RL either constructs solutions or supports heuristic and exact algorithms. Digital twins provide an optional deployment and evaluation layer for training, testing, and validating these methods. We critically synthesize recent advances across these roles and uses, highlighting their advantages, implementation requirements, limitations, and challenges. Finally, we consolidate guidance on when the reviewed approaches may be considered and outline directions for future research on the methodological and practical integration of RL and OR.