This study investigates how the Harrow-Hassidim-Lloyd (HHL) Algorithm, the quantum algorithm for solving linear systems of equations (LSE), performs within the Policy Iteration of model-based Reinforcement Learning (RL). The HHL Algorithm offers an exponential speed-up over the best known classical algorithm for solving LSE. We simulate the algorithm numerically with Python and conduct an error analysis for an example RL application, by examining the Policy Iteration outputs produced by the HHL Algorithm. We address the pre-requests of the HHL Algorithm and how they affect our RL problem, and finally present problem mappings for our example application.