Updated and formatted README

README.md

@@ -26,8 +26,8 @@ model-index:
 
 # *Agent using DQN to play ALE/Pacman-v5*
 
->## Update 20 May 2024: Latest DQN model is version 2.8
->***NOTE:** Video preview is the best model of version 2.8 playing for 10,000 steps. Evaluation metrics are self-reported based on 10 episodes of evaluation. Can be found in `agents/dqn_v2-8/evals.txt`*
+## Update 20 May 2024: Latest DQN model is version 2.8
+***NOTE:** Video preview is the best model of version 2.8 playing for 10,000 steps. Evaluation metrics are self-reported based on 10 episodes of evaluation. Can be found in `agents/dqn_v2-8/evals.txt`*
 
 This is an agent that is trained using Stable Baselines3 as part of the capstone project for South Hills School in Spring 2024. The goal of this project is to gain familiarity with reinforcement learning concepts and tools, and to train an agent to score up into the 400-500 point range in Pac-Man.
 
@@ -50,7 +50,7 @@ After cloning the repository to your local device, run:
 ```bash
 pip install -r requirements.txt
 ```
->***NOTE:** The `requirements.txt` file will install all the extra dependencies for Stable Baselines and the entire version of TensorFlow. This is for ease of use for Stable Baselines and to ensure that extra data points and tools are available in TensorBoard. If you wish to install dependencies as needed, you can simply skip the `requirements.txt` file and install packages via `pip` as desired.*
+***NOTE:** The `requirements.txt` file will install all the extra dependencies for Stable Baselines and the entire version of TensorFlow. This is for ease of use for Stable Baselines and to ensure that extra data points and tools are available in TensorBoard. If you wish to install dependencies as needed, you can simply skip the `requirements.txt` file and install packages via `pip` as desired.*
 
 --- 
 
@@ -153,42 +153,37 @@ python <script_name> --help
 
 ##### *watch_agent.py*
 
->This will render the specified agent in real-time. Does not save any evaluation information.
+- This will render the specified agent in real-time. Does not save any evaluation information.
 
 ##### *evaluate_agent.py*
 
->This will evaluate a specified agent and append the results to a specified log file.
+- This will evaluate a specified agent and append the results to a specified log file.
 
 ##### *get_config.py*
 
->This will pull configuration information from the specified agent and save it in JSON format. The data is pulled from the data file in the agent's zip file and strips out the serialized data to make the data more human-readable. The default save file will save to the directory from which the command is run. Best practice is to save the file to the agent's directory.
+- This will pull configuration information from the specified agent and save it in JSON format. The data is pulled from the data file in the agent's zip file and strips out the serialized data to make the data more human-readable. The default save file will save to the directory from which the command is run. Best practice is to save the file to the agent's directory.
 
 ##### *plot_improvement.py*
 
->This plots the average score and standard deviation of the `dqn_v2` agent over all evaluation episodes during a training run as a bar graph with each training run shown as one bar. Removes the lowest and highest episode scores from each evaluation.
+- This plots the average score and standard deviation of the `dqn_v2` agent over all evaluation episodes during a training run as a bar graph with each training run shown as one bar. Removes the lowest and highest episode scores from each evaluation.
 
 ##### *record_video.py*
 
->This will record a video of a specified agent being evaluated. Does not save any evaluation information. *Currently in major development. Currently located in development branch.*
+- This will record a video of a specified agent being evaluated. Does not save any evaluation information. *Currently in major development. Currently located in development branch.*
 
 ##### *plot_evaluations.py*
 
->This will plot the evaluation data that was gathered during the training run of the specified agent using MatPlotLib. Charts can be saved to a directory of the user's choosing. *Currently in major development. Currently located in development branch.*
+- This will plot the evaluation data that was gathered during the training run of the specified agent using MatPlotLib. Charts can be saved to a directory of the user's choosing. *Currently in major development. Currently located in development branch.*
 
 ---
 
 ## *External References*
 
-- [Foundations of Deep RL -- 6-lecture series by Pieter Abbeel](https://www.youtube.com/playlist?list=PLwRJQ4m4UJjNymuBM9RdmB3Z9N5-0IlY0)
-    - This is an excellent introduction to some of the concepts behind Deep RL Algorithms. Pieter Abbeel is a machine learning and robotics researcher at UC Berkeley.
-- [Training AI to Play Pokemon with Reinforcement Learning](https://www.youtube.com/watch?v=DcYLT37ImBY)
-    - Peter Whidden's video of using Proximal Policy Optimization was a major inspiration for this project and has some fantastic visualizations of the agent learning.
-- [Frame Skipping and Pre-Processing for Deep Q-Networks on Atari 2600 Games](https://danieltakeshi.github.io/2016/11/25/frame-skipping-and-preprocessing-for-deep-q-networks-on-atari-2600-games/)
-    - Daniel Takeshi wrote an excellent post that helped me better understand some of the terminology around frame skipping.
-- [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/abs/1312.5602)
-    - This paper on Deep Q Networks is a landmark in the field of reinforcement learning.
-- [Hugging Face Deep Reinforcement Learning Course](https://huggingface.co/learn/deep-rl-course/unit0/introduction)
-    - Another inspiration for this project and a great place to get hands-on experience.
+- [Foundations of Deep RL -- 6-lecture series by Pieter Abbeel](https://www.youtube.com/playlist?list=PLwRJQ4m4UJjNymuBM9RdmB3Z9N5-0IlY0). *This is an excellent introduction to some of the concepts behind Deep RL Algorithms. Pieter Abbeel is a machine learning and robotics researcher at UC Berkeley.*
+- [Training AI to Play Pokemon with Reinforcement Learning](https://www.youtube.com/watch?v=DcYLT37ImBY). *Peter Whidden's video of using Proximal Policy Optimization was a major inspiration for this project and has some fantastic visualizations of the agent learning.*
+- [Frame Skipping and Pre-Processing for Deep Q-Networks on Atari 2600 Games](https://danieltakeshi.github.io/2016/11/25/frame-skipping-and-preprocessing-for-deep-q-networks-on-atari-2600-games/). *Daniel Takeshi wrote an excellent post that helped me better understand some of the terminology around frame skipping.*
+- [Playing Atari with Deep Reinforcement Learning](https://arxiv.org/abs/1312.5602). *This paper on Deep Q Networks is a landmark in the field of reinforcement learning.*
+- [Hugging Face Deep Reinforcement Learning Course](https://huggingface.co/learn/deep-rl-course/unit0/introduction). *Another inspiration for this project and a great place to get hands-on experience.*
 - [Stable Baselines3](https://stable-baselines3.readthedocs.io/en/master/)
 - [RL Zoo](https://rl-baselines3-zoo.readthedocs.io/en/master/)
 - [Gymnasium](https://gymnasium.farama.org/)
@@ -197,4 +192,4 @@ python <script_name> --help
 
 ## *Contact*
 
-Please feel free to contact me on [Twitter](https://x.com/ledmands) or [LinkedIn](https://linkedin.com/in/lucasedmands) or in in the Discussion section on the Community tab of this repository!
+Please feel free to contact me on [Twitter](https://x.com/ledmands) or [LinkedIn](https://linkedin.com/in/lucasedmands) or in the Discussion section on the Community tab of this repository!