agentchat_MathChat.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/microsoft/autogen/blob/main/notebook/agentchat_MathChat.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Auto Generated Agent Chat: Using MathChat to Solve Math Problems\n",
    "\n",
    "AutoGen offers conversable agents powered by LLM, tool or human, which can be used to perform tasks collectively via automated chat. This framwork allows tool use and human participance through multi-agent conversation. Please find documentation about this feature [here](https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat).\n",
    "\n",
    "MathChat is an experimental convesational framework for math problem solving. In this notebook, we demonstrate how to use MathChat to solve math problems. MathChat uses the `AssistantAgent` and `MathUserProxyAgent`, which is similar to the usage of `AssistantAgent` and `UserProxyAgent` in other notebooks (e.g., [Automated Task Solving with Code Generation, Execution & Debugging](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_auto_feedback_from_code_execution.ipynb)). Essentially, `MathUserProxyAgent` implements a different auto reply mechanism corresponding to the MathChat prompts. You can find more details in the paper [An Empirical Study on Challenging Math Problem Solving with GPT-4](https://arxiv.org/abs/2306.01337) or the [blogpost](https://microsoft.github.io/autogen/blog/2023/06/28/MathChat).\n",
    "\n",
    "## Requirements\n",
    "\n",
    "AutoGen requires `Python>=3.8`. To run this notebook example, please install the [mathchat] option.\n",
    "```bash\n",
    "pip install \"pyautogen[mathchat]\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %pip install \"pyautogen[mathchat]~=0.1.1\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set your API Endpoint\n",
    "\n",
    "The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import autogen\n",
    "\n",
    "config_list = autogen.config_list_from_json(\n",
    "    \"OAI_CONFIG_LIST\",\n",
    "    filter_dict={\n",
    "        \"model\": {\n",
    "            \"gpt-4\",\n",
    "            \"gpt4\",\n",
    "            \"gpt-4-32k\",\n",
    "            \"gpt-4-32k-0314\",\n",
    "            \"gpt-4-32k-v0314\",\n",
    "            \"gpt-3.5-turbo\",\n",
    "            \"gpt-3.5-turbo-16k\",\n",
    "            \"gpt-3.5-turbo-0301\",\n",
    "            \"chatgpt-35-turbo-0301\",\n",
    "            \"gpt-35-turbo-v0301\",\n",
    "            \"gpt\",\n",
    "        }\n",
    "    }\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It first looks for environment variable \"OAI_CONFIG_LIST\" which needs to be a valid json string. If that variable is not found, it then looks for a json file named \"OAI_CONFIG_LIST\". It filters the configs by models (you can filter by other keys as well).\n",
    "\n",
    "The config list looks like the following:\n",
    "```python\n",
    "config_list = [\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your OpenAI API key here>',\n",
    "    },\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your Azure OpenAI API key here>',\n",
    "        'api_base': '<your Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-06-01-preview',\n",
    "    },\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your Azure OpenAI API key here>',\n",
    "        'api_base': '<your Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-06-01-preview',\n",
    "    },\n",
    "]\n",
    "```\n",
    "\n",
    "If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose \"upload file\" icon.\n",
    "\n",
    "You can set the value of config_list in other ways you prefer, e.g., loading from a YAML file."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct agents for MathChat\n",
    "\n",
    "We start by initialzing the `AssistantAgent` and `MathUserProxyAgent`. The system message needs to be set to \"You are a helpful assistant.\" for MathChat. The detailed instructions are given in the user message. Later we will use the `MathUserProxyAgent.generate_init_message` to combine the instructions and a math problem for an initial message to be sent to the LLM assistant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from autogen.agentchat.contrib.math_user_proxy_agent import MathUserProxyAgent\n",
    "\n",
    "autogen.ChatCompletion.start_logging()\n",
    "\n",
    "# 1. create an AssistantAgent instance named \"assistant\"\n",
    "assistant = autogen.AssistantAgent(\n",
    "    name=\"assistant\", \n",
    "    system_message=\"You are a helpful assistant.\",\n",
    "    llm_config={\n",
    "        \"request_timeout\": 600,\n",
    "        \"seed\": 42,\n",
    "        \"config_list\": config_list,\n",
    "    }\n",
    ")\n",
    "\n",
    "# 2. create the MathUserProxyAgent instance named \"mathproxyagent\"\n",
    "# By default, the human_input_mode is \"NEVER\", which means the agent will not ask for human input.\n",
    "mathproxyagent = MathUserProxyAgent(\n",
    "    name=\"mathproxyagent\", \n",
    "    human_input_mode=\"NEVER\",\n",
    "    code_execution_config={\"use_docker\": False},\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 1\n",
    "\n",
    "Problem: Find all $x$ that satisfy the inequality $(2x+10)(x+3)<(3x+9)(x+8)$. Express your answer in interval notation.\n",
    "\n",
    "Correct Solution: \n",
    "We have \\begin{align*} (2x+10)(x+3)&<(3x+9)(x+8) \\quad \\Rightarrow\n",
    "\\\\ 2(x+5)(x+3)&<3(x+3)(x+8) \\quad \\Rightarrow\n",
    "\\\\ 2(x+5)(x+3)-3(x+3)(x+8)&<0 \\quad \\Rightarrow\n",
    "\\\\ (2x+10-(3x+24))(x+3)&<0 \\quad \\Rightarrow\n",
    "\\\\ (-x-14)(x+3)&<0 \\quad \\Rightarrow\n",
    "\\\\ (x+14)(x+3)&>0.\n",
    "\\end{align*} This inequality is satisfied if and only if $(x+14)$ and $(x+3)$ are either both positive or both negative.  Both factors are positive for $x>-3$ and both factors are negative for $x<-14$.  When $-14<x<-3$, one factor is positive and the other negative, so their product is negative.   Therefore, the range of $x$ that satisfies the inequality is $ \\boxed{(-\\infty, -14)\\cup(-3,\\infty)} $."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# given a math problem, we use the mathproxyagent to generate a prompt to be sent to the assistant as the initial message.\n",
    "# the assistant receives the message and generates a response. The response will be sent back to the mathproxyagent for processing.\n",
    "# The conversation continues until the termination condition is met, in MathChat, the termination condition is the detect of \"\\boxed{}\" in the response.\n",
    "math_problem = \"Find all $x$ that satisfy the inequality $(2x+10)(x+3)<(3x+9)(x+8)$. Express your answer in interval notation.\"\n",
    "mathproxyagent.initiate_chat(assistant, problem=math_problem)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 2\n",
    "Problem: For what negative value of $k$ is there exactly one solution to the system of equations \\begin{align*}\n",
    "y &= 2x^2 + kx + 6 \\\\\n",
    "y &= -x + 4?\n",
    "\\end{align*}\n",
    "\n",
    "Correct Solution: Setting the two expressions for $y$ equal to each other, it follows that $2x^2 + kx + 6 = -x + 4$. Re-arranging, $2x^2 + (k+1)x + 2 = 0$. For there to be exactly one solution for $x$, then the discriminant of the given quadratic must be equal to zero. Thus, $(k+1)^2 - 4 \\cdot 2 \\cdot 2 = (k+1)^2 - 16 = 0$, so $k+1 = \\pm 4$. Taking the negative value, $k = \\boxed{-5}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "math_problem = \"For what negative value of $k$ is there exactly one solution to the system of equations \\\\begin{align*}\\ny &= 2x^2 + kx + 6 \\\\\\\\\\ny &= -x + 4?\\n\\\\end{align*}\"\n",
    "mathproxyagent.initiate_chat(assistant, problem=math_problem)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 3\n",
    "Problem: Find all positive integer values of $c$ such that the equation $x^2-7x+c=0$ only has roots that are real and rational. Express them in decreasing order, separated by commas.\n",
    "\n",
    "Correct Solution: For the roots to be real and rational, the discriminant must be a perfect square. Therefore, $(-7)^2-4 \\cdot 1 \\cdot c = 49-4c$ must be a perfect square. The only positive perfect squares less than 49 are $1$, $4$, $9$, $16$, $25$, and $36$. The perfect squares that give a integer value of $c$ are $1$, $9$, and $25$. Thus, we have the equations $49-4c=1$, $49-4c=9$, and $49-4c=25$. Solving, we get that the positive integer values of c are $\\boxed{12, 10, 6}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "math_problem = \"Find all positive integer values of $c$ such that the equation $x^2-7x+c=0$ only has roots that are real and rational. Express them in decreasing order, separated by commas.\"\n",
    "mathproxyagent.initiate_chat(assistant, problem=math_problem)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------\n",
    "### Using other prompts\n",
    "\n",
    "MathChat allows different prompts that instruct assistant to solve the problem.\n",
    "\n",
    "Check out `MathUserProxyAgent.generate_init_message(problem, prompt_type='default', customized_prompt=None)`:\n",
    "- You may choose from `['default', 'python', 'two_tools']` for parameter `prompt_type`.  We include two more prompts in the paper: \n",
    "    1. `'python'` is a simplified prompt from the default prompt that uses Python only. \n",
    "    2. `'two_tools'` further allows the selection of Python or Wolfram Alpha based on this simplified `python` prompt. Note that this option requries a Wolfram Alpha API key and put it in `wolfram.txt`.\n",
    "\n",
    "- You can also input your customized prompt if needed: `mathproxyagent.generate_init_message(problem, customized_prompt=\"Your customized prompt\")`. Since this mathproxyagent detects '\\boxed{}' as termination, you need to have a similar termination sentence in the prompt: \"If you get the answer, put the answer in \\\\boxed{}.\". If the customized is provided, the `prompt_type` will be ignored.\n",
    "\n",
    "\n",
    "### Example 4 (Use the \"python\" prompt):\n",
    "\n",
    "Problem: If $725x + 727y = 1500$ and $729x+ 731y = 1508$, what is the value of $x - y$ ?\n",
    "\n",
    "Correct Solution: Subtracting the two equations gives: \n",
    "\\begin{align*}\n",
    "(729x+731y)-(725x+727y) &= 1508-1500\\\\\n",
    "\\Rightarrow\\qquad 4x+4y &= 8\\\\\n",
    "\\Rightarrow\\qquad x+y &= 2.\n",
    "\\end{align*}\n",
    "\n",
    "Multiplying this equation by 725 and subtracting this equation from $725x+727y=1500$ gives \\begin{align*}\n",
    "(725x+727y) - 725(x+y) &= 1500-725(x+y) \\implies \\\\\n",
    "2y &= 50.\n",
    "\\end{align*}So we can write $x-y$ as $(x+y) - 2y$, which equals  $2 - 50 = \\boxed{-48}$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we set the prompt_type to \"python\", which is a simplied version of the default prompt.\n",
    "math_problem = \"Problem: If $725x + 727y = 1500$ and $729x+ 731y = 1508$, what is the value of $x - y$ ?\"\n",
    "mathproxyagent.initiate_chat(assistant, problem=math_problem, prompt_type=\"python\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5 (Use the \"two_tools\" prompt)\n",
    "\n",
    "Problem: Find all numbers $a$ for which the graph of $y=x^2+a$ and the graph of $y=ax$ intersect. Express your answer in interval notation.\n",
    "\n",
    "\n",
    "Correct Solution: If these two graphs intersect then the points of intersection occur when  \\[x^2+a=ax,\\] or  \\[x^2-ax+a=0.\\] This quadratic has solutions exactly when the discriminant is nonnegative: \\[(-a)^2-4\\cdot1\\cdot a\\geq0.\\] This simplifies to  \\[a(a-4)\\geq0.\\] This quadratic (in $a$) is nonnegative when $a$ and $a-4$ are either both $\\ge 0$ or both $\\le 0$. This is true for $a$ in $$(-\\infty,0]\\cup[4,\\infty).$$ Therefore the line and quadratic intersect exactly when $a$ is in $\\boxed{(-\\infty,0]\\cup[4,\\infty)}$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The wolfram alpha appid is required for this example (the assistant may choose to query Wolfram Alpha).\n",
    "import os\n",
    "if \"WOLFRAM_ALPHA_APPID\" not in os.environ:\n",
    "    os.environ[\"WOLFRAM_ALPHA_APPID\"] = open(\"wolfram.txt\").read().strip()\n",
    "\n",
    "# we set the prompt_type to \"two_tools\", which allows the assistant to select wolfram alpha when necessary.\n",
    "math_problem = \"Find all numbers $a$ for which the graph of $y=x^2+a$ and the graph of $y=ax$ intersect. Express your answer in interval notation.\"\n",
    "mathproxyagent.initiate_chat(assistant, problem=math_problem, prompt_type=\"two_tools\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "flaml_dev",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}