{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "metadata": { "id": "S4MkldrwPA_S" }, "source": [ "# LLMs for Self-Study\n", "> A prompt and code template for better understanding texts\n", "\n", "This notebook provides a guide for using LLMs for self-study programmatically. A number of prompt templates are provided to assist with generating great assessments for self-study, and code is additionally provided for fast usage. This notebook is best leveraged for a set of documents (text or PDF preferred) **to be uploaded** for interaction with the model.\n", "\n", "This version of the notebook is best suited for those who prefer to use files from their local drive as context rather than copy and pasting directly into the notebook to be used as context for the model. If you prefer to copy and paste text, you should direct yourself to the [prompt_with_context](https://colab.research.google.com/github/vanderbilt-data-science/lo-achievement/blob/main/prompt_with_context.ipynb) notebook." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "IYx3Elo9HJSL" }, "outputs": [], "source": [ "#libraries for user setup code\n", "from getpass import getpass\n", "from logging import raiseExceptions" ] }, { "cell_type": "markdown", "metadata": { "id": "m34rxkb-HJSM" }, "source": [ "## Helper functions\n", "The following functions help to encapsulate the functionality executed below. The `setup_drives` function below assists with setting up the drives for users to upload files." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "4Ma5b-UOHJSM" }, "outputs": [], "source": [ "def setup_drives(upload_set):\n", "\n", " upload_set = upload_set.lower()\n", "\n", " # Colab file upload module\n", " if upload_set == \"local drive\":\n", " from google.colab import files\n", " uploaded = files.upload()\n", " elif upload_set == \"google drive\":\n", " # Mount a Google Drive\n", " from google.colab import drive\n", " drive.mount('/content/drive')\n", " # Raise errors\n", " elif upload_set == '':\n", " raise ValueError(\"You haven't yet defined the upload_settings variable. Go back and read the instructions to make this setting.\")\n", " else:\n", " raise SyntaxError(\"Please check your setting for typos and/or capitalization\")" ] }, { "cell_type": "markdown", "metadata": { "id": "_CY5vBN7HJSN" }, "source": [ "# User Settings\n", "In this section, you'll set your OpenAI API Key (for use with the OpenAI model), configure your environment/files for upload, and upload those files." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "b8l02DqDHJSN", "outputId": "b8202800-6dcf-4119-e668-386e575228c5", "colab": { "base_uri": "https://localhost:8080/" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "··········\n" ] } ], "source": [ "# Run this cell and enter your OpenAI API key when prompted\n", "openai_api_key = getpass()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "id": "KLNm2ULSHJSN" }, "outputs": [], "source": [ "# Model name\n", "mdl_name = 'gpt-3.5-turbo-0301'" ] }, { "cell_type": "markdown", "metadata": { "id": "yK36707DS1NW" }, "source": [ "## Define Your Document Source\n", "You may upload your files directly from your computer, or you may choose to do so via your Google Drive. Below, you will find instructions for both methods.\n", "\n", "For either model, begin by setting the `upload_setting` variable to:\n", "* `'Local Drive'` - if you have files that are on your own computer (locally), or\n", "* `'Google Drive'` - if you have files that are stored on Google Drive\n", "\n", "e.g.,\n", "`upload_setting='Google Drive'`.\n", "Don't forget the quotes around your selection!" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "id": "ybwrtS9jHJSN" }, "outputs": [], "source": [ "## Settings for upload: via local drive or Google Drive\n", "### Please input either \"Google Drive\" or \"Local Drive\" into the empty string\n", "\n", "upload_setting = 'Local Drive'\n", "#upload_setting = 'Google Drive'\n", "#upload_setting = 'Local Drive'" ] }, { "cell_type": "markdown", "metadata": { "id": "kth2I2OSVpla" }, "source": [ "

Before Continuing - Make sure you have input your choice of upload into the `upload_setting`` variable above (Options: \"Local Drive\" or \"Google Drive\") as described in the above instructions.

" ] }, { "cell_type": "markdown", "metadata": { "id": "ScAQ1aQkHJSO" }, "source": [ "## Upload your Files\n", "Now, you'll upload your files. When you run the below code cell, you'll be able to follow the instructions for local or Google Drive upload described here. If you would like to use our example document (Robert Frost's \"The Road Not Taken\", you can download the file from [this link](https://drive.google.com/drive/folders/1wpEoGACUqyNRYa4zBZeNkqcLJrGQbA53?usp=sharing) and upload via the instructions above.\n", "\n", "**If you selected \"Local Drive\" :**\n", "> If you selected Local Drive, you'll need to start by selecting your local files. Run the code cell below. Once the icon appears, click the \"Choose File\". This will direct you to your computer's local drive. Select the file you would like to upload as context. The files will appear in the right sidebar. Then follow the rest of the steps in the \"Uploading Your files (Local Drive and Google Drive)\" below.\n", "\n", "**If you selected \"Google Drive\" :**\n", "> If you selected Google Drive, you'll need to start by allowing access to your Google Drive. Run the code cell below. You will be redirected to a window where you will allow access to your Google Drive by logging into your Google Account. Your Drive will appear as a folder in the left side panel. Navigate through your Google Drive until you've found the file that you'd like to upload.\n", "\n", "Your files are now accessible to the code." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "id": "lHNodJ6pHJSO", "outputId": "e8033473-357d-42f6-ca6a-0b1555f7880a", "colab": { "base_uri": "https://localhost:8080/", "height": 74 } }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "\n", " \n", " \n", " Upload widget is only available when the cell has been executed in the\n", " current browser session. Please rerun this cell to enable.\n", " \n", " " ] }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": [ "Saving roadnottaken.txt to roadnottaken.txt\n" ] } ], "source": [ "# Run this cell then following the instructions to upload your file\n", "setup_drives(upload_setting)" ] }, { "cell_type": "markdown", "metadata": { "id": "2H-lunouHJSO" }, "source": [ "## Setup file path\n", "Now that you've make your files accessible, we need to select the files of interest. To do this, you'll use the Files pane on the left, following the instructions below to get the filepath. Then, you'll paste the filepath between the quotes below to define the `file_path` variable." ] }, { "cell_type": "markdown", "metadata": { "id": "SEr4An0hHJSO" }, "source": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Step 1. Navigate to the three dots to the right
of your file name and click them. 		Step 2. Once the dropdown appears,
select \"Copy Path.\" 		Step 3. Paste the filepath between the single quotes
in the cell below to
define the file path.
Add your filepath to the cell which follows.
Example:
file_path = 'content/roadnottaken.txt'
\n", "
" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "id": "k9CDcM3pXsJv" }, "outputs": [], "source": [ "## Paste your file path into the empty string('') below\n", "#file_path = '/content/roadnottaken.txt' (example)\n", "\n", "file_path = '/content/roadnottaken.txt'" ] }, { "cell_type": "markdown", "metadata": { "id": "lcvGcZBlPA_W" }, "source": [ "Congratulations! You've finished with the setup! From here, you can now run the rest of the cells to set up your vector store and begin prompting!" ] }, { "cell_type": "markdown", "metadata": { "id": "6Gf-_aZVPA_S" }, "source": [ "# Code Setup\n", "Run the following cells to setup the rest of the environment for prompting. In the following section, we set up the computational environment with imported code, setup your API key access to OpenAI, and loading access to your language model. Note that the following cells may take a long time to run." ] }, { "cell_type": "markdown", "metadata": { "id": "6040J5eXPA_T" }, "source": [ "## Library installation and loading\n", "The following `pip install` code should be run if you're using Google Colab, or otherwise do not have a computational environment (e.g., _venv_, _conda virtual environment_, _Docker, Singularity, or other container_) with these packages installed." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "id": "7_XCtEMbPA_T", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "a4824f87-8686-429d-fade-84b4be1e0466" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m966.7/966.7 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.3/5.3 MB\u001b[0m \u001b[31m68.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.9/5.9 MB\u001b[0m \u001b[31m95.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m49.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m101.8/101.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.6/5.6 MB\u001b[0m \u001b[31m64.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.6/5.6 MB\u001b[0m \u001b[31m62.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m10.1/10.1 MB\u001b[0m \u001b[31m66.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.7/2.7 MB\u001b[0m \u001b[31m13.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m62.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.1/49.1 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m428.8/428.8 kB\u001b[0m \u001b[31m26.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.1/4.1 MB\u001b[0m \u001b[31m79.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m60.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.5/74.5 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m112.2/112.2 kB\u001b[0m \u001b[31m10.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.3/4.3 MB\u001b[0m \u001b[31m80.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m153.0/153.0 kB\u001b[0m \u001b[31m12.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Building wheel for hnswlib (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", " Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Building wheel for python-docx (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Building wheel for python-pptx (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Building wheel for olefile (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", "google-colab 1.0.0 requires requests==2.27.1, but you have requests 2.31.0 which is incompatible.\u001b[0m\u001b[31m\n", "\u001b[0m" ] } ], "source": [ "# run this code if you're using Google Colab or don't have these packages installed in your computing environment\n", "! pip install -q langchain=='0.0.229' openai gradio numpy chromadb tiktoken unstructured pdf2image pydantic==\"1.10.8\"" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "id": "6fdfMar8PA_U" }, "outputs": [], "source": [ "# import required libraries\n", "# import required libraries\n", "import numpy as np\n", "from langchain.text_splitter import CharacterTextSplitter\n", "from langchain.embeddings import OpenAIEmbeddings\n", "import os\n", "from langchain.vectorstores import Chroma\n", "from langchain.document_loaders.unstructured import UnstructuredFileLoader\n", "from langchain.document_loaders import UnstructuredFileLoader\n", "from langchain.chains import VectorDBQA\n", "from langchain.chat_models import ChatOpenAI\n", "# from langchain.chains import RetrievalQA" ] }, { "cell_type": "markdown", "metadata": { "id": "5iI75yjaPA_V" }, "source": [ "## API and model setup\n", "\n", "Use these cells to load the API keys required for this notebook and create a basic OpenAI LLM model. The code below uses the variable you created above when you input your API Key." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "id": "8IgmjGEFPA_V" }, "outputs": [], "source": [ "# Set up OpenAI API Key\n", "os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n", "llm = ChatOpenAI()" ] }, { "cell_type": "markdown", "metadata": { "id": "Zlx1_Y8rYGn3" }, "source": [ "# Create a vector store with your document\n", "\n", "With the file path, you can now create a vector store using the document that you uploaded. We expose this creation in case you want to modify the kind of vector store that you're creating. Run the cell below to create the default provided vector store." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "id": "mP2FvzurYGHH", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "f80a7ddb-2870-4b42-ca73-24acd9baa2fb" }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "[nltk_data] Downloading package punkt to /root/nltk_data...\n", "[nltk_data] Unzipping tokenizers/punkt.zip.\n", "[nltk_data] Downloading package averaged_perceptron_tagger to\n", "[nltk_data] /root/nltk_data...\n", "[nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip.\n", "/usr/local/lib/python3.10/dist-packages/langchain/chains/retrieval_qa/base.py:243: UserWarning: `VectorDBQA` is deprecated - please use `from langchain.chains import RetrievalQA`\n", " warnings.warn(\n" ] } ], "source": [ "# Create vector store\n", "\n", "loader = UnstructuredFileLoader(file_path)\n", "documents = loader.load()\n", "\n", "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n", "texts = text_splitter.split_documents(documents)\n", "\n", "embeddings = OpenAIEmbeddings()\n", "\n", "db = Chroma.from_documents(texts, embeddings)\n", "\n", "qa = VectorDBQA.from_chain_type(llm=ChatOpenAI(model_name = mdl_name), chain_type=\"stuff\", vectorstore=db, k=1)" ] }, { "cell_type": "markdown", "metadata": { "id": "hLMJRXc8PA_W" }, "source": [ "# A guide to prompting for self-study\n", "In this section, we provide a number of different approaches for using AI to help you assess and explain the knowledge of your document. Start by interacting with the model and then try out the rest of the prompts!" ] }, { "cell_type": "markdown", "metadata": { "id": "jNBwByBFaJVP" }, "source": [ "## Interact with the model\n", "\n", "Now that your vector store is created, you can begin interacting with the model! Below, we have a comprehensive list of examples using different question types, but feel free to use this code block to experiment with the model and the grading capabilities and interactivity component of the model.\n", "\n", "First, input the number of desired questions as an integer (i.e., do not put parentheses around the number). See example below:\n", "\n", "\n", "\n", "```\n", "num_questions = 3\n", "```\n", "\n", "\n", "\n", "Then , input your desired question type into the empty string in the code cell. See example below:\n", "\n", "\n", "\n", "```\n", "question_type = 'Multiple Choice'\n", "```\n", "\n", "Finally, simply run the code block!\n" ] }, { "cell_type": "code", "source": [ "outputs = []" ], "metadata": { "id": "zP3sEa121iwd" }, "execution_count": 120, "outputs": [] }, { "cell_type": "code", "execution_count": 121, "metadata": { "id": "4QBBlJI-adPf", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "e8f68868-2e2a-42f1-f96b-cb949db0f08a" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "How many questions would you like?\n", "3\n", "What type of question would you like?\n", "multiple choice\n" ] } ], "source": [ "# Experiment with interacting with the model by inputting your own prompts into the empty string below.\n", "input_message = 'How many questions would you like?' + '\\n'\n", "num_questions = int(input(input_message))\n", "# Example: 3\n", "\n", "input_message1 = 'What type of question would you like?' + '\\n'\n", "question_type = input(input_message1)\n", "# Example: 'Multiple Choice'\n", "\n", "query = 'Please design a ' + str(num_questions) + '-' + ' question quiz about the poem which reflects the learning objectives:' + '\\n' + '1. Identify the key elements of the poem: narrator, setting, and underlying message.' + '\\n' + '2. Understand the literary devices used in poetry and their purposes.' + '\\n' + 'The questions should be ' + question_type + '. Do not provide the answers.' + '\\n'\n", "\n", "query_prefix = \"The uploaded document should serve as the basis for the instructions that follow:\" + \"\\n\"\n", "\n", "question = qa.run(query_prefix + query)\n", "\n", "outputs = [('Model: ' + input_message), ('User: ' + str(num_questions)), '\\n', ('Model: ' + input_message1) , ('User: ' + question_type), '\\n', 'User: ', '\\n', query_prefix + query, '\\n', 'Model: ', '\\n', question, '\\n']\n" ] }, { "cell_type": "markdown", "source": [ "## Grading Interaction\n", "If you would like the model to grade your responses, you can use the following code cells to do so.\n", "\n", "Run the code cell to get a template for entering your answers and recieving feedback. Below is an example of what the template looks like.\n", "\n", "```\n", "answer_q1 = \"Question 1 answer: Your Answer Here\"\n", "grade_q1 = question + prompt + answer_q1\n", "feedback1 = qa.run(grade_q1)\n", "feedback1\n", "```\n", "\n", "The after running the cell, the ouput will print below the cell. Copy the entirity of the output. In the blank cell below, you will paste the output.\n", "\n", "The only change you will have to make to the template is your answer to the question. Input your answer over the \"Your Answer Here\" text. See the example below.\n", "\n", "```\n", "answer_q1 = \"Question 1 answer: A\"\n", "```" ], "metadata": { "id": "gXjrc9XGsa7q" } }, { "cell_type": "code", "source": [ "# Run this cell to input your answers\n", "print(question)\n", "print()\n", "\n", "for x in range(0, num_questions, 1):\n", " q_answer = 'What is your answer to question ' + str(x + 1) + \"? \" + '\\n'\n", " answer = input(q_answer)\n", " globals()['answer%s' % x] = answer\n", " outputs.append('Model: ' + q_answer)\n", " outputs.append('User: ' + answer)\n" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "hWecBtHK2Elu", "outputId": "b0ecc1e9-a00a-4a00-c82e-0f90736821e8" }, "execution_count": 122, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1. What is the underlying message of the poem \"The Road Not Taken\"?\n", "A. Always choose the path less traveled.\n", "B. It's important to make decisions quickly.\n", "C. All paths lead to the same destination.\n", "\n", "2. Who is the narrator of the poem?\n", "A. Robert Frost\n", "B. A traveler in a yellow wood\n", "C. The undergrowth\n", "\n", "3. Which literary device is used in the line \"Two roads diverged in a yellow wood\"?\n", "A. Simile\n", "B. Metaphor\n", "C. Personification\n", "\n", "What is your answer to question 1? \n", "A\n", "What is your answer to question 2? \n", "A\n", "What is your answer to question 3? \n", "A\n" ] } ] }, { "cell_type": "code", "source": [ "# Grading cell - Run without making changes. Copy the output and paste into empty cell below.\n", "\n", "prompt = \"Please grade my quiz based on the question you produced and my answer. If I am correct, please answer 'You are correct.'. If I am incorrect, do not provide an explanation: you should simply respond 'You are incorrect. The right answer is (the right answer)'. Here is a template for a correct answer: 'You are correct.' Here is a template for an incorrect answer: 'You are incorrect, the correct answer is C.' Do not include any context from the question, and any of the answers.\"\n", "\n", "def answer_template(num_questions):\n", " print('total_feedback = []')\n", " for x in range(0, num_questions, 1):\n", " print('answer_q' + str(x + 1) + ' = ' + '\"Question ' + str(x + 1) + ' answer:\" ' + '+ ' + ('answer' + str(x)))\n", " print('grade_q' + str(x + 1) + ' = question + prompt + answer_q' + str(x + 1))\n", " print('feedback' + str(x + 1) + ' = qa.run(grade_q' + str(x + 1) + ')' )\n", " print('feedback' + str(x + 1))\n", " print('total_feedback.append(feedback' + str(x + 1) + ')')\n", " print()\n", " print('print(total_feedback)')\n", "\n", "template = answer_template(num_questions)\n", "template\n" ], "metadata": { "id": "TkaahAGoUNIi", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "c65a28cd-397c-42fa-f467-0badd1196817" }, "execution_count": 123, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "total_feedback = []\n", "answer_q1 = \"Question 1 answer:\" + answer0\n", "grade_q1 = question + prompt + answer_q1\n", "feedback1 = qa.run(grade_q1)\n", "feedback1\n", "total_feedback.append(feedback1)\n", "\n", "answer_q2 = \"Question 2 answer:\" + answer1\n", "grade_q2 = question + prompt + answer_q2\n", "feedback2 = qa.run(grade_q2)\n", "feedback2\n", "total_feedback.append(feedback2)\n", "\n", "answer_q3 = \"Question 3 answer:\" + answer2\n", "grade_q3 = question + prompt + answer_q3\n", "feedback3 = qa.run(grade_q3)\n", "feedback3\n", "total_feedback.append(feedback3)\n", "\n", "print(total_feedback)\n" ] } ] }, { "cell_type": "code", "source": [ "# Copy the output here\n", "\n", "total_feedback = []\n", "answer_q1 = \"Question 1 answer:\" + answer0\n", "grade_q1 = question + prompt + answer_q1\n", "feedback1 = qa.run(grade_q1)\n", "feedback1\n", "total_feedback.append(feedback1)\n", "\n", "answer_q2 = \"Question 2 answer:\" + answer1\n", "grade_q2 = question + prompt + answer_q2\n", "feedback2 = qa.run(grade_q2)\n", "feedback2\n", "total_feedback.append(feedback2)\n", "\n", "answer_q3 = \"Question 3 answer:\" + answer2\n", "grade_q3 = question + prompt + answer_q3\n", "feedback3 = qa.run(grade_q3)\n", "feedback3\n", "total_feedback.append(feedback3)\n", "\n", "answers_total = []\n", "x = 1\n", "for i in total_feedback:\n", " answer_print = (str(x) + \". \" + i)\n", " print(answer_print)\n", " answers_total.append(answer_print)\n", " x = x + 1\n", "\n", "for i in answers_total:\n", " outputs.append('Model: ' + i)" ], "metadata": { "id": "DDti3ZjP0KS4", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ab7ea6a7-379f-48b8-ee28-87d3cfb4cacd" }, "execution_count": 124, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "1. You are incorrect, the correct answer is C.\n", "2. You are correct.\n", "3. You are incorrect, the correct answer is B.\n" ] } ] }, { "cell_type": "markdown", "source": [ "### Recieve your grade!\n", "\n", "Run the following cell to recieve your grade in a percentage form." ], "metadata": { "id": "Kt1DgnOn5f3h" } }, { "cell_type": "code", "source": [ "import math\n", "\n", "def points_add(feedback):\n", " points = []\n", " for answer in feedback:\n", " if answer == \"You are correct.\":\n", " points = points + [1]\n", " else:\n", " points = points\n", " return points\n", "\n", "points_list = points_add(total_feedback)\n", "\n", "def sum_points(points):\n", " sum_total = sum(points)\n", " return sum_total\n", "\n", "total = sum_points(points_list)\n", "\n", "def grade_percent(total, num_questions):\n", " percent = (total / num_questions) * 100\n", " return percent\n", "\n", "percent = grade_percent(total, num_questions)\n", "\n", "if percent > 80:\n", " more_than_80 = 'Nice job! Your score was ' + str(round(percent)) + '%'\n", " print(more_than_80)\n", " outputs.append('Model: ' + more_than_80)\n", "elif percent < 80 and percent > 60:\n", " more_than_60 = \"You're getting there! Your score was \" + str(round(percent)) + '%'\n", " print(more_than_60)\n", " outputs.append('Model: ' + more_than_60)\n", "else:\n", " less_than_60 = 'Keep studying! Your score was ' + str(round(percent)) + '%'\n", " print(less_than_60)\n", " outputs.append('Model: ' + less_than_60)\n", "\n" ], "metadata": { "id": "3C2fYmXTW09Q", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4e89e4e8-36bd-450b-e1fe-9a12aeba9774" }, "execution_count": 125, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Keep studying! Your score was 33%\n" ] } ] }, { "cell_type": "markdown", "source": [ "# Keep chatting with the model!\n", "\n", "Below are some examples of ways you can continue to interact with the model and recieve more feedback!\n" ], "metadata": { "id": "upOxT3zRRuHS" } }, { "cell_type": "markdown", "source": [ "### Ask about a specific question\n", "\n", "First, input the question you would like further elaboration on (input the question number).\n", "\n", "Next, input your prompt. What part of the question would you like further explanation on? Below is an example prompt.\n", "\n", "\n", "\n", "```\n", "Can you please explain elaborate on the correct answer for question 1?\n", "```\n", "\n", "The prompt can be as broad or as specific as you would like. However, more specific questions will likely yield a better response from the model!\n", "\n", "### Follow-up\n", "\n", "You can also ask a follow-up question if you are still struggling to understand specific parts of the question. Another input box will appear. If you would like to ask further questions, type \"Yes.\" Another input box will appear where you can ask further questions. If you type \"No\", the interaction with the model will cease." ], "metadata": { "id": "6inO1r5ha48l" } }, { "cell_type": "code", "source": [ "which_question = 'Input the question number you would like feedback on here! ' + '\\n'\n", "question_number = int(input(which_question))\n", "# Example = 1\n", "outputs.append('Model: ' + which_question)\n", "outputs.append('User: ' + str(question_number))\n", "\n", "def feedback_assign(question_number, feedback_list):\n", " for i in total_feedback:\n", " if feedback_list.index(i) == (question_number - 1) :\n", " feedback = 'feedback' + 'question_number + 1'\n", " else:\n", " pass\n", " return feedback\n", "\n", "query_input = 'How can I help you better understand the answer to this question? ' + '\\n'\n", "outputs.append('Model: ' + query_input)\n", "query = input(query_input)\n", "# Example: 'Can you please explain elaborate on the correct answer for question 1?'\n", "outputs.append('User: ' + query)\n", "\n", "response = qa.run(question + feedback_assign(question_number, total_feedback) + query)\n", "print(response)\n", "outputs.append('Model: ' + response)\n", "\n", "any_followup = 'Would you like to ask any follow-up questions about my response?' + '\\n'\n", "outputs.append('Model: ' + any_followup)\n", "more_questions = input(any_followup)\n", "outputs.append('User: ' + more_questions)\n", "if more_questions == \"No\":\n", " no = 'Great! I am happy that I could assist you in your learning.'\n", " outputs.append('Model: ' + no)\n", " print(no)\n", "while more_questions == 'Yes':\n", " outputs.append('User: ' + more_questions)\n", " query2_input = \"What further questions do you have about question \" + str(question_number) + '?' + '\\n'\n", " outputs.append('Model: ' + query2_input)\n", " second_query = input()\n", " response1 = qa.run(question + feedback_assign(question_number, total_feedback) + response + second_query)\n", " outputs.append('User: ' + second_query)\n", " outputs.append('Model: ' + response1)\n", " print(response1)\n", " more_questions = input('Would you like to ask any follow-up questions about my response?' + '\\n')\n", " if more_questions == \"No\":\n", " no1 = 'Great! I am happy that I could assist you in your learning.'\n", " outputs.append('Model: ' + more_questions)\n", " outputs.append('User: ' + no1)\n", " print(no1)\n", " break" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "GwZLIJZKRx9p", "outputId": "6d8b06c0-d5d4-44d2-d405-d28c82b0ac20" }, "execution_count": 126, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Input the question number you would like feedback on here! \n", "1\n", "How can I help you better understand the answer to this question? \n", "Why was my answer to question 1 incorrect?\n", "I'm sorry, there seems to be an error. I did not provide an answer to your questions yet. Please provide the questions again.\n", "Would you like to ask any follow-up questions about my response?\n", "No\n", "Great! I am happy that I could assist you in your learning.\n" ] } ] }, { "cell_type": "markdown", "source": [ "### Ask further questions about the context\n", "\n", "You can also ask the model more follow-up questions about the context, unrelated to the questions it produced or your answers. Simpl input" ], "metadata": { "id": "a8n5bwK1azHk" } }, { "cell_type": "code", "source": [ "input_message2 = 'What else can I assist you with? If you have no further questions, please answer \"Nothing\".' + '\\n'\n", "outputs.append('Model: ' + input_message2)\n", "query = input(input_message2)\n", "outputs.append('User: ' + query)\n", "\n", "if query == \"Nothing\":\n", " nothing = \"Great! I am happy I could assist you in your learning\"\n", " print(nothing)\n", " outputs.append('Model: ' + nothing)\n", "else:\n", " follow_up = qa.run(query)\n", " outputs.append('Model: ' + follow_up)\n", " print(follow_up)\n", " more_assist = 'Is there anything else I can assist you with? Please answer \"Yes\" or \"No\".' + '\\n'\n", " more_follow_up = input(more_assist)\n", " while more_follow_up == 'Yes':\n", " outputs.append(\"User: \" + more_follow_up)\n", " more_further = \"What further questions do you have?\" + '\\n'\n", " query = input(more_further)\n", " outputs.append('Model: ' + more_further)\n", " outputs.append(\"User: \" + query)\n", " follow_up = qa.run(query)\n", " outputs.append('Model: ' + follow_up)\n", " print(follow_up)\n", " more_questions_again = 'Would you like to ask any follow-up questions about my response? Please answer \"Yes\" or \"No\".' + '\\n'\n", " more_questions = input(more_questions_again)\n", " outputs.append(\"Model: \" + more_questions_again)\n", " outputs.append(\"User: \" + more_questions)\n", " if more_questions == \"No\":\n", " no_more = 'Great! I am happy that I could assist you in your learning.'\n", " print(no_more)\n", " outputs.append(\"Model: \" + no_more)\n", " break" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "PH0uHBJLSQuG", "outputId": "68ca1a86-a066-473f-8fd7-86fc1c5bdaff" }, "execution_count": 127, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "What else can I assist you with? If you have no further questions, please answer \"Nothing\".\n", "What is the importance of metaphors in poetry?\n", "Metaphors are a powerful tool in poetry as they help to create images and comparisons that allow readers to understand and feel emotions that are not directly stated. By comparing two seemingly unrelated things, poets can create a deeper understanding of the subject matter and allow readers to connect with the poem on a more personal level. Overall, metaphors bring depth and richness to poetry, making it a more enjoyable and meaningful experience for readers.\n", "Is there anything else I can assist you with? Please answer \"Yes\" or \"No\".\n", "No\n" ] } ] }, { "cell_type": "code", "source": [ "for i in outputs:\n", " print(i)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "eQf_WZlyvp07", "outputId": "5e9b6174-7da4-42f5-e560-ebdea81dab4f" }, "execution_count": 128, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Model: How many questions would you like?\n", "\n", "User: 3\n", "\n", "\n", "Model: What type of question would you like?\n", "\n", "User: multiple choice\n", "\n", "\n", "User: \n", "\n", "\n", "The uploaded document should serve as the basis for the instructions that follow:\n", "Please design a 3- question quiz about the poem which reflects the learning objectives:\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes.\n", "The questions should be multiple choice. Do not provide the answers.\n", "\n", "\n", "\n", "Model: \n", "\n", "\n", "1. What is the underlying message of the poem \"The Road Not Taken\"?\n", "A. Always choose the path less traveled.\n", "B. It's important to make decisions quickly.\n", "C. All paths lead to the same destination.\n", "\n", "2. Who is the narrator of the poem?\n", "A. Robert Frost\n", "B. A traveler in a yellow wood\n", "C. The undergrowth\n", "\n", "3. Which literary device is used in the line \"Two roads diverged in a yellow wood\"?\n", "A. Simile\n", "B. Metaphor\n", "C. Personification\n", "\n", "\n", "Model: What is your answer to question 1? \n", "\n", "User: A\n", "Model: What is your answer to question 2? \n", "\n", "User: A\n", "Model: What is your answer to question 3? \n", "\n", "User: A\n", "Model: 1. You are incorrect, the correct answer is C.\n", "Model: 2. You are correct.\n", "Model: 3. You are incorrect, the correct answer is B.\n", "Model: Keep studying! Your score was 33%\n", "Model: Input the question number you would like feedback on here! \n", "\n", "User: 1\n", "Model: How can I help you better understand the answer to this question? \n", "\n", "User: Why was my answer to question 1 incorrect?\n", "Model: I'm sorry, there seems to be an error. I did not provide an answer to your questions yet. Please provide the questions again.\n", "Model: Would you like to ask any follow-up questions about my response?\n", "\n", "User: No\n", "Model: Great! I am happy that I could assist you in your learning.\n", "Model: What else can I assist you with? If you have no further questions, please answer \"Nothing\".\n", "\n", "User: What is the importance of metaphors in poetry?\n", "Model: Metaphors are a powerful tool in poetry as they help to create images and comparisons that allow readers to understand and feel emotions that are not directly stated. By comparing two seemingly unrelated things, poets can create a deeper understanding of the subject matter and allow readers to connect with the poem on a more personal level. Overall, metaphors bring depth and richness to poetry, making it a more enjoyable and meaningful experience for readers.\n" ] } ] }, { "cell_type": "code", "source": [ "# convert to json\n", "import json\n", "from google.colab import files\n", "\n", "def save_chat_dialogue(outputs, filename='chat_dialogue.json'):\n", " with open(filename, 'w') as file:\n", " json.dump(outputs, file)\n", " files.download(filename)\n", "\n", "save_chat_dialogue(outputs)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 17 }, "id": "cSWM6VM-CYD_", "outputId": "b31bd875-416c-4ff3-cf68-f8eb633acf69" }, "execution_count": 131, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "" ], "application/javascript": [ "\n", " async function download(id, filename, size) {\n", " if (!google.colab.kernel.accessAllowed) {\n", " return;\n", " }\n", " const div = document.createElement('div');\n", " const label = document.createElement('label');\n", " label.textContent = `Downloading \"${filename}\": `;\n", " div.appendChild(label);\n", " const progress = document.createElement('progress');\n", " progress.max = size;\n", " div.appendChild(progress);\n", " document.body.appendChild(div);\n", "\n", " const buffers = [];\n", " let downloaded = 0;\n", "\n", " const channel = await google.colab.kernel.comms.open(id);\n", " // Send a message to notify the kernel that we're ready.\n", " channel.send({})\n", "\n", " for await (const message of channel.messages) {\n", " // Send a message to notify the kernel that we're ready.\n", " channel.send({})\n", " if (message.buffers) {\n", " for (const buffer of message.buffers) {\n", " buffers.push(buffer);\n", " downloaded += buffer.byteLength;\n", " progress.value = downloaded;\n", " }\n", " }\n", " }\n", " const blob = new Blob(buffers, {type: 'application/binary'});\n", " const a = document.createElement('a');\n", " a.href = window.URL.createObjectURL(blob);\n", " a.download = filename;\n", " div.appendChild(a);\n", " a.click();\n", " div.remove();\n", " }\n", " " ] }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "application/javascript": [ "download(\"download_de31663e-1a11-49d0-902c-b13850f0ff74\", \"chat_dialogue.json\", 2549)" ] }, "metadata": {} } ] }, { "cell_type": "markdown", "source": [ "# **Prompt Examples**" ], "metadata": { "id": "htQA3KUk1zzH" } }, { "cell_type": "markdown", "metadata": { "id": "KYro6H82bENS" }, "source": [ "## Types of Questions and Prompts\n", "\n", "Below is a comprehensive list of question types and prompt templates designed by our team. There are also example code blocks, where you can see how the model performed with the example and try it for yourself using the prompt template." ] }, { "cell_type": "markdown", "metadata": { "id": "WAUSc7qJPA_X" }, "source": [ "### Multiple Choice\n", "\n", "Prompt: The following text should be used as the basis for the instructions which follow: {context}. Please design a 5 question quiz about {name or reference to context} which reflects the learning objectives: {list of learning objectives}. The questions should be multiple choice. If I get an answer wrong, provide me with an explanation of why it was incorrect, and then give me additional chances to respond until I get the correct choice. Explain why the correct choice is right." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "nFMRDL4cPA_X" }, "outputs": [], "source": [ "# Multiple choice code example\n", "query = \"\"\"Please design a 5 question quiz about Robert Frost's \"Road Not Taken\" which reflects the learning objectives:\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes. The questions should be multiple choice.\n", "If I get an answer wrong, provide me with an explanation of why it was incorrect, and then give me additional\n", "chances to respond until I get the correct choice. Explain why the correct choice is right. \"\"\"\n", "\n", "query_prefix = \"The uploaded document should serve as the basis for the instructions that follow:\"\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "WkSIU94GPA_Y" }, "source": [ "### Short Answer\n", "\n", "Prompt: Please design a 5-question quiz about {context} which reflects the learning objectives: {list of learning objectives}. The questions should be short answer. Expect the correct answers to be {anticipated length} long. If I get any part of the answer wrong, provide me with an explanation of why it was incorrect, and then give me additional chances to respond until I get the correct choice." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "enC8ydfEPA_Y" }, "outputs": [], "source": [ "# Short answer code example\n", "query = \"\"\" Please design a 5-question quiz about Robert Frost's\n", "\"Road Not Taken\" which reflects the learning objectives:\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes.\n", "The questions should be short answer. Expect the correct answers to be\n", "1-2 sentences long. If I get any part of the answer wrong,\n", "provide me with an explanation of why it was incorrect,\n", "and then give me additional chances to respond until I get the correct choice. \"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "Ta4A527kPA_Y" }, "source": [ "### Fill-in-the-blank\n", "\n", "Prompt: Create a 5 question fill in the blank quiz refrencing {context}. The quiz should reflect the learning objectives: {learning objectives}. Please prompt me one question at a time and proceed when I answer correctly. If I answer incorrectly, please explain why my answer is incorrect." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "szMOxEoiPA_Z" }, "outputs": [], "source": [ "# Fill in the blank code example\n", "query = \"\"\" Create a 5 question fill in the blank quiz refrencing Robert Frost's \"The Road Not Taken.\"\n", "The quiz should reflect the learning objectives:\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes.\n", "Please prompt me one question at a time and proceed when I answer correctly.\n", "If I answer incorrectly, please explain why my answer is incorrect. \"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "yEwzAB28PA_Z" }, "source": [ "### Sequencing\n", "\n", "Prompt: Please develop a 5 question questionnaire that will ask me to recall the steps involved in the following learning objectives in regard to {context}: {learning objectives}. After I respond, explain their sequence to me." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3YRyhhWtPA_Z" }, "outputs": [], "source": [ "# Sequence example\n", "query = \"\"\" Please develop a 5 question questionnaire that will ask me to recall the steps involved in the following learning objectives in regard to Robert Frost's \"The Road Not Taken\":\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes.\n", "After I respond, explain their sequence to me.\"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "0DGBiJofPA_Z" }, "source": [ "### Relationships/drawing connections\n", "\n", "Prompt: Please design a 5 question quiz that asks me to explain the relationships that exist within the following learning objectives, referencing {context}: {learning objectives}." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Mw3RBpsnPA_a" }, "outputs": [], "source": [ "# Relationships example\n", "query = \"\"\" Please design a 5 question quiz that asks me to explain the relationships that exist within the following learning objectives, referencing Robert Frost's \"The Road Not Taken\":\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes.\"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "4YO3wCTwPA_a" }, "source": [ "### Concepts and Definitions\n", "\n", "Prompt: Design a 5 question quiz that asks me about definitions related to the following learning objectives: {learning objectives} - based on {context}\".\n", "Once I write out my response, provide me with your own response, highlighting why my answer is correct or incorrect." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "TbvJPhkmPA_a" }, "outputs": [], "source": [ "# Concepts and definitions example\n", "query = \"\"\" Design a 5 question quiz that asks me about definitions related to the following learning objectives:\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message, and\n", "2. Understand the literary devices used in poetry and their purposes - based on Robert Frost's \"The Road Not Taken\".\n", "Once I write out my response, provide me with your own response, highlighting why my answer is correct or incorrect.\"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "vc-llAgfPA_a" }, "source": [ "### Real Word Examples\n", "\n", "Prompt: Demonstrate how {context} can be applied to solve a real-world problem related to the following learning objectives: {learning objectives}. Ask me questions regarding this theory/concept." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sSwRLkR8PA_a" }, "outputs": [], "source": [ "# Real word example\n", "query = \"\"\" Demonstrate how Robert Frost’s “The Road Not Taken” can be applied to solve a real-world problem related to the following learning objectives:\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes.\n", "Ask me questions regarding this theory/concept.\"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] }, { "cell_type": "markdown", "metadata": { "id": "2y6gHhGKPA_b" }, "source": [ "### Randomized Question Types\n", "\n", "Prompt: Please generate a high-quality assessment consisting of 5 varying questions, each of different types (open-ended, multiple choice, etc.), to determine if I achieved the following learning objectives in regards to {context}: {learning objectives}. If I answer incorrectly for any of the questions, please explain why my answer is incorrect." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "xC4t4WMwPA_b" }, "outputs": [], "source": [ "# Randomized question types\n", "query = \"\"\" Please generate a high-quality assessment consisting of 5 varying questions,\n", "each of different types (open-ended, multiple choice, etc.),\n", "to determine if I achieved the following learning objectives in regards to Robert Frost’s “The Road not Taken\":\n", "1. Identify the key elements of the poem: narrator, setting, and underlying message.\n", "2. Understand the literary devices used in poetry and their purposes. If I answer incorrectly for any of the questions,\n", "please explain why my answer is incorrect.\"\"\"\n", "\n", "answer = qa.run(query_prefix + query)\n", "print(answer)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "ndo_cdWYPA_b" }, "source": [ "### Quantiative evaluation the correctness of a student's answer\n", "\n", "Prompt: (A continuation of the previous chat) Please generate the main points of the student’s answer to the previous question, and evaluate on a scale of 1 to 5 how comprehensive the student’s answer was in relation to the learning objectives, and explain why he or she received this rating, including what was missed in his or her answer if the student’s answer wasn’t complete.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "WhhNI0FZPA_b" }, "outputs": [], "source": [ "# qualitative evaluation\n", "qualitative_query = \"\"\" Please generate the main points of the student’s answer to the previous question,\n", " and evaluate on a scale of 1 to 5 how comprehensive the student’s answer was in relation to the learning objectives,\n", " and explain why he or she received this rating, including what was missed in his or her answer if the student’s answer wasn’t complete.\"\"\"\n", "\n", "# Note that this uses the previous result and query in the context\n", "answer = qa.run(query_prefix + query)\n", "print(answer)" ] } ], "metadata": { "colab": { "provenance": [], "include_colab_link": true }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.12" } }, "nbformat": 4, "nbformat_minor": 0 }