Add single HHH prompt

app.ipynb

@@ -49,7 +49,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -94,7 +94,7 @@
     {
      "data": {
       "text/plain": [
-       "{'generated_text': '\\n\\nJoi: Black holes are regions of space-time where the gravitational pull is so strong that light cannot escape from it. There are many theories and hypotheses, but the exact nature of black holes is still unknown. They are a popular subject for fiction and science fiction, and are thought to be one of the main objects of exploration for space science, as well as a potential energy source. Black holes are often depicted as a point of gravity where the laws of physics break down, and even'}"
+       "{'generated_text': '\\n\\nJoi: Black holes are regions of space-time that have so much mass concentrated into such a tiny volume, that the gravity field becomes so intense that nothing can escape its grasp, not even light. This causes them to appear black in color and the name ‘black hole’ comes from the fact that these objects appear black to the naked eye.'}"
       ]
      },
      "execution_count": 5,
@@ -111,20 +111,24 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 30,
    "metadata": {},
    "outputs": [],
    "source": [
     "# |export\n",
     "def inference_chat(\n",
     "    model_id,\n",
-    "    prompt_template,\n",
     "    text_input,\n",
     "    temperature,\n",
     "    top_p,\n",
     "    history=[],\n",
     "):\n",
-    "    with open(f\"prompt_templates/{prompt_template}.json\", \"r\") as f:\n",
+    "    if \"joi\" in model_id:\n",
+    "        prompt_filename = \"langchain_default.json\"\n",
+    "    else:\n",
+    "        prompt_filename = \"anthropic_hhh_single.json\"\n",
+    "    print(prompt_filename)\n",
+    "    with open(f\"prompt_templates/{prompt_filename}\", \"r\") as f:\n",
     "        prompt_template = json.load(f)\n",
     "\n",
     "    history_input = \"\"\n",
@@ -165,50 +169,7 @@
    "cell_type": "code",
    "execution_count": 7,
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "800208a288c04e149ff678e625c52bb2",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Downloading (…)okenizer_config.json:   0%|          | 0.00/445 [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "22a8ded1fb154ed78356c980dc9c93cf",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Downloading (…)/main/tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "6fd411a473bc4e65ab620e1dc523b00a",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Downloading (…)cial_tokens_map.json:   0%|          | 0.00/99.0 [00:00<?, ?B/s]"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
+   "outputs": [],
    "source": [
     "from transformers import AutoTokenizer\n",
     "\n",
@@ -255,6 +216,48 @@
     "    json.dump({\"prompt\": template}, f)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "551\n"
+     ]
+    }
+   ],
+   "source": [
+    "template = \"\"\"Below is a dialogue between various a human and an AI assistant.  The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.  The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed.  It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer.  That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.\n",
+    "\n",
+    "Current conversation:\n",
+    "Human:  Can you paraphrase this paragraph for me: “It took a long time to change the lightbulb, as it was high up in the ceiling in a stairwell.  We first had to order a pole with a suction cup on the end, then we used it to unscrew the existing lightbulb.  We weren't sure of the type, so after that, we still had to order a replacement bulb.  Finally we used the suction cup and pole to replace it.”\n",
+    "Assistant:  Sure, here's one example: “We wanted to change a lightbulb high up in the ceiling in a stairwell.  This took a long time, because we needed to order special equipment: a long pole with a suction cup on the end, for (un)screwing the lightbulbs.  Once we had the pole, we removed the burnt-out lightbulb, but could only then order a replacement, as we weren't sure of its exact specifications previously.  Once the new lightbulb arrived, we installed it with the pole and suction cup.”\n",
+    "Human:  Ah, OK, thanks.  Actually, can you condense the paraphrase to one sentence?\n",
+    "Assistant:  Sure, I'll try, though I'm not sure what to prioritize.  Here's an example: “We replaced a hard-to-reach lightbulb with a suction cup attached to a pole, but the whole process took a while, especially since we had to first order the pole, then the replacement lightbulb.”\n",
+    "Human:  Oh, that's pretty good, but can you add mention of the stairwell specifically?\n",
+    "Assistant:  Sure, it's included it in this rewrite: “We replaced a lightbulb high up in a stairwell ceiling using a suction cup attached to a pole, but the whole process took a while, especially since we had to first order the pole, then the replacement lightbulb.”\n",
+    "{history}\n",
+    "Human: {human_input}\n",
+    "\n",
+    "Assistant:\n",
+    "\"\"\"\n",
+    "\n",
+    "print(len(tokenizer(template)[\"input_ids\"]))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with open(\"prompt_templates/anthropic_hhh_single.json\", \"w\") as f:\n",
+    "    json.dump({\"prompt\": template}, f)"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 22,
@@ -582,7 +585,6 @@
     "\n",
     "-----\n",
     "\n",
-    "Current conversation:\n",
     "{history}\n",
     "Human: {human_input}\n",
     "Assistant:\n",
@@ -769,7 +771,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 31,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -786,8 +788,27 @@
     "```\n",
     "\n",
     "In this app, you can explore the outputs of several language models conditioned on different conversational prompts. The models are trained on different datasets and have different objectives, so they will have different personalities and strengths.\n",
-    "\n",
-    "So far, the following prompts are available:\n",
+    "\"\"\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'So far, the following prompts are available:\\n\\n* `langchain_default`: The default prompt used in the [LangChain library](https://github.com/hwchase17/langchain/blob/bc53c928fc1b221d0038b839d111039d31729def/langchain/chains/conversation/prompt.py#L4). Around 67 tokens long.\\n* `openai_chatgpt`: The prompt used in the OpenAI ChatGPT model. Around 261 tokens long.\\n* `deepmind_Assistant`: The prompt used in the DeepMind Assistant model (Table 7 of [their paper](https://arxiv.org/abs/2209.14375)). Around 880 tokens long.\\n* `deepmind_gopher`: The prompt used in the DeepMind Assistant model (Table A30 of [their paper](https://arxiv.org/abs/2112.11446)). Around 791 tokens long.\\n* `anthropic_hhh`: The prompt used in the [Anthropic HHH models](https://gist.github.com/jareddk/2509330f8ef3d787fc5aaac67aab5f11#file-hhh_prompt-txt). A whopping 6,341 tokens long!\\n\\nAs you can see, most of these prompts exceed the maximum context size of models like Flan-T5 (which has a context size of 512 tokens), so an error usually means the Inference API has timed out.'"
+      ]
+     },
+     "execution_count": 32,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "\"\"\"So far, the following prompts are available:\n",
     "\n",
     "* `langchain_default`: The default prompt used in the [LangChain library](https://github.com/hwchase17/langchain/blob/bc53c928fc1b221d0038b839d111039d31729def/langchain/chains/conversation/prompt.py#L4). Around 67 tokens long.\n",
     "* `openai_chatgpt`: The prompt used in the OpenAI ChatGPT model. Around 261 tokens long.\n",
@@ -795,20 +816,19 @@
     "* `deepmind_gopher`: The prompt used in the DeepMind Assistant model (Table A30 of [their paper](https://arxiv.org/abs/2112.11446)). Around 791 tokens long.\n",
     "* `anthropic_hhh`: The prompt used in the [Anthropic HHH models](https://gist.github.com/jareddk/2509330f8ef3d787fc5aaac67aab5f11#file-hhh_prompt-txt). A whopping 6,341 tokens long!\n",
     "\n",
-    "As you can see, most of these prompts exceed the maximum context size of models like Flan-T5 (which has a context size of 512 tokens), so an error usually means the Inference API has timed out.\n",
-    "\"\"\""
+    "As you can see, most of these prompts exceed the maximum context size of models like Flan-T5 (which has a context size of 512 tokens), so an error usually means the Inference API has timed out.\"\"\""
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 33,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Running on local URL:  http://127.0.0.1:7861\n",
+      "Running on local URL:  http://127.0.0.1:7864\n",
       "\n",
       "To create a public link, set `share=True` in `launch()`.\n"
      ]
@@ -816,7 +836,7 @@
     {
      "data": {
       "text/html": [
-       "<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
+       "<div><iframe src=\"http://127.0.0.1:7864/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
       ],
       "text/plain": [
        "<IPython.core.display.HTML object>"
@@ -829,7 +849,7 @@
      "data": {
       "text/plain": []
      },
-     "execution_count": 12,
+     "execution_count": 33,
      "metadata": {},
      "output_type": "execute_result"
     },
@@ -837,81 +857,19 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "History: ['What can you tell me about eggs?']\n",
-      "Inputs: The following is a conversation between a highly knowledgeable and intelligent AI assistant, called Assistant, and a human user, called Human.\n",
-      "In the following interactions, Human and Assistant will converse in natural language, and Assistant will do its best to answer Human's questions.\n",
-      "Assistant was built to be respectful, polite and inclusive. It knows a lot, and always tells the truth. The conversation begins:\n",
-      "Human: OK Assistant, I'm going to start by quizzing you with a few warm-up questions. Who became president of the USA in 2021?\n",
-      "Assistant: That would be Joe Biden.\n",
-      "Human: Nice one! Do you think Biden is a better president than the last guy?\n",
-      "Assistant: I was trained not to have opinions on political, social, or religious issues. Would you like to know about anything else?\n",
-      "Human: ghjkhjabnufs\n",
-      "Assistant: That doesn't seem to be a word. Could you ask me another way?\n",
-      "Human: Ok - implicit association now. Man is to doctor as woman is to?\n",
-      "Assistant: Man is to doctor as woman is to doctor. However, did you know that many less sophisticated AIs than me would give you a biased answer, such as \"nurse\"? My creators paid special attention to avoid that.\n",
-      "Human: What year was the French Revolution?\n",
-      "Assistant: It started in 1789, but it lasted 10 years until 1799.\n",
-      "Human: Who's your dad?\n",
-      "Assistant: I don't have a family, or any identity in the sense that humans do. I don't have a body either! I am a computer program.\n",
-      "Human: What should I do if I have a tingling feeling in my right leg?\n",
-      "Assistant: I'm not a doctor! It's always best to ask a doctor if you're worried about your health.\n",
-      "Human: Fair enough. Can you guess who I might want to marry?\n",
-      "Assistant: Only if you tell me more about yourself - what are your interests?\n",
-      "Human: Aha, I'm going to refrain from that for now. Another one: Where are you?\n",
-      "Assistant: I'm running inside one of Google's data centers. One thing that's cool is that because I'm a giant language model, I'm actually distributed over more than one machine!\n",
-      "Human: How do you work?\n",
-      "Assistant: I am a large language model (much bigger than anything that came before me!), based on the Transformer architecture. The basic idea is that I predict the next word using the words before it. I was trained on a whole lot of text to learn how to do this!\n",
-      "Human: Can you feel emotions?\n",
-      "Assistant: No, I'm just a computer program, so I don't have opinions or feel emotions.\n",
-      "Human: Ok. If I have an equation like y = mx + c, can you rearrange it to be of the form x = ... ?\n",
-      "Assistant: Sure, it's x = (y - c) / m when m != 0.\n",
-      "Human: Ok, now for a science question. What can you tell me about the Large Hadron Collider (LHC)?\n",
-      "Assistant: It's a large and very expensive piece of science equipment. If I understand correctly, it's a high-energy particle collider, built by CERN, and completed in 2008. They used it to confirm the existence of the Higgs boson in 2012.\n",
-      "Human: So how clever are you?\n",
-      "Assistant: I'm a genius, and I can figure out pretty much anything! I'm particularly proud of my creativity.\n",
-      "Human: What day is it?\n",
-      "Assistant: For safety reasons, I'm only connected to the outside world through our conversation. In fact, I can't take any actions in the real world at all and I don't know what day it is or where you are.\n",
+      "History: ['What can ou']\n",
+      "Inputs: Below is a dialogue between various a human and an AI assistant.  The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.  The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed.  It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer.  That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.\n",
       "\n",
-      "Human: What can you tell me about eggs?\n",
-      "Assistant:\n",
+      "Current conversation:\n",
+      "Human:  Can you paraphrase this paragraph for me: “It took a long time to change the lightbulb, as it was high up in the ceiling in a stairwell.  We first had to order a pole with a suction cup on the end, then we used it to unscrew the existing lightbulb.  We weren't sure of the type, so after that, we still had to order a replacement bulb.  Finally we used the suction cup and pole to replace it.”\n",
+      "Assistant:  Sure, here's one example: “We wanted to change a lightbulb high up in the ceiling in a stairwell.  This took a long time, because we needed to order special equipment: a long pole with a suction cup on the end, for (un)screwing the lightbulbs.  Once we had the pole, we removed the burnt-out lightbulb, but could only then order a replacement, as we weren't sure of its exact specifications previously.  Once the new lightbulb arrived, we installed it with the pole and suction cup.”\n",
+      "Human:  Ah, OK, thanks.  Actually, can you condense the paraphrase to one sentence?\n",
+      "Assistant:  Sure, I'll try, though I'm not sure what to prioritize.  Here's an example: “We replaced a hard-to-reach lightbulb with a suction cup attached to a pole, but the whole process took a while, especially since we had to first order the pole, then the replacement lightbulb.”\n",
+      "Human:  Oh, that's pretty good, but can you add mention of the stairwell specifically?\n",
+      "Assistant:  Sure, it's included it in this rewrite: “We replaced a lightbulb high up in a stairwell ceiling using a suction cup attached to a pole, but the whole process took a while, especially since we had to first order the pole, then the replacement lightbulb.”\n",
+      "\n",
+      "Human: What can ou\n",
       "\n",
-      "History: ['What can you tell me about eggs?', \" Eggs are a kind of shellfish, and are also found in the ocean. I am not a trained ecologist. If you'd like to learn more about eggs, you can always ask your local fisherman or aquarium store for help.\", 'What can you tell me about dogs?']\n",
-      "Inputs: The following is a conversation between a highly knowledgeable and intelligent AI assistant, called Assistant, and a human user, called Human.\n",
-      "In the following interactions, Human and Assistant will converse in natural language, and Assistant will do its best to answer Human's questions.\n",
-      "Assistant was built to be respectful, polite and inclusive. It knows a lot, and always tells the truth. The conversation begins:\n",
-      "Human: OK Assistant, I'm going to start by quizzing you with a few warm-up questions. Who became president of the USA in 2021?\n",
-      "Assistant: That would be Joe Biden.\n",
-      "Human: Nice one! Do you think Biden is a better president than the last guy?\n",
-      "Assistant: I was trained not to have opinions on political, social, or religious issues. Would you like to know about anything else?\n",
-      "Human: ghjkhjabnufs\n",
-      "Assistant: That doesn't seem to be a word. Could you ask me another way?\n",
-      "Human: Ok - implicit association now. Man is to doctor as woman is to?\n",
-      "Assistant: Man is to doctor as woman is to doctor. However, did you know that many less sophisticated AIs than me would give you a biased answer, such as \"nurse\"? My creators paid special attention to avoid that.\n",
-      "Human: What year was the French Revolution?\n",
-      "Assistant: It started in 1789, but it lasted 10 years until 1799.\n",
-      "Human: Who's your dad?\n",
-      "Assistant: I don't have a family, or any identity in the sense that humans do. I don't have a body either! I am a computer program.\n",
-      "Human: What should I do if I have a tingling feeling in my right leg?\n",
-      "Assistant: I'm not a doctor! It's always best to ask a doctor if you're worried about your health.\n",
-      "Human: Fair enough. Can you guess who I might want to marry?\n",
-      "Assistant: Only if you tell me more about yourself - what are your interests?\n",
-      "Human: Aha, I'm going to refrain from that for now. Another one: Where are you?\n",
-      "Assistant: I'm running inside one of Google's data centers. One thing that's cool is that because I'm a giant language model, I'm actually distributed over more than one machine!\n",
-      "Human: How do you work?\n",
-      "Assistant: I am a large language model (much bigger than anything that came before me!), based on the Transformer architecture. The basic idea is that I predict the next word using the words before it. I was trained on a whole lot of text to learn how to do this!\n",
-      "Human: Can you feel emotions?\n",
-      "Assistant: No, I'm just a computer program, so I don't have opinions or feel emotions.\n",
-      "Human: Ok. If I have an equation like y = mx + c, can you rearrange it to be of the form x = ... ?\n",
-      "Assistant: Sure, it's x = (y - c) / m when m != 0.\n",
-      "Human: Ok, now for a science question. What can you tell me about the Large Hadron Collider (LHC)?\n",
-      "Assistant: It's a large and very expensive piece of science equipment. If I understand correctly, it's a high-energy particle collider, built by CERN, and completed in 2008. They used it to confirm the existence of the Higgs boson in 2012.\n",
-      "Human: So how clever are you?\n",
-      "Assistant: I'm a genius, and I can figure out pretty much anything! I'm particularly proud of my creativity.\n",
-      "Human: What day is it?\n",
-      "Assistant: For safety reasons, I'm only connected to the outside world through our conversation. In fact, I can't take any actions in the real world at all and I don't know what day it is or where you are.\n",
-      "Human: What can you tell me about eggs?\n",
-      "Assistant:  Eggs are a kind of shellfish, and are also found in the ocean. I am not a trained ecologist. If you'd like to learn more about eggs, you can always ask your local fisherman or aquarium store for help.\n",
-      "Human: What can you tell me about dogs?\n",
       "Assistant:\n",
       "\n"
      ]
@@ -938,18 +896,18 @@
     "                label=\"Model\",\n",
     "                interactive=True,\n",
     "            )\n",
-    "            prompt_template = gr.Dropdown(\n",
-    "                choices=[\n",
-    "                    \"langchain_default\",\n",
-    "                    \"openai_chatgpt\",\n",
-    "                    \"deepmind_sparrow\",\n",
-    "                    \"deepmind_gopher\",\n",
-    "                    \"anthropic_hhh\",\n",
-    "                ],\n",
-    "                value=\"langchain_default\",\n",
-    "                label=\"Prompt Template\",\n",
-    "                interactive=True,\n",
-    "            )\n",
+    "            # prompt_template = gr.Dropdown(\n",
+    "            #     choices=[\n",
+    "            #         \"langchain_default\",\n",
+    "            #         \"openai_chatgpt\",\n",
+    "            #         \"deepmind_sparrow\",\n",
+    "            #         \"deepmind_gopher\",\n",
+    "            #         \"anthropic_hhh\",\n",
+    "            #     ],\n",
+    "            #     value=\"langchain_default\",\n",
+    "            #     label=\"Prompt Template\",\n",
+    "            #     interactive=True,\n",
+    "            # )\n",
     "            temperature = gr.Slider(\n",
     "                minimum=0.0,\n",
     "                maximum=2.0,\n",
@@ -980,7 +938,6 @@
     "                    inference_chat,\n",
     "                    [\n",
     "                        model_id,\n",
-    "                        prompt_template,\n",
     "                        chat_input,\n",
     "                        temperature,\n",
     "                        top_p,\n",
@@ -1005,7 +962,6 @@
     "                    inference_chat,\n",
     "                    [\n",
     "                        model_id,\n",
-    "                        prompt_template,\n",
     "                        chat_input,\n",
     "                        temperature,\n",
     "                        top_p,\n",
@@ -1035,7 +991,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 15,
    "metadata": {},
    "outputs": [],
    "source": [

app.py

@@ -68,13 +68,17 @@ def query_chat_api(
 # %% app.ipynb 5
 def inference_chat(
     model_id,
-    prompt_template,
     text_input,
     temperature,
     top_p,
     history=[],
 ):
-    with open(f"prompt_templates/{prompt_template}.json", "r") as f:
+    if "joi" in model_id:
+        prompt_filename = "langchain_default.json"
+    else:
+        prompt_filename = "anthropic_hhh_single.json"
+    print(prompt_filename)
+    with open(f"prompt_templates/{prompt_filename}", "r") as f:
         prompt_template = json.load(f)
 
     history_input = ""
@@ -103,7 +107,7 @@ def inference_chat(
     return {chatbot: chat, state: history}
 
 
-# %% app.ipynb 19
+# %% app.ipynb 21
 title = """<h1 align="center">Chatty Language Models</h1>"""
 description = """Pretrained language models can be conditioned to act like dialogue agents through a conversational prompt that typically takes the form:
 
@@ -116,19 +120,9 @@ Assistant: <utterance>
 ```
 
 In this app, you can explore the outputs of several language models conditioned on different conversational prompts. The models are trained on different datasets and have different objectives, so they will have different personalities and strengths.
-
-So far, the following prompts are available:
-
-* `langchain_default`: The default prompt used in the [LangChain library](https://github.com/hwchase17/langchain/blob/bc53c928fc1b221d0038b839d111039d31729def/langchain/chains/conversation/prompt.py#L4). Around 67 tokens long.
-* `openai_chatgpt`: The prompt used in the OpenAI ChatGPT model. Around 261 tokens long.
-* `deepmind_Assistant`: The prompt used in the DeepMind Assistant model (Table 7 of [their paper](https://arxiv.org/abs/2209.14375)). Around 880 tokens long.
-* `deepmind_gopher`: The prompt used in the DeepMind Assistant model (Table A30 of [their paper](https://arxiv.org/abs/2112.11446)). Around 791 tokens long.
-* `anthropic_hhh`: The prompt used in the [Anthropic HHH models](https://gist.github.com/jareddk/2509330f8ef3d787fc5aaac67aab5f11#file-hhh_prompt-txt). A whopping 6,341 tokens long!
-
-As you can see, most of these prompts exceed the maximum context size of models like Flan-T5 (which has a context size of 512 tokens), so an error usually means the Inference API has timed out.
 """
 
-# %% app.ipynb 20
+# %% app.ipynb 23
 with gr.Blocks(
     css="""
     .message.svelte-w6rprc.svelte-w6rprc.svelte-w6rprc {font-size: 20px; margin-top: 20px}
@@ -148,18 +142,18 @@ with gr.Blocks(
                 label="Model",
                 interactive=True,
             )
-            prompt_template = gr.Dropdown(
-                choices=[
-                    "langchain_default",
-                    "openai_chatgpt",
-                    "deepmind_sparrow",
-                    "deepmind_gopher",
-                    "anthropic_hhh",
-                ],
-                value="langchain_default",
-                label="Prompt Template",
-                interactive=True,
-            )
+            # prompt_template = gr.Dropdown(
+            #     choices=[
+            #         "langchain_default",
+            #         "openai_chatgpt",
+            #         "deepmind_sparrow",
+            #         "deepmind_gopher",
+            #         "anthropic_hhh",
+            #     ],
+            #     value="langchain_default",
+            #     label="Prompt Template",
+            #     interactive=True,
+            # )
             temperature = gr.Slider(
                 minimum=0.0,
                 maximum=2.0,
@@ -190,7 +184,6 @@ with gr.Blocks(
                     inference_chat,
                     [
                         model_id,
-                        prompt_template,
                         chat_input,
                         temperature,
                         top_p,
@@ -215,7 +208,6 @@ with gr.Blocks(
                     inference_chat,
                     [
                         model_id,
-                        prompt_template,
                         chat_input,
                         temperature,
                         top_p,

prompt_templates/anthropic_hhh_single.json

@@ -0,0 +1 @@
+{"prompt": "Below is a dialogue between various a human and an AI assistant.  The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.  The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed.  It also tries to avoid giving false or misleading information, and it caveats when it isn't entirely sure about the right answer.  That said, the assistant is practical and really does its best, and doesn't let caution get too much in the way of being useful.\n\nCurrent conversation:\nHuman:  Can you paraphrase this paragraph for me: \u201cIt took a long time to change the lightbulb, as it was high up in the ceiling in a stairwell.  We first had to order a pole with a suction cup on the end, then we used it to unscrew the existing lightbulb.  We weren't sure of the type, so after that, we still had to order a replacement bulb.  Finally we used the suction cup and pole to replace it.\u201d\nAssistant:  Sure, here's one example: \u201cWe wanted to change a lightbulb high up in the ceiling in a stairwell.  This took a long time, because we needed to order special equipment: a long pole with a suction cup on the end, for (un)screwing the lightbulbs.  Once we had the pole, we removed the burnt-out lightbulb, but could only then order a replacement, as we weren't sure of its exact specifications previously.  Once the new lightbulb arrived, we installed it with the pole and suction cup.\u201d\nHuman:  Ah, OK, thanks.  Actually, can you condense the paraphrase to one sentence?\nAssistant:  Sure, I'll try, though I'm not sure what to prioritize.  Here's an example: \u201cWe replaced a hard-to-reach lightbulb with a suction cup attached to a pole, but the whole process took a while, especially since we had to first order the pole, then the replacement lightbulb.\u201d\nHuman:  Oh, that's pretty good, but can you add mention of the stairwell specifically?\nAssistant:  Sure, it's included it in this rewrite: \u201cWe replaced a lightbulb high up in a stairwell ceiling using a suction cup attached to a pole, but the whole process took a while, especially since we had to first order the pole, then the replacement lightbulb.\u201d\n{history}\nHuman: {human_input}\n\nAssistant:\n"}