option to vary prompt

app.py

@@ -13,7 +13,25 @@ st.write("""
 
 import urllib.request 
 import os
-result_file = os.environ["POST_PROCESSING_JSON"]
+prefix_post_processing = os.environ["POST_PROCESSING_JSON"]
+
+
+prompt_options_dict = {
+    'Prompt option 0 (with typos and grammatical errors, two shot prompting)': 'prompt_option_0.json',
+    'Prompt option 1 (Zero shot prompting)': 'prompt_option_1.json',
+    'Prompt option 2 (Prompt option 2 with two shot prompting)': 'prompt_option_2.json',
+    'Prompt option 3 (Prompt option 0 with minor fixes)': 'prompt_option_3.json'
+}
+t_result_file = st.selectbox(
+     'Select result file:',
+     list(prompt_options_dict.keys()))
+
+result_file = prefix_post_processing + prompt_options_dict[t_result_file]
+
+prompt_option = int(prompt_options_dict[t_result_file].split('_')[-1].split('.')[0])
+
+# with open(result_file, 'r') as f:
+#     data_dict = json.load(f)
 
 with urllib.request.urlopen(result_file) as url:
     data_dict = json.load(url)
@@ -27,7 +45,7 @@ overall_performance = round(data_dict["Overall performance"]*100, 2)
 st.header("Prompt")
 st.write("For each data point in the evaluation dataset, we create a prompt for LLM by adding the context and the question to the below prompt template, while following the same structure of the prompt template.")
 # File uploader
-with open('prompt_0.txt', "r") as file:
+with open(f'prompt_{prompt_option}.txt', "r") as file:
     file_contents = file.read()
     # st.write(file_contents)
 st.text_area("Prompt template:", value=file_contents, height=300)
@@ -116,7 +134,7 @@ embedder_categories = data_dict['Embedder categories']
 
 option = st.selectbox(
      'Select characteristic:',
-     list(embedder_categories.keys()))
+     sorted(list(embedder_categories.keys())))
 
 
 st.write('The following are the categories:')

prompt_1.txt

@@ -0,0 +1,7 @@
+Instructions:  
+- Given a context and a question, your goal is to answer the question by extracting information from the context.
+- Your answer must be a segment or span of any of the sentences in the provided context, or "<NO ANSWER>" if the question cannot be answered using only the given context.
+- In cases where the answer is within the context, your answer should not be a full sentence but rather contain only the relevant span or segment of the sentence that answers the question.
+- Your answer should be succinct and concise, not descriptive. Ensure that your answer has no additional content or filler text.
+- The context begins after 'Context:' and ends before 'Question:', while the question is delimited by 'Question:' and 'Answer:'.
+

prompt_2.txt

@@ -0,0 +1,19 @@
+Instructions:  
+- Given a context and a question, your goal is to answer the question by extracting information from the context.
+- Your answer must be a segment or span of any of the sentences in the provided context, or "<NO ANSWER>" if the question cannot be answered using only the given context.
+- In cases where the answer is within the context, your answer should not be a full sentence but rather contain only the relevant span or segment of the sentence that answers the question.
+- Your answer should be succinct and concise, not descriptive. Ensure that your answer has no additional content or filler text.
+- The context begins after 'Context:' and ends before 'Question:', while the question is delimited by 'Question:' and 'Answer:'.
+
+Context: The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse ("Norman" comes from "Norseman") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries.
+
+Question: In what country is Normandy located?
+
+Answer: France
+
+Context: The English name "Normans" comes from the French words Normans/Normanz, plural of Normant, modern French normand, which is itself borrowed from Old Low Franconian Nortmann "Northman" or directly from Old Norse Norðmaðr, Latinized variously as Nortmannus, Normannus, or Nordmannus (recorded in Medieval Latin, 9th century) to mean "Norseman, Viking".
+
+Question: What name comes from the English words Normans/Normanz?
+
+Answer: <NO ANSWER>
+

prompt_3.txt

@@ -0,0 +1,20 @@
+Instructions:
+- The user message will give context and question. 
+- If the answer exists, you must answer ONLY from the span of the context that starts after 'Context:' and ends before 'Question:'.
+- DO NOT ADD any additional content or filler text. 
+- If the question cannot be answered from the context alone or if you do not understand the question, then output "<NO ANSWER>".
+
+Examples:
+
+Context: The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse ("Norman" comes from "Norseman") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries.
+
+Question: In what country is Normandy located?
+
+Answer: France
+
+Context: The English name "Normans" comes from the French words Normans/Normanz, plural of Normant, modern French normand, which is itself borrowed from Old Low Franconian Nortmann "Northman" or directly from Old Norse Norðmaðr, Latinized variously as Nortmannus, Normannus, or Nordmannus (recorded in Medieval Latin, 9th century) to mean "Norseman, Viking".
+
+Question: What name comes from the English words Normans/Normanz?
+
+Answer: <NO ANSWER>
+