Update README.md

README.md

@@ -1,12 +1,22 @@
 ---
-language:
-- it
 license: apache-2.0
 datasets:
 - DeepMount00/gquad_it
+language:
+- it
 pipeline_tag: question-answering
 ---
 
+## SQuAD-it Evaluation
+
+The Stanford Question Answering Dataset (SQuAD) in Italian (SQuAD-it) is used to evaluate the model's reading comprehension and question-answering capabilities. The following table presents the F1 score and Exact Match (EM) metrics, including the percentage improvements:
+
+| Model                                        | F1 Score | Exact Match (EM) | 
+|----------------------------------------------|----------|------------------|
+| **DeepMount00/Gemma_QA_ITA_v3**              | **77.24%**   | **64.60%**       | 
+| **DeepMount00/Gemma_QA_ITA_v2**              | **77.17%**   | **63.82%**       |
+| **DeepMount00/mamba_790_hf_qa**              | **69.72%**   | **58.56%**       | 
+| **DeepMount00/Gemma_QA_ITA**                 | **59.59%**   | **40.68%**       | 
 
 ## How to Use
 How to use mamba q&a
@@ -28,11 +38,11 @@ def predict(contesto, domanda):
 
     input_ids = tokenizer([prompt], return_tensors="pt").to(device)
 
-    generate_ids = model.generate(**input_ids, max_new_tokens=150, eos_token_id=0)
+    generate_ids = model.generate(**input_ids, max_new_tokens=150, eos_token_id=8112)
 
     answer = tokenizer.batch_decode(generate_ids)
     try:
-        final_answer = answer[0].split("##RISPOSTA: ")[1].split('\n', 1)[0]
+        final_answer = answer[0].split("##RISPOSTA: ")[1].split("##END")[0].strip("\n")
     except IndexError:
         final_answer = ""
     return final_answer