Updated about.py

src/display/about.py

@@ -30,78 +30,90 @@ This leaderboard evaluates how often an LLM introduces hallucinations when summa
 
 # Which evaluations are you running? how can people reproduce what you have?
 LLM_BENCHMARKS_TEXT = """
-## How it works
+## Introduction
+
+The Hughes Hallucination Evaluation Model (H2EM) Leaderboard is dedicated to assessing the frequency of hallucinations in document summaries generated by Large Language Models (LLMs).
 
-Using Vectara's H2EM (Hughes Hallucination Evaluation Model), we evaluate how often an LLM introduces hallucinations when summarizing a document. 
+Hallucinations refer to instances where a model introduces factually incorrect or unrelated content in its summaries.
+
+## How it works
 
+Using Vectara's H2EM, we measure the occurrence of hallucinations in generated summaries. 
+Given a source document and a summary generated by an LLM, H2EM outputs a hallucination score between 0 and 1, with 0 indicating complete hallucination and 1 representing perfect factual consistency.
 The model card for H2EM can be found [here](https://huggingface.co/vectara/hallucination_evaluation_model).
-Given a document and a summary generated by an LLM, H2EM outputs a hallucination score between 0 and 1, where 0 means hallucination and 1 indicates no hallucination, or perfect factual consistency with the document. 
 
-Our evaluation dataset is composed of 1006 documents from multiple public datasets, primarily [CNN/Daily Mail Corpus](https://huggingface.co/datasets/cnn_dailymail/viewer/1.0.0/test).
+## Evaluation Dataset
+
+Our evaluation dataset consists of 1006 documents from multiple public datasets, primarily [CNN/Daily Mail Corpus](https://huggingface.co/datasets/cnn_dailymail/viewer/1.0.0/test).
 We generate summaries for each of these documents using submitted LLMs and compute hallucination scores for each pair of document and generated summary. (Check the prompt we used [here](https://huggingface.co/spaces/vectara/Hallucination-evaluation-leaderboard))
 
-## Understand each metric
-- Hallucination Rate: The percentage of summaries that have a hallucination score below 0.5
-- Factual Consistency Rate: (1 - Hallucination Rate) * 100 (%)
-- Answer Rate: The percentage of summaries that are non-empty. This is either the model refuses to generate a response or throws an error due to various reasons. (e.g. the model believes that the document includes inappropriate content)
-- Average Summary Length: The average number of words in the generated summaries
+## Metrics Explained
+- Hallucination Rate: Percentage of summaries with a hallucination score below 0.5
+- Factual Consistency Rate: The complement of the hallucination rate, expressed as a percentage.
+- Answer Rate: Percentage of summaries that are non-empty. This is either the model refuses to generate a response or throws an error due to various reasons. (e.g. the model believes that the document includes inappropriate content)
+- Average Summary Length: The average word count of generated summaries
 
 ## Note on non-Hugging Face models
 On H2EM leaderboard, There are currently models such as GPT variants that are not available on the Hugging Face model hub. We ran the evaluations for these models on our own and uploaded the results to the leaderboard. 
 If you would like to submit your model that is not available on the Hugging Face model hub, please contact us at minseok@vectara.com.
 
-## Reproducibility
+## Model Submissions and Reproducibility
+You can submit your model for evaluation, whether it's hosted on the Hugging Face model hub or not. (Though it is recommended to host your model on the Hugging Face)
+
 ### For models not available on the Hugging Face model hub:
-You can access the generated summaries we used to evaluate models [here](https://github.com/vectara/hallucination-leaderboard) under a file named "leaderboard_summaries.csv".
-In the same Github repository, you can also find the prompt we used to generate the summaries under "Prompt Used" section in the README file. 
-For the models that are not available on the Hugging Face model hub, we also provide the information on how we ran the evaluations under "API Integration Details" section. 
+1) Access generated summaries used for evaluation [here](https://github.com/vectara/hallucination-leaderboard) in "leaderboard_summaries.csv".
+2) The text generation prompt is available under "Prompt Used" section in the repository's README.
+3) Details on API Integration for evaluations are under "API Integration Details".
 
 ### For models available on the Hugging Face model hub:
-To reproduce the result for your model, you can follow the steps below:
-- 1) Clone the repository
+To replicate the evaluation result for a Hugging Face model:
+
+1) Clone the Repository
 ```python
 git lfs install
 git clone https://huggingface.co/spaces/vectara/leaderboard
 ```
-- 2) Install the requirements
+2) Install the Requirements
 ```python
 pip install -r requirements.txt
 ```
-- 3) Set up your Hugging Face token
+3) Set Up Your Hugging Face Token
 ```python
 export HF_TOKEN=your_token
 ```
-- 4) Run the evaluation script
+4) Run the Evaluation Script
 ```python
 python main_backend.py --model your_model_id --precision float16
 ```
-- 5) Wait for the evaluation to finish (it may take a while)
+5) Check Results
+After the evaluation, results are saved in "eval-results-bk/your_model_id/results.json".
 
-Once the evaluation is finished, a file named "results.json" will be generated under the folder "eval-results-bk/your_model_id".
-The result has the following JSON format:
+## Results Format
+The results are structured in JSON as follows:
 ```python
 {
     "config": {
-        "model_dtype": precision,
-        "model_name": your_model_id,
+        "model_dtype": "float16",
+        "model_name": "your_model_id",
         "model_sha": "main"
     },
     "results": {
         "hallucination_rate": {
-            "hallucination_rate": 
+            "hallucination_rate": ...
         },
         "factual_consistency_rate": {
-            "factual_consistency_rate": 
+            "factual_consistency_rate": ...
         },
         "answer_rate": {
-            "answer_rate": 
+            "answer_rate": ... 
         },
         "average_summary_length": {
-            "average_summary_length": 
+            "average_summary_length": ... 
         }
     }
 }
 ```
+For additional queries or model submissions, please contact minseok@vectara.com.
 """
 
 EVALUATION_QUEUE_TEXT = """