Update README.md

README.md

@@ -28,71 +28,43 @@ inference: false
 </p>
 
 <div style="margin:auto; text-align:center">
-  <h1 style="margin-bottom: 0; font-size: 4em;">🐐 FinGEITje 7B</h1>
-  <em style="font-size: 2em;">A large open Dutch Financial language model.</em>
+  <h1 style="margin-bottom: 0; font-size: 3em;">🐐 FinGEITje 7B</h1>
+  <em style="font-size: 1em;">A large open Dutch Financial language model.</em>
 </div>
 
 This model is a fine-tuned version of [BramVanroy/GEITje-7B-ultra](https://huggingface.co/BramVanroy/GEITje-7B-ultra) on the [snoels/FinGEITje-sft](https://huggingface.co/datasets/snoels/FinGEITje-sft) dataset.
 
-It achieves the following results on the evaluation set:
-- **Loss**: 0.3928
-
 ## Model Description
 
-FinGEITje 7B is a large open Dutch financial language model with 7 billion parameters, based on Mistral 7B. It has been further trained on Dutch financial texts, enhancing its proficiency in the Dutch language and its knowledge of financial topics. As a result, FinGEITje provides more accurate and relevant responses in the domain of finance, including areas such as banking, investment, insurance, and economic policy.
-
-FinGEITje builds upon the foundation of GEITje models, which are Dutch language models based on Mistral 7B and have been further pretrained on extensive Dutch corpora.
-
-## Intended Uses & Limitations
-
-### Intended Use
-
-- **Educational Purposes**: Researching Dutch financial language patterns, generating financial texts for study, or aiding in financial education.
-- **Content Generation**: Assisting in drafting Dutch financial reports, summaries, articles, or other finance-related content.
-- **Financial Chatbots**: Integrating into chatbots to provide responses related to Dutch financial matters, customer service, or financial advice simulations.
-- **Financial Analysis**: Supporting analysis by generating insights or summarizing financial data in Dutch.
-
-### Limitations
-
-- **Not for Real-time Financial Decisions**: The model does not have access to real-time data and may not reflect the most current financial regulations or market conditions. It should not be used for making actual financial decisions.
-- **Accuracy**: While trained on financial data, the model may still produce incorrect or nonsensical answers, especially for prompts outside its training scope.
-- **Bias**: The model may reflect biases present in its training data, potentially affecting the neutrality of its responses.
-- **Ethical Use**: Users should ensure that the model's outputs comply with ethical standards and do not promote misinformation or harmful content.
-
-### Ethical Considerations
-
-- **License Restrictions**: FinGEITje is released under a non-commercial license (CC BY-NC 4.0). Commercial use is prohibited without obtaining proper permissions.
-- **Misinformation**: Users should verify the information generated by the model, particularly for critical applications.
-- **Data Privacy**: The model should not be used to generate or infer personal identifiable information or confidential data.
+FinGEITje 7B is a large open Dutch financial language model with 7 billion parameters, based on Mistral 7B. It has been further trained on Dutch financial texts, enhancing its proficiency in the Dutch language and its knowledge of financial topics. As a result, FinGEITje provides more accurate and relevant responses in the domain of finance.
 
 ## Training and Evaluation Data
 
 ### Training Data
 
-FinGEITje 7B was fine-tuned on the [snoels/FinGEITje-sft](https://huggingface.co/datasets/snoels/FinGEITje-sft) dataset, which consists of translated and processed Dutch financial texts. This dataset includes a wide range of financial topics and instruction tuning data, ensuring that the model gains a comprehensive understanding of the financial domain in Dutch.
+FinGEITje 7B was fine-tuned on the [snoels/FinGEITje-sft](https://huggingface.co/datasets/snoels/FinGEITje-sft) dataset, which consists of translated and processed Dutch financial texts. This dataset includes a wide range of financial topics and instruction tuning data.
 
 #### Data Processing Steps
 
 1. **Translation**: Original instruction tuning datasets were translated into Dutch using a specialized translation service to maintain the integrity of financial terminology.
 2. **Post-processing**: The translated data underwent post-processing to correct any translation inconsistencies and to format it according to the original dataset structure.
 3. **Formatting**: The data was formatted to match the style and requirements of instruction tuning datasets, ensuring compatibility with the fine-tuning process.
-4. **Filtering**: A Dutch language check and predefined validation checks were applied to filter out any low-quality or irrelevant data, enhancing the overall dataset quality.
+4. **Filtering**: A Dutch language check and predefined validation checks were applied to filter out any low-quality or irrelevant data.
 
 ### Evaluation Data
 
 The model was evaluated using:
 
-- **[snoels/FinDutchBench](https://huggingface.co/datasets/snoels/FinDutchBench)**: A Dutch financial benchmark dataset designed to assess the model's performance on various financial tasks. It includes tasks that test the model's understanding of financial concepts, its ability to generate financial texts, and its comprehension of Dutch financial regulations.
+- **[snoels/FinDutchBench](https://huggingface.co/datasets/snoels/FinDutchBench)**: A Dutch financial benchmark dataset designed to assess the model's performance on various financial tasks. 
 
 ## Training Procedure
 
-FinGEITje was trained following the methodology described in the [Alignment Handbook](https://github.com/huggingface/alignment-handbook), utilizing a structured training pipeline to ensure optimal model performance.
+FinGEITje was trained following the methodology described in the [Alignment Handbook](https://github.com/huggingface/alignment-handbook).
 
 ### Training Configuration
 
-- The training configuration is based on the recipe outlined in the alignment handbook and can be found in the `config_qlora.yml` file.
+- The training configuration is based on the recipe outlined in the alignment handbook and can be found in the [config_qlora.yaml](https://github.com/snoels/fingeit/blob/master/src/training/sft/config_qlora.yaml) file.
 - The model was further trained using **QLoRA** (Quantized LoRA) for efficient fine-tuning with reduced computational resources.
-- Training was conducted on a multi-GPU setup to handle the large model size and extensive dataset.
 
 ### Training Hyperparameters
 
@@ -120,8 +92,8 @@ The following hyperparameters were used during training:
 
 The evaluation package includes a set of metrics defined per task, grouped per dataset to evaluate the model's performance across different financial domains. The evaluation notebooks are available:
 
-- **[Evaluation in Dutch](https://github.com/your-repo/evaluation_nl.ipynb)**: Assesses the model's performance on the Dutch financial benchmark dataset.
-- **[Evaluation in English](https://github.com/your-repo/evaluation_en.ipynb)**: Evaluates the model's performance on English financial benchmarks for comparison purposes.
+- **[Evaluation in Dutch](https://github.com/snoels/fingeit/blob/master/notebooks/evaluation_nl.ipynb)**: Assesses the model's performance on the Dutch financial benchmark dataset.
+- **[Evaluation in English](https://github.com/snoels/fingeit/blob/master/notebooks/evaluation_en.ipynb)**: Evaluates the model's performance on English financial benchmarks for comparison purposes.
 
 ### Framework Versions
 
@@ -156,7 +128,7 @@ tokenizer = AutoTokenizer.from_pretrained("BramVanroy/GEITje-7B-ultra", use_fast
 base_model = AutoModelForCausalLM.from_pretrained("BramVanroy/GEITje-7B-ultra", device_map='auto')
 
 # Load the FinGEITje model with PEFT adapters
-model = PeftModel.from_pretrained(base_model, "snoels/FinGEITje-7B", device_map='auto')
+model = PeftModel.from_pretrained(base_model, "snoels/FinGEITje-7B-sft", device_map='auto')
 ```
 
 ### Generating Text
@@ -173,34 +145,6 @@ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
 print(response)
 ```
 
-### Interactive Conversation
-
-```python
-from transformers import pipeline, Conversation
-
-# Initialize the conversational pipeline
-chatbot = pipeline(
-    "conversational",
-    model=model,
-    tokenizer=tokenizer,
-    device_map='auto'
-)
-
-# Start a conversation
-start_messages = [
-    {"role": "system", "content": "Je bent een behulpzame financiële assistent."},
-    {"role": "user", "content": "Kun je me iets vertellen over de huidige rentevoeten voor hypotheekleningen in Nederland?"}
-]
-
-conversation = Conversation()
-for msg in start_messages:
-    conversation.add_message(**msg)
-
-# Get the assistant's response
-response = chatbot(conversation)
-print(response)
-```
-
 ## Limitations and Future Work
 
 While FinGEITje 7B demonstrates significant improvements in understanding and generating Dutch financial content, certain limitations exist:
@@ -209,6 +153,8 @@ While FinGEITje 7B demonstrates significant improvements in understanding and ge
 - **Accuracy Concerns**: The model may generate incorrect or outdated information. Users should verify critical information with reliable sources.
 - **Biases**: Potential biases in the training data may affect the neutrality and fairness of the model's responses.
 - **Language Scope**: Primarily designed for Dutch; performance in other languages is not optimized.
+- **Ethical Use**: Users should ensure that the model's outputs comply with ethical standards and do not promote misinformation or harmful content.
+
 
 ### Future Work
 
@@ -223,7 +169,7 @@ We would like to thank:
 
 - **Rijgersberg** ([GitHub](https://github.com/Rijgersberg)) for creating [GEITje](https://github.com/Rijgersberg/GEITje), one of the first Dutch foundation models, and for contributing significantly to the development of Dutch language models.
 - **Bram Vanroy** ([GitHub](https://github.com/BramVanroy)) for creating [GEITje-7B-ultra](https://huggingface.co/BramVanroy/GEITje-7B-ultra), an open-source Dutch chat model, and for sharing training, translation, and evaluation resources.
-- **Contributors of the Alignment Handbook** for providing valuable resources that guided the development and training process of FinGEITje.
+- **Contributors of the [Alignment Handbook](https://github.com/huggingface/alignment-handbook)** for providing valuable resources that guided the development and training process of FinGEITje.
 
 ## Citation
 
@@ -243,13 +189,6 @@ If you use FinGEITje in your work, please cite:
 
 This model is licensed under the [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/) license.
 
-### Summary
-
-- **Attribution**: You must give appropriate credit, provide a link to the license, and indicate if changes were made.
-- **NonCommercial**: You may not use the material for commercial purposes.
-
-For the full license text, please refer to the [license document](https://creativecommons.org/licenses/by-nc/4.0/legalcode).
-
 ## Contact
 
 For any inquiries or questions, please contact [Sander Noels](mailto:sander.noels@ugent.be).