README.md

---
license: other
license_name: stem.ai.mtl
license_link: LICENSE
language:
- en
tags:
- phi-2
- electrical engineering
- Microsoft
datasets:
- STEM-AI-mtl/Electrical-engineering
- garage-bAInd/Open-Platypus
task_categories:
  - question-answering
  - text-generation
pipeline_tag: text-generation
widget:
  - text: "Enter your instruction here"
inference: true
auto_sample: true
inference_code: chat-GPTQ.py
library_tag: transformers
---
# For the electrical engineering community

A unique, deployable and efficient 2.7 billion parameters model in the field of electrical engineering. This repo contains the adapters from the LoRa fine-tuning of the phi-2 model from Microsoft. It was trained on the [STEM-AI-mtl/Electrical-engineering](https://huggingface.co/datasets/STEM-AI-mtl/Electrical-engineering) dataset combined with [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus).

- **Developed by:** STEM.AI
- **Model type:** Q&A and code generation
- **Language(s) (NLP):** English
- **Finetuned from model:** [microsoft/phi-2](https://huggingface.co/microsoft/phi-2)


### Direct Use

Q&A related to electrical engineering, and Kicad software. Creation of Python code in general, and for Kicad's scripting console.

Refer to [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) model card for recommended prompt format.

### Inference script

[Standard](https://github.com/STEM-ai/Phi-2/blob/4eaa6aaa2679427a810ace5a061b9c951942d66a/chat.py)

[GPTQ format](https://github.com/STEM-ai/Phi-2/blob/ab1ced8d7922765344d824acf1924df99606b4fc/chat-GPTQ.py)

## Training Details

### Training Data

Dataset related to electrical engineering: [STEM-AI-mtl/Electrical-engineering](https://huggingface.co/datasets/STEM-AI-mtl/Electrical-engineering)
It is composed of queries, 65% about general electrical engineering, 25% about Kicad (EDA software) and 10% about Python code for Kicad's scripting console.

In additionataset related to STEM and NLP: [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus)

### Training Procedure 
[LoRa script](https://github.com/STEM-ai/Phi-2/blob/4eaa6aaa2679427a810ace5a061b9c951942d66a/LoRa.py)

A LoRa PEFT was performed on a 48 Gb A40 Nvidia GPU.

## Model Card Authors

STEM.AI: stem.ai.mtl@gmail.com\
[William Harbec](https://www.linkedin.com/in/william-harbec-56a262248/)