license: mit
SuperCOT LoRA
SuperCOT is a LoRA I trained with the aim of making LLaMa follow prompts for Langchain better, by infusing chain-of-thought datasets, code explanations and instructions, snippets, logical deductions and Alpaca GPT-4 prompts. Trained against LLaMa 30B 4-bit for 3 epochs with cutoff length 1024, using a mixture of the following datasets:
https://huggingface.co/datasets/QingyiSi/Alpaca-CoT
Chain of thought QED
Chain of thought Aqua
CodeAlpaca
https://huggingface.co/datasets/neulab/conala
Code snippets
https://huggingface.co/datasets/yahma/alpaca-cleaned
Alpaca GPT4
You should prompt the LoRA the same way you would prompt Alpaca or Alpacino:
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
<instruction>
### Input:
<any additional context. Remove this if it's not neccesary>
### Response:
<make sure to leave a single new-line here for optimal results>
7B version coming soon
1024 and 512 variants of 13B coming soon
merged 13B ggml model coming soon
Citations
Alpaca COT datasets
@misc{alpaca-cot,
author = {Qingyi Si, Zheng Lin },
school = {Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China},
title = {Alpaca-CoT: An Instruction Fine-Tuning Platform with Instruction Data Collection and Unified Large Language Models Interface},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/PhoebusSi/alpaca-CoT}},
}
Stanford Alpaca
@misc{alpaca,
author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto },
title = {Stanford Alpaca: An Instruction-following LLaMA model},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
}
Google FLAN
@inproceedings{weifinetuned,
title={Finetuned Language Models are Zero-Shot Learners},
author={Wei, Jason and Bosma, Maarten and Zhao, Vincent and Guu, Kelvin and Yu, Adams Wei and Lester, Brian and Du, Nan and Dai, Andrew M and Le, Quoc V},
booktitle={International Conference on Learning Representations}
}