Hercules-Qwen1.5-14B

• Model Details

  Model Description

  This model has capabilities in math, coding, function calling, roleplay, and more. We fine-tuned it using 700,000 examples of Hercules-v4.

  Developed by: M4-ai

  Language(s) (NLP): English and maybe Chinese

  License: tongyi-qianwen license

  Finetuned from model: Qwen1.5-14B

• Uses

  General purpose assistant, question answering, chain-of-thought, etc..

  Recommendations

  Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

• Evaluation

  Coming soon

• Training Details

  Training Data

  https://huggingface.co/datasets/Locutusque/hercules-v4.0

  Training Hyperparameters

  Training regime: bf16 non-mixed precision

• Technical Specifications

  Hardware

  We used 8 Kaggle TPUs, and we trained at a global batch size of 128 and sequence length of 1024

• Contributions

  Thanks to @Tonic, @aloobun, @fhai50032, and @Locutusque for their contributions to this model.