---
license: creativeml-openrail-m
datasets:
- amphora/QwQ-LongCoT-130K
language:
- en
base_model:
- Qwen/Qwen2.5-3B-Instruct
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- long-CoT
- safetensors
- 3B
- Instruct
- QwQ
- Qwen2.5
---
### **QwQ-LCoT-3B-Instruct Model Card**

The **QwQ-LCoT-3B-Instruct** model is a lightweight, instruction-tuned language model designed for complex reasoning and explanation tasks. It is fine-tuned on the **Qwen2.5-3B-Instruct** base model using the **QwQ-LongCoT-130K** dataset, focusing on **long-chain-of-thought (LCoT)** reasoning for enhanced logical comprehension and detailed output generation.

| **File Name**                         | **Size**       | **Description**                                  | **Upload Status**  |
|----------------------------------------|----------------|-------------------------------------------------|--------------------|
| `.gitattributes`                       | 1.57 kB        | Specifies LFS tracking for large files.         | Uploaded           |
| `README.md`                            | 267 Bytes      | Basic project information file.                 | Updated            |
| `added_tokens.json`                    | 657 Bytes      | Custom tokens added to the tokenizer.           | Uploaded           |
| `config.json`                          | 859 Bytes      | Configuration file for the model.               | Uploaded           |
| `generation_config.json`               | 281 Bytes      | Configuration file for text generation settings.| Uploaded           |
| `merges.txt`                           | 1.82 MB        | Contains the byte-pair encoding (BPE) merges.   | Uploaded           |
| `pytorch_model-00001-of-00002.bin`     | 4.96 GB        | First shard of the model weights in PyTorch format. | Uploaded (LFS)     |
| `pytorch_model-00002-of-00002.bin`     | 1.21 GB        | Second shard of the model weights in PyTorch format. | Uploaded (LFS)     |
| `pytorch_model.bin.index.json`         | 36 kB          | Index mapping for sharded model weights.        | Uploaded           |
| `special_tokens_map.json`              | 644 Bytes      | Maps special tokens to their roles.             | Uploaded           |
| `tokenizer.json`                       | 11.4 MB        | Serialized tokenizer data.                      | Uploaded (LFS)     |
| `tokenizer_config.json`                | 7.73 kB        | Tokenizer configuration settings.               | Uploaded           |
| `vocab.json`                           | 2.78 MB        | Vocabulary file for the tokenizer.              | Uploaded           |

### **Sample Long CoT:**

![Screenshot 2024-12-13 211732.png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/Mgm9LmQZlFZmglKYwEDYA.png)

### **Key Features:**

1. **Long Chain-of-Thought Reasoning:**  
   - Specifically designed to generate comprehensive, step-by-step explanations for complex queries.

2. **Lightweight and Efficient:**  
   - With only 3 billion parameters, it is optimized for systems with limited computational resources without compromising reasoning capabilities.

3. **Instruction Optimization:**  
   - Fine-tuned to follow prompts and provide concise, actionable, and structured responses.

---

### **Training Details:**

- **Base Model:** [Qwen2.5-3B-Instruct](#)  
- **Dataset:** [amphora/QwQ-LongCoT-130K](#)  
  - Comprising 133,000 annotated samples focusing on logical tasks and structured thinking.
---

### **Capabilities:**

1. **Text Generation:**  
   - Provides detailed, structured, and logical text outputs tailored to user prompts.

2. **Reasoning Tasks:**  
   - Solves step-by-step problems in math, logic, and science.

3. **Educational Assistance:**  
   - Generates coherent explanations for academic and research purposes.

4. **Dialogue and Summarization:**  
   - Handles conversational queries and summarizes long documents effectively.

---

### **Usage Instructions:**

1. **Setup:**
   Download all model files and ensure compatibility with the Hugging Face Transformers library.

2. **Loading the Model:**
   ```python
   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "prithivMLmods/QwQ-LCoT-3B-Instruct"
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   model = AutoModelForCausalLM.from_pretrained(model_name)
   ```

3. **Generate Long-Chain Reasoning Outputs:**
   ```python
   input_text = "Explain the process of photosynthesis step-by-step."
   inputs = tokenizer(input_text, return_tensors="pt")
   outputs = model.generate(**inputs, max_length=300, temperature=0.5)
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   ```

4. **Customize Output Generation:**  
   Modify the `generation_config.json` file for different scenarios:
   - **`temperature`**: Controls randomness (lower = deterministic, higher = creative).  
   - **`max_length`**: Sets response length.  
   - **`top_p`**: Adjusts sampling for diversity in outputs.

---