forcemultiplier/fastapply-xml-cot-3b-adapter

Credits and Acknowledgments

TURBOPASTA is built upon the excellent work of Fast Apply by Kortix AI. Our model leverages their dataset and builds on their pioneering approach to code merging and transformation. Key inspirations include:

• Dataset structure and generation methodology
• XML-based prompt engineering approach
• Evaluation metrics and benchmarking approaches

Special thanks to:

• The Kortix AI team for open-sourcing Fast Apply
• Their foundational work on high-speed code transformation models
• The comprehensive dataset they've made available to the community

While TURBOPASTA introduces its own innovations, the groundwork laid by Fast Apply was instrumental in making this project possible. We encourage users interested in code transformation models to also check out the original Fast Apply models:

• FastApply-7B-v1.0
• FastApply-1.5B-v1.0
• FastApply-dataset-v1.0

This project is licensed under Apache-2.0, consistent with Fast Apply's open-source ethos.

---

Based on a dataset inspired by https://www.kortix.ai/

TURBOPASTA LoRA Adapter for Qwen2.5-3B

A LoRA adapter for unsloth/Qwen2.5-3B that merges code updates using chain-of-thought reasoning and maintains strict adherence to original code structure and formatting.

Technical Specifications

Base Model

• Model: unsloth/Qwen2.5-3B
• LoRA Rank: 64
• Target Modules: v_proj, o_proj, down_proj, up_proj, q_proj, k_proj, gate_proj
• Task: CAUSAL_LM
• Dropout: 0
• Alpha: 32

Input/Output Format

Input XML structure:

<instruction>You are a coding assistant that helps merge code updates, ensuring every modification is fully integrated. Merge all changes from the snippet into the code. Preserve the code's structure, order, comments, and indentation exactly.</instruction>

<fastapply>
  <code>
    {original_code}
  </code>
  <update>
    {update_snippet}
  </update>
  <finalcode>
    {merged_result}
  </finalcode>
</fastapply>

The model supports multiple <fastapply> blocks for few-shot context learning. Use your stop token as </fastapply>.

Deployment

VLLM Server Setup

export VLLM_ALLOW_RUNTIME_LORA_UPDATING=1
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1

vllm serve unsloth/qwen2.5-3b \
  --gpu-memory-utilization=1 \
  --port 6002 \
  --served-model-name="turbopasta" \
  --trust-remote-code \
  --max-model-len 8192 \
  --disable-log-requests \
  --enable-lora \
  --lora-modules lora=./dataset/output/turbopasta/lora_model \
  --max-lora-rank 64

Client Implementation

import requests

def merge_code(original_code: str, update_snippet: str, vllm_url: str = "http://localhost:6002/v1/completions") -> dict:
    xml_content = (
        '<instruction>You are a coding assistant that helps merge code updates, ensuring every modification is fully '
        'integrated. Merge all changes from the snippet into the code. Preserve the code\'s structure, order, comments, '
        'and indentation exactly.</instruction>\n'
        '<fastapply>\n'
        '  <code>\n'
        f'{original_code}\n'
        '  </code>\n'
        '  <update>\n'
        f'{update_snippet}\n'
        '  </update>'
    )
    
    response = requests.post(
        vllm_url,
        json={
            "prompt": xml_content,
            "max_tokens": 6000,
            "temperature": 0.1,
            "model": "lora",
            "stop": ["</fastapply>"]
        },
        timeout=30000
    )
    
    completion = response.json()["choices"][0]["text"]
    
    # Parse XML tags
    import re
    def extract_tag(tag: str) -> str:
        match = re.search(f'<{tag}>(.*?)</{tag}>', completion, re.DOTALL)
        return match.group(1).strip() if match else ""
        
    return {
        "merged_code": extract_tag("finalcode")
    }

Batch Processing

The model works with the included data processor for parallel processing of code updates:

from request_processor import RequestProcessor

processor = RequestProcessor(
    input_file="updates.jsonl",
    output_file="merged.jsonl",
    num_threads=24
)
processor.process_file()

Input JSONL format:

{
    "id": "update_id",
    "original_code": "...",
    "update_snippet": "...",
    "file_path": "path/to/file"
}

Output JSONL format:

{
    "id": "update_id",
    "original_code": "...",
    "update_snippet": "...",
    "merged_code": "...",
    "file_path": "path/to/file",
    "processed_at": "2024-10-24 02:52:33"
}

Implementation and Performance Considerations

• Uses thread pooling for parallel processing
• Atomic writes with file locking
• Progress tracking with tqdm
• Automatic error handling and logging
• Configurable thread count for optimization
• Temperature set to 0.1 for consistent merges

Error Handling

Errors are captured in the output JSONL:

{
    "error": "error message",
    "processed_at": "timestamp"
}

Monitor errors in real-time:

tail -f merged.jsonl | grep error

Model Training Details

This model was trained using Force Multiplier's autotuning pipeline with the following key characteristics:

• Base Model: unsloth/Qwen2.5-3B
• Training Type: Few-shot learning with chain-of-thought reasoning
• Special Focus: Code structure preservation and merge accuracy
• LoRA Configuration: Optimized for code understanding and generation

Limitations

• Maximum context length of 8192 tokens
• Best suited for single-file code changes
• May require multiple passes for complex refactoring
• Not recommended for binary file merges