shreyasmeher/llama-3.1-terrorism-classifier

GTD-Finetuned Llama 3.1 8B - LoRA Weights

• Model Type: LoRA adaptation weights
• Base Model: unsloth/meta-llama-3.1-8b-bnb-4bit
• Use Case: Combine with base model for classification

Training Data

• Dataset: Global Terrorism Database (GTD)
• Time Period: Events before January 1, 2017
• Format: Multi-label classification with probabilities
• Labels:
  • Assassination
  • Armed Assault
  • Bombing/Explosion
  • Hijacking
  • Hostage Taking (Barricade Incident)
  • Hostage Taking (Kidnapping)
  • Facility/Infrastructure Attack
  • Unarmed Assault
  • Unknown

Data Processing

1. Date Filtering:

   • Filtered events occurring before 2017-01-01
   • Properly handled missing month/day values

2. Data Cleaning:

   • Removed entries with missing summaries
   • Removed entries with missing primary attack types
   • Handled multi-label cases (up to 3 labels per event)

3. Label Processing:

   • Primary attack type: Assigned higher probability (0.8)
   • Secondary attack type: Assigned medium probability (0.5)
   • Tertiary attack type: Assigned lower probability (0.3)
   • Probabilities normalized to sum to 1.0

4. Training Format:

   • Input: Event summaries in natural language
   • Output: JSON object with attack types and probabilities
   • Instruction: Consistent prompt for classification task
   • Added EOS tokens for proper generation

Training Details

• Optimizer: AdamW 8-bit
• Learning Rate: 2e-4
• Batch Size: 2 per device
• Gradient Accumulation Steps: 4
• LR Scheduler: Linear
• Weight Decay: 0.01
• LoRA Configuration:
  • Rank: 16
  • Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  • Alpha: 16
  • Dropout: 0

Intended Use

This model is designed for:

1. Multi-label classification of terrorist events
2. Probability estimation for different attack types
3. Research purposes in terrorism studies and security analysis
4. Comparative analysis with ConfliBERT and other models

Limitations

1. Training data limited to pre-2017 events
2. May not capture recent changes in attack patterns
3. Performance dependent on quality of event descriptions
4. Subject to biases present in the GTD dataset

Evaluation Results

[To be added after evaluation]

Ethical Considerations

1. Model trained on sensitive data about terrorist events
2. Should be used responsibly for research and analysis
3. Not intended for operational security decisions
4. Results should be interpreted with appropriate context

Citation and Attribution

@misc{gtd-llama,
  author = {Meher, Shreyas},
  title = {GTD-Finetuned Llama 3.1 8B},
  year = {2024},
  publisher = {HuggingFace},
  note = {Based on Meta's Llama 3.1 and GTD Dataset}
}

Acknowledgments

• Unsloth for optimization framework
• Hugging Face for transformers and TRL library
• Global Terrorism Database team
• Meta AI for Llama 3.1 base model