---
version: main
family: smollm2-1.7b
model_name: -all_raw_folders_metadata-600B
license: mit
tags:
  - model
  - transformer
  - smollm2
---
# SmolLM2 -all_raw_folders_metadata-600B (Version: main)

## Model Details
- **Architecture:** SmolLM2
- **Parameters:** 1.7B

## Training Configuration
```yaml
optimizer:
  class_path: torch.optim.AdamW
  init_args:
    lr: 0.0005
    weight_decay: 0.01
precision: bf16-mixed
seed: 42
train:
  global_batch_size: 1024
  max_seq_length: 2048
  max_tokens: 600000000000
  micro_batch_size: 8

```

## Model Loading and Revision System

This repository hosts multiple revisions of the model.
To load a specific revision, use the `revision` parameter. For example:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("locuslab/-all_raw_folders_metadata-600B", revision="final")
tokenizer = AutoTokenizer.from_pretrained("locuslab/-all_raw_folders_metadata-600B", revision="final")
```

Replace `"final"` with the desired revision.