Model Card for Model-Demo-35M

This model is used for a demo task for 35M.

Task type

Protein-level Regression

Model input type

SA Sequence

LoRA config

• r: 8
• lora_dropout: 0.0
• lora_alpha: 16
• target_modules: ['value', 'output.dense', 'query', 'intermediate.dense', 'key']
• modules_to_save: ['classifier']

Training config

• optimizer:
  • class: AdamW
  • betas: (0.9, 0.98)
  • weight_decay: 0.01
• learning rate: 0.0005
• epoch: 5
• batch size: 1
• precision: 16-mixed

Framework versions

• PEFT 0.10.0