maywell

/

Synatra-RP-Orca-2-7b-v0.1

like 6

How can I load this model?

With this code

model = AutoModelForCausalLM.from_pretrained(
        model_id,
        device_map="auto",
        use_flash_attention_2=True,
        torch_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
gen_cfg = GenerationConfig.from_model_config(model.config)
gen_cfg.pad_token_id = tokenizer.eos_token_id

I got error on loading tokenizer

ValueError: Couldn't instantiate the backend tokenizer from one of: 
(1) a `tokenizers` library serialization file, 
(2) a slow tokenizer instance to convert or 
(3) an equivalent slow tokenizer class to instantiate and convert. 
You need to have sentencepiece installed to convert a slow tokenizer to a fast one.