tokenizer in InternLMXComposer2ForCausalLM is always None

by kkkknddnjfks - opened Jul 5

Jul 5

https://huggingface.co/internlm/internlm-xcomposer2d5-7b/blob/eb12cdf26df23c26b0f485bbb8c674d7ceb3a02e/modeling_internlm_xcomposer2.py#L103

kkkknddnjfks changed discussion title from tokensizer in InternLMXComposer2ForCausalLM is always None to tokenizer in InternLMXComposer2ForCausalLM is always None Jul 5

RBBB1122

Aug 12

Hello, have you solved this problem? I am also stuck with this “None”.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment