There is some issue with tokenization...
❯ mlx_lm.server --model nisten/deepseek-r1-qwen32b-mlx-6bit --log-level DEBUG
2025-01-20 19:43:30,142 - DEBUG - Starting new HTTPS connection (1): huggingface.co:443
2025-01-20 19:43:30,356 - DEBUG - https://huggingface.co:443 "GET /api/models/nisten/deepseek-r1-qwen32b-mlx-6bit/revision/main HTTP/1.1" 200 4367
Fetching 10 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 47393.27it/s]
Traceback (most recent call last):
File "/opt/anaconda3/envs/mlx/bin/mlx_lm.server", line 8, in
sys.exit(main())
^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/mlx_lm/server.py", line 752, in main
run(args.host, args.port, ModelProvider(args))
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/mlx_lm/server.py", line 134, in init
self.load("default_model")
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/mlx_lm/server.py", line 161, in load
model, tokenizer = load(
^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/mlx_lm/utils.py", line 739, in load
tokenizer = load_tokenizer(
^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/mlx_lm/tokenizer_utils.py", line 367, in load_tokenizer
AutoTokenizer.from_pretrained(model_path, **tokenizer_config_extra),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 862, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2089, in from_pretrained
return cls._from_pretrained(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2311, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 124, in init
super().init(
File "/opt/anaconda3/envs/mlx/lib/python3.12/site-packages/transformers/tokenization_utils_fast.py", line 111, in init
fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: data did not match any variant of untagged enum ModelWrapper at line 757491 column 3
~/Desktop/mlx-model ▓▒░