Error while trying to use the inference API with this model.
I tried using the inference API using this model and got the following error:
Can't load tokenizer using from_pretrained, please update its configuration: Can't load tokenizer for 'hivemind/gpt-j-6B-8bit'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'hivemind/gpt-j-6B-8bit' is the correct path to a directory containing all relevant files for a GPT2TokenizerFast tokenizer.
This error occurs even when using the inference box in the model card page.
Something is wrong here, could you please have a look?
I get the same error trying to run locally on my machine.
I had the same problem. There's a example notebook in model card (https://colab.research.google.com/drive/1ft6wQU0BhqG5PRlwgaZJv2VukKKjU4Es) that shows usage of this model. You have to use tokenizer from EleutherAI/gpt-j-6B and custom GPTJForCausalLM class that pass params.
For the record:
- this model did not support hosted API, it could only run via the colab notebook (see model card)
- currently, there is a better way to run gpt-J in 8 bit (also see model card)