Spaces:
Duplicated from Aston-xMAD/1bit_llama3_instruct_xmad_chatbot

xmadai
/

1bit_llama3_instruct_xmad_chatbot

Sleeping

App Files Files Community

1bit_llama3_instruct_xmad_chatbot / utils /test_module /custom_tokenization.py

Aston-xMAD's picture

init commit

b37c16f verified 4 months ago

88 Bytes

	from transformers import BertTokenizer


	class CustomTokenizer(BertTokenizer):
	pass