Update tokenization_indictrans.py

#2
by VarunGumma - opened

Update Tokenization as per new HF Tokenizer

prajdabre changed pull request status to merged

Sign up or log in to comment