All numbers are converted to Chinese numerals.

#4
by cookiexND - opened

Since all numbers are converted to Chinese numerals, they are difficult to read and impractical.

clu-ling org

@cookiexND , unfortunately our lab has no plans to publish an updated version of this model, since we're no longer using it in any applications.

If you're interested in fine-tuning a model that represents numerals as you desire, I would suggest modifying the transcripts for Common Voice v18 (https://commonvoice.mozilla.org/ja/datasets) to account for your desired output (see https://huggingface.co/datasets/mozilla-foundation/common_voice_17_0/raw/main/transcript/ja/train.tsv for a sample). You'll probably want to take care that you have sufficient examples where digits should not be converted to Arabic numerals (ex. 伊丹 十三 → 伊丹 十三 vs. 三種類 → 3種類 ). To ensure that sort of coverage, you may need to record additional data to supplement Common Voice.

An alternative would be to post-process your output using something like https://github.com/nagataaaas/Kanjize (you could also fine-tune a seq2seq transformer or a generative LLM to accomplish this).

Thank you for your kind reply. Understood.

cookiexND changed discussion status to closed

Sign up or log in to comment