matsten/Glot500-m-iuseg

The model Glot500-m-iuseg is a fine-tuned version of the Glot500-m model. It was fine-tuned to segment Inuktitut words by morpheme boundaries and is intended to be used as a pre-processing tool for the language.

The model found in this repository is our best performing fine-tuned model described in the paper: "Surface-Level Morphological Segmentation of Low-resource Inuktitut Using Pre-trained Large Language Models" (link will be added when published)

Datasets used: The Nunavut Hansard Inuktitut–English Parallel Corpus 3.0 with Preliminary Machine Translation Results: https://aclanthology.org/2020.lrec-1.312/

Method used: LLMSegm: Surface-level Morphological Segmentation Using Large Language Model: https://aclanthology.org/2024.lrec-main.933/