license: other
license_name: gpt-sw3
license_link: LICENSE
base_model: AI-Sweden-Models/gpt-sw3-6.7b
ICELANDIC GPT-SW3 FOR SPELL AND GRAMMAR CHECKING
This is a model for correcting spelling and grammar errors in Icelandic text. It is a GPT-SW3 model (https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b) finetuned on Icelandic and particularly on the spell and grammar checking task.
Provided here is the model along with a script for running it locally.
To run the script you will need a python3 environment. Install the required dependencies by running
pip install -r requirements.txt
The current version of transformers includes a bug which has to be fixed in the user's environment before the model can be run. To fix it, change "gpt-sw3-7b" in line no. 138 in transformers/models/gpt_sw3/tokenization_gpt_sw3.py to "gpt-sw3-6.7b".
After that you can run the script with an input file consisting of text to correct.
The model is fine-tuned on the following three tasks:
- Task 1: The model evaluates one text with regards to e.g. grammar and spelling, and returns all errors in the input text as a list, with their position in the text and their corrections.
- Task 2: The model evaluates two texts and chooses which one is better with regards to e.g. grammar and spelling.
- Task 3: The model evaluates one text with regards to e.g. grammar and spelling, and returns a corrected version of the text.
The script which runs the model takes the following three arguments:
- --task: A number (1-3) representing the intended task. The script includes prompts for each task.
- --input-file: A file containing text to be evaluated. The format of the input file differs between tasks, and is described further below.
- --output-file: A path to a desired output file to be created by the script. The format of the file differs between tasks, and is described further below.
An input file for tasks 1 and 3 should be a .txt file consisting of texts per line. An example of both files can be found under ./example_inputs. An input file for task 2 should be a .jsonl file, where each line is a dictionary object showing two texts. Keys in the dictionary are "a" and "b" and texts to be evaluated are their values. An example of this file can be found under ./example_inputs.
All output files are .txt files and output examples for each task are shown in ./example_outputs. An output file for task 1 shows each text which was evaluated, followed by a list of corrections. Text outputs are separated by an empty line. An output file for task 2 shows 'A' or 'B' for which text is preferred, one choice per line. An output file for task 3 shows the corrected text, one text per line.
Run the script with
python run_model.py --task 3 --input-file example_inputs/task3_example.txt --output-file example_outputs/task3_example.txt
The script we provide runs in CPU-only mode and should work on most systems that have enough RAM to load the model. Users that wish to accelerate their corrections with specialized hardware (eg GPUs) will need to install appropriate support packages for their hardware. We refer to the PyTorch documentation: https://pytorch.org/get-started/locally/ . After the extra packages are installed, add the device parameter to the pipeline constructor. See the HuggingFace documentation (https://huggingface.co/docs/transformers/main_classes/pipelines) for more details.