Instruction Following for benchmarks
Hi,
Thanks for your work on LLaMAX3.
I'm currently trying to figure out how to get the model to follow instructions more clearly.
Here's an example of what I mean:
This is my prompt
### Instruction:
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
Translate the following sentences from English to German.If not specified otherwise, the form of address of the response is informal and direct.
If there are vaiables (words prefixed with %) in the input, keep them in the response.
Do not add punctuations to the response, that is not present in the input.
Here are additional instructions:
### Input:
The quick brown fox jumps over the lazy dog
### Response:
The Response is Der schnelle braune Fuchs springt über den faulen Hund.
As you can see the model added a dot at the end of the response, even though it was tasked not to do so.
Here's another example:
### Instruction:
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
Translate the following sentences from English to Spanish.If not specified otherwise, the form of address of the response is informal and direct.
If there are vaiables (words prefixed with %) in the input, keep them in the response.
Do not add punctuations to the response, that is not present in the input.
Here are additional instructions:
### Input:
Hello, world!
### Response:
The Response is Hola mundo!
You can see here that the comma between the words was dropped.
I'm trying to figure out if there's anything I can do to improve the instruction following or if there's something I'm doing wrong.
What are your thoughts on this?
Thank you for your interest in our work.
Generally, if you want to improve the model's adherence to instructions, you can consider the following methods when designing prompts:
- Place important instructions at the beginning and end of the prompt.
- Emphasize the instruction multiple times.
- Break down complex instructions into multiple steps, allowing the model to complete them step by step. For example: (1) Translate the following sentences from English to Spanish. (2) Check whether to add punctuations to the response that are not present in the input.
Additionally, we only used Alpaca data during the SFT stage. You can try using more instruction fine-tuning data or synthetic data to train a stronger instruction-following model based on our base model(https://huggingface.co/LLaMAX/LLaMAX3-8B).
If you have any further questions, please let me know.