euclaise/Memphis-scribe-3B

Memphis-scribe 3B is a finetune of Memphis-CoT 3B on more creative data, which itself is a finetune of StableLM 3B 4e1t.

It is trained further on TinyCoT, but also on

• 5000 comments from reddit-instruct-curated
• 20000 comments from writingprompts-curated
• 2000 examples of converting MathQA problems to Python snippets
• 2000 examples of shorter booksum cases (both chapter->summary and summary->chapter tasks)
• 2000 examples from mathoverflow-accepted comments with >10 upvotes
• 2000 examples from coedit-reworded-deduped
• 500 examples from SQuAD, for generating QA pairs given the context
• 500 examples from ROPES, for generating scenario+QA triplets given the context
• conala
• 500 examples from logician
• 500 examples from goodwiki, for generating article given the title and description
• 2000 examples from rpguild
• Curated subset of oasst2
• LimaRP

Training procedure

I started from Memphis-CoT 3B, which used a novel iterative contrastive finetuning procedure to improve reasoning ability.

I first generated completions just as in each of the Memphis-CoT cycles.

Then, for each example in the dataset, I sampled a correct and incorrect completion. I applied the same ranking loss over these completions (with a weight of 0.2), but applied the cross-entropy loss over the example tokens instead of the completion tokens.

Finally, I averaged it with the Memphis-CoT model prior to the additional training, again with spherical linear interpolation, this time with a weight of 0.8.

Prompt formats

### User:
[insert instruction here]
### Assistant:
[insert response here]
### User:
...

Alternatively:

### System:
[Insert system message here, focused on roleplay]
### User:
[insert instruction here]
### Assistant:
[insert response here]
### User:
...

Benchmarks

This model performs significantly worse than Memphis-CoT on benchmarks, despite being better suited to chat and creative writing tasks. This is an expected tradeoff, especially for small models.

Model	GSM8K (5-shot)	AGIEval (English/Nous subset, acc_norm)	BIG Bench Hard (CoT, few-shot*)
StableLM 3B Base	2.05%	25.14%	36.75%
Memphis-CoT 3B	18.8%	27.22%	36.92%
Memphis-scribe 3B	9.55%	24.78%
*5-shot, as performed automatically by LM Evaluation Harness bbh_cot_fewshot even with num_fewshot=0