These models underwent supervised fine-tuning and direct preference optimization for instruction following on top of Microsoft's Phi 1.5 base LLM