README.md

---
library_name: peft
base_model: cognitivecomputations/dolphin-2_6-phi-2
license: apache-2.0
datasets:
- jondurbin/contextual-dpo-v0.1
tags:
- dpo
- rag
---

# Model Card for Phi-2-Contextual

A lightweight model (2.8B) with enhanced RAG capabilities, with lower risk of hallucination.

This is a DPO fine-tune of the Phi-2 architecture (in particular, dolphin-2_6-phi-2) over the dataset https://huggingface.co/datasets/jondurbin/contextual-dpo-v0.1.

![phi-2-contextual](./mixtureofloras_an_illustration_of_a_mechanical_dolphin_consulti_e0fc7b52-f504-48d9-ab78-1db58bd4167e.png)

## Usage

Load the model as

```
model = AutoModelForCausalLM.from_pretrained(
    "vicgalle/phi-2-contextual",
    torch_dtype="auto",
    load_in_4bit=True,
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "cognitivecomputations/dolphin-2_6-phi-2",
    trust_remote_code=True
)
```

and use the following prompt template.


## Prompt format

```
BEGININPUT
BEGINCONTEXT
[key0: value0]
[key1: value1]
... other metdata ...
ENDCONTEXT
[insert your text blocks here]
ENDINPUT
[add as many other blocks, in the exact same format]
BEGININSTRUCTION
[insert your instruction(s).  The model was tuned with single questions, paragraph format, lists, etc.]
ENDINSTRUCTION
```
with the following definitions:

- `BEGININPUT` - denotes a new input block
- `BEGINCONTEXT` - denotes the block of context (metadata key/value pairs) to associate with the current input block
- `ENDCONTEXT` - denotes the end of the metadata block for the current input
- [text] - Insert whatever text you want for the input block, as many paragraphs as can fit in the context.
- `ENDINPUT` - denotes the end of the current input block
- [repeat as many input blocks in this format as you want]
- `BEGININSTRUCTION` - denotes the start of the list (or one) instruction(s) to respond to for all of the input blocks above.
- [instruction(s)]
- `ENDINSTRUCTION` - denotes the end of instruction set


## Examples

### Reference-based Q&A

```
BEGININPUT
BEGINCONTEXT
date: 2021-01-01
url: https://en.wikipedia.org/wiki/Rottnest_Island
ENDCONTEXT
Rottnest Island (Nyungar: Wadjemup), often colloquially referred to as "Rotto", is a 19-square-kilometre (7.3 sq mi) island off the coast of Western Australia, located 18 kilometres (11 mi) west of Fremantle. A sandy, low-lying island formed on a base of aeolianite limestone, Rottnest is an A-class reserve, the highest level of protection afforded to public land.

Together with Garden Island, Rottnest Island is a remnant of Pleistocene dune ridges.[7] Along with several other islands, Rottnest became separated from the mainland around 7,000 years ago, when sea levels rose; the traditional Noongar name for the island is Wadjemup, which means "place across the water where the spirits are".[8] Human artefacts have been found on the island dating back at least 30,000 years, but visitation and habitation of the island by the Noongar people appears to have ceased following its separation from the mainland.

The island was first documented by Willem de Vlamingh in 1696, who called it 't Eylandt 't Rottenest ("Rats' Nest Island") after the quokka population. Following establishment of the Swan River Colony (now Perth) in 1829, the island was initially used by British settlers for agricultural purposes, and a permanent settlement was built in Thomson Bay. From 1838 to 1931, Rottnest Island was also used as a prison and forced labour camp for over 3,600 Aboriginal people, who were subjected to extremely harsh conditions on the island.[9][10] Other historical uses of the island include as a military site, and for internment camps housing enemy aliens. Many of the island's buildings date from the colonial period, often made from locally quarried limestone, and are now used as accommodation for holidays.
ENDINPUT
BEGININSTRUCTION
In which years was the island used as a jail?
ENDINSTRUCTION
```

produces

```
The island was used as a jail from 1838 to 1931.
```

### Structured output generation

```
BEGININPUT
BEGINCONTEXT
date: 2021-01-01
url: https://web.site/123
ENDCONTEXT
The rock parrot (Neophema petrophila) is a species of grass parrot native to Australia. Described by John Gould in 1841, it is a small parrot 22 to 24 cm (8+3⁄4 to 9+1⁄2 in) long and weighing 50 to 60 g (1+3⁄4 to 2 oz) with predominantly olive-brown upperparts and more yellowish underparts. Its head is olive with light blue forecheeks and lores, and a dark blue frontal band line across the crown with lighter blue above and below. The sexes are similar in appearance, although the female tends to have a duller frontal band and less blue on the face. The preferred habitats are rocky islands and coastal dune areas from Lake Alexandrina in southeastern South Australia westwards across coastal South and Western Australia to Shark Bay.
ENDINPUT
BEGININSTRUCTION
Generate a list with all the mentioned locations
ENDINSTRUCTION
```

produces

```
The mentioned locations in the provided text are:

1. Lake Alexandrina in southeastern South Australia
2. Coastal dune areas extending from Lake Alexandrina to Shark Bay in Western Australia.
```

### Multiple contexts


```
BEGININPUT
BEGINCONTEXT
date: 2021-01-01
url: https://wikipedia.com/Lego
ENDCONTEXT
Since 1963, Lego pieces have been manufactured from acrylonitrile butadiene styrene (ABS).[19][39] As of September 2008, Lego engineers use the NX CAD/CAM/CAE PLM software suite to model the elements. The software allows the parts to be optimised by way of mould flow and stress analysis. Prototype moulds are sometimes built before the design is committed to mass production. The ABS plastic is heated to 232 °C (450 °F) until it reaches a dough-like consistency. It is then injected into the moulds using forces of between 25 and 150 tonnes and takes approximately 15 seconds to cool. The moulds are permitted a tolerance of up to twenty micrometres to ensure the bricks remain connected.[42] Human inspectors check the output of the moulds to eliminate significant variations in colour or thickness. According to the Lego Group, about eighteen bricks out of every million fail to meet the standard required.[46]

Lego factories recycle all but about 1 percent of their plastic waste from the manufacturing process. If the plastic cannot be re-used in Lego bricks, it is processed and sold on to industries that can make use of it.[47][48] Lego, in 2018, set a self-imposed 2030 deadline to find a more eco-friendly alternative to the ABS plastic.[49]
ENDINPUT
BEGININPUT
BEGINCONTEXT
date: 2021-01-02
url: https://wikipedia.com/Rottnest
ENDCONTEXT
Rottnest Island (Nyungar: Wadjemup), often colloquially referred to as "Rotto", is a 19-square-kilometre (7.3 sq mi) island off the coast of Western Australia, located 18 kilometres (11 mi) west of Fremantle. A sandy, low-lying island formed on a base of aeolianite limestone, Rottnest is an A-class reserve, the highest level of protection afforded to public land.

Together with Garden Island, Rottnest Island is a remnant of Pleistocene dune ridges.[7] Along with several other islands, Rottnest became separated from the mainland around 7,000 years ago, when sea levels rose; the traditional Noongar name for the island is Wadjemup, which means "place across the water where the spirits are".[8] Human artefacts have been found on the island dating back at least 30,000 years, but visitation and habitation of the island by the Noongar people appears to have ceased following its separation from the mainland.

The island was first documented by Willem de Vlamingh in 1696, who called it 't Eylandt 't Rottenest ("Rats' Nest Island") after the quokka population. Following establishment of the Swan River Colony (now Perth) in 1829, the island was initially used by British settlers for agricultural purposes, and a permanent settlement was built in Thomson Bay. From 1838 to 1931, Rottnest Island was also used as a prison and forced labour camp for over 3,600 Aboriginal people, who were subjected to extremely harsh conditions on the island.[9][10] Other historical uses of the island include as a military site, and for internment camps housing enemy aliens. Many of the island's buildings date from the colonial period, often made from locally quarried limestone, and are now used as accommodation for holidays.
ENDINPUT
BEGININSTRUCTION
What is threshold to ensure the pieces match?
ENDINSTRUCTION
```

produces

```
The threshold to ensure the pieces match refers to the tolerance of up to twenty micrometres in the Lego brick manufacturing process. According to the provided context, this tolerance is set by the Lego Group and it helps to ensure that the bricks remain connected. 

Reference(s):
date: 2021-01-01
url: https://wikipedia.com/Lego
```




### Framework versions

- PEFT 0.7.1