matthayes commited on
Commit
bed6170
1 Parent(s): 7a630fd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -4
README.md CHANGED
@@ -32,7 +32,7 @@ In a Databricks notebook you could run:
32
  ```
33
 
34
  The instruction following pipeline can be loaded using the `pipeline` function as shown below. This loads a custom `InstructionTextGenerationPipeline`
35
- found in the model repo [here](https://huggingface.co/databricks/dolly-v2-7b/blob/main/instruct_pipeline.py), which is why `trust_remote_code=True` is required.
36
  Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
37
  It is also fine to remove it if there is sufficient memory.
38
 
@@ -46,22 +46,74 @@ generate_text = pipeline(model="databricks/dolly-v2-7b", torch_dtype=torch.bfloa
46
  You can then use the pipeline to answer instructions:
47
 
48
  ```
49
- generate_text("Explain to me the difference between nuclear fission and fusion.")
 
50
  ```
51
 
52
- Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/databricks/dolly-v2-7b/blob/main/instruct_pipeline.py),
53
  store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
54
 
55
  ```
 
56
  from instruct_pipeline import InstructionTextGenerationPipeline
57
  from transformers import AutoModelForCausalLM, AutoTokenizer
58
 
59
  tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-7b", padding_side="left")
60
- model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-7b", device_map="auto")
61
 
62
  generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
63
  ```
64
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
 
66
  ## Known Limitations
67
 
 
32
  ```
33
 
34
  The instruction following pipeline can be loaded using the `pipeline` function as shown below. This loads a custom `InstructionTextGenerationPipeline`
35
+ found in the model repo [here](https://huggingface.co/databricks/dolly-v2-3b/blob/main/instruct_pipeline.py), which is why `trust_remote_code=True` is required.
36
  Including `torch_dtype=torch.bfloat16` is generally recommended if this type is supported in order to reduce memory usage. It does not appear to impact output quality.
37
  It is also fine to remove it if there is sufficient memory.
38
 
 
46
  You can then use the pipeline to answer instructions:
47
 
48
  ```
49
+ res = generate_text("Explain to me the difference between nuclear fission and fusion.")
50
+ print(res[0]["generated_text"])
51
  ```
52
 
53
+ Alternatively, if you prefer to not use `trust_remote_code=True` you can download [instruct_pipeline.py](https://huggingface.co/databricks/dolly-v2-3b/blob/main/instruct_pipeline.py),
54
  store it alongside your notebook, and construct the pipeline yourself from the loaded model and tokenizer:
55
 
56
  ```
57
+ import torch
58
  from instruct_pipeline import InstructionTextGenerationPipeline
59
  from transformers import AutoModelForCausalLM, AutoTokenizer
60
 
61
  tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-7b", padding_side="left")
62
+ model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-7b", device_map="auto", torch_dtype=torch.bfloat16)
63
 
64
  generate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)
65
  ```
66
 
67
+ ### LangChain Usage
68
+
69
+ To use the pipeline with LangChain, you must set `return_full_text=True`, as LangChain expects the full text to be returned
70
+ and the default for the pipeline is to only return the new text.
71
+
72
+ ```
73
+ import torch
74
+ from transformers import pipeline
75
+
76
+ generate_text = pipeline(model="databricks/dolly-v2-7b", torch_dtype=torch.bfloat16,
77
+ trust_remote_code=True, device_map="auto", return_full_text=True)
78
+ ```
79
+
80
+ You can create a prompt that either has only an instruction or has an instruction with context:
81
+
82
+ ```
83
+ from langchain import PromptTemplate, LLMChain
84
+ from langchain.llms import HuggingFacePipeline
85
+
86
+ # template for an instrution with no input
87
+ prompt = PromptTemplate(
88
+ input_variables=["instruction"],
89
+ template="{instruction}")
90
+
91
+ # template for an instruction with input
92
+ prompt_with_context = PromptTemplate(
93
+ input_variables=["instruction", "context"],
94
+ template="{instruction}\n\nInput:\n{context}")
95
+
96
+ hf_pipeline = HuggingFacePipeline(pipeline=generate_text)
97
+
98
+ llm_chain = LLMChain(llm=hf_pipeline, prompt=prompt)
99
+ llm_context_chain = LLMChain(llm=hf_pipeline, prompt=prompt_with_context)
100
+ ```
101
+
102
+ Example predicting using a simple instruction:
103
+
104
+ ```
105
+ print(llm_chain.predict(instruction="Explain to me the difference between nuclear fission and fusion.").lstrip())
106
+ ```
107
+
108
+ Example predicting using an instruction with context:
109
+
110
+ ```
111
+ context = """George Washington (February 22, 1732[b] – December 14, 1799) was an American military officer, statesman,
112
+ and Founding Father who served as the first president of the United States from 1789 to 1797."""
113
+
114
+ print(llm_context_chain.predict(instruction="When was George Washington president?", context=context).lstrip())
115
+ ```
116
+
117
 
118
  ## Known Limitations
119