nm-testing

/

TinyLlama-1.1B-Chat-v1.0-pruned50-quant-ds

like 0

The model is for running on the DeepSparse Engine, like this:

from deepsparse import TextGeneration # pip install deepsparse-nightly[llm]

prompt = "How to make banana bread?"
formatted_prompt =  f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"

model = TextGeneration(model_path="hf:nm-testing/TinyLlama-1.1B-Chat-v1.0-pruned50-quant-ds")
print(model(formatted_prompt, max_new_tokens=200).generations[0].text)

"""
Sure, here's a recipe for making banana bread:

Ingredients:
- 1 cup of all-purpose flour
- 1 cup of unsalted butter
- 1 cup of unsalted sugar
- 1 cup of mashed bananas
- 1 cup of milk
- 1/2 cup of egg whites
- 1/4 cup of melted butter

Instructions:
1. Preheat oven to 375°F (150°C).
2. In a large mixing bowl, combine flour, sugar, mashed bananas, milk, egg whites, and butter. Mix well.
3. Add melted butter and mix again.
4. Add melted sugar and mix again.
5. Add melted milk and mix again.
6. Add egg whites and mix

"""