590m / README.md
leaderboard-pr-bot's picture
Adding Evaluation Results
4e5e3d6
|
raw
history blame
7.6 kB
metadata
license: cc-by-nc-4.0
datasets:
  - tatsu-lab/alpaca
language:
  - en

Model Card for Model ID

This is a finetuned model of Cerebras 590M model using DataBricksLabs Dolly Framework

Model Details

Model Description

  • Developed by: Finetuned by Corianas (me) using open source tools
  • Shared by [optional]: [More Information Needed]
  • Model type: [More Information Needed]
  • Language(s) (NLP): EN
  • License: cc-by-nc-4.0
  • Finetuned from model [optional]: https://huggingface.co/cerebras/Cerebras-GPT-590M

Uses

This is a simple GPT chatbot that has been finetuned to understand instructions. Its knowledge about facts about the world is should be considered almost non-existant.

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

  • Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

Instruction: Write a love letter to Edgar Allan Poe.

Dear Edgar,

I hope you are doing well. As a person of your age and background, you are always learning new things and never too sure of your own best skills. May your future brighten up your days with the learning you have to offer. I can't wait to read your book and see the way it influenced you.

Thanks for the inspiration. It is so much fun to explore your special talents.

Sincerely,


Instruction: Write a tweet announcing Dolly, a large language model from Databricks.

Dolly is the world's first massive language model, available now in India. #IndiaDolly #Databricks


Instruction: I'm selling my Nikon D-750, write a short blurb for my ad.

I'm selling my Nikon D-750, someone whose dedication to imaging means he/she is always ready to answer questions, be creative and take chances on what really matters. A custom-designed top hat, a wide-angle lens, and an affordable ultrabuck remote offer the ultimate in quality. He/She can even go as far as to order a custom-made outdoor halogen pass. And if he/She is more than willing to take on any challenge, there is no doubt that with a creative lens, personal shots, and a custom top hat, photography is at its peak. As a satisfied customer, I would


Instruction: Explain to me the difference between nuclear fission and fusion.

Nuclear fission is a process that occurs during the production of nuclear fuel, while fusion is a process that occurs in the fusion industry. Nuclear fission is used to create the active ingredient in nuclear fuel, such as fission products, fusion plutonium, and decontamination services. Fusion requires a lower pressure in the fusion process and is more flexible compared to nuclear fission. Nuclear fission fuses with fission products, fission plutonium, and decontamination services.


Instruction: Give me a list of 5 science fiction books I should read next.

  1. How to Translate a Face into a System
  2. How to Fly a Surrealist
  3. How to Survive an Army Wildlife Hunt
  4. How to Strike an iPhone
  5. How to Teach a Hackathon

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

  • Hardware Type: [More Information Needed]
  • Hours used: [More Information Needed]
  • Cloud Provider: [More Information Needed]
  • Compute Region: [More Information Needed]
  • Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 25.62
ARC (25-shot) 24.15
HellaSwag (10-shot) 31.91
MMLU (5-shot) 26.61
TruthfulQA (0-shot) 42.19
Winogrande (5-shot) 48.38
GSM8K (5-shot) 0.08
DROP (3-shot) 6.03