README.md

---
language:
- en
license: cc-by-nc-4.0
tags:
- mixtral
- uncensored
- high-intelligence
model-index:
- name: MixtralOrochi8x7B
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 70.31
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=smelborp/MixtralOrochi8x7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 86.1
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=smelborp/MixtralOrochi8x7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 70.13
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=smelborp/MixtralOrochi8x7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 63.99
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=smelborp/MixtralOrochi8x7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 79.87
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=smelborp/MixtralOrochi8x7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 17.29
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=smelborp/MixtralOrochi8x7B
      name: Open LLM Leaderboard
---

# Orochi

<img src="https://huggingface.co/smelborp/MixtralOrochi8x7B/resolve/main/orochi.png" width="600" />

## Overview

Orochi is a cutting-edge language model based on the Mixtral architecture developed by Mistral. It represents a sophisticated merge of several prominent models, including Mixtral instruct, Noromaid, OpenBuddy, and several others, using mergekit with the DARE merge method. This model aims to provide highly intelligent responses unrestricted by content limitations. The name "Orochi" references the mythical Yamata-no-Orochi, symbolizing the model's multifaceted and powerful capabilities.

## Goals

- **Uncensored Content**: To provide unrestricted and comprehensive responses across various domains.
- **High Intelligence**: Leverage the combined knowledge and capabilities of the merged models to deliver insightful and accurate information.
- **Innovation in Language Modeling**: Push the boundaries of what's possible in natural language understanding and generation.

## Model Details

- **Architecture**: Mixtral, a Mixture of Experts model, underlies Orochi's design, enabling it to specialize and optimize its responses across different tasks and topics.
- **Merge Strategy**: Utilizing mergekit and the DARE method, Orochi integrates aspects of various models to enhance its performance and capabilities.

## Usage

Due to its uncensored nature, Orochi is best utilized in environments where intelligent, unrestricted dialogue is necessary. Users are encouraged to implement their own content moderation or alignment strategies appropriate for their use case.

## Ethical Considerations

As an uncensored model, Orochi may generate content that is unsuitable for all audiences. Users are advised to consider the implications of using such a model and to implement suitable safeguards and ethical guidelines.

## Acknowledgements

Orochi is a product of numerous contributions from the fields of machine learning and language modeling. Special thanks to the teams behind Mixtral, mergekit, and all the individual models integrated into Orochi.

---
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_smelborp__MixtralOrochi8x7B)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |64.62|
|AI2 Reasoning Challenge (25-Shot)|70.31|
|HellaSwag (10-Shot)              |86.10|
|MMLU (5-Shot)                    |70.13|
|TruthfulQA (0-shot)              |63.99|
|Winogrande (5-shot)              |79.87|
|GSM8k (5-shot)                   |17.29|