Edit model card

Neural-4-ARC-7b

Neural-4-ARC-7b is a merge of the following models using LazyMergekit:

🧩 Configuration

models:
  - model: Kukedlc/NeuralSirKrishna-7b
    # No parameters necessary for base model
  - model: paulml/OmniBeagleSquaredMBX-v3-7B
    parameters:
      density: 0.66
      weight: 0.2
  - model: nlpguy/AlloyIngotNeoX
    parameters:
      density: 0.55
      weight: 0.2
  - model: Gille/StrangeMerges_21-7B-slerp
    parameters:
      density: 0.55
      weight: 0.2
  - model: Kukedlc/Jupiter-k-7B-slerp
    parameters:
      density: 0.44
      weight: 0.2
  - model: Kukedlc/NeuralSirKrishna-7b
    parameters:
      density: 0.66
      weight: 0.2
merge_method: dare_ties
base_model: Kukedlc/NeuralSirKrishna-7b
parameters:
  int8_mask: true
dtype: bfloat16

πŸ’» Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "Kukedlc/Neural-4-ARC-7b"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
668
Safetensors
Model size
7.24B params
Tensor type
BF16
Β·
Inference API
Model is too large to load in Inference API (serverless). To try the model, launch it on Inference Endpoints (dedicated) instead.

Merge of