mradermacher

/

model_requests

like 68

Hi,

I tried to run MMLU Pro benchmark against Meta-Llama-3.1-8B-Instruct-abliterated.i1-Q6_K.gguf from:
https://huggingface.co/mradermacher/Meta-Llama-3.1-8B-Instruct-abliterated-i1-GGUF
I used llama-server from llama.cpp, but I get a mangled response like this:

"step step the \"step is the step the step the is the correct the step the step the step the  the question 1 the following a \"the \"the correct the correct the \"the answer the correct the answer the correct the step of a given a given a correct the correct the correct the correct the answer the correct a the step of a given the correct the answer the correct the answer the correct to the step of the is the answer the corrects the correct the answer the correct the answer the correct the..."

I also tried q8_0 and f16 from the static quant:
https://huggingface.co/mradermacher/Meta-Llama-3.1-8B-Instruct-abliterated-GGUF
What's interesting is that the model from bartowski has the same problem.
https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-abliterated-GGUF
However, the model from mlx-community doesn't have this problem.
https://huggingface.co/mlx-community/Meta-Llama-3.1-8B-Instruct-abliterated-4bit
All three repos quantized from the same model:
https://huggingface.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated
I tried the original model from mlabonne with transformers, and it didn't have the problem.

I'm also attaching the full prompt for one question at the bottom. The answers in the five pairs of Q&A in the beginning are not generated by the model. They are 5-shot CoT examples to guide the model.

Do you mind looking into this?
Thanks so much!

[
    {
        "role": "system",
        "content": "The following are multiple choice questions (with answers) about biology. Think step by step and then finish your answer with \"the answer is (X)\" where X is the correct letter choice."
    },
    {
        "role": "user",
        "content": "Question: Which of the following represents an accurate statement concerning arthropods?\nOptions: A. They possess an exoskeleton composed primarily of peptidoglycan.\nB. They possess an open circulatory system with a dorsal heart.\nC. They are members of a biologically unsuccessful phylum incapable of exploiting diverse habitats and nutrition sources.\nD. They lack paired, jointed appendages."
    },
    {
        "role": "assistant",
        "content": "Answer: Let's think step by step. Peptidoglycan is known to comprise the plasma membrane of most bacteria, rather than the exoskeleton of arthropods, which is made of chitin, which rules out (A). The answer (C) is false because arthropods are a highly successful phylum. Likewise, arthropods have paired, jointed appendages, which rules out (D). The only remaining option is (B), as arthropods have an open circulatory system with a dorsal tubular heart. The answer is (B)."
    },
    {
        "role": "user",
        "content": "Question: In a given population, 1 out of every 400 people has a cancer caused by a completely recessive allele, b. Assuming the population is in Hardy-Weinberg equilibrium, which of the following is the expected proportion of individuals who carry the b allele but are not expected to develop the cancer?\nOptions: A. 19/400\nB. 1/400\nC. 40/400\nD. 38/400\nE. 2/400\nF. 1/200\nG. 20/400\nH. 50/400"
    },
    {
        "role": "assistant",
        "content": "Answer: Let's think step by step. According to the Hardy Weinberg Law, $p^2 + 2 p q + q^2 = 1$, and $p + q = 1$ where $p$ is the frequency of the dominant allele, $q$ is the frequency of the recessive allele, and $p^2$, $q^2$, and $2pq$ are the frequencies of dominant homozygous, recessive homozygous, and heterozygous individuals, respectively. \u200bThe frequency of the recessive allele (q) is $\\sqrt{\\frac{1}{400}} = 0.05$. We have $p = 1 - q = 0.95$. The frequency of heterozygous individuals is $2pq = 2 \\cdot 0.05 \\cdot 0.95 = 0.095$. The number of heterozygous individuals is equal to the frequency of heterozygous individuals times the size of the population, or $0.095 * 400 = 38$. So we end up with 38/400. The answer is (D)."
    },
    {
        "role": "user",
        "content": "Question: A mutation in a bacterial enzyme changed a previously polar amino acid into a nonpolar amino acid. This amino acid was located at a site distant from the enzyme\u2019s active site. How might this mutation alter the enzyme\u2019s substrate specificity?\nOptions: A. By changing the enzyme\u2019s pH optimum\nB. By changing the enzyme's molecular weight\nC. An amino acid change away from the active site increases the enzyme's substrate specificity.\nD. By changing the shape of the protein\nE. By changing the enzyme's temperature optimum\nF. By altering the enzyme's ability to be denatured\nG. By changing the enzyme\u2019s location in the cell\nH. By changing the enzyme's color\nI. An amino acid change away from the active site cannot alter the enzyme\u2019s substrate specificity.\nJ. By altering the enzyme's rate of reaction"
    },
    {
        "role": "assistant",
        "content": "Answer: Let's think step by step. A change in an amino acid leads to a change in the primary structure of the protein. A change in the primary structure may lead to a change in the secondary and the tertiary structure of the protein. A change in the tertiary structure means a change in the shape of the protein, so (C) has to be correct. Since the change does not affect the active site of the enzyme, we do not expect the activity of the enzyme to be affected. The answer is (D)."
    },
    {
        "role": "user",
        "content": "Question: Which of the following is not a way to form recombinant DNA?\nOptions: A. Translation\nB. Conjugation\nC. Specialized transduction\nD. Transformation"
    },
    {
        "role": "assistant",
        "content": "Answer: Let's think step by step. The introduction of foreign DNA or RNA into bacteria or eukaryotic cells is a common technique in molecular biology and scientific research. There are multiple ways foreign DNA can be introduced into cells including transformation, transduction, conjugation, and transfection. In contrast, (A) is not a way to form DNA: during translation the ribosomes synthesize proteins from RNA. The answer is (A)."
    },
    {
        "role": "user",
        "content": "Question: Which of the following is not known to be involved in the control of cell division?\nOptions: A. Microtubules\nB. Checkpoints\nC. DNA polymerase\nD. Centrosomes\nE. Cyclins\nF. Mitochondria\nG. Protein kinases\nH. Fibroblast cells"
    },
    {
        "role": "assistant",
        "content": "Answer: Let's think step by step. Normal cells move through the cell cycle in a regulated way. At the checkpoint stage, they use information about their own internal state and cues from the environment around them to decide whether to proceed with cell division. Cues like these act by changing the activity of core cell cycle regulators inside the cell. The most common regulators are cyclins and cyclin-dependent kinases. Fibroblast cells do not play any role in cell division. The answer is (H)."
    },
    {
        "role": "user",
        "content": "Question: Which of the following would most likely provide examples of mitotic cell divisions?\nOptions: A. cross section of muscle tissue\nB. longitudinal section of a shoot tip\nC. longitudinal section of a leaf vein\nD. cross section of a fruit\nE. cross section of a leaf\nF. longitudinal section of a petal\nG. longitudinal section of a seed\nH. cross section of an anther (site of pollen production in a flower)"
    }
]