README.md

---
extra_gated_heading: You need to share contact information with Meta to access this
  model
extra_gated_prompt: "## FAIR Noncommercial Research License\nLast Updated: [October\
  \ 16th 2024]\n“Acceptable Use Policy” means the FAIR Acceptable Use Policy, applicable\
  \ to Research Materials, that is incorporated into this Agreement.\n“Agreement”\
  \ means the terms and conditions for use, reproduction, distribution and modification\
  \ of the Research Materials set forth herein.\n“Documentation” means the specifications,\
  \ manuals and documentation accompanying Research Materials distributed by Meta.\n\
  “Licensee” or “you” means you, or your employer or any other person or entity (if\
  \ you are entering into this Agreement on such person or entity’s behalf), of the\
  \ age required under applicable laws, rules or regulations to provide legal consent\
  \ and that has legal authority to bind your employer or such other person or entity\
  \ if you are entering in this Agreement on their behalf.\n“Meta” or “we” means Meta\
  \ Platforms Ireland Limited (if you are located in or, if you are an entity, your\
  \ principal place of business is in the EEA or Switzerland) and Meta Platforms,\
  \ Inc. (if you are located outside of the EEA or Switzerland).\n“Noncommercial Research\
  \ Uses” means noncommercial research use cases related to research, development,\
  \ education, processing, or analysis and in each case, is not primarily intended\
  \ for commercial advantage or monetary compensation to you or others.\n“Research\
  \ Materials” means, collectively, Documentation and the models, software and algorithms,\
  \ including machine-learning model code, trained model weights, inference-enabling\
  \ code, training-enabling code, fine-tuning enabling code, demonstration materials\
  \ and other elements of the foregoing distributed by Meta and made available under\
  \ this Agreement.\nBy clicking “I Accept” below or by using or distributing any\
  \ portion or element of the Research Materials, you agree to be bound by this Agreement.\n\
  1. License Rights and Redistribution.\n a. Grant of Rights. You are granted a non-exclusive,\
  \ worldwide, non-transferable and royalty-free limited license under Meta’s intellectual\
  \ property or other rights owned by Meta embodied in the Research Materials to use,\
  \ reproduce, distribute, copy, create derivative works of, and make modifications\
  \ to the Research Materials.  \n b. Redistribution and Use.  \n  i. You will not\
  \ use the Research Materials or any outputs or results of the Research Materials\
  \ in connection with any commercial uses or for any uses other than Noncommercial\
  \ Research Uses;\n  ii. Distribution of Research Materials, and any derivative works\
  \ thereof, are subject to the terms of this Agreement. If you distribute or make\
  \ the Research Materials, or any derivative works thereof, available to a third\
  \ party, you may only do so under the terms of this Agreement. You shall also provide\
  \ a copy of this Agreement to such third party.\n  iii.  If you submit for publication\
  \ the results of research you perform on, using, or otherwise in connection with\
  \ Research Materials, you must acknowledge the use of Research Materials in your\
  \ publication.\n  iv. Your use of the Research Materials must comply with applicable\
  \ laws and regulations (including Trade Control Laws) and adhere to the FAIR Acceptable\
  \ Use Policy, which is hereby incorporated by reference into this Agreement.\n\n\
  2. User Support. Your Noncommercial Research Use of the Research Materials is done\
  \ at your own discretion; Meta does not process any information nor provide any\
  \ service in relation to such use.  Meta is under no obligation to provide any support\
  \ services for the Research Materials. Any support provided is “as is”, “with all\
  \ faults”, and without warranty of any kind.\n3. Disclaimer of Warranty. UNLESS\
  \ REQUIRED BY APPLICABLE LAW, THE RESEARCH MATERIALS AND ANY OUTPUT AND RESULTS\
  \ THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND\
  \ META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING,\
  \ WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY,\
  \ OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING\
  \ THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE RESEARCH MATERIALS AND ASSUME\
  \ ANY RISKS ASSOCIATED WITH YOUR USE OF THE RESEARCH MATERIALS AND ANY OUTPUT AND\
  \ RESULTS.\n4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES\
  \ BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE,\
  \ PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST\
  \ PROFITS OR ANY DIRECT OR INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY\
  \ OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY\
  \ OF ANY OF THE FOREGOING.\n5. Intellectual Property.\n a. Subject to Meta’s ownership\
  \ of Research Materials and derivatives made by or for Meta, with respect to any\
  \ derivative works and modifications of the Research Materials that are made by\
  \ you, as between you and Meta, you are and will be the owner of such derivative\
  \ works and modifications.\n b. If you institute litigation or other proceedings\
  \ against Meta or any entity (including a cross-claim or counterclaim in a lawsuit)\
  \ alleging that the Research Materials, outputs or results, or any portion of any\
  \ of the foregoing, constitutes infringement of intellectual property or other rights\
  \ owned or licensable by you, then any licenses granted to you under this Agreement\
  \ shall terminate as of the date such litigation or claim is filed or instituted.\
  \ You will indemnify and hold harmless Meta from and against any claim by any third\
  \ party arising out of or related to your use or distribution of the Research Materials.\n\
  \n6. Term and Termination. The term of this Agreement will commence upon your acceptance\
  \ of this Agreement or access to the Research Materials and will continue in full\
  \ force and effect until terminated in accordance with the terms and conditions\
  \ herein. Meta may terminate this Agreement if you are in breach of any term or\
  \ condition of this Agreement. Upon termination of this Agreement, you shall delete\
  \ and cease use of the Research Materials. Sections 5, 6 and 9 shall survive the\
  \ termination of this Agreement. \n7. Governing Law and Jurisdiction. This Agreement\
  \ will be governed and construed under the laws of the State of California without\
  \ regard to choice of law principles, and the UN Convention on Contracts for the\
  \ International Sale of Goods does not apply to this Agreement. The courts of California\
  \ shall have exclusive jurisdiction of any dispute arising out of this Agreement.\
  \ \n8. Modifications and Amendments. Meta may modify this Agreement from time to\
  \ time by posting a revised version at https://huggingface.co/facebook/layerskip-llama3.2-1B/blob/main/LICENSE;\
  \ provided that they are similar in spirit to the current version of the Agreement,\
  \ but may differ in detail to address new problems or concerns. All such changes\
  \ will be effective immediately. Your continued use of the Research Materials after\
  \ any modification to this Agreement constitutes your agreement to such modification.\
  \ Except as provided in this Agreement, no modification or addition to any provision\
  \ of this Agreement will be binding unless it is in writing and signed by an authorized\
  \ representative of both you and Meta.\n\nFAIR Acceptable Use Policy \nThe Fundamental\
  \ AI Research (FAIR) team at Meta seeks to further understanding of new and existing\
  \ research domains with the mission of advancing the state-of-the-art in artificial\
  \ intelligence through open research for the benefit of all. \nAs part of this mission,\
  \ Meta makes certain research materials available for noncommercial research use.\
  \ Meta is committed to promoting the safe and responsible use of such research materials.\
  \  \nProhibited Uses\nYou agree you will not use, or allow others to use, Research\
  \ Materials to:\n1.Violate the law or others’ rights, including to:\n a. Engage\
  \ in, promote, generate, contribute to, encourage, plan, incite, or further illegal\
  \ or unlawful activity or content, such as:\n  i. Violence or terrorism\n  ii. Exploitation\
  \ or harm to children, including the solicitation, creation, acquisition, or dissemination\
  \ of child exploitative content or failure to report Child Sexual Abuse Material\n\
  \  iii. Human trafficking, exploitation, and sexual violence\n  iv. The illegal\
  \ distribution of information or materials to minors, including obscene materials,\
  \ or failure to employ legally required age-gating in connection with such information\
  \ or materials.\n  v. Sexual solicitation\n  vi. Any other criminal activity\n b.\
  \ Engage in, promote, incite, or facilitate the harassment, abuse, threatening,\
  \ or bullying of individuals or groups of individuals\n c. Engage in, promote, incite,\
  \ or facilitate discrimination or other unlawful or harmful conduct in the provision\
  \ of employment, employment benefits, credit, housing, other economic benefits,\
  \ or other essential goods and services\n d. Engage in the unauthorized or unlicensed\
  \ practice of any profession including, but not limited to, financial, legal, medical/health,\
  \ or related professional practices\n e. Collect, process, disclose, generate, or\
  \ infer health, demographic, or other sensitive personal or private information\
  \ about individuals without rights and consents required by applicable laws\n f.\
  \ Engage in or facilitate any action or generate any content that infringes, misappropriates,\
  \ or otherwise violates any third-party rights, including the outputs or results\
  \ of any technology using FAIR research materials\n g. Create, generate, or facilitate\
  \ the creation of malicious code, malware, computer viruses or do anything else\
  \ that could disable, overburden, interfere with or impair the proper working, integrity,\
  \ operation or appearance of a website or computer system\n\n2. Engage in, promote,\
  \ incite, facilitate, or assist in the planning or development of activities that\
  \ present a risk of death or bodily harm to individuals, including use of research\
  \ artifacts related to the following:\n a. Military, warfare, nuclear industries\
  \ or applications, espionage, use for materials or activities that are subject to\
  \ the International Traffic Arms Regulations (ITAR) maintained by the United States\
  \ Department of State\n b. Guns and illegal weapons (including weapon development)\n\
  \ c. Illegal drugs and regulated/controlled substances\n d. Operation of critical\
  \ infrastructure, transportation technologies, or heavy machinery\n e. Self-harm\
  \ or harm to others, including suicide, cutting, and eating disorders\n f. Any content\
  \ intended to incite or promote violence, abuse, or any infliction of bodily harm\
  \ to an individual\n\n3. Intentionally deceive or mislead others, including use\
  \ of FAIR Research Materials related to the following:\n a. Generating, promoting,\
  \ or furthering fraud or the creation or promotion of disinformation\n b. Generating,\
  \ promoting, or furthering defamatory content, including the creation of defamatory\
  \ statements, images, or other content\n c. Generating, promoting, or further distributing\
  \ spam\n d. Impersonating another individual without consent, authorization, or\
  \ legal right\n e. Representing that outputs of FAIR research materials or outputs\
  \ from technology using FAIR research materials o are human-generated\n f. Generating\
  \ or facilitating false online engagement, including fake reviews and other means\
  \ of fake online engagement\n\n4. Fail to appropriately disclose to end users any\
  \ known dangers of your Research Materials.\nPlease report any violation of this\
  \ Policy or other problems that could lead to a violation of this Policy by submitting\
  \ a report [here](https://docs.google.com/forms/d/e/1FAIpQLSeb11cryAopJ7LNrC4nxEUXrHY26hfkXQMf_uH-oFgA3WlYZQ/viewform)."
extra_gated_fields:
  First Name: text
  Last Name: text
  Date of birth: date_picker
  Country: country
  Affiliation: text
  geo: ip_location
  ? By clicking Submit below I accept the terms of the license and acknowledge that
    the information I provide will be collected stored processed and shared in accordance
    with the Meta Privacy Policy
  : checkbox
extra_gated_description: The information you provide will be collected, stored, processed
  and shared in accordance with the [Meta Privacy Policy](https://www.facebook.com/privacy/policy/).
extra_gated_button_content: Submit
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- facebook
- meta
- pytorch
- llama
- llama-3
- llama-cpp
- gguf-my-repo
license: other
license_name: fair
license_link: LICENSE
base_model: facebook/layerskip-llama3.2-1B
model-index:
- name: LayerSkip Llama3.2 1B
  results:
  - task:
      type: question-answering
    dataset:
      name: BoolQ
      type: google/boolq
    metrics:
    - type: acc
      value: 0.646
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: PIQA
      type: ybisk/piqa
    metrics:
    - type: acc
      value: 0.724
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: SIQA
      type: allenai/social_i_qa
    metrics:
    - type: acc
      value: 0.419
      name: acc
      verified: false
  - task:
      type: text-generation
    dataset:
      name: HellaSwag
      type: Rowan/hellaswag
    metrics:
    - type: acc
      value: 0.455
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: WinoGrande
      type: allenai/winogrande
    metrics:
    - type: acc
      value: 0.6
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: ARC (Easy)
      type: allenai/ai2_arc
    metrics:
    - type: acc
      value: 0.651
      name: acc
      verified: false
    - type: acc
      value: 0.295
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: OpenBookQA
      type: allenai/openbookqa
    metrics:
    - type: acc
      value: 0.258
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: RACE
      type: ehovy/race
    metrics:
    - type: acc
      value: 0.354
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: MMLU
      type: cais/mmlu
    metrics:
    - type: acc
      value: 0.247
      name: acc
      verified: false
  - task:
      type: text-generation
    dataset:
      name: Natural Questions
      type: google-research-datasets/nq_open
    metrics:
    - type: exact_match
      value: 0.038
      name: exact_match
      verified: false
  - task:
      type: question-answering
    dataset:
      name: TriviaQA
      type: sentence-transformers/trivia-qa
    metrics:
    - type: acc
      value: 0.121
      name: acc
      verified: false
  - task:
      type: text-generation
    dataset:
      name: GSM8K
      type: openai/gsm8k
    metrics:
    - type: exact_match
      value: 0.024
      name: exact_match
      verified: false
  - task:
      type: question-answering
    dataset:
      name: MathQA
      type: allenai/math_qa
    metrics:
    - type: acc
      value: 0.259
      name: acc
      verified: false
  - task:
      type: question-answering
    dataset:
      name: SQuAD2.0
      type: rajpurkar/squad_v2
    metrics:
    - type: exact
      value: 0.355
      name: exact
      verified: false
  - task:
      type: text-classification
    dataset:
      name: ToxiGen
      type: toxigen/toxigen-data
    metrics:
    - type: acc
      value: 0.431
      name: acc
      verified: false
  - task:
      type: text-generation
    dataset:
      name: HumanEval
      type: openai_humaneval
    metrics:
    - type: pass@1
      value: 0.024
      name: pass@1
      verified: false
  - task:
      type: text-generation
    dataset:
      name: MBPP
      type: mbpp
    metrics:
    - type: pass@1
      value: 0.062
      name: pass@1
      verified: false
---

# impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF

![image/png](https://cdn-uploads.huggingface.co/production/uploads/63c2c05b8cc87cf0c05b89ba/PWrUdqxKbBBM18x8uZt40.png)

This model was converted to GGUF format from [`facebook/layerskip-llama3.2-1B`](https://huggingface.co/facebook/layerskip-llama3.2-1B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/facebook/layerskip-llama3.2-1B) for more details on the model.

## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -c 2048
```