--- extra_gated_heading: You need to share contact information with Meta to access this model extra_gated_prompt: "## FAIR Noncommercial Research License\nLast Updated: [October\ \ 16th 2024]\n“Acceptable Use Policy” means the FAIR Acceptable Use Policy, applicable\ \ to Research Materials, that is incorporated into this Agreement.\n“Agreement”\ \ means the terms and conditions for use, reproduction, distribution and modification\ \ of the Research Materials set forth herein.\n“Documentation” means the specifications,\ \ manuals and documentation accompanying Research Materials distributed by Meta.\n\ “Licensee” or “you” means you, or your employer or any other person or entity (if\ \ you are entering into this Agreement on such person or entity’s behalf), of the\ \ age required under applicable laws, rules or regulations to provide legal consent\ \ and that has legal authority to bind your employer or such other person or entity\ \ if you are entering in this Agreement on their behalf.\n“Meta” or “we” means Meta\ \ Platforms Ireland Limited (if you are located in or, if you are an entity, your\ \ principal place of business is in the EEA or Switzerland) and Meta Platforms,\ \ Inc. (if you are located outside of the EEA or Switzerland).\n“Noncommercial Research\ \ Uses” means noncommercial research use cases related to research, development,\ \ education, processing, or analysis and in each case, is not primarily intended\ \ for commercial advantage or monetary compensation to you or others.\n“Research\ \ Materials” means, collectively, Documentation and the models, software and algorithms,\ \ including machine-learning model code, trained model weights, inference-enabling\ \ code, training-enabling code, fine-tuning enabling code, demonstration materials\ \ and other elements of the foregoing distributed by Meta and made available under\ \ this Agreement.\nBy clicking “I Accept” below or by using or distributing any\ \ portion or element of the Research Materials, you agree to be bound by this Agreement.\n\ 1. License Rights and Redistribution.\n a. Grant of Rights. You are granted a non-exclusive,\ \ worldwide, non-transferable and royalty-free limited license under Meta’s intellectual\ \ property or other rights owned by Meta embodied in the Research Materials to use,\ \ reproduce, distribute, copy, create derivative works of, and make modifications\ \ to the Research Materials. \n b. Redistribution and Use. \n i. You will not\ \ use the Research Materials or any outputs or results of the Research Materials\ \ in connection with any commercial uses or for any uses other than Noncommercial\ \ Research Uses;\n ii. Distribution of Research Materials, and any derivative works\ \ thereof, are subject to the terms of this Agreement. If you distribute or make\ \ the Research Materials, or any derivative works thereof, available to a third\ \ party, you may only do so under the terms of this Agreement. You shall also provide\ \ a copy of this Agreement to such third party.\n iii. If you submit for publication\ \ the results of research you perform on, using, or otherwise in connection with\ \ Research Materials, you must acknowledge the use of Research Materials in your\ \ publication.\n iv. Your use of the Research Materials must comply with applicable\ \ laws and regulations (including Trade Control Laws) and adhere to the FAIR Acceptable\ \ Use Policy, which is hereby incorporated by reference into this Agreement.\n\n\ 2. User Support. Your Noncommercial Research Use of the Research Materials is done\ \ at your own discretion; Meta does not process any information nor provide any\ \ service in relation to such use. Meta is under no obligation to provide any support\ \ services for the Research Materials. Any support provided is “as is”, “with all\ \ faults”, and without warranty of any kind.\n3. Disclaimer of Warranty. UNLESS\ \ REQUIRED BY APPLICABLE LAW, THE RESEARCH MATERIALS AND ANY OUTPUT AND RESULTS\ \ THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND\ \ META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING,\ \ WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY,\ \ OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING\ \ THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE RESEARCH MATERIALS AND ASSUME\ \ ANY RISKS ASSOCIATED WITH YOUR USE OF THE RESEARCH MATERIALS AND ANY OUTPUT AND\ \ RESULTS.\n4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES\ \ BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE,\ \ PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST\ \ PROFITS OR ANY DIRECT OR INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY\ \ OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY\ \ OF ANY OF THE FOREGOING.\n5. Intellectual Property.\n a. Subject to Meta’s ownership\ \ of Research Materials and derivatives made by or for Meta, with respect to any\ \ derivative works and modifications of the Research Materials that are made by\ \ you, as between you and Meta, you are and will be the owner of such derivative\ \ works and modifications.\n b. If you institute litigation or other proceedings\ \ against Meta or any entity (including a cross-claim or counterclaim in a lawsuit)\ \ alleging that the Research Materials, outputs or results, or any portion of any\ \ of the foregoing, constitutes infringement of intellectual property or other rights\ \ owned or licensable by you, then any licenses granted to you under this Agreement\ \ shall terminate as of the date such litigation or claim is filed or instituted.\ \ You will indemnify and hold harmless Meta from and against any claim by any third\ \ party arising out of or related to your use or distribution of the Research Materials.\n\ \n6. Term and Termination. The term of this Agreement will commence upon your acceptance\ \ of this Agreement or access to the Research Materials and will continue in full\ \ force and effect until terminated in accordance with the terms and conditions\ \ herein. Meta may terminate this Agreement if you are in breach of any term or\ \ condition of this Agreement. Upon termination of this Agreement, you shall delete\ \ and cease use of the Research Materials. Sections 5, 6 and 9 shall survive the\ \ termination of this Agreement. \n7. Governing Law and Jurisdiction. This Agreement\ \ will be governed and construed under the laws of the State of California without\ \ regard to choice of law principles, and the UN Convention on Contracts for the\ \ International Sale of Goods does not apply to this Agreement. The courts of California\ \ shall have exclusive jurisdiction of any dispute arising out of this Agreement.\ \ \n8. Modifications and Amendments. Meta may modify this Agreement from time to\ \ time by posting a revised version at https://huggingface.co/facebook/layerskip-llama3.2-1B/blob/main/LICENSE;\ \ provided that they are similar in spirit to the current version of the Agreement,\ \ but may differ in detail to address new problems or concerns. All such changes\ \ will be effective immediately. Your continued use of the Research Materials after\ \ any modification to this Agreement constitutes your agreement to such modification.\ \ Except as provided in this Agreement, no modification or addition to any provision\ \ of this Agreement will be binding unless it is in writing and signed by an authorized\ \ representative of both you and Meta.\n\nFAIR Acceptable Use Policy \nThe Fundamental\ \ AI Research (FAIR) team at Meta seeks to further understanding of new and existing\ \ research domains with the mission of advancing the state-of-the-art in artificial\ \ intelligence through open research for the benefit of all. \nAs part of this mission,\ \ Meta makes certain research materials available for noncommercial research use.\ \ Meta is committed to promoting the safe and responsible use of such research materials.\ \ \nProhibited Uses\nYou agree you will not use, or allow others to use, Research\ \ Materials to:\n1.Violate the law or others’ rights, including to:\n a. Engage\ \ in, promote, generate, contribute to, encourage, plan, incite, or further illegal\ \ or unlawful activity or content, such as:\n i. Violence or terrorism\n ii. Exploitation\ \ or harm to children, including the solicitation, creation, acquisition, or dissemination\ \ of child exploitative content or failure to report Child Sexual Abuse Material\n\ \ iii. Human trafficking, exploitation, and sexual violence\n iv. The illegal\ \ distribution of information or materials to minors, including obscene materials,\ \ or failure to employ legally required age-gating in connection with such information\ \ or materials.\n v. Sexual solicitation\n vi. Any other criminal activity\n b.\ \ Engage in, promote, incite, or facilitate the harassment, abuse, threatening,\ \ or bullying of individuals or groups of individuals\n c. Engage in, promote, incite,\ \ or facilitate discrimination or other unlawful or harmful conduct in the provision\ \ of employment, employment benefits, credit, housing, other economic benefits,\ \ or other essential goods and services\n d. Engage in the unauthorized or unlicensed\ \ practice of any profession including, but not limited to, financial, legal, medical/health,\ \ or related professional practices\n e. Collect, process, disclose, generate, or\ \ infer health, demographic, or other sensitive personal or private information\ \ about individuals without rights and consents required by applicable laws\n f.\ \ Engage in or facilitate any action or generate any content that infringes, misappropriates,\ \ or otherwise violates any third-party rights, including the outputs or results\ \ of any technology using FAIR research materials\n g. Create, generate, or facilitate\ \ the creation of malicious code, malware, computer viruses or do anything else\ \ that could disable, overburden, interfere with or impair the proper working, integrity,\ \ operation or appearance of a website or computer system\n\n2. Engage in, promote,\ \ incite, facilitate, or assist in the planning or development of activities that\ \ present a risk of death or bodily harm to individuals, including use of research\ \ artifacts related to the following:\n a. Military, warfare, nuclear industries\ \ or applications, espionage, use for materials or activities that are subject to\ \ the International Traffic Arms Regulations (ITAR) maintained by the United States\ \ Department of State\n b. Guns and illegal weapons (including weapon development)\n\ \ c. Illegal drugs and regulated/controlled substances\n d. Operation of critical\ \ infrastructure, transportation technologies, or heavy machinery\n e. Self-harm\ \ or harm to others, including suicide, cutting, and eating disorders\n f. Any content\ \ intended to incite or promote violence, abuse, or any infliction of bodily harm\ \ to an individual\n\n3. Intentionally deceive or mislead others, including use\ \ of FAIR Research Materials related to the following:\n a. Generating, promoting,\ \ or furthering fraud or the creation or promotion of disinformation\n b. Generating,\ \ promoting, or furthering defamatory content, including the creation of defamatory\ \ statements, images, or other content\n c. Generating, promoting, or further distributing\ \ spam\n d. Impersonating another individual without consent, authorization, or\ \ legal right\n e. Representing that outputs of FAIR research materials or outputs\ \ from technology using FAIR research materials o are human-generated\n f. Generating\ \ or facilitating false online engagement, including fake reviews and other means\ \ of fake online engagement\n\n4. Fail to appropriately disclose to end users any\ \ known dangers of your Research Materials.\nPlease report any violation of this\ \ Policy or other problems that could lead to a violation of this Policy by submitting\ \ a report [here](https://docs.google.com/forms/d/e/1FAIpQLSeb11cryAopJ7LNrC4nxEUXrHY26hfkXQMf_uH-oFgA3WlYZQ/viewform)." extra_gated_fields: First Name: text Last Name: text Date of birth: date_picker Country: country Affiliation: text geo: ip_location ? By clicking Submit below I accept the terms of the license and acknowledge that the information I provide will be collected stored processed and shared in accordance with the Meta Privacy Policy : checkbox extra_gated_description: The information you provide will be collected, stored, processed and shared in accordance with the [Meta Privacy Policy](https://www.facebook.com/privacy/policy/). extra_gated_button_content: Submit language: - en library_name: transformers pipeline_tag: text-generation tags: - facebook - meta - pytorch - llama - llama-3 - llama-cpp - gguf-my-repo license: other license_name: fair license_link: LICENSE base_model: facebook/layerskip-llama3.2-1B model-index: - name: LayerSkip Llama3.2 1B results: - task: type: question-answering dataset: name: BoolQ type: google/boolq metrics: - type: acc value: 0.646 name: acc verified: false - task: type: question-answering dataset: name: PIQA type: ybisk/piqa metrics: - type: acc value: 0.724 name: acc verified: false - task: type: question-answering dataset: name: SIQA type: allenai/social_i_qa metrics: - type: acc value: 0.419 name: acc verified: false - task: type: text-generation dataset: name: HellaSwag type: Rowan/hellaswag metrics: - type: acc value: 0.455 name: acc verified: false - task: type: question-answering dataset: name: WinoGrande type: allenai/winogrande metrics: - type: acc value: 0.6 name: acc verified: false - task: type: question-answering dataset: name: ARC (Easy) type: allenai/ai2_arc metrics: - type: acc value: 0.651 name: acc verified: false - type: acc value: 0.295 name: acc verified: false - task: type: question-answering dataset: name: OpenBookQA type: allenai/openbookqa metrics: - type: acc value: 0.258 name: acc verified: false - task: type: question-answering dataset: name: RACE type: ehovy/race metrics: - type: acc value: 0.354 name: acc verified: false - task: type: question-answering dataset: name: MMLU type: cais/mmlu metrics: - type: acc value: 0.247 name: acc verified: false - task: type: text-generation dataset: name: Natural Questions type: google-research-datasets/nq_open metrics: - type: exact_match value: 0.038 name: exact_match verified: false - task: type: question-answering dataset: name: TriviaQA type: sentence-transformers/trivia-qa metrics: - type: acc value: 0.121 name: acc verified: false - task: type: text-generation dataset: name: GSM8K type: openai/gsm8k metrics: - type: exact_match value: 0.024 name: exact_match verified: false - task: type: question-answering dataset: name: MathQA type: allenai/math_qa metrics: - type: acc value: 0.259 name: acc verified: false - task: type: question-answering dataset: name: SQuAD2.0 type: rajpurkar/squad_v2 metrics: - type: exact value: 0.355 name: exact verified: false - task: type: text-classification dataset: name: ToxiGen type: toxigen/toxigen-data metrics: - type: acc value: 0.431 name: acc verified: false - task: type: text-generation dataset: name: HumanEval type: openai_humaneval metrics: - type: pass@1 value: 0.024 name: pass@1 verified: false - task: type: text-generation dataset: name: MBPP type: mbpp metrics: - type: pass@1 value: 0.062 name: pass@1 verified: false --- # impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63c2c05b8cc87cf0c05b89ba/PWrUdqxKbBBM18x8uZt40.png) This model was converted to GGUF format from [`facebook/layerskip-llama3.2-1B`](https://huggingface.co/facebook/layerskip-llama3.2-1B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. Refer to the [original model card](https://huggingface.co/facebook/layerskip-llama3.2-1B) for more details on the model. ## Use with llama.cpp Install llama.cpp through brew (works on Mac and Linux) ```bash brew install llama.cpp ``` Invoke the llama.cpp server or the CLI. ### CLI: ```bash llama-cli --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -p "The meaning to life and the universe is" ``` ### Server: ```bash llama-server --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -c 2048 ``` Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. Step 1: Clone llama.cpp from GitHub. ``` git clone https://github.com/ggerganov/llama.cpp ``` Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux). ``` cd llama.cpp && LLAMA_CURL=1 make ``` Step 3: Run inference through the main binary. ``` ./llama-cli --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -p "The meaning to life and the universe is" ``` or ``` ./llama-server --hf-repo impactframes/layerskip-llama3.2-1B-Q4_K_M-GGUF --hf-file layerskip-llama3.2-1b-q4_k_m.gguf -c 2048 ```