README.md

model-index:
 - name: xmanii/llama-3-8b-instruct-bnb-4bit-persian
 description: |
 **Model Information**

 **Developed by:** xmanii
 **License:** Apache-2.0
 **Finetuned from model:** unsloth/llama-3-8b-instruct-bnb-4bit

 **Model Description**

 This LLaMA model was fine-tuned on a unique Persian dataset of Alpaca chat conversations, consisting of approximately 8,000 rows. Our training process utilized two H100 GPUs, completing in just under 1 hour. We leveraged the power of Unsloth and Hugging Face's TRL library to accelerate our training process by 2x.

 ![Unsloth Made with Love](https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png)

 **Training Resources**

 * 2x H100 GPUs
 * Unsloth and Hugging Face's TRL library

 **Dataset**

 * Unique Persian dataset of Alpaca chat conversations
 * Approximately 8,000 rows

 **Open-Source Contribution**

 This model is open-source, and we invite the community to use and build upon our work. The fine-tuned LLaMA model is designed to improve Persian conversation capabilities, and we hope it will contribute to the advancement of natural language processing in the Persian language.

 **Using Adapters with Unsloth**

 To run the model with adapters, you can use the following code:

 ```python
 import torch
 from unsloth import FastLanguageModel
 from unsloth.chat_templates import get_chat_template

 model_save_path = "path to the download folder" #the hugging face folder path pulled.

 model, tokenizer = FastLanguageModel.from_pretrained(
 model_name=model_save_path,
 max_seq_length=4096,
 load_in_4bit=True,
 )
 FastLanguageModel.for_inference(model) # Enable native 2x faster inference

 tokenizer = get_chat_template(
 tokenizer,
 chat_template="llama-3", # use the llama-3 template
 mapping={"role": "from", "content": "value", "user": "human", "assistant": "gpt"}, # mapping the messages.
 )

 messages = [{"from": "human", "value": "your prompt"}]#add your prompt here as human
 inputs = tokenizer.apply_chat_template(
 messages,
 tokenize=True,
 add_generation_prompt=True, # Must add for generation
 return_tensors="pt",
 ).to("cuda")

 outputs = model.generate(input_ids=inputs, max_new_tokens=2048, use_cache=True)
 response = tokenizer.batch_decode(outputs, skip_special_tokens=True)
 print(response)
 ```

 **Full 16-bit Merged Model**

 For a full 16-bit merged model, please check out xmanii/Llama3-8b-simorgh-16bit.

 **Future Work**

 We are working on quantizing the models and bringing them to ollama.