Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,134 @@
+
+---
+
+language:
+- en
+- it
+license: apache-2.0
+tags:
+- text-generation-inference
+- transformers
+- unsloth
+- llama
+- trl
+- sft
+
+---
+
+[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://huggingface.co/QuantFactory)
+
+
+# QuantFactory/Meta-Llama-3.1-8B-Text-to-SQL-GGUF
+This is quantized version of [ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL](https://huggingface.co/ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL) created using llama.cpp
+
+# Original Model Card
+
+
+# Meta LLaMA 3.1 8B 4-bit Finetuned Model
+
+This model is a fine-tuned version of `Meta-Llama-3.1-8B`, developed by **ruslanmv** for text generation tasks. It leverages 4-bit quantization, making it more efficient for inference while maintaining strong performance in natural language generation.
+
+---
+
+## Model Details
+
+- **Base Model**: `unsloth/meta-llama-3.1-8b-bnb-4bit`
+- **Finetuned by**: ruslanmv
+- **Language**: English
+- **License**: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+- **Tags**: 
+ - text-generation-inference
+ - transformers
+ - unsloth
+ - llama
+ - trl
+ - sft
+
+---
+
+## Model Usage
+
+### Installation
+
+To use this model, you will need to install the necessary libraries:
+
+```bash
+pip install transformers accelerate 
+```
+
+### Loading the Model in Python
+
+Here’s an example of how to load this fine-tuned model using Hugging Face's `transformers` library:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+# Load the model and tokenizer
+model_name = "ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL"
+
+# Ensure you have the right device setup
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+# Load the model and tokenizer from the Hugging Face Hub
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+# Initialize the tokenizer (adjust the model name as needed)
+# Define EOS token for terminating the sequences
+EOS_TOKEN = tokenizer.eos_token
+
+# Define Alpaca-style prompt template
+alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+
+### Instruction:
+{}
+
+### Input:
+{}
+
+### Response:
+"""
+
+# Format the prompt without the response part
+prompt = alpaca_prompt.format(
+ "Provide the SQL query",
+ "Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"
+)
+# Tokenize the prompt and generate text
+inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
+outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)
+
+# Decode the generated text
+generated_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
+
+# Extract the generated response only (remove the prompt part)
+response_start = generated_text.find("### Response:") + len("### Response:\n")
+response = generated_text[response_start:].strip()
+
+# Print the response (excluding the prompt)
+print(response)
+
+
+
+```
+and the answer is 
+
+```
+SELECT * FROM table1 WHERE anni = 2020
+```
+### Model Features
+
+- **Text Generation**: This model is fine-tuned to generate coherent and contextually accurate text based on the provided input. 
+
+
+### License
+
+This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). You are free to use, modify, and distribute this model, provided that you comply with the license terms.
+
+### Acknowledgments
+
+This model was fine-tuned by **ruslanmv** based on the original work of `unsloth` and the `meta-llama-3.1-8b-bnb-4bit` model.
+
+
+