Update README.md

README.md

@@ -5,14 +5,95 @@ datasets:
 - IlyaGusev/ru_turbo_alpaca_evol_instruct
 - IlyaGusev/ru_turbo_alpaca
 - IlyaGusev/ru_turbo_saiga
+- RussianNLP/russian_super_glue 
 language:
 - ru
 pipeline_tag: question-answering
 ---
-The model was trained on part of the datasets IlyaGusev/gazeta , IlyaGusev/ru_turbo_alpaca_evol_instruct, IlyaGusev/ru_turbo_alpaca, IlyaGusev/ru_turbo_saiga using LoRA
+The model was trained on part of the datasets 
 
-##Base_model NousResearch/Yarn-Llama-2-7b-64k
+*IlyaGusev/gazeta* , 
 
-Need cuda > 11.4 
+*IlyaGusev/ru_turbo_alpaca_evol_instruct*, 
+
+*IlyaGusev/ru_turbo_alpaca*, 
+
+*IlyaGusev/ru_turbo_saiga* , 
+
+*RussianNLP/russian_super_glue (muserc)* 
+
+using LoRA
+
+#### Base_model NousResearch/Yarn-Llama-2-7b-64k
+
+#### Need cuda > 11.4 
+
+### GPU A100
+
+
+
+
+```python
+  
+!pip install peft
+!pip install flash-attn --no-build-isolation
+!pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
+
+```
+```python
+model = AutoModelForCausalLM.from_pretrained(
+            'geldarr/saiga-Yarn-Llama-2-7b-64k',
+            trust_remote_code=True,
+            torch_dtype=torch.float16,
+            device_map={'':0}
+        )
+tokenizer = AutoTokenizer.from_pretrained('geldarr/saiga-Yarn-Llama-2-7b-64k', use_fast=False)
+
+```
+
+```python
+big_prompts = '''<s>system\nТы — Сайга, русскоязычный автоматический ассистент. Ты разговариваешь с людьми и помогаешь им.</s>\n
+<s>user
+Дай ответы на вопрос основываясь только на тексте ниже:\n
+вопрос?
+
+Текст <65536 tokens
+
+</s>
+<s>bot
+'''
+```python
+gen_config = {
+        "pad_token_id": 0,
+        "bos_token_id": 1,
+        "eos_token_id": 2,
+        "temperature": 0.4,
+        "top_p": 0.9,
+        "top_k": 50,
+        "do_sample": True,
+        "max_new_tokens": 15360,
+        "repetition_penalty": 1.1,
+        "no_repeat_ngram_size": 15,
+    }
+generation_config = GenerationConfig.from_dict(gen_config)
+```
+
+```python
+def generate(model, tokenizer, prompt, generation_config):
+    data = tokenizer(prompt, return_tensors="pt")
+
+    data = {k: v.to(model.device) for k, v in data.items()}
+    output_ids = model.generate(
+        **data,
+        generation_config=generation_config
+    )[0]
+    output_ids = output_ids[len(data["input_ids"][0]):]
+    output = tokenizer.decode(output_ids)
+    return output.strip()
+
+output = generate(model, tokenizer, big_prompts, generation_config)
+
+print(output)
+
+```
 
-GPU A100