metadata
license: apache-2.0
base_model: mhenrichsen/danskgpt-tiny
tags:
- generated_from_trainer
DanskGPT-chat-tiny
DanskGPT-chat-tiny er chat-varianten af mhenrichsen/danskgpt-tiny trænet på private chat datasæt.
Model beskrivelse
Modellen er beregnet til at være en lightweight version af DanskGPT, der kan køre på næsten alle enheder.
Prompt template
Modellen er trænet med ChatML format (samme som OpenAI's modeller), og skal bruges på følgende måde:
<|im_start|>system\nDu er en hjælpsom assistent.<|im_end|>\n<|im_start|>user\nHvad er skak?<|im_end|>\n<|im_start|>assistant
Inferens
vLLM
pip install vllm
from vllm import LLM, SamplingParams
sampling_params = SamplingParams(temperature=0.8, top_p=0.95, max_tokens=1024)
llm = LLM(model="mhenrichsen/danskgpt-tiny-chat")
system_message = "Du er en hjælpsom assistent."
conversation_history = f"<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n"
while True:
prompt = input("Bruger: ")
new_prompt = f"{conversation_history}{prompt}<|im_end|>\n<|im_start|>assistant\n"
outputs = llm.generate(new_prompt, sampling_params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(f"AI: {generated_text!r}")
conversation_history = f"{prompt}{generated_text!r}<|im_end|>\n<|im_start|>user\n"
Endpoint med openai
pip install openai
python -m vllm.entrypoints.openai.api_server --model mhenrichsen/danskgpt-tiny-chat
og brugt som erstatning for OpenAI's endpoints:
from openai import OpenAI
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
chat_response = client.chat.completions.create(
model="mhenrichsen/danskgpt-tiny-chat",
messages=[
{"role": "system", "content": "Du er en hjælpsom assistent. Giv mig et langt svar."},
{"role": "user", "content": "Fortæl mig om Danmark."},
]
)
print("AI:", chat_response)
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 4
- total_train_batch_size: 256
- total_eval_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- num_epochs: 3
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
1.3599 | 0.0 | 1 | 1.4118 |
0.7896 | 0.25 | 136 | 0.7813 |
0.7339 | 0.5 | 272 | 0.7490 |
0.7378 | 0.75 | 408 | 0.7285 |
0.7112 | 1.0 | 544 | 0.7146 |
0.6377 | 1.23 | 680 | 0.7135 |
0.6192 | 1.49 | 816 | 0.7133 |
0.5985 | 1.74 | 952 | 0.7073 |
0.6067 | 1.99 | 1088 | 0.7026 |
0.5139 | 2.22 | 1224 | 0.7167 |
0.5099 | 2.47 | 1360 | 0.7193 |
0.5217 | 2.72 | 1496 | 0.7168 |
Framework versions
- Transformers 4.37.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0