README.md

metadata

base_model: unsloth/gemma-1.1-2b-it-bnb-4bit
language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - gemma
  - trl
  - sft

functions_metadata = [
    {
      "type": "function",
      "function": {
        "name": "get_temperature",
        "description": "get temperature of a city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "name"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
]

messages = [
    { "role": "user", "content": f"""Bạn là một trợ lý hữu ích có quyền truy cập vào các chức năng sau. Sử dụng chúng nếu cần -\n{str(functions_metadata)}"""},
    { "role": "user", "content": "What is the temperature in Tokyo right now?"},
    # You will get the previous prediction, extract it will the tag <functioncall>
    # execute the function and append it to the messages like below:
    { "role": "assistant", "content": """<functioncall> {"name": "get_temperature", "arguments": '{"city": "Tokyo"}'} </functioncall>"""},    
    { "role": "user", "content": """<function_response> {"temperature":30 C} </function_response>"""}
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)


outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
# >> The current temperature in Tokyo is 30 degrees Celsius.

Uploaded model

• Developed by: hiieu
• License: apache-2.0
• Finetuned from model : unsloth/gemma-1.1-2b-it-bnb-4bit

This gemma model was trained 2x faster with Unsloth and Huggingface's TRL library.