llama3.2-1B-Function-calling

⚠️ Important: This model is still under development and has not been fully fine-tuned. It is not yet suitable for use in production and should be treated as a work-in-progress. The results and performance metrics shared here are preliminary and subject to change.

Model description

This model was trained from scratch on an unknown dataset and is intended for function-calling tasks. As it is still in early stages, further development is required to optimize its performance.

Intended uses & limitations

Currently, this model is not fully trained or optimized for any specific task. It is intended to handle function-calling tasks but should not be used in production until more comprehensive fine-tuning and evaluation are completed.

Training and evaluation data

More information is needed regarding the dataset used for training. The model has not yet been fully evaluated, and additional testing is required to confirm its capabilities.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.3083	0.9997	1687	0.3622
0.202	2.0	3375	0.2844
0.1655	2.9997	5061	0.1491

These results are preliminary, and further training will be necessary to refine the model's performance.

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0