Qwen2-1.5B-stepbasin-books
this was finetuned at 16384 context length
This is an experiment on long context text generation (i.e. 6k+ tokens generated) to evaluate if/when generation breaks down, etc. As such, all the data on which this model has been fine-tuned are full-length books.
Details
This model is a fine-tuned version of Qwen/Qwen2-1.5B on https://github.com/stepbasin/books/tree/master/books
It achieves the following results on the evaluation set:
- Loss: 2.8110
- Accuracy: 0.4298
- Num Input Tokens Seen: 44040192
- Downloads last month
- 16
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for BEE-spoke-data/Qwen2-1.5B-stepbasin-books
Base model
Qwen/Qwen2-1.5B