---
datasets:
- JeanKaddour/minipile
language:
- en
license: mpl-2.0
---

# RWKV Minipile

## Model Specifications

- **Architecture**: RWKV
- **Vocabulary Size**: 65,536
- **Embedding Size**: 768
- **Number of Layers**: 12
- **Context Length**: 512
- **Data Type**: bfloat16
- **Dataset**: Minipile
- **Tokens**: 20,643,840 (20 Million)

The model underwent a rigorous training regimen, completing 30 epochs to optimize performance.


## Inference
```bash
pip install torch numpy

python inference.py
```