|
--- |
|
license: other |
|
--- |
|
# Manticore-13b-Landmark-GPTQ |
|
|
|
## Key Features |
|
- **[Landmark Attention](https://arxiv.org/pdf/2305.16300v1.pdf)** |
|
- **[Large Context Size (~18k)](https://i.ibb.co/tLLGLNc/image.jpg)** |
|
- **[4-Bit Quantization](https://arxiv.org/pdf/2210.17323.pdf)** |
|
## Composition |
|
Manticore-13b-Landmark-GPTQ is a blend of: |
|
- [Manticore-13B](https://huggingface.co/openaccess-ai-collective/manticore-13b) |
|
- [Manticore-13B-Landmark-QLoRA](https://huggingface.co/Honkware/Manticore-13b-Landmark-QLoRA) |
|
- [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) |
|
## Using [Oobabooga](https://github.com/oobabooga/text-generation-webui) |
|
- Trust Remote Code - **(Enabled)** |
|
- Add the bos_token to the beginning of prompts - **(Disabled)** |
|
- Truncate the prompt up to this length - **(Increased)** |
|
## Landmark Training Code |
|
See [GitHub](https://github.com/eugenepentland/landmark-attention-qlora) for the training code. |
|
|
