---
license: apache-2.0
datasets:
- wikitext
- ptb_text_only
language:
- en
metrics:
- perplexity
pipeline_tag: text-generation
model-index:
- name: distilgpt2
  results:
  - task:
      type: text-generation
    dataset:
      name: penn_treebank
      type: ptb_text_only
    metrics:
    - name: perlexity@BASELINE
      type: dmx-perlexity
      value: 63.45857238769531
    - name: perlexity@FALLBACK
      type: dmx-perlexity
      value: 64.36720275878906
  - task:
      type: text-generation
    dataset:
      name: wikitext2
      type: wikitext-2-raw-v1
    metrics:
    - name: perlexity@BASELINE
      type: dmx-perlexity
      value: 46.05925369262695
    - name: perlexity@FALLBACK
      type: dmx-perlexity
      value: 46.570838928222656
---
This is a d-Matrix functional reference of the GPT2 model family, of the following revisions: 
- [`distilgpt2`](https://huggingface.co/distilbert/distilgpt2)
- [`gpt2`](https://huggingface.co/openai-community/gpt2)
- [`gpt2-medium`](https://huggingface.co/openai-community/gpt2-medium) 
- [`gpt2-large`](https://huggingface.co/openai-community/gpt2-large) 
- [`gpt2-xl`](https://huggingface.co/openai-community/gpt2-xl) (default) 

The reference provides the following functional configurations:
  Configuration | Explanation
  :-- | :-- 
  **`BASELINE`** | a reference functionally equivalent to the original model
  **`BASIC`** | all linear algebraic operations quantized to `BFP16-64`, and all other operations transformed to corresponding functional kernel simulations


### Usage

Prerequisites:
- Install d-Matrix ML Tools: `pip install dmx-mltools`
- clone this repo. `cd` to the cloned repo.

```python
>>> import os
>>> from mltools import dmx
>>> from transformers import pipeline
>>> import evaluate
>>> from datasets import load_dataset

>>> pipe = pipeline(
>>>     "text-generation",
>>>     model="d-matrix/gpt2",
>>>     revision="gpt2-xl", 
>>>     use_auth_token=os.environ.get("HUGGING_FACE_HUB_TOKEN"),
>>>     trust_remote_code=True,
>>>     # device_map="auto",  # enabling model parallel on multi-GPU nodes
>>> )
>>> pipe.model = dmx.Model(
>>>     pipe.model, monkey_patched=False, hf=True, input_names=["input_ids", "labels"]
>>> )

>>> pipe.model.transform("/path/to/BASIC.yaml")

>>> perplexity = evaluate.load("d-matrix/dmx_perplexity", module_type="metric")
>>> input_texts = load_dataset("ptb_text_only", "penn_treebank", split="test")["sentence"]
>>> results = perplexity.compute(model=pipe.model.body, references=input_texts)
>>> print(results)
{'loss': 3.0451672077178955, 'perplexity': 21.0135440826416}
```

### Evaluation results