Cannot load tokenizer: Exception: data did not match any variant of untagged enum PostProcessorWrapper at line 2395 column 3

#143
by huangleiabcde - opened

Hi, today when I use llama3 on my computer, I find that I can not load tokenizer. The error message is

Exception: data did not match any variant of untagged enum PostProcessorWrapper at line 2395 column 3

Can anyone know how to solve this problem?

Here is the snapshot of my result:

image.png

image.png

I just tried the same code, and it worked well. Please make sure to use an updated transformers version.

Hi, today when I use llama3 on my computer, I find that I can not load tokenizer. The error message is

Exception: data did not match any variant of untagged enum PostProcessorWrapper at line 2395 column 3

Can anyone know how to solve this problem?

Here is the snapshot of my result:

image.png

image.png

@huangleiabcde Hi, I had the same issue, have you solved the problem?

@huangleiabcde same issue here, anyone tried finding the soln

@RodneyWizard @Daiqingq

Hi, I solved this problem by upgrade transformers to the lastest version. Besides, I still have something wrong in the process of Transformers upgrade. So I just uninstall Anaconda and upgrade to the lastest Anaconda. And then install all packages necessary.

Sign up or log in to comment