Unable to load tokenizer

#2
by FearfulOstrich - opened

Hello,
As of yesterday, we have had issues loading the tokenizer.
Using the classic AutoTokenizer class to load the tokenizer results in an Exception:
Screenshot 2024-06-04 at 12.24.11.png
Is this issue known ?

my environment:

  • python: 3.11.9
  • transformers: 4.39.3

Code for error:

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("almanach/camembert-bio-base")
ALMAnaCH (Inria) org

Hello @FearfulOstrich thank you for reporting this ! It should be fixed now with 86cc3e73fb06ef10690902215dcf9e8518419fa8
Tell me if that doesn't solve the issue

FearfulOstrich changed discussion status to closed

Sign up or log in to comment