ValueError: Non-consecutive added token '<pad>' found. Should have index 32100 but has index 0 in saved vocabulary.

[celll1]

When running App.py, the following error occurs in the "PixArtAlphaPipeline.from_pretrained".
This error was also reproduced when trying to run "PixArtAlphaPipeline.from_pretrained" by itself.
Could you give me a solution? Thank you.

h:\Pixart_alpha\Pixart\PixArt-alpha
g:\Anaconda3\envs\pixart\lib\site-packages\IPython\core\magics\osm.py:393: UserWarning: using bookmarks requires you to install the pickleshare library.
bkms = self.shell.db.get('bookmarks', {})
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
g:\Anaconda3\envs\pixart\lib\site-packages\diffusers\pipelines\pipeline_utils.py:1691: FutureWarning: You are trying to load the model files of the variant=fp16, but no such modeling files are available.The default model files: {'text_encoder/model.safetensors.index.json', 'text_encoder/model-00002-of-00002.safetensors', 'text_encoder/model-00001-of-00002.safetensors', 'transformer/diffusion_pytorch_model.safetensors', 'vae/diffusion_pytorch_model.safetensors'} will be loaded instead. Make sure to not load from variant=fp16if such variant modeling files are not available. Doing so will lead to an error in v0.24.0 as defaulting to non-variantmodeling files is deprecated.
deprecate("no variant default", "0.24.0", deprecation_message, standard_warn=False)

Loading pipeline components...: 0%| | 0/5 [00:00<?, ?it/s]

Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]

Loading checkpoint shards: 50%|█████ | 1/2 [00:01<00:01, 1.08s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:02<00:00, 1.04s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:02<00:00, 1.05s/it]

Loading pipeline components...: 20%|██ | 1/5 [00:02<00:08, 2.23s/it]
Loading pipeline components...: 40%|████ | 2/5 [00:02<00:03, 1.07s/it]
Loading pipeline components...: 60%|██████ | 3/5 [00:03<00:02, 1.25s/it]
Loading pipeline components...: 60%|██████ | 3/5 [00:04<00:02, 1.34s/it]
Traceback (most recent call last):
File "h:\Pixart_alpha\Pixart\PixArt-alpha\scripts\app.py", line 98, in
pipe = PixArtAlphaPipeline.from_pretrained(
File "g:\Anaconda3\envs\pixart\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1249, in from_pretrained
loaded_sub_model = load_sub_model(
File "g:\Anaconda3\envs\pixart\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 502, in load_sub_model
loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
File "g:\Anaconda3\envs\pixart\lib\site-packages\transformers\tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "g:\Anaconda3\envs\pixart\lib\site-packages\transformers\tokenization_utils_base.py", line 2044, in _from_pretrained
raise ValueError(
ValueError: Non-consecutive added token '' found. Should have index 32100 but has index 0 in saved vocabulary.

OS: windows 11 64bit
conda list (summary of an excerpt) :
accelerate 0.17.0 pypi_0 pypi
diffusers 0.22.1 pypi_0 pypi
gradio 3.38.0 pypi_0 pypi
optimum 1.14.0 pypi_0 pypi
pillow 9.3.0 pypi_0 pypi
sentencepiece 0.1.99 pypi_0 pypi
torch 2.0.0+cu117 pypi_0 pypi
transformers 4.30.0 pypi_0 pypi

[lawrence-cj]

h:\Pixart_alpha\Pixart\PixArt-alpha
g:\Anaconda3\envs\pixart\lib\site-packages\IPython\core\magics\osm.py:393: UserWarning: using bookmarks requires you to install the pickleshare library.
bkms = self.shell.db.get('bookmarks', {})
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'

Maybe try pip install triton.

[yujincheng08]

upgrade transformer.

[celll1]

Thanks for your great work.
I just upgraded transformer and accelerate and now this code works.
I wondered why the code worked without installing triton.
However, I had a Gradio error occurred at startup. I will post an entry at a later.