【dreambooth】RuntimeError: mat1 and mat2 must have the same dtype, but got Long and Half

[pipipiapia]

Follow the tutorial but get this erro below ( the 't_freq' dtype is int64), want to know how to fix it. Thanks!
Traceback (most recent call last):
File "/home/tanyang/PixArt-alpha/train_scripts/train_dreambooth.py", line 312, in
train()
File "/home/tanyang/PixArt-alpha/train_scripts/train_dreambooth.py", line 84, in train
loss_term = train_diffusion.training_losses(model, clean_images, timesteps, model_kwargs=dict(y=y, mask=y_mask, data_info=data_info))
File "/home/tanyang/PixArt-alpha/diffusion/model/respace.py", line 97, in training_losses
return super().training_losses(self._wrap_model(model), *args, **kwargs)
File "/home/tanyang/PixArt-alpha/diffusion/model/gaussian_diffusion.py", line 759, in training_losses
model_output = model(x_t, t, **model_kwargs)
File "/home/tanyang/PixArt-alpha/diffusion/model/respace.py", line 134, in call
return self.model(x, timestep=new_ts, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", line 1519, in forward
else self._run_ddp_forward(*inputs, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", line 1355, in _run_ddp_forward
return self.module(*inputs, **kwargs) # type: ignore[index]
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/accelerate/utils/operations.py", line 687, in forward
return model_forward(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/accelerate/utils/operations.py", line 675, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/amp/autocast_mode.py", line 16, in decorate_autocast
return func(*args, **kwargs)
File "/home/tanyang/PixArt-alpha/diffusion/model/nets/PixArtMS.py", line 166, in forward
t = self.t_embedder(timestep) # (N, D)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/tanyang/PixArt-alpha/diffusion/model/nets/PixArt_blocks.py", line 272, in forward
t_emb = self.mlp(t_freq)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/container.py", line 215, in forward
input = module(input)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/tanyang/miniconda3/envs/pixart/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 must have the same dtype, but got Long and Half

[pipipiapia]

replacing
s_freq = self.timestep_embedding(s, self.frequency_embedding_size).to(s.dtype)
with
s_freq = self.timestep_embedding(s, self.frequency_embedding_size).to(torch.float16)
works

I don't know whether it is the best way.

[lawrence-cj]

Bug fixed