Decoding Without a Decoder: Direct Pixel Prediction from Diffusion Transformers
In Submission
TL;DR: We introduce DWD, which adds a separate linear pixel head to diffusion transformers while keeping the denoising head responsible for sampling. Finetuning only the last few transformer blocks removes the decoder from latent image and video models while maintaining or improving visual quality. The same approach also improves texture detail in pixel diffusion models.