Post

Article URL: https://tuna-ai.org/tuna-2/ Comments URL: https://news.ycombinator.com/item?id=47938702 Points: 1 # Comments: 0

Article URL: https://tuna-ai.org/tuna-2/

Comments URL: https://news.ycombinator.com/item?id=47938702

* Joint first authors, listed alphabetically by last name

Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the two tasks and preventing fully end-to-end optimization from raw pixels. We introduce Tuna-2 , a native unified multimodal model that performs visual understanding and generation directly based on pixel embeddings. Tuna-2 drastically simplifies the model architecture by employing simple patch embedding layers to encode visual input, completely discarding the modular vision encoder designs such as the VAE or the representation encoder. Experiments show that Tuna-2 achieves state-of-the-art performance in multimodal benchmarks, demonstrating that unified pixel-space modelling can fully compete with latent-space approaches for high-quality image generation. Moreover, while the encoder-based variant converges faster in early pretraining, Tuna-2's encoder-free design achieves stronger multimodal understanding at scale, particularly on tasks requiring fine-grained visual perception. These results show that pretrained vision encoders are not necessary for multimodal modelling, and end-to-end pixel-space learning offers a scalable path toward stronger visual representations for both generation and perception.

We propose \model, a native unified multimodal model that supports multimodal understanding and generation with encoder-free designs, achieving state-of-the-art performance across a wide range of understanding and generation benchmarks.

With sufficient end-to-end vision pretraining, the encoder-free Tuna-2 consistently outperforms the encoder-based Tuna-R on multimodal understanding, especially on fine-grained, perception-heavy benchmarks—demonstrating that large-scale pixel-level training can fully replace pretrained vision encoders.

Through controlled comparison under the same unified framework, we reveal that scaling up vision pretraining is the key to closing and surpassing the gap between encoder-free and encoder-based designs, offering actionable insights for future native unified multimodal models.

Tuna-2 discards the VAE module and operates the vision-language backbone and flow matching head entirely in pixel space. We adopt the x-prediction and v-loss paradigm for pixel-space flow matching, employing rectified flow and its linear schedule to construct noisy samples directly in pixel space.