VAE: The Unexpected Origin of Image Generation AI — and the Future of Physical AI

Introduction

Hello, this is Easygoing.

In this article, I’d like to explore the VAE, the foundational technology behind today’s image generation AI—and one that is also set to become a key building block supporting Japan’s future manufacturing.

2013: The Arrival of the VAE

The history of image generation begins with the VAE, which appeared in December 2013.

VAE (Variational Autoencoder)

gantt
    title VAE to Physical AI Roadmap
    dateFormat YYYY-MM-DD
    axisFormat %Y

	section VAE
        VAE : 2013-12-20, 2026-09-25

	section GAN
        GAN : 2014-06-10, 2026-09-25

    section Diffusion Model
        Diffusion Model : 2015-03-12, 2026-09-25
        Diffusion Model (DDPM) : 2020-06-19, 2026-09-25
        Diffusion Model (SDE) : 2020-11-26, 2026-09-25

    section Image Generation AI<br>(VAE + Diffusion Model)
        Stable Diffusion 1 : 2022-08-22, 2026-09-25
        Stable Diffusion XL : 2023-07-26, 2026-09-25
        Flux.1 : 2024-08-01, 2026-09-25

    section Video Generation AI
        Wan-2.1 : 2025-02-25, 2026-09-25
        Cosmos-Predict2 : 2025-04-30, 2026-09-25

    section Physical AI
		Cosmos3 : 2026-05-31, 2026-09-25

Introductory AI learning video by 3Blue1Brown Japan

A VAE captures the features of an input image and arranges them in an orderly fashion on a map known as the latent space.

For example, the following diagram shows a VAE trained to recognize digits. When various digit features are captured and arranged, you can see that the same digits cluster together in nearby groups.

• Features of the input digits are used to arrange them on the central map

The VAE Becomes the Prototype of Generative AI

Once a VAE has been sufficiently trained, selecting a single point in the latent space that does not exist in the training data allows it to output a new digit.

The newly generated digit inherits features from multiple learned digits, yet it is not a direct copy of any original.

In this way, AI that creates new things by referencing learned data is called generative AI, and the VAE has become the prototype of today’s image generation AI.

VAE → GAN → Diffusion Models

While the VAE generates new images based on learned data, its outputs inherit features from multiple images, resulting in a tendency to be unclear and blurry.

Image generation AI subsequently evolved into GANs and diffusion models.

gantt
    title VAE to Physical AI Roadmap
    dateFormat YYYY-MM-DD
    axisFormat %Y

	section VAE
        VAE : 2013-12-20, 2026-09-25

	section GAN
        GAN : 2014-06-10, 2026-09-25

    section Diffusion Model
        Diffusion Model : 2015-03-12, 2026-09-25
        Diffusion Model (DDPM) : 2020-06-19, 2026-09-25
        Diffusion Model (SDE) : 2020-11-26, 2026-09-25

    section Image Generation AI<br>(VAE + Diffusion Model)
        Stable Diffusion 1 : 2022-08-22, 2026-09-25
        Stable Diffusion XL : 2023-07-26, 2026-09-25
        Flux.1 : 2024-08-01, 2026-09-25

    section Video Generation AI
        Wan-2.1 : 2025-02-25, 2026-09-25
        Cosmos-Predict2 : 2025-04-30, 2026-09-25

    section Physical AI
		Cosmos3 : 2026-05-31, 2026-09-25

GAN (Generative Adversarial Networks)

• Uses a separate AI model during training to point out mistakes
• Limited output variety
• Fast inference

Diffusion Model

• Learns the process of gradually adding noise
• Rich output variety
• Heavy inference cost

GANs can generate higher-quality images than VAEs and are still widely used in many AI upscaling models today.

About AI upscaling models that use GANs

Meanwhile, diffusion models produce images with rich variety and high quality. Combined with algorithmic advances that improved speed, they became the mainstream of image generation AI.

Stable Diffusion Changed the World!

In August 2022, Stability AI released a new image generation AI called Stable Diffusion as open source, sending shockwaves around the world.

Stable Diffusion specialized the VAE for compression and reconstruction, inventing a new approach that runs the diffusion model in the compressed space (latent space).

The experience of running image generation—previously requiring dedicated AI workstations—on a personal laptop stunned the world. From that point on, image generation AI developed at a rapid pace.

VAEs Continue to Evolve

After the arrival of Stable Diffusion, many image generation AIs were released, and VAEs also advanced rapidly.

Among the major breakthroughs, two stand out in particular: Flux.1_vae and Wan-2.1_vae.

gantt
    title VAE to Physical AI Roadmap
    dateFormat YYYY-MM-DD
    axisFormat %Y

	section VAE
        VAE : 2013-12-20, 2026-09-25

	section GAN
        GAN : 2014-06-10, 2026-09-25

    section Diffusion Model
        Diffusion Model : 2015-03-12, 2026-09-25
        Diffusion Model (DDPM) : 2020-06-19, 2026-09-25
        Diffusion Model (SDE) : 2020-11-26, 2026-09-25

    section Image Generation AI<br>(VAE + Diffusion Model)
        Stable Diffusion 1 : 2022-08-22, 2026-09-25
        Stable Diffusion XL : 2023-07-26, 2026-09-25
        Flux.1 : 2024-08-01, 2026-09-25

    section Video Generation AI
        Wan-2.1 : 2025-02-25, 2026-09-25
        Cosmos-Predict2 : 2025-04-30, 2026-09-25

    section Physical AI
		Cosmos3 : 2026-05-31, 2026-09-25

VAE Comparison

• VAEs compress information by a factor of 48

VAE Reconstruction Quality (Author’s Measurements)

• Flux.1_vae and Wan-2.1_vae offer excellent reconstruction quality

Flux.1_vae

• Increased latent space channels from 4 → 16
• Significant improvement in image quality (reconstruction quality)
• Apache-2.0 open license

Wan-2.1_vae

• Reconstruction quality surpassing Flux.1_vae
• Added temporal compression
• Enabled video generation AI

About Flux.1_vae

Released in August 2024 by Germany’s Black Forest Labs, Flux.1_vae increased the number of latent space channels from 4 to 16, greatly improving image quality, and was released under the Apache-2.0 open license, allowing subsequent companies to freely use the technology.

Then, in February 2025, China’s Alibaba released Wan-2.1_vae, which offered even higher reconstruction quality than Flux.1_vae and added temporal compression, becoming a driving force in the evolution of image generation AI into video generation AI.

Physical AI Demands Reliability

When AI is used for industrial applications such as autonomous driving or factory robot control, high reliability and processing speed are required.

AI used in such industrial applications is called Physical AI. Training Physical AI requires AI that understands the laws of physics and repeated simulations in virtual space.

Video Generation AI Understands the Laws of Physics!

Teaching AI human common sense, including the laws of physics, was long considered extremely difficult.

Yet this problem was solved from an unexpected direction.

Feathers Fall More Slowly Than Iron Balls

When a video generation AI creates a video of an iron ball and a feather being dropped simultaneously, the feather falls more slowly than the iron ball due to air resistance.

In a Vacuum, Feathers and Iron Balls Fall at the Same Speed

However, when the same video is generated in the vacuum of the lunar surface, the feather and iron ball fall at the same speed.

Through learning vast amounts of video data, video generation AI naturally acquired the physical law that, in a vacuum with no air resistance, objects fall at the same speed.

NVIDIA Provides the World Foundation Model

The fact that video generation AI understands the laws of physics caught the strong attention of NVIDIA, the world’s largest GPU manufacturer.

NVIDIA named the foundational models for Physical AI optimized for its own GPU environment the World Foundation Model, and released the Cosmos-Predict1 model in January 2025.

Then, in April 2025, it released the Cosmos-Predict2 model, which incorporates the video generation AI Wan-2.1_vae.

Cosmos-Predict2 Model Architecture

By incorporating Wan-2.1_vae, Cosmos-Predict2 achieved a significant improvement in processing efficiency. This became a symbolic event demonstrating that NVIDIA is absorbing excellent open-source technology and rebuilding the World Foundation Model on a video generation AI foundation.

• Video generation AI → World Foundation Model → Physical AI

Cosmos3 Is an Omnimodal Model

The World Foundation Model continued to evolve, and the latest Cosmos3 model was released in June 2026.

Introduction video for NVIDIA’s World Foundation Model (Cosmos)

Cosmos3 Model Architecture

The Cosmos3 model integrates the text encoder and Transformer into a Qwen3-VL-based Cosmos3 Reasoner & Generator, and uses Wan-2.2_vae for image and video output.

Cosmos3 advances beyond previous video generation AI to become an omnimodal model that accepts various inputs and produces outputs in any desired format.

Cosmos3 Model Inputs and Outputs

flowchart LR

A1(Image)
A2(Video)
A3(Text)
A4(Audio)
A5(Action)

subgraph MoT architecture

B1("Cosmos3<br>Reasoner & Generator<br>(Custom extension of Qwen3-VL)")
B2(Wan-2.2_vae)

end

C1(Image)
C2(Video)
C3(Text)
C4(Audio)
C5(Action)

B1-->B2

A1-->B1
A2-->B1
A3-->B1
A4-->B1
A5-->B1

B2-->C1
B2-->C2
B1--->C3
B1--->C4
B1--->C5

• An omnimodal model accepts various inputs and produces outputs in any format

The Cosmos3 Reasoner & Generator and VAE adopt a MoT (Mixture of Transformers) architecture in which only the necessary parts activate depending on the input, achieving the response speed required for Physical AI.

Cosmos3 Features a New-Era Open License

As a World Foundation Model, Cosmos3 is the first to adopt the new-era open license OpenMDW-1.1.

OpenMDW-1.1 (Open Model, Data & Weights License Agreement version 1.1)

• The model can be freely used, modified, and redistributed
• If a patent or copyright lawsuit is filed, the license terminates
• Outputs can be freely used

The OpenMDW-1.1 license is essentially an extension of the previous open-license standard Apache-2.0, adapted for the AI model era. It was developed by the Linux Foundation, a highly trusted authority in open source, and is expected to become the standard open license for next-generation AI models.

The author had been skeptical of NVIDIA’s open-source strategy at the time of Cosmos-Predict2. However, with the adoption of OpenMDW-1.1 in Cosmos3, the author now believes it is safe to base Japan’s Physical AI infrastructure on NVIDIA’s World Foundation Model.

About the dual licensing of the Anima model and NVIDIA’s license restrictions regarding Cosmos-Predict2

Competing Models to the World Foundation Model

Finally, let’s consider models that could compete with NVIDIA’s World Foundation Model as a foundation for Physical AI.

World Foundation Model and Competing Models

Wan-2.2 Model

Wan-2.2 is the model that formed the basis of NVIDIA’s Cosmos series and can be considered the origin of open-source video generation AI.

However, starting with the successor Wan-2.3 and beyond, the Wan series has shifted to a closed strategy—including the VAE—so future community development is expected to be ceded to NVIDIA.

MiniMax-H3 Model

MiniMax-H3 is an open-weight video generation AI released in August 2026 that achieves performance far surpassing previous models, particularly in anime video generation.

However, the MiniMax-H3 Community License assigned to the model has a unique restriction: licenses are not granted to companies or organizations in the United States, Europe, the United Kingdom, or South Korea. Even for Japanese companies, industrial use is considered difficult if future overseas expansion is taken into account.

Noetra

Noetra is a government-led domestic AI development project launched in July 2026, involving 44 major Japanese companies.

Noetra Domestic AI Foundation Model Roadmap

Noetra plans to introduce approximately 27,500 of the latest NVIDIA Rubin GPUs and aims to release a domestic AI model that will serve as the foundation for Physical AI every year starting from fiscal year 2026.

However, there are many challenges facing domestic AI: whether a project involving as many as 44 companies can coordinate effectively, and whether a culture of Japanese companies openly releasing their own technology as libraries will take root.

Still, considering the timing—Cosmos3, which previously used a proprietary license, was open-licensed just before the Japanese government partnered with NVIDIA—maintaining domestic technology even at significant cost may not be wasted as a bargaining chip in the AI era.

UiT Architecture

There is also the view that once AI performance improvements and efficiency advance sufficiently, it may become possible to omit the VAE entirely and handle all tasks with a single model.

The image is of an ultra-fast ChatGPT controlling both car driving and factory robots.

UiT (Unified Transformer) Architecture

flowchart LR

A1(Image)
A2(Video)
A3(Text)
A4(Audio)
A5(Action)

subgraph UiT architecture

B1(Unified Transformer)

end

C1(Image)
C2(Video)
C3(Text)
C4(Audio)
C5(Action)

A1-->B1
A2-->B1
A3-->B1
A4-->B1
A5-->B1

B1-->C1
B1-->C2
B1-->C3
B1-->C4
B1-->C5

In a previous article, I introduced an image generation AI model based on the UiT architecture. Similar technology may expand to video and audio, and its introduction into Physical AI may not be far off.

About image generation AI using the UiT architecture

Summary: It All Started with the VAE

• The VAE compresses the world
• Wan-vae has become the de facto standard
• Physical AI ≈ World Foundation Model ≈ Video Generation AI

In this article, we looked back from the birth of the VAE to Physical AI.

Since the arrival of Stable Diffusion, the author has been deeply absorbed in image generation. Now that this technology is actually being applied to industrial AI, it feels deeply moving.

TBS CROSS DIG special on NVIDIA and Physical AI

The Japanese government has announced a policy of fully investing in Physical AI, and in July 2026 NVIDIA CEO Jensen Huang visited Japan, making headlines by visiting a familiar SEGA game center and holding an izakaya gathering with semiconductor industry veterans.

Entrusting Japan’s future to NVIDIA’s single ecosystem is a high-stakes gamble. Yet, for Japan’s survival going forward, it can already be said that the die has been cast.

In an era when concerns are growing that engineers will lose their jobs due to AI advancement, looking at the direction Japan’s future is heading, new job openings may emerge in the fields of video generation and Physical AI over the next few years.

Thank you for reading to the end!