Hook
A maps company just open-sourced one of the most interesting video models of the year. It does not just make the picture. It makes the sound at the same time, in the same network, in sync. And it hands you the weights under Apache 2.0.
The Story
The model is DreamX-Creator 1.0. It comes from the DreamX Team at AMap — yes, Alibaba’s navigation and maps division (AMAP-ML on GitHub). The paper landed on arXiv on August 31, and the weights and inference code dropped on September 3.
Here is what makes it stand out. Most AI video tools make a silent clip first. Then you bolt sound on afterward — a separate model, a separate step, and lip-sync that never quite lands. DreamX-Creator is native audio-video. You give it a first frame and a text prompt. It denoises the video and the audio together, so the footsteps hit when the foot hits, the voice matches the mouth, and the room tone belongs to the room.
The engine is a 7B joint generator — a video DiT and an audio DiT running side by side. For the first half of the network the two streams stay separate, each an expert in its own medium. Then they meet through a trick the team calls Gated Cross-Modal Attention: small learned gates decide, token by token and head by head, how much the sound should listen to the picture and the other way around. It is a clean way to let two very different signals share information without drowning each other out.
The base model tops out at a working resolution, so a second model does the polish: an Autoregressive 1-Step 2K Refiner (a 5B SR-DiT). It takes a bidirectional, multi-step teacher and distills it into a student that upscales to a crisp 2K in a single step. Fast, and sharp. The whole thing is built on top of the open Wan2.2 foundations — shared text encoder, video VAE and tokenizer — with a dedicated audio VAE (CreatorDACVAE) for the sound.
To get the sync tight, the team trains with reinforcement learning that scores three things at once: is the audio good, is the video good, and do they actually agree? That last reward — the cross-modal one — is the hard part, and it is where most stitched-together pipelines fall apart.
Why You Should Care
Sound is the half of video that AI kept skipping. A silent generated clip is a mood board. A clip that arrives with its own synced audio is a shot. For anyone building trailers, animatics, game cutscenes or short films, native A/V removes an entire painful stage of the pipeline.
And then there is the license. Apache 2.0 is about as permissive as it gets — commercial use, no MAU cliff, no regional carve-outs. That is a real gift next to the more restrictive community licenses on some rival 3D and video models. Because it sits on Wan2.2, it should also feel familiar to anyone already living in that ecosystem, and it is a natural fit for a ComfyUI graph.
The bigger picture is who shipped it. AMap is a maps company. It also has DreamX-World, an interactive world model, in the same family. A navigation team quietly turning into a serious open creative-AI lab tells you how wide the spatial-intelligence race has become.
Try It / Follow Them
The code and weights are live now. The main generator is single-GPU capable, with CPU-offload options for tighter VRAM.
Repo: github.com/AMAP-ML/DreamX-Creator
Weights: huggingface.co/GD-ML/DreamX-Creator
Paper: arXiv 2608.31106
The team: AMAP-ML on GitHub
Two modules, two folders. Run audio_video_generation for the joint clip, then feed the output to video_refiner for the 2K pass. Each has its own requirements file and README.
IK3D Lab Take
This is the release we get quietly excited about. No launch hype, no waitlist — just a strong idea (generate sound and picture as one thing) shipped as open weights under the friendliest license around. The Gated Cross-Modal Attention design is elegant, and building on Wan2.2 means it plugs straight into workflows people already run at home.
The honest caveat: the paper reports competitive results but no hard benchmark tables yet, and native ComfyUI nodes aren’t confirmed — for now it is inference scripts, not a one-click graph. But the trajectory is clear. The silent-clip era is ending, and the model that ends it came from the people who map the roads. We’ll be running it.
