This video provides a practical exploration of Alibaba's newly released open-source video generation model, Huan 2.2, highlighting its improvements over previous versions. The presenter details architectural enhancements, including a Mixture of Experts (MoE) design in the larger 14-billion parameter variant, and notes significant increases in training data. The video focuses on testing the 5-billion parameter model, accessible to consumer GPUs (24GB VRAM), demonstrating its text-to-video and image-to-video capabilities. The presenter also shares personal installation challenges and their workarounds. While the model shows improvements in general quality and motion, limitations such as "horroresque" facial features, lack of prompt adherence (e.g., panning shots), and slow generation times on consumer hardware are observed. The potential for more efficient performance via Comfy UI's native offloading is also mentioned.
Huan 2.2: A Deep Dive into Alibaba's New Open-Source Video Generation Model
0:03 Huan 2.2 Release: Alibaba has launched Huan 2.2, an improved open-source video generation model, building on the previous Huan 2.1.
0:31 Comfy UI Workflow: The model's GitHub repository includes a Comfy UI workflow, notably allowing the 5-billion parameter version to run on 8GB of VRAM with native offloading, a significant reduction in VRAM utilization compared to command-line execution.
1:20 Enhanced Training Data: Huan 2.2 benefits from substantially improved training data, with approximately 65% more images and 83% more videos, leading to better model generalization.
1:46 Mixture of Experts (MoE) Architecture: The 14-billion parameter version of Huan 2.2 now utilizes a Mixture of Experts (MoE) architecture, boasting 27 billion total parameters but only 14 billion active per step, maintaining similar inference computation and GPU memory to the non-MoE 14B model of Huan 2.1. This MoE design features a "high noise expert" for early stages (layout) and a "low noise expert" for later stages (detail refinement).
3:22 Focus on 5B Parameter Model: The video's testing is concentrated on the 5-billion parameter version, which is compatible with consumer GPUs (e.g., 24GB VRAM cards like the 3090 Ti).
3:34 Installation Workarounds: The presenter encountered two main installation hiccups:
Flash Attention Failure: Resolved by installing Flash Attention manually after other packages listed in requirements.txt.
Out of Memory Error: Countered by exporting the PyTorch flag PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:512 to optimize VRAM allocation.
5:01 Text-to-Video: Anthropomorphic Cats: Testing the default sample prompt ("two anthropomorphic cats...fighting intensely"), Huan 2.2 produced higher quality and more intricate details (e.g., eyes) compared to its predecessor, Huan 2.1.
6:58 Text-to-Video: Recycled Computers Fighting: A custom prompt ("old computers being recycled...folks doing recycling are fighting") resulted in a 5-second, 720p, 24fps video. While the background was interesting and relevant, facial features were consistently "horroresque."
8:08 Text-to-Video: Panning Aerial Shot (Alien Beach): An intricate prompt requesting a "wide cinematic panning aerial shot of a crowded beach in an alien world" yielded a somewhat static video that lacked the requested panning motion. However, elements like water movement, shadows, and overall scene composition were deemed acceptable.
10:03 Text-to-Video: Gym POV: The prompt "POV of a person working out at a gym" produced a reasonable result with proper gym elements, good body and hair movement, but surprisingly lacked a visible "sled" despite the implied action.
11:53 Image-to-Video: Robot Handshake: Converting a static image of a person shaking hands with a robot into video yielded impressive robot movement, accurately animating its joints. The camera motion mimicked an amateur filming, though a ring unexpectedly appeared on the person's finger.
13:32 Overall Performance & Limitations: Each video generation using the 5-billion parameter model took approximately 13.5 minutes on a single 3090 Ti, which is considered relatively slow. The model showed a consistent struggle with generating accurate human facial features, often making them look "terrifying." The more powerful 14-billion parameter MoE variant was not tested due to its demanding hardware requirements.
16:00 Conclusion: The presenter concludes that having open-source video models like Huan 2.2 is valuable, especially given the high cost of state-of-the-art closed-source alternatives, and offers an enjoyable platform for experimentation.