Major anime producer Bilibili has released its open-source AI animated video generation model, AniSora, promising “one-click creation of video shots across diverse anime styles including series episodes, Chinese original animations, manga adaptations, VTuber content, anime PVs, mad-style parodies(鬼畜动画), and more!” Bilibili is well known for producing hundreds of animated titles, including the ongoing Aharen-san wa Hakarenai (sole producer), To Be Hero X, Your Forma, The Beginning After The End, and other series like:
- Fire Force (Season 1 and 2)
- Link Click
- The Heike Story
The video model uses over 10 million “high-quality clips” from 1 million raw animation videos. Each is paired with descriptive text to guide the video generation. Bilibili s the effectiveness of its model using outputs inspired by styles from Studio Ghibli’s films, Angel’s Egg, She and Her Cat, Finding Nemo, SpongeBob SquarePants, Despicable Me, Tangled, Big Fish & Begonia, The Lion King, and more. The open-source model allows anyone to generate animation across several art styles from text or reference images. Examples of animated comic s were also shown. You can check out some use cases below.
Bilibili’s motivation was to meet the growing demand for animation and the “need for efficient production processes,” echoing the AI stances of companies like Toei Animation and KADOKAWA. “Traditionally, creating high-quality animation has required extensive manual effort for tasks like creating storyboards, generating keyframes, and inbetweening, making the process labor-intensive and time-consuming,” Bilibili says. “Previous efforts to incorporate computer vision techniques have assisted animators in generating in-between frames for animation. However, these methods often show effectiveness only within certain artistic styles, limiting their applicability to the varied demands of modern animations.“
ALSO READ:
KADOKAWA Is ‘Actively Investing’ in AI for Anime Production, CEO Reveals How & Why
AniSora’s release garnered a mix of reactions, with some praising its potential for cutting down the time and people spent on production, and others critical, due to its unauthorized training on animation, which its outputs will compete with in the market.
Bilibili is a major Chinese anime streaming service and animation producer. It s fellow streamer Netflix in funding AI research with animation potential. Netflix released a paper in January in collaboration with its subsidiary Eyeline Studios, as well as Stony Brook University, the University of Maryland, and Stanford University. The model, named Go-with-the-Flow, is similarly capable of some of the above use cases, including video generation from a single frame. Sony’s anime streaming and production company, Crunchyroll, recently ruled out using AI in anime production. Sony holds a 4.98% stake in Bilibili.
Source: AniSora via ArXiv
Featured image © 2001 Hayao Miyazaki/Studio Ghibli, NDDTM
Participate In Discussions