An illustration of the Meta AI logo captured on May 20, 2024. Reuters
Meta, the parent company of Facebook, recently introduced a groundbreaking AI model called Movie Gen. This new technology has the capability to generate realistic video and audio clips based on user prompts, positioning itself as a competitor to existing tools from companies like OpenAI and ElevenLabs.
In a demonstration, Meta showcased samples of what Movie Gen can create. These included videos of animals swimming and surfing, as well as clips using real photos of individuals, illustrating them engaged in activities such as painting. One of the model's notable features is its ability to generate background music and sound effects that sync seamlessly with the visuals in the videos. For example, in one video, Movie Gen inserted pom-poms into the hands of a man running alone in a desert. In another, it transformed a dry parking lot into a water-covered surface while a man skateboarded through it.
The technology can produce videos lasting up to 16 seconds and audio clips of up to 45 seconds. In blind tests, Meta’s model showed promising results when compared to similar products from other startups, including Runway, OpenAI, ElevenLabs, and Kling. These findings highlight Movie Gen's potential in the competitive landscape of AI-driven content creation.
This announcement comes at a time when Hollywood is grappling with how to effectively integrate generative AI video technology into the filmmaking process. Earlier this year, OpenAI unveiled Sora, a product capable of creating feature film-like videos in response to text prompts. This has sparked excitement among technologists in the entertainment sector, who see the potential for such tools to streamline and enhance the filmmaking experience. However, there are also concerns about the ethical implications of using AI systems that may have been trained on copyrighted materials without proper permission.
Additionally, lawmakers are increasingly worried about the use of AI-generated fakes, or deepfakes, particularly in elections worldwide, including in countries like the U.S., Pakistan, India, and Indonesia. These concerns underscore the need for responsible use and regulation of such powerful technologies.
Despite the capabilities of Movie Gen, Meta representatives indicated that the company is unlikely to make this tool available for developers to use openly, unlike its Llama series of large-language models. Instead, Meta plans to work directly with the entertainment industry and content creators to explore practical applications of Movie Gen. They also announced that the tool will be incorporated into Meta's products sometime next year.
According to Meta’s blog post and accompanying research paper, the company utilized a combination of licensed and publicly available datasets to develop Movie Gen. Meanwhile, OpenAI has been in discussions with Hollywood executives and agents about potential partnerships involving its Sora technology. However, no formal agreements have been made as of yet. The apprehension around these discussions grew in May when actress Scarlett Johansson accused OpenAI of imitating her voice without her consent for its chatbot.
In another move within the entertainment sector, Lions Gate Entertainment, known for films like "The Hunger Games" and "Twilight," announced that it would provide AI startup Runway access to its film and television library. This collaboration aims to train an AI model, allowing filmmakers to utilize it in their creative processes.
Meta's introduction of Movie Gen signifies a significant step forward in the AI content creation space. As the technology develops, it could reshape how videos and audio are produced, sparking both excitement and caution in the entertainment industry.
