Products You May Like
OpenAI showed the power of generative AI and launched a new computational arms race with its image and text-generating bots. Now, it’s going after another vestige of human creativity: video. The company has announced its newest AI model, known as Sora. You won’t be able to try it just yet, but OpenAI has posted a large batch of sample videos, and they’re extremely impressive. In some cases, you’d never guess these were entirely created by AI.
Sora is a diffusion model like several popular image generators. It starts with a video consisting of random noise and then modifies repeatedly to remove the noise. At the end of that process, Sora spits out a video based on the user’s text prompt. Sora builds on the recaptioning technique that made DALL-E 3 so capable. OpenAI generates descriptions for training data, which allows the model to follow user prompts with much greater accuracy.
OpenAI has posted over three dozen sample videos, most almost indistinguishable from real video footage or human-created 3D animations. The model understands different visual styles, so you can tell it to create a Pixar-style render or a cinematic clip that looks like it was shot on 35mm film. Sora can create entirely new videos from scratch or extend an existing video clip the way DALL-E can use “outpainting” to expand a static image. The model can even generate multiple videos featuring consistent settings or subjects.
Most of the sample videos show the best of Sora, but OpenAI did include a few mistakes to show the model’s limitations. The company says that people or animals can “spontaneously appear” in scenes with large numbers of objects. The model also gets a bit confused about how physics works, rendering impossible motions that give the footage away as AI-generated.
Still, most of what OpenAI has shown is incredible. It’s so incredible that you’re right to worry about misuse. The company says it’s planning to build in all its usual guardrails to block celebrity likenesses, sexual content, hateful images, and so on. It will also create tools to help identify Sora-generated videos. OpenAI plans to test Sora with a small group of trusted experts to ensure it’s safe to deploy widely. It does not give a timeline for opening up access.
AI video generation is not new, but none of the competitors are anywhere near as capable as what we’ve seen of Sora. Even if the company succeeds in locking down Sora to prevent misuse, others will build on this advance to create unrestricted versions. At that point, reality is whatever you type.