OpenAI’s Sora evaluation: Marques Brownlee breaks down the AI video mannequin

Some of the highly-anticipated AI-related merchandise has simply arrived: OpenAI’s AI video generator Sora launched on Monday as a part of the corporate’s 12 Days of OpenAI occasion.

OpenAI has supplied sneak peeks at Sora’s output up to now. However, how completely different is it at launch? OpenAI has definitely been laborious at work to replace and enhance its AI video generator in preparation for its public launch.

YouTuber Marques Brownlee had a first take a look at Sora, releasing his video evaluation of the newest OpenAI product hours earlier than OpenAI even formally introduced the launch. What did Brownlee suppose?

What Sora is sweet at

In line with Brownlee, his Sora testing discovered that the AI video generator excels at creating landscapes. AI generated overhead, drone-like photographs of nature or well-known landscapes look identical to real-life inventory footage. After all, as Brownlee factors out, in case you are particularly well-versed in how the environment of a landmark look, one may be capable to spot the variations. Nonetheless, there’s not an excessive amount of that appears distinctly AI-generated in some of these Sora-created clips.

Maybe the kind of video Sora is greatest capable of create, in keeping with Brownlee, are summary movies. Background or screensaver sort summary artwork could be made fairly nicely by Sora even with particular directions.

Brownlee additionally discovered that Sora-generated sure kinds of animated content material, like stop-motion or claymation sort animation, look satisfactory at occasions because the generally jerky actions that also plague AI video appear like stylistic selections.

Most surprisingly, Brownlee discovered that Sora was capable of deal with very particular animated textual content visuals. Phrases usually present up as garbled textual content in different AI picture and video era fashions. With Sora, Brownlee discovered that so long as the textual content was particular, say just a few phrases on title card, Sora was capable of generate the visible with appropriate spelling.

The place Sora goes improper

Sora, nonetheless, nonetheless presents lots of the identical issues that every one AI video mills that got here earlier than it have struggled with.

The very first thing Brownlee mentions is object permanence. Sora has points with displaying, say, a selected object in a person’s hand all through the runtime of the video. Generally the thing will transfer or simply all of a sudden disappear. Similar to with AI textual content, Sora’s AI video suffers from hallucinations. 

Which brings Brownlee to Sora’s largest downside: Physics on the whole. Photorealistic video appears to be fairly difficult for Sora as a result of it could’t simply appear to get motion down proper. An individual merely strolling will begin slowing down or dashing up in unnatural methods. Physique elements or objects will all of a sudden warp into one thing fully completely different at occasions as nicely.

And, whereas Brownlee did point out these enhancements with textual content, until you might be getting very particular, Sora nonetheless garbles the spelling of any kind of background textual content such as you may see on buildings or avenue indicators.

Sora may be very a lot an ongoing work, as OpenAI shared through the launch. Whereas it could provide a step up from different AI video mills, it is clear that there are just a few areas the place all AI video fashions are going to search out difficult.