How Do We Use AI in Production?

Concept art created by Limehouse through Midjourney

On 28 September 2022, Open AI’s image generator DALL-E 2 was opened to the public. This moment had a number of precedents, with Midjourney’s similar artificial intelligence programme open to the public from July 12 of this same year, and an open source emulation of DALL-E called Dall-E Mini (now renamed Craiyon blowing up social media around the same time).

2022 was, in many ways then, the year of AI. The public was able to engage with the software more readily than ever before; and did so en masse, using it to create everything from Bigfoot performing at Woodstock, to Karl Marx getting slimed at the Kids Choice Awards.

But it was also a year where the creative industry, which had already been engaging with AI for some time, began to ramp up its usage significantly.

A major area where artificial intelligence has found a creative home is in production. Speaking to LBB, Limehouse’s ECD & head of retouching James Lucas was quick to extoll its virtues.

“Every single job we've had recently, we start by putting it through Midjourney to see what it comes out with. Just straight off the bat, what kind of ideas has it got towards colour, composition, lighting etc.”

Limehouse has been using similar AI for concepting for a while now, and James is a real advocate of its role in concepting. 

“If you were mocking up some sort of creative idea, you’re gonna have to assign a real amount of time to it. Midjourney can do it in fifteen seconds. And then you can sift through all the various options it gives you, and just curate. ‘Oh I like this bit, and that bit,’ and steadily you put something brand new together.”

Heath George, founder and executive producer at Clockwork films, takes a similar stance.

“You can effectively do a whole storyboard now in a fairly short amount of time. It's a little hit and miss, granted, but what is a real game changer is the fact that you can now input your own photos. And so, instead of generating random art from the existing input databases, I can have photos of my actors, and give parameters like ‘my actor in this scenario; doing this action; in the style of this artist.” Now I have an amazing piece of pre-visualisation and I get a really good storyboard.”

But, for all that AI may make production easier. Heath worries what it could mean for work within the creative industry.

“A DOP or director could theoretically do that storyboarding process. And then where does that leave the storyboard artist?... 

There is no doubt that, over the last 10 years, as technology has gotten smaller and better, crews have also gotten smaller. Even just the improvements in the cameras have had a knock on effect. You don't need as many grips; you don't need as-big equipment to mount the cameras; you don't need as-powerful lights. I think AI is going to do exactly the same thing.”

James, on the other hand, is somewhat less concerned. “I used to work with a lot of hand lettering guys back in the past, and then computers came in. Now that essentially did put a small group of people out of their jobs, but the rest learnt how to use the computers and became an operator.”

At the very least, Heath insists that what awaits production companies is a wholly transformative process. “ From a crew point of view, it means that you might not need a second or a third camera assistant. You've got drones and gyros now that can stabilise and move shots that are better than a dolly. So, because you’ve got AI stabilising the gyro, the gripping department gets smaller.”

All this is without even getting into the ways AI is used in post production. Limehouse’s managing director and founder, Duncan Harriss, listed off the myriad ways the technology has changed the process. These ranged from:

• Autocolouring through Devinci; 
• Music creation based on extending existing tracks with Adobe Premier Remix;
• Automatic rotoscoping and cleaning up shots;
• Using AI to help convert assets in to AR & VR
• Video upscaling through AI like Topaz’s Video Enhance AI - Image or Video Enhance. You can also use AI to rid the footage of artefacts such as moire, macro blocking, aliasing, and other issues that can afflict various lower quality cameras and footage.

He continued, speaking specifically about AI’s use in the realm of animation. “In the CGI world, they're building tools that allow you to, for example, populate a scene. You can create 500 cars, or animals, or people, and populate it. This way, you don’t have to buy or manually create models. I also know people who are using it in animation to map faces, and rather than having to change the lip syncing for language translations, you can automate it through AI.”

Duncan calls this sort of process “heavy lifting”, which, as far as I can tell, is an apt description of how AI is being employed in production. The key thing that the technology is allowing for is for the hard, time-consuming, manual parts of production to be “filled in”. The future implications of this change are, undoubtedly, vast, and still uncertain.

But AI is here to stay. And no aspect of production will go unchanged.