Amimal Logic's Alex Timbs
Animal Logic, the Australian-based digital studio behind movies such as Babe, The Great Gatsby and Happy Feet, knows a thing or two about data management.
Its head of IT, Alex Timbs, has plenty of experience dealing with the storage and networking demands of movie making. He chatted with CIO Australia about how an increased focus on data management is transforming the way the company makes movies.
Currently, he manages the IT demands of a team that is working on five projects, including two animated features: Lego Batman Movie, which is slated for release early this year, and Lego Ninjago Movie, which is slated for mid-year.
In discussing his ups and downs as head of IT, Timbs said he recognises the enormous responsibility he holds leading the company’s tech strategy and master IT vision. The design, animation and visual effects company employs 500 staff in Sydney, 100 in Vancouver and 70 in Los Angeles.
“It is an incredibly privileged position to be around so many creative people and working for a company that is at the forefront of animation and visual effects. It is up there globally and certainly on a path of expansion. I’m not creative in the drawing or traditional artistic sense, but we have to consider ourselves creative in the technical space,” Timbs told CIO.
“We face huge challenges and we need to find creative ways to address those issues. It is incredibly challenging because of the breadth of responsibility and size of the infrastructure. We definitely need to nurture high end IT personnel that can deal with the stress and challenges. At the end of the day, all of them are all creative because they’re finding creative solutions to crazy problems that come up every day.”
He said dealing with the ongoing demands of networking, storage and data management top the list of things that keep him up at night.
“Most of our challenges relate to resources or resourcing the cyclic nature of our business. We do very large projects in animated feature, which can take three years from story to finished images. We have a very steep ramp up and sometimes that ramp up can occur quite late in the film and can require a very significant amount of extra resources which includes storage, render (in particular), and human resources,” he said.
HR drives things like workstation needs, licensing, and dealing with the increase in network throughput (increased number of storage nodes and farm nodes).
“It is challenging in the sense that sometimes we need these massive amounts of resources for a relatively short periods of time - it is really bursty. You could go from needing double the amount of storage that you had six months ago, and you might only need that for six months and then you can start archiving.”
Looking back at earlier films, storage and data management was clunkier and monolithic. “There were lots of silos of storage - monolithic, large storage systems where you had a controller and a myriad of discs,” he said.
“At the end of making Happy Feet, we were running 14 silos of storage that had to be accessed by a combination of the farm and users. In order to manage those resources we had two full time people managing them around the clock.
"They were managing performance and capacity and to manage it they had to move data around. If there was a big render that occurred at night across five of these silos, and they were almost full, they would have to move the data over to ones that were less full. They were constantly moving stuff around. It was a big job and quite inefficient.”
Thankfully, he said today’s storage architecture is much more granular, enabling building blocks that can scale up in relatively small chunks.
“We don’t have these huge, chunky, monolithic whole rack type investments needed to to grow storage,” he said.
He explained that the organisation appreciates the “one bucket managed, single file system” approach and, as as result, is running Dell EMC's Isilon network-attached storage platform with 7 petabytes of primary and DR storage.
“We are really pushing it in terms of performance,” said Timbs, adding there are new features in terms of the operating system which unlock hidden potential in the hardware via creative software development. “One of the most recent was L3 cache, which allows us to more fully utilise the SSDs that are in the appliances.”
Tech advancements
When asked about the main benefits to movie making from the latest technology enhancements, Timbs said the core value proposition is around iterating.
“The more you iterate in the pipeline (the more cycles you can go through the pipeline), the more you can distil the final creative product so you get a better looking image at the end of it.
“Some of the shots we work on are done hundreds of times, over and over again, either because of a fundamental creative change by the director or art director, or because we want a different camera angle - so they keep iterating until they get the best final creative output.
"The amount of iterations we can do and the speed at which we can iterate is many times faster these days - a large part of that is due to technology and automation.”
During the making of The Lego Movie, for example, he required massive amounts of data - the film peaked at 345 terabytes.
No doubt, storage has been his big focus given the escalating data demands.
“It’s consumed very rapidly and because of the size, the growth of company at the moment, and the number of projects we’ve taken on and the complexity of the techniques that are used to generate those images - people want 2K, HDR, stereo, 4K (there are a lot of changes in technology driven by the consumer side) - people expect a very high visual quality. We have invested heavily in areas of technology that are cost-effective but give us an increase in efficiency and capacity to be able to deliver more of those iterations through the pipeline,” he said.
The company has also virtualised its core services and updated its core infrastructure and edge switching with VDX switches.
“Our networking gives us the ability to deliver data from one place to another very quickly - the primary areas we want to move data is the render farms and storage and then ultimately back to the users to view the rendered material.”
He said the company also relies heavily on its asset management system, whereby assets can be checked in by a user or by an automated process.
“That allows us, in an automated way, to scan the file system and automatically take things offline once they’ve been protected to tape or once they've been archived. Or if it’s a file not considered an asset, and only needed for a short period of time to make something else or if it was a deprecated version, we have automated processes to scrub that off the system,” he said.
“As you can imagine a human trying to look at two million files just for one show. Lego had 185 million files that were under 2 megabytes on the file system at peak. The current shows are three to four times that size, like Lego Ninjago and Lego Batman, so the problems are getting bigger and more complex. We definitely rely on improvements in technology and data analytics and the ability to automate a lot of those processes based on policy to stay on top of it.”
Asked his next steps, he said the virtualisation of infrastructure is on the cards.
“Virtualising the render farm would be a potential for us," he said. "As compute gets more and more dense, it becomes more and more challenging to chop up your environment in a way that makes sense and so virtualising and completely abstracting the hardware layer is something of interest. The same can be said for storage - we are definitely keen on looking at more modular, versatile storage.”
The company is also looking at ways to enable the fastest, most efficient way of deploying infrastructure for short periods, which could mean adopting a cloud strategy.
“Running out there without a really clear understanding of what your strategy is and why you might be moving data or a particular process to the cloud, for example, is pretty futile. You really have to understand what the core value proposition is,” he said.
“It’s about monitoring automation - looking to see what opportunities may exist in the cloud.”
Security is another area of focus and the organisation is eyeing next-generation security and identity access management, multi-factor authentication, best-of-breed firewalls, and network segregation initiatives.
And while his head is squarely in the IT space, his main mission is to push the envelope, be one step ahead of industry changes and help the team provide groundbreaking visual effects and animation.
“Every movie seems to be trying to push the boundaries. People want to see something new, something more spectacular - and creatively that’s definitely what directors and studios and ultimately the end consumers want to see.
"They want to be wowed by a good story and amazing visuals and with that comes the constant desire and challenges to reach over the bar. Once you set the bar to a certain height, you want to exceed that and push the bar higher.”
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.