Success Stories

Framestore Clusters EonStor to Build Europe's Most Powerful Render Farm for Computer Generated Images

Organization

Framestore, a computer generated image studio

Challenges

Deliver a system scalable from 50TB to 200TB, cost effectively, with guaranteed storage for mission critical digital image data

Solution

EonStor^® RAID subsystems met the challenge with:

High capacity
Scalability
Open architecture
Excellent price/performance
Free choice of disk drives

Framestore Clusters EonStor to Build Europe's Most Powerful Render Farm for Computer Generated Images

“80% of the rendering for the film was completed in the last three months of production. We designed the systems to scale. The project started with 50TB of storage which we grew by an additional 150TB as we worked,” said Prescott. He continued, “With this capacity we believe we are, by some margin, running the most powerful CGI render farm in Europe.” - Steve Prescott Director of Technology, Framestore

Framestore

Based in London, Framestore is Europe's largest animation, visual effects and computer generated imagery studio, with a team of over 800 staff and numerous awards including an Oscar and a Bafta. Framestore's feature animation division recently completed its first full length animated feature film, The Tale of Despereaux for Universal Studios – an immense project requiring new IT resources.

Rendering cinema and broadcast quality images consumes huge volumes of data storage. With the production of a full length film, in addition to visual effects work, advertising and post production processing, Framestore needed to be able to rapidly expand storage for its render farm and to guarantee the safety of this valuable data.

Instantly Scalable Storage a Must

Framestore's in-house IT team was tasked with building a new render farm to manage the demands. The film required a unique look, so to meet the tight deadlines animation would proceed alongside the development of the custom processing algorithms needed to render the final look.

This parallel approach would increase storage and render farm demands as many intermediate stages of production data would be retained to avoid unnecessary animation rework whilst fine tuning of the lighting effects and colour proceeded. The storage system had to have the capability to grow with the project.

With tens of thousands of man hours being invested, RAID 6 was seen as essential for data resilience and a mirror facility was required to provide off site back up. To achieve the rapid scaling anticipated Framestore had selected Lustre, the open source Linux cluster file system, so needed open standards based storage controllers.

Eonstor: A Trusted Choice

Infortrend's EonStor range offered Framestore high performance RAID 6 controllers with a choice of interfaces and the freedom to select the optimal drives for each part of the system. These attributes coupled with open standards based hardware and proven reliability secured Infortrend's selection for this challenging project.

“Producing cinema quality animation generates huge volumes of valuable data. At peak working we were generating 5TB per night, so reliable high performance storage was central to our system design. We were very keen to use RAID 6 to give us extra resilience,” said Steve Prescott, Technology Director, Framestore.
“Doing Despereaux we doubled the amount of cores we used on the render farm and quadrupled the amount of data that we were producing. We had 250 artists working on it, 6000 cores in the render farm and 200 terabytes of storage which had to be smooth and accessible for the final stages of the movie”.

A 600TB System

Framestore's final system comprised a 6000 core render farm closely coupled by 4Gbit/s Fibre Channel to 200TB of Infortrend EonStor configured as a RAID 6 protected clustered Linux file system.

Production data was mirrored on a second 400TB cluster which provided enough capacity to mirror and snapshot the entire project. In each cluster the metadata servers used two 4Gbit/s Fibre Channel-attached EonStor RAID arrays loaded with SAS drives for speed. The bulk storage was provided by eight object storage servers each having multiple 24 bay Infortrend EonStor arrays attached via dual 4Gbit/s Fibre Channel links, and populated with SATA drives. Faster, more expensive SAS drives were not necessary as Prescott explains, "We got the aggregate bandwidth we needed because we had so many spindles." EonStor's free choice of drives enabled Framestore to make a significant cost saving through this intelligent design choice.

Mirroring and snapshots of the primary storage to the second cluster were performed across a 10Gb/s network infrastructure built on dark fibre which Framestore installed between its three Soho production offices and a London Docklands data centre. This infrastructure enabled the entire project to be replicated and backed up across the multiple locations each day.

At the peak of production, rendering on the Tale of Despereaux was using 4700 of Framestores 6000 processor cores – with other projects running in parallel using the remainder. During production, the Tale of Despereaux team generated 90 million files, over 200TB of data in total.

About Framestore

Framestore have in excess of 20 years of experience in digital film and video technology. Employing over 800 staff, the company works out of three buildings in London's Soho district, with another office based in New York. 2008 was a landmark year for Framestore, with the company winning both the Oscar and BAFTA for its visual effects work on The Golden Compass. Over the years, the company has won numerous international awards including two Technical Oscars, three BAFTA Craft Awards and thirteen Primetime Emmy Awards. Recent film projects include: The Chronicles of Narnia: Prince Caspian, The Dark Knight, Quantum of Solace and Australia. In 2006, Framestore set up a feature animation division, with the aim of producing its own digitally animated films. The first of these, The Tale of Despereaux, commenced full production in 2007, and opened worldwide in December 2008.