By Robert Goldrich

CULVER CITY, Calif. --

During a town hall event this past Tuesday evening (10/21) on the Sony Pictures Studios lot in Culver City, Calif., the Association of Independent Commercial Producers (AICP) Tech Committee made the first public presentation of findings from its Digital Asset Management (DAM) survey of member production company executives, CTOs/lead techs and librarians/vault managers, followed by a panel discussion moderated by SHOOT publisher and editorial director Roberta Griefer which delved further into DAM and related issues.

The need for exploring DAM is rooted in the seismic shift in image acquisition. Prior to 2009, 90-plus percent of commercials were shot on film, a steadfast archival medium yielding a 100-plus year preservation master. Anyone in the production or post chain could “always go back to the neg.” But in three short years, commercial production has become 90 percent digital, which carries storage implications as the shelf life of a hard drive, for instance, is more precarious and considerably less than what the film medium can provide.

Tech Committee member Paul Korver, executive producer/principal at post house Cinelicious, cited an example underscoring the importance of digital asset management to the client, agency, production and post communities. He related the true story of a Fortune 500 company’s newly hired CMO who, wanting to better understand the brand’s history, requested all broadcast masters for spots from the past 10 years be delivered to him. Agencies scrambled but could not fulfill the request on their own so they turned to production company and postproduction vendors who too were unable to locate a complete set of masters. Weeks later many of the spots retrieved were delivered to the CMO in the form of poorly compressed links to YouTube videos. Korver noted that as a result, agencies were fired and the client blackballed the vendors. Clearly there’s some question as to whose responsibility it is for archiving and maintaining content. And while the vendors in this case may not have been legally responsible, they found themselves punished nonetheless.

Wiredrive president Bill Sewell, another Tech Committee member, then presented findings of the DAM survey, which was conducted to get a better handle on the state of digital production and preservation in the commercial community, and to identify and evaluate technological, legal and other issues resulting from digital production and delivery.

An overview of the survey findings shared by Sewell and Korver were…

Production companies shoot, store and deliver large amounts of video for their clients, with most shoots generating between 250GB and 1,500 GB per day.
All footage (raw digital negative shot predominately with ARRI Alexa) is transferred from cards (which are wiped) to hard drives (which are saved). 82 percent of production companies store their raw digital negative on single hard drives kept on a shelf. 69 percent store finished masters in the same way, making hard drive storage a critical part of the media lifecycle.

According to the members surveyed, hard drive storage carries considerable risk.
78% of members surveyed expect hard drives to last less than five years
18% expect failure within six to ten years
96% expect failure before 10 years
27% reported data loss that significantly impacted the company in a negative way
51% feel “safe” about the digital storage options they use
27% have “fingers crossed”
12% feel they’re “probably going to lose some data”
Only 6% felt “Very safe”

This timeframe becomes a problem for finished assets:
69% of production companies store finished assets on single hard drives; while more than 53% expect finished masters to be available for a minimum of 10 years.
31% noted they’d like their finished masters to last over 50 years. 
People expect LTO (Linear Tape Open) storage to last longer, with 43% expecting failure before 10 years. 
71% do not use LTO.
51% have a digital archive policy and implement it on every job.
33% don’t have a real one and “could use some guidance coming up with best practices”.

Managing technology to provide long-term media storage is a challenge. Managing expectations of legal liability should be much more straightforward, but is also quite unclear:
20% of surveyed production companies have contractual and legally binding language to provide clarity for long term storage responsibility.
53% rely on mutual assumptions and industry standard practices.  25% claim: “it’s unclear”.
75% of production companies save both RAW digital negative and uncompressed finished masters “just in case,” even without contractual obligation.
69% believe responsibility lies with the agency or client. 17% are unsure and 13% believe no one is contractually bound to store assets.
79% were interested in helping AICP define a standard set of practices. 

The study concluded that there’s a strong need for an agreed-upon media storage policy for production, postproduction and agencies that addresses technological and legal realities. 
Masters that need to last for 50-plus years should not be stored on hard drives rated for five years. Long term storage solutions like LTO are not common practice and will require education. Production companies which are not contractually obligated to manage large and expensive media libraries need to operate without fear of clients who may hold them responsible for those projects. AICP is actively working on behalf of its members in an effort to address this issue and find solutions. These efforts also entail teaming with AICE to develop best practices and solutions.

A more detailed rundown of survey results appears later in this story. Tech Committee member Jessie Nagel of communications/PR agency Hype, who welcomed guests, noted that an initiative a couple of years ago to more substantively integrate AICP associate and general (production company) members laid the groundwork for what became this DAM survey, an example of associate members taking a proactive role to benefit the overall industry, including the core production house membership.

DAM Dialogue
SHOOT’s Griefer moderated a discussion featuring panelists Melissa Davies, co-founder and CEO of Arc 9 which offers creative media collaboration tools to review/proof, file share, present and manage in one simple application; Michele deLorimier, freelance digital imaging technician (DIT); Todd Makurath, CEO of branded entertainment production company Bullitt, and a member of both the AICP Tech Committee and the AICP/West Board; Ryan Schnizlein, head of operations & technology, Cutters Studios, and head of the AICE Technology Committee; and attorney Bret Siciliano, associate, corporate department of Venable LLP.  Griefer kicked off the discussion by reading an excerpt from a POV column that Paul Korver had  written for SHOOT upon his return from the 2013 NAB Convention.  She noted that since that column, which addressed the need to plan for long term digital storage, there are more cameras, more workflows and more storage options than ever before and commended the AICP for taking steps towards formulating best practices by conducting the survey and holding the meeting.  Thanking the AICP for inviting SHOOT to moderate the panel, Griefer mentioned the great timing as SHOOT had just published an article on Digital Workflow Solutions and had plans for more regular coverage of workflow-related issues.  She let the audience know that while the panel would touch on several areas of digital asset management, begining with acqusition and the path data takes through storage, they would primarily focus on long term digital storage.

Schnizlein noted that AICE has been working to address the issues brought up in the AICP Tech Committee survey and that he was gratified to be invited to this panel discussion, reflecting a joint effort on the part of AICP and AICE to make progress on this front. He said it’s critical that awareness be raised regarding the amount of data being created, how it should be protected and ultimately archived to a suitable long-term storage medium such as LTO tapes. Then there has to be meaningful discussion on how to pay for it. Somewhere along the file-based workflow path the industry has been on for the last 10 years, we forgot to replace the longevity of film and videotape.

As for the path that digital assets take at Cutters, Schnizlein said it all starts with a call with the DIT on production, touching base regarding what files and formats Cutters will be receiving. He noted that in 99 percent of the jobs, Cutters gets one hard drive copy of the material created in production–the original camera files, LUTs, production audio, script notes, etc. Cutters then writes the shoot drive assets to LTFS (Linear Tape File System)-formatted LTO tape. This protects these assets through the post process and creates a long-term storage solution for production data. While in post, assets are on protected/backed-up storage solutions. Upon wrapping its work, Cutters archives non-proprietary content it has created to LTFS-formatted LTO tapes. Next, Cutters ships the original shoot drive and its LTOs to the client’s designated long-term storage vendor/facility.  In response to Griefer's question asking if he could provide some specific companies offering various long term storage options, Shnizlein mentioned open source software (free) to entry level that starts at about $500 and while he was not endorsing any specific one, some well known products include bru pe-tolis and yoyotta id ltfs –for appliances, cache-a and strongbox and for storage integrated solutions, there are vendors such as  Quantum.

Makurath said that LTO tape is common in the feature world. “We haven’t been talking about it [LTO] in the commercial world [as reflected in AICP survey findings]. Maybe we should be.” He added that the DIT making an LTO on set might merit some consideration.

DIT deLorimier said that it could prove too expensive to create an LTO tape during the production process; rather, it might be more economically feasible to do so at a brick-and-mortar post house. In her response to Griefer's question regarding the role of the DIT compared to that of a media manager or digital loader,  deLorimier described the media manager or digital loader as “a more passive position in the camera department,” involved in downloading and verifying footage. By contrast, the DIT is “a more proactive position,” involved in creating, for example, a link with postproduction so the editor will get what he or she needs in terms of deliverables–the same for online and VFX as well. The DIT job starts in pre-pro, selecting recording format and connecting the dots between departments. The goal of the DIT is to match the right equipment to the best interests of the job, perform media management and look management as well as technical assistance during the shoot, preserving the integrity of the data for the producer and maintaining the creative intent of the cinematographer through the hand-off to postproduction.  Griefer also asked about re-accessing and re-reprocessing footage at a later date if for example the camera used can capture high dynamic range, something that is not going to be used now in the finished work.  deLorimier discussed that when, "we shoot raw there is the opportunity to re-debayer the footage as better debayer algorithms are developed."

Makurath assessed that if there’s one takeaway from the DAM discussion, it’s that “a hard drive on a shelf is not the place to store your media.” He said all parties need to explore how do properly protect this footage over the long haul, citing such possible viable options as LTO and the Cloud. Makurath added that masters are important not just for client archival purposes but also in terms of contractual commitments to directors whereby production companies need to provide them access to copies of their work, particularly if they move onto another production house.  To Griefer's question asking about the differences in how he handled storage in the past as COO of a large multi-national production company (Caviar)  that produced great amounts of content, he joked how now as a much smaller company, he is using hard drives but will be looking towards more suitable long-term storage in the future.  

Davies noted that entertainment industry studios are getting more involved in Cloud storage while commercials have not really embraced Cloud technology. The Cloud option offers certain flexibility, dovetailing with other software products such as Aspera or how a company like MediaMax Cloud, in which she is in an investor, offers other services such as electronic press kits for the entertainment industry.  In response to Griefer's question about the benefits of long term cloud storage, Davies said, "the primary benefits are offering backup, long-term preservation and disaster recovery."   When asked about specific companies/services, Davies mentioned Amazon S3 and suggested, "that a good option for production companies and agencies could be to look to outsource library management/Cloud storage and distribution services with companies such as LAC and Extreme Reach." Schnizlein chimed in that the Cloud is a viable option given that digital content databases will grow enormously in the years to come. 

Attorney Siciliano said that contractual language relative to DAM should cover who’s going to store the data and for how long, and who’s going to pay for it.  Griefer asked about his thoughts on the various AICP documents and production contracts that he reviewed prior to the meeting as they related to spelling out guidelines for long term digital storage.  He commented that "once the association had best practices in place, clear language regarding responsibilities and financial obligations could most likely be added in a few paragraphs to existing documents to replace the sometimes vague language that exists now if it exists at all." 

Griefer also shared with the audience a number of legal considerations based on an exchange she had with attorney Greg Boyd, a partner at Frankfurt Kurnit Klein & Selz (FKKS). Boyd provided a list of five considerations for long-term storage of digital assets:

1.  What is it?  Is it art or is there any personal information in there (as is often in some types of digital assets).  Photos can be personal information under COPPA for instance.  The nature of the material will change almost every elements of the storage.

2.  How long do we store it?  What is the asset life cycle?  This depends on contractual obligations and also on what it is.  You don’t want to store personal information any longer than necessary.  Is it encrypted or should it be?  Who has access to it?

3.  Who pays for it?  The agency, the client, or the production company?

4.  How do we store licenses along with it?  Releases and code licenses should be preserved, preferably in the same place.

5.  Confidentiality level – are these just basic assets or are they shoe/car prototypes for the next 2 years of production.  Is it personal information/financial or healthcare assets? 

In wrapping up the panel discussion, Griefer summarized the panelists' consensus that while it is generally understood that agencies are responsible for the maintance of long-term digital assets, there is a need to map out standard practices that clarify responsibilities and financial obligations and incorporate that information into the legal documents involved in production.  What exists now is often vague and does not follow uniform guidelines.  And, while production companies may not be responsible for the overall long-term storage of a project, they still want to maintain a record of their work and be able to have their directors' work accessible should they need it; so clearly there is the need to replace hard drive storage with longer-term storage that is secure, easily accessible and cost efficient.

Survey details
Here’s a more comprehensive rundown of AICP DAM survey findings, with feedback to questions:

Digital Asset Management General Member Survey 2014
Who took the survey?
Survey Background
Sent June 23 – July 8, 2014
56 general members reached
49 general members responded
88% response rate

48% of respondents are from production company management
30% of respondents are librarians/vault managers
23% of respondents are CTO/lead tech personnel

Broadcast spots per year (48 respondents)
10 respondents do 1-10 broadcast spots per year
12 respondents do 11-25 broadcast spots per year
13 respondents do 26-60 broadcast spots per year
10 respondents to 61-100 broadcast spots per year
3 respondents do 100-plus spots per year

Choose Top 4 Camera Formats You Use (respondents allowed to select multiple options–total of 49 respondents)
ARRI Alexa HD ProRes 92%  45 respondents
RED 4K/5K RAW 55%  27 respondents
ARRI Alexa 3K RAW 53%  26 respondents
Canon 5D HD H.264 43%  21 respondents
Phantom Cine Raw 43%  21 respondents
RED 2K Raw 41%  20 respondents
Canon C300 HD 29%  14 respondents
Sony F55 4K RAW 16%  8 respondents
Canon C500 4K RAW 10%  5 respondents
Go Pro 10% 5 respondents
Sony F5 Sony F65 4K RAW 4%  2 respondents

Do you have an interest in helping AICP define a standard set of digital practices? Total respondents: 48
79 percent responded Yes
21 percent responded No

What digital media assets do you store? Respondents allowed to select multiple options Total Respondents: 47
Finished Masters 72%  34 respondents
All RAW Digital Negative 68%  32 respondents
HD Dailies (transcoded media) 57%  27 respondents
Selects/Circle takes of RAW Digital Negative 13%  6 respondents

How and where do you store RAW Digital Negative? Respondents allowed to select multiple options Total Respondents: 49
Individual hard drives on a shelf in our office 82%  40 respondents
Individual hard drives at external vault location 18%  9 respondents
Local redundant RAID Storage 12%  6 respondents
LTO tape on a shelf in our office 10%  5 respondents
We don’t store 10%  5 respondents
LTO Tape on a shelf at an external vault vendor 6%  3 respondents
Cloud Storage 6%  3 respondents
LTO tape in a Robotic Tape Library at our office 2%  1 respondent

Estimate amount of Raw Digital Neg per shoot day (GB). Total Respondents: 48
250 or less 10%  5 respondents
251-500 GB 21%  10 respondents
501-750 GB 17%  8 respondents
751- 1500 GB 27%  13 respondents
1501 – 3000 GB 8%  4 respondents
Over 3000 GB 4%  2 respondents
I do not know. 13%  6 respondents

Does your company currently use any on-set or near-set digital asset management system to archive/track digital media at point of capture? Total Respondents: 49
No; DIT provides 92 percent
Yes 8 percent

How and where do you store Finished Masters? Respondents allowed to select multiple options Total Respondents: 49
Single hard drives on a shelf in our office 69%  34 respondents
Local Redundant RAID storage 41%  20 respondents
Single hard drives at an external Vault vendor 16%  8 respondents
LTO tape on a shelf in our office 8%  4 respondents
Cloud Storage 8%  4 respondents
We don’t 8%  4 respondents
LTO tape on a shelf at an external vault vendor 6%  3 respondents
HD Videotape Master (i.e. HDCAM-SR) 4%  2 respondents
LTO tape in a robotic library at our office 2%  1 respondent

What format is your Finished Master? Respondents allowed to select multiple options Total Respondents: 47
ProRes 422 HQ (or 4444) Quicktime 83%  39 respondents
Uncompressed Quicktime 53%  25 respondents
Uncompressed DPX/TIF Sequence 21%  10 respondents
Avid DNxHD 9%  4 respondents
HD Videotape (i.e. HDCAM-SR) 9%  4 respondents

If your company uses LTO tapes as part of its storage strategy, what tape format do you write to? Total Respondents: 42
We don’t use LTO tapes 71%  30 respondents
Don’t know 12%  5 respondents
LTFS 10%  4 respondents
TAR 5%  2 respondents
BRU 2%  1 respondent
PAX 0%  0 respondents

Who is responsible?
On each job that your company produces, how would you describe the clarity regarding whose responsibility it is to provide long-term storage of digital assets? Total Respondents: 49
Non-Contractual (but based on mutual assumptions and industry standard practices) 53%
Contractual and legally binding  20%
Unclear 24%
No Idea 2%

Do you own the rights to content you are hired to create? Total Respondents: 49
Rarely 51%
Never 29%
Sometimes 18%
Yes 2%

Who is responsible to store the RAW Digital Negative? Respondents allowed to select multiple options Total Respondents: 48
Agency/Client 69%  33 respondents
Editorial 19%  9 respondents
I’m not sure. 17%  8 respondents
Post Production/Finishing 15%  7 respondents
No one is contractually bound to store it. 13%  6 respondents
Production Company 8%  4 respondents

Is your company contractually bound (to either your directors or clients) to store the RAW Digital Negative of the content you produce? Total Respondents: 48
Never 48%
Rarely 29%
Sometimes 13%
Yes 10%

We are required to store footage on behalf of:
Total Respondents: 33
Directors 42%
Clients 24%
Both Directors & Clients 33%

What are storage lifecycle expectations?
How long do you feel that the RAW Digital Negative of the content you’ve produced should be accessible? Total Respondents: 47
0-5 Years 24 respondents
5-9 Years 13 respondents
10-20 Years 4 respondents
21-50 Years 2 respondents
More than 50 Years 4 respondents

How long do you feel that the Finished Masters of the content you’ve produced should be accessible? Total Respondents: 49
0-5 Years 11 respondents
5-9 Years 12 respondents
10-20 Years 8 respondents
21-50 Years 3 respondents
More than 50 Years 15 respondents

How long are you required to store these assets? Total Respondents: 44
0-1 Years 3 respondents
1-5 Years 3 respondents
6-10 Years 0 respondents
Over 10 years 0 respondents
No agreement 38 respondents

What digital assets do you tend to store “just in case”–even though your are not contractually bound to do so? Respondents allowed to select multiple options Total Respondents: 48
RAW Digital Negative of all shot footage 75%  36 respondents
Finished Masters in uncompressed/ProRes Format 75% 36 respondents
Dailies/Finished Masters in H.264 proxy/Web format 46% 22 respondents
Selects/Circle takes of RAW Digital Negative 19% 9 respondents

Please check all options that characterize your company’s internal digital storage policy or protocol: Total Respondents: 49
We have a digital archive policy and implement it on every job. 55%  27 respondents
We don’t have a real one & could use some guidance coming up with best practices. 33%  16 respondents
We don’t have one and don’t care. 12%  6 respondents
We have one but it’s followed inconsistently and varies from job to job. 10%  5 respondents

How safe is your storage?
How do you feel about the safety of the digital storage options you use? Total Respondents: 49
Safe 51%
Fingers crossed 27%
Going to lose data 12%
Very safe 6%
Haven’t considered 4%

Have you ever lost any digital assets that impacted your company in a negative way? Total Respondents: 49
No 73%
Yes 27%

How long do you expect your storage to last? Total Respondents: 49
0-2 Years
Drives (off) 22%
Drives (exercised) 21%
Drives (Raid) 29%
LTO 14%
Videotape 22%
Cloud Storage 11%

2-5 Years
Drives (off) 55%
Drives (exercised) 35%
Drives (Raid) 31%
LTO 3%
Videotape 13%
Cloud Storage 5%

6-10 Years
Drives (off) 18%
Drives (exercised) 40%
Drives (Raid) 29%
LTO 26%
Videotape 26%
Cloud Storage 20%

11-20 Years
Drives (off) 2%
Drives (exercised) 2%
Drives (Raid) 4%
LTO 33%
Videotape 22%
Cloud Storage 14%

21-49 Years
LTO 12%
Videotape 9%
Cloud Storage 9%

Over 50 Years
Drives (off) 2%
Drives (exercised) 2%
Drives (Raid) 6%
LTO 12%
Videotape 13%
Cloud Storage 41%

As a company, do you like the idea of Cloud Storage or would you prefer in-house storage? Total Respondents: 47
In-house storage 60%
Cloud Storage 26%
Both 15%

How connected is your office?
Internet/Broadband Connection Protocol – How does your company primarily access the outside world? Total Respondents: 49
Cable Modem (i.e. Time Warner Cable; Roadrunner) 41%  20 respondents
Shared Fiber (i.e. Verizon FiOS) 19% 9 respondents
DSL (i.e. T1, T3) 14% 7 respondents
Wireless Broadband Provider (i.e. Towerstream) 10% 5 respondents
Direct Fiber/Dark Fiber (i.e. direct link to “One Wilshire”) 4% 2 respondents
Multiple Providers 12% 6 respondents

What players do you subscribe to when showing your content to the outside world? Due to Multiple selections, numbers do not add up to 100% Total Respondents: 49
Wiredrive 73% 36 respondents
Vimeo Plus/Pro 61% 30 respondents
YouTube 20% 10 respondents
Simian 16% 8 respondents
Interdubs 10% 5 respondents
Another type of player 8% 4 respondents
Non-Branded Flash Player 4% 2 respondents

What is your company’s approach to 4K production? Total Respondents: 49
We’re going to wait and see if this becomes a deliverable spec. 53% 26 respondents
We shoot 4K for image quality reasons, but master only to HD or lower resolution. 35% 17 respondents
We care and are future proofing for 4K by shooting 4K resolution masters. 12% 6 respondents

Have you seen 4K content displayed on a 4K television? Total Respondents: 49
No 53%
Yes 47%

MySHOOT Profiles