IBC News Recap: HEVC, Cloud Workflows, Media Management, Video Encoding & Optimization

I wasn’t at the IBC show this year, but here’s a run down of the news announcements that I saw. Lots of focus on HEVC as expected, but less talk about 4K compared to last year. If I had to pick one theme from this show it would be cloud based media management platforms. I found two announcements to be particularly interesting; Brightcove’s new stand-alone video player platform, de-coupled from their OVP services and Microsoft Azure Media Services new live streaming platform. I’ll have more details and thoughts on both of these new offerings later in the week.

Sponsored by

Inside Apple’s Live Event Stream Failure, And Why It Happened: It Wasn’t A Capacity Issue

Apple’s live stream of the unveiling of the iPhone 6 and Watch was a disaster today right from the start, with many users like myself having problems trying to watch the event. While at first I assumed it must be a capacity issue pertaining to Akamai, a deeper look at the code on Apple’s page and some other elements from the event shows that decisions made by Apple pertaining to their website, and problems with how they set up storage on Amazon’s S3 service, contributed the biggest problems to the event.

Unlike the last live stream Apple did, this time around Apple decided to add some Javascript JSON (JavaScript Object Notation) code to the apple.com page which added an interactive element on the bottom showing tweets about the event. As a result, this was causing the page to make refresh calls every few milliseconds. By Apple making the decision to add the JSON JavaScript code, it made the apple.com website un-cachable. By contrast, Apple usually has Akamai caching the page for their live events but this time around there would have been no way for Akamai to have done that, which causes a huge impact on the performance when it comes to loading the page and the stream. And since Apple embeds their video directly in the web page, any performance problems in the page also impacts the video. Akamai didn’t return my call asking for more details, but looking at the code shows there was no way Akamai could have cached it. This is also one of the reasons why when I tried to load the Apple live event page on my iPad, it would make Safari quit. That’s a problem with the code on the page, not with the video.

Because of all the refresh calls from the JSON-related JavaScript code, it looks like it artificially forced the player to degrade the quality of the video, dropping it down to a lower bitrate, because it thought there were more requests for the stream than there was. As for the foreign language translation that we heard for the first 27 minutes of the event, that’s all on Apple as they do the encoding themselves for their events, from the location the event is at. Clearly someone on Apple’s side didn’t have the encoder set up right and their primary and backup streams were also way out of sync. So whatever Apple sent to Akamai’s CDN is what got delivered and in this case, the video was overlaid with a foreign language track. I also saw at least one instance where I believe that Apple’s encoder(s) were rebooted after the event had already started which probably also contributed to the “could not load movie” and “you don’t have permission to access” error messages.

Looking at the metadata from the event page, you could see that Apple was hosting content from the interactive element on the apple.com event page on Amazon’s S3 cloud storage service. From what I can tell, it looks like Apple set up the content in a single bucket on S3 with little to no cache hit ratio, with poor bucket configuration. Amazon didn’t reply to my request for more info, but it’s clear that Apple didn’t set up their S3 storage correctly, which caused huge performance issues when all the requests hit Amazon’s network in a single location.

As for Akamai’s involvement in the event, they were the only CDN Apple used. Traceroutes from all over the planet (thanks to all who sent them in to me) showed that Apple relied solely on Akamai for the delivery. Without Akamai being able to cache Apple’s webpage, the performance to the videos took a huge hit. If Akamai can’t cache the website at the edge, then all requests have to go back to a central location, which defeats the whole purpose of using Akamai or any other CDNs to begin with. All CDNs architecture is based on being able to cache content, which in this case, Akamai clearly was not able to do. The below chart from third-party web performance provider Cedexis shows Akamai’s availability dropping to 98.5% in Eastern Europe during the event, which isn’t surprising if no caching is being used.

akamaiThe bottom line with this event is that the encoding, translation, JavaScript code, the video player, the call to S3 single storage location and the millisecond refreshes all didn’t work properly together and was the root cause of Apple’s failed attempt to make the live stream work without any problems. So while it would be easy to say it was a CDN capacity issue, which was my initial thought considering how many events are taking place today and this week, it does not appear that a lack of capacity played any part in the event not working properly. Apple simply didn’t provision and plan for the event properly.

Updated Thursday Sept. 9th: From talking to transit providers & looking at DeepField data, Apple’s live video stream did 6-8Tbps at peak. World Cup peak on Akamai was 6.8Tbps. So the idea that this was a capacity issue isn’t accurate and the event didn’t generate some of the numbers I see people saying, like “hundreds of millions” watching the stream.

Updated Thursday Sept. 9th: While some in the comments section want to argue with me that problems with the Apple.com webpage didn’t impact the video, here is another post from someone who explains, in much better detail than me, many of the problems Apple had with their website, that contributed to the live stream issues. See: Learning from Apple’s livestream perf fiasco

Internet Traffic Records Could Be Broken This Week Thanks To Apple, NFL, Sony, Xbox, EA and Others

Thanks to so many large scale live events and large file downloads taking place this week, it’s going to be a huge week of traffic on the Internet with content delivery networks and last mile providers preparing for what’s to come. Tomorrow in particular will be a big day on the net with so many things taking place all on the same day. Apple product announcements always make for a busy day on the web and while iOS 8 won’t be available for download tomorrow, here is a list of all the other events taking place tomorrow, or this week.

  • Monday Night Football (WatchESPN)
  • Apple’s Product Announcement (Tuesday)
  • Microsoft Security Patches (Tuesday)
  • NFL Now/Game Rewind Highlights (Tuesday is busiest day for NFL videos)
  • Yahoo! Aerosmith Concert (Tuesday)
  • Bungie’s Release Of Destiny Game (Tuesday)
  • EA Sports Fifa 15 Game Beta (Tuesday)
  • EA Sports NHL 15 Game (Tuesday)
  • League Of Legends NA/Europe Seed Matches (Tuesday)
  • Xbox Free Game Releases (including Halo: Reach)
  • Sony PS4 White Destiny Bundle (Tuesday)
  • New York Fashion Week Live (Monday-Thursday)
  • Apple’s iOS 8 Download (Probably Thursday)
  • President’s ISIS Speech (Wednesday)

Delivering video over the Internet at the same scale and quality that you do over a cable network isn’t possible. The Internet is not a cable network and if you think otherwise, you will be proven wrong this week. We’re going to see long download times, more buffering of streams, more QoS issues and ISPs that will take steps to deal with the traffic, knowing it will have a negative impact on the user experience. When iOS 8 comes out, some last mile providers are going to struggle and some will rate limit their network connections, as we saw the last time Apple’s iOS download was available. For some ISPs, iOS 7 downloads took up 40% of their traffic. Also, all other content providers are going to have to compete with this traffic and many I spoke to about it are keeping an eye on their quality guarantees and SLAs with their CDNs this week.

As for which CDNs are delivering all this content, I’ll be doing a lot of traceroutes this week, but Akamai, Limelight and Level 3 are all in the mix. I know Akamai will see the most web traffic from news sites covering Apple’s product announcements. Last time I looked, Microsoft’s security patches were being delivered by Akamai, Limelight and Level 3. Yahoo’s concert is being done by Akamai. Level 3 and others do the Xbox releases, Limelight was doing a lot of Sony downloads last I checked and when iOS 8 is available, I expect a lot of it to be delivered by Apple themselves, with Akamai and maybe also Level 3. Many of these events, both live and downloads, push more than 1Tbps via a single CDN, let alone those that use dual vendors, so it’s going to be a very busy week on the web for CDNs.

For all the talk of paid interconnects being such a bad idea, or causing great harm to the Internet, none of what is going to take place this week would be possible if these paid connections between CDNs and ISPs were not in place. So complain all you want, but it’s why the Internet works the way it does and why hopefully, all will go smoothly this week.

I will be collecting as many traceroutes as I can from multiple regions during all of these events, so if you can do a traceroute from your location, please send it to me at mail@danrayburn.com

Telstra To Acquire OVP Provider Ooyala: 5x Revenue Valuation, Will Remain Independent

Online video platform provider Ooyala announced that Australia based telco provider Telstra will acquire the company, which will become a subsidiary of Telstra, and will operate as an independent business under the leadership of its existing management team. Telstra, which already owned 23% of Ooyala via a previous $61M investment, is paying $270M to have a 98% ownership in the company. In the past seven years, Ooyala had raised $122M in funding from Telstra and other investors and the company is projected to do $65M in revenue this year, which puts Ooyala’s valuation at just above 5x projected 2014 revenue. That’s a nice multiple considering that Ooyala’s largest competitor in the space, Brightcove, has a market cap of just under $200M, but is expected to do around $122M this year, or double Ooyala’s revenue.

This is a smart deal for Ooyala and the right time to do it as the company has been experiencing growing pains with their business that has negatively impacted some of their customers. Any small company can only manage growth so well with the resources they have and Ooyala simply needs more resources from a larger company to continue to grow. Now with Telstra’s backing, Ooyala should be able to scale their business much faster and avoid some of the headaches they have experienced, hopefully enabling them to provide a much more consistent quality service to every customer. As Ooyala’s CEO mentioned in a letter to employees, scale is the key to their business, and that’s what Telstra gives them. Ooyala couldn’t get there by simply taking more money; they needed a partner and backing of a much larger company that can take their cloud offering and integrate it into a much larger platform.

Vendors that want to compete in the MSO space have to have deep pockets and spend a lot of money on people, infrastructure, R&D and all the accompanying costs that go along with a cloud based service. To put it in perspective, OVP providers Ooyala, Brightcove and Kaltura have raised more than $400M in funding combined, but are on track to just over half that in 2014 revenue. Their business models require a lot of cash and more importantly, scale. To win in this market, they have to build a much larger platform, like what Cisco did when it acquired Inlet Technologies, Extend Media and BNI Video to create their Videoscape TV everywhere solution. When the major players in any market are companies like Cisco, which to date has invested half a billion dollars into their TV everywhere platform, the writing is on the wall for smaller companies to become part of a larger ecosystem.

While I believe this deal is good for Ooyala and their customers, as an industry, we have to set the proper expectations on the size and growth of the different markets that vendors sell into. Companies are always so quick to throw out huge numbers on how big a market will grow to in a few years, but they are always wrong. Brightcove made this mistake when in their S-1 filing they said the “total potential market opportunity was approximately $2.3 billion in 2011, growing to approximately $5.8 billion in 2015.” This set wrong expectations with not just those on Wall Street, but also with those who track the industry. In an open letter to Ooyala’s employees, their CEO said, “The market for the technologies and services we provide is will be worth tens of billions in the next few years.” Those are not realistic market sizing numbers.

If you combine the revenue of all the companies in this space that are selling cloud based services similar to Ooyala, it is somewhere in the ballpark of $500M. And that number includes a large percentage of revenue from reselling bandwidth, not platform license fees. [See: OVPs Still Getting Too Much Of Their Revenue From The Re-Sale Of Bandwidth] So even if the industry as a whole grows 30% a year, for the next three years, that’s combined revenue of about $1.3B globally. That’s a long way away from Brightcove’s “$5.8B” number or Ooyala’s “tens of billions” estimate.

While Ooyala still has to execute properly in the market, Telstra’s backing gives Ooyala everything they need to scale their business to a level that most other competitors won’t be able to get to. They can hire more employees, spend more on R&D, have a true global presence, create very large and meaningful partnerships and drive faster adoption for their services. I would argue they have more work to do now than before the acquisition, but with the new backing and new resources, Ooyala should be able to grow and scale their business at a level they simply would not have accomplished on their own.

Here’s The Program For Streaming Media West Show: Now Placing Speakers

Screen Shot 2014-08-06 at 4.49.26 PMThe advance program for the 2014 Streaming Media West show (#smwest) taking place Nov. 18-19, at the Hyatt Regency Huntington Beach Resort & Spa in Huntington Beach, CA is now complete.[view the entire agenda here] Speaker placement has officially started and our first batch of speakers from Netflix, FOX Networks, DIRECTV, NY Times and YouTube have already been confirmed. Speaking spots will go fast, so if you are interested in being considered for one of the speaking spots, or placing someone from a company on a session, please contact me. As always, vendors who bring customers will always get picked before vendors that don’t bring customers.

If you have any questions at any time about the program, what a session will cover, how the selection process works, or need help choosing the right session, you can call me (917-523-4562) or send email. However I find it is much easier to discuss it on the phone rather than email and many times, I can give you an answer on the spot. So please feel free to call at any time.

You can view the entire agenda here, and below is a list of all the session titles:

  • How To: Selecting The Right Video Management Technology
  • On-Premises Or In The Cloud: Choosing The Best Live Video Workflow
  • Best Practices for Webcasting Sporting Events
  • Netflix’s Video Workflow: Transcoding, Codecs, and 4K Streaming
  • How YouTube Lost at Becoming The Next TV Network
  • Designing The Right OTT Workflow For Premium Content Owners
  • How To: Building A Chromecast Application
  • Streaming Live Events When There Is No Room For Failure
  • The Business Of Building An Enterprise Online Video Network
  • How To: Multi-Cam Shooting, Editing and Delivery On An iPad
  • TV Everywhere Trends And What It Means For The Future Of Television
  • Using YouTube For Original Content Distribution
  • How To: H.265 vs. H.264: Choosing The Best Options
  • Content Management Strategies for Enterprise Content Platforms
  • Measuring The ROI On An HEVC Deployment
  • Best Practices for Search & Discovery in a Connected World
  • How To: Producing and Distributing HEVC
  • Replacing Flash: Adaptive Streaming and DRM in HTML5
  • Streaming Deployment Architectures in Higher Education
  • How To: Building Enhanced Video With JW Player
  • The Risks and Rewards of Employee-Generated Video
  • The Business of Delivering Multi-Platform Content
  • How To: Building a Streaming HTML5 Video Player
  • Using Media Optimization To Improve Streaming Performance
  • Building Streaming Apps: Choosing The Right Devices & Platforms
  • How To: Changing Viewer Engagement with Second Screen Experiences
  • Building an Immersive Online Television Experience

Thursday Webinar – Enterprise Video: On-Premises vs InThe Cloud

Thursday at 2pm ET, I’ll be moderating another StreamingMedia.com webinar, this time on the topic of, “Enterprise Video: On-Premises vs Cloud.” In this webinar, you’ll learn how enterprise IT organizations can make the right decisions about moving from on-premises video platforms to cloud based solutions. Hear why on-premises vs. cloud is not only about security but also relates to functionality and total cost of ownership. Learn about security controls that will scale to hybrid and public clouds, how smartphones and tablets are driving the consumption of video within the enterprise and get a thorough understanding of the technical requirements for securing a cloud based infrastructure.

In this webinar, speakers from MediaPlatform, Kaltura, VBrick and Qumu will also discuss:

  • How to choose your deployment wisely with new hybrid models
  • The importance of deployment flexibility
  • The benefits of the cloud for scalability, elasticity and redundancy
  • Practical tips for moving from on-premises to the cloud
  • How content is secured in a cloud environment
  • Using existing WAN for economical, bandwidth-efficient and secure video delivery

These questions and many more will be covered in this webinar so REGISTER NOW to join us for this FREE Web event.

WAN Optimization & CDN Provider Aryaka Carves Out A Niche To Address Enterprises’ Content Delivery Problems

Aryaka_logoAryaka Networks recently launched a new CDN in the market that is purpose built for B2B enterprise content delivery, something a bit different than what most CDNs are offering today. Those who track the CDN market have probably never heard of Aryaka before because the company focuses on cloud based WAN solutions that have been deployed across 3000 customer sites in 280 cities and 48 countries. Their value proposition is an enterprise grade private core network with a built-in proprietary software stack for WAN optimization and application acceleration.

Aryaka’s founder and CEO, Ajit Gupta earlier founded Speedera Networks, which was a acquired by Akamai for $130M in 2005. Aryaka brings together an integrated optimization platform for all applications, both, inside and outside the firewall with best practices from WAN Optimization and CDN to introduce a content and application acceleration solution on a private network. A CDN purpose built for the enterprise, which aims to be disruptive in the market.

As websites have become increasingly personalized and interactive, the percentage of dynamic content on the web continues to grow and replace static content as the predominant form of content on the web. In parallel, the push for applications into the cloud is becoming real. Enterprises are putting a web front end on CRM and ERP applications to make them easier to use from any browser worldwide. Providing acceptable user experience to these applications is critical to enhancing brand reputation and sales conversion capabilities.

Traditional CDNs from the late 1990s were built for a read-only web where they delivered faster load times by caching static content closer to the end users. Today, most traditional CDN vendors continue to serve static content along with streaming video while a handful have successfully expanded their products to offer dynamic site acceleration services through middle mile improvement with TCP optimization, intelligent routing, connection management, SSL acceleration and dynamic compression. These solutions are built over the public internet and congestion, packet loss, latency and jitter are common on intercontinental Internet links.

Aryaka says the global cache eviction policy of these CDN vendors dictates that B2B enterprise content suffers heavily as it competes with cache space that is monopolized by large content providers/aggregators whose content is more frequently accessed because of its popularity. This means that with many traditional CDNs, B2B enterprises don’t get the same level of value as content providers with popular content would get, which is why Aryaka is focusing on B2B enterprise content delivery with their new offering.

Aryaka’s says that one of their differentiators is that their CDN delivers content over a stable, low latency, private core network built for enterprise content delivery. Unlike traditional CDNs offering static caching or dynamic acceleration, the middle mile is both intelligent and private, a key differentiator. This ensures that customers are not subject to the vagaries of the public internet, especially across long distances. The private core offers consistency in performance compared to the Internet, no matter the time of day or traffic spikes due to global events. Aryaka’s customers use the private core network for data acceleration as well as real-time TCP and UDP based traffic.

Aryaka has also designed and implemented a multi-segment TCP optimization stack optimized differently across the first, middle and last mile – not just from the origin out to the edge like some other CDNs. The multi-segment stack is unique because it takes advantage of the underlying network characteristics which are known and predictable on the private middle mile. As a result, Aryaka says this results in superior bi-directional dynamic content acceleration for enterprise customers who continue to deploy web and IP based applications, as well as real-time data and APIs in their websites. With the growing importance of user generated content in the form of uploads, posts, forms, etc., Aryaka promises their customers a much faster experience for all traffic.

All of Aryaka’s customers get access to a multi-tier cache system providing what they say has better performance, significant gains for increasing cache hit rate and reduces connections to the origin.  This allows Aryaka’s CDN to achieve significant gains over other vendors in reducing the total load that the customer’s server needs to handle per user for both cached and dynamic content. For static content, Aryaka has implemented a large memory cache that enables faster content access compared to physical disks that are used by many CDN vendors. Each customer is assigned certain resources which ensures they are not affected by unfair cache eviction due to the popularity of other customers’ content even if delivered off the same server.

Aryaka’s CDN is purpose-built for the B2B enterprise and they are not focused on broadcasters and social media companies with large bandwidth and commodity delivery requirements. B2B enterprises do not have to fear competition for cache performance due to large video, social media or ecommerce and retail content that is popular and therefore more frequently accessed. Aryaka’s network has 25 globally distributed POPs in Tier 3+ datacenters worldwide and are strategically located just 30 milliseconds away from 90% of the world’s business users.

While some CDN players still continue to commoditize and crowd the low end of the industry, while trying to move up the value add services chain, it’s certainly interesting to see Aryaka Networks taking a different approach by going after customers at the high-end of the market. They have become an interesting company to keep an eye on and I’m watching them closely for when they share data points on how quickly they are growing their business and what their CDN deployments look like. You don’t hear the term B2B used often when talking about CDN services, and Aryaka is out in the market to prove that they have a unique and unparalleled understanding of specific customer needs for B2B enterprises when it comes to content delivery.