Disney+ Tech Problems: What We’ve Learned and What Needs Fixing
With the launch of Disney+ being such a big deal for consumers and the streaming media industry, many have been giving their take on what caused the problems, why it happened, or want to point to poor planning on Disney’s part. But the fact is, the majority of those commenting don’t have any details on what Disney has built, don’t know what needs to be fixed, and have no experience themselves in building such services – at scale.
I’m not going to speculate myself as to what caused all of the different problems as I don’t know all the details. But from talking to some folks involved and looking at all of the different problems I’ve seen personally, or that have been reported by users, there are some things we do know.
Some want to immediately suggest that Disney, “doesn’t have enough servers”, or “isn’t doing load balancing right”, or “needs a multi-CDN strategy”. For starters, public data shows Disney has a multi-CDN strategy, using at least six CDNs for different parts of the workflow from vendors including Verizon Media, Akamai, Fastly, Limelight Networks, CenturyLink and AWS. Almost no one is complaining about the videos streaming, the problems are around non-streaming issues like account login, setting up profiles, apps freezing or crashing, media not able to be found and other user management issues.
Disney announced today that more they got more than 10M signups for Disney+ and even if 100% of them were all streaming at the same time, delivering 10M video streams across so many different CDNs would not be a problem. 10M simultaneous streams across multiple CDNs is a very low number so it’s not a “capacity” problem when it comes to video. Disney is also using services from AWS in at least three regions including us-east-1, us-west-2 and eu-west-1. So the idea that Disney isn’t load balancing the website or other parts of their platform is not true and suggesting Disney isn’t purchasing enough capacity or resources from cloud and CDN providers is simply not accurate. [side note: For all the media outlets that are reporting that Disney+ got more than 10M signups “on day one”, or “10M subscribers”, that’s not accurate. Subscribers were signing up for Disney+ months in advance and I have 2 accounts, but am only 1 “subscriber”. Hence why Disney correctly said “sign ups”, not subs.]
To build a platform like Disney+ there are more pieces in the backend than most are ever aware of. The website, authentication, billing, subscription management, user profiles, account personalization, error beacons, API calls, Swagger tools, support for Widevine and Fairplay, QoS monitoring by Conviva, metadata, transcoding, storage (of more than just video), plus all the delivery, playback, reporting and analytics, etc. and that’s only scratching the surface. All of these components have to talk to one another and some can have issues and not impact other parts of the workflow, while others pieces when they fail, cascade down the line causing other unintended problems. That’s the nature of the streaming media workflow when building such a complex platform.
While I expected there to be some issues on day one, I was disappointed in how widespread the problems were and the number of different issues viewers experienced. No doubt Disney Streaming Services is learning a lot from this, as have all of the OTT services that have launched in the market. But for those to suggest that Disney Streaming Services didn’t plan in advance, or didn’t test things first, that’s simply not accurate. You can do all the testing you want at scale in a controlled environment, but until you put real traffic over the platform, you can only know so much. No amount of pre-testing can replicate what will happen when a service goes live, which is why Disney+ was rolled out in the Netherlands so they could get some real-world feedback.
What problems Disney saw in the Netherlands (few were reported) and how much of an insight that gave them at scale, I don’t know. I also don’t know to what degree Disney tested each portion of the workflow separately or together, and at what scale. But we’ve seen everyone including HBO, WWE, Sling TV, AT&T, Hulu, Amazon Prime Video, CBS and now Disney all have problems, of varying degrees. It’s also worth noting that HBO had repeated problems with Game Of Thrones, which we now know from AT&T’s presentation last month, peaked at 5M simultaneous viewers. What this shows us is that it’s difficult and complex to build out a consumer video service, even when you are talking 5-10M subscribers.
While I don’t know why Disney made some of the decisions they did, I was surprised that they decided to push out the Disney+ app across all platforms, at the same time. Many video services in the past would allow users to download and login to the app before the service launched, so any authentication issues could be worked out in advance. That seemed like a risky approach, but maybe there was a good reason why Disney did it. I also found it odd that platforms like Roku and Fire TV were promoting the Disney+ app, but you couldn’t download the app as it wasn’t yet in the store since that takes time to propagate. We know the app can’t show up at once for everyone, but why not wait to showcase it on the main screen until it’s actually available for download? The very first experience I get with Disney+ is that the app can’t be found.
For me, the big failure on Disney’s part, was the lack of communication and setting customers’ expectations properly. Complaints where coming in for almost 4 hours before Disney put out a short statement on Twitter saying “The consumer demand for Disney+ has exceeded our high expectations. We are working to quickly resolve the current user issue. We appreciate your patience.” But they didn’t give a list or acknowledge which issues they were working on or an ETA as to when they would be resolved. Blaming customers for signing up for a service you ask them to buy is a really bad strategy. It also didn’t help that wait times with support would say 20 minutes, but hours later still you could not get through. Disney set expectations and then didn’t live up to them and saying on Twitter, “please note that due to the overwhelming response to Disney+, wait time may be high for customer service”, is a really poor reply. Especially since the support chat window says, “Please expect wait times to be longer than normal.” What is normal? It’s day one of the service.
As of now, Disney has 30 different error codes listed on their website, but yesterday when I was on their help page looking for specific ones, they didn’t come up. Today they do, but some are confusing as they give two very different responses, to the same error. Error code 9 says they can’t process my payment, but it also says it could be a not logged in error. Also, if you want to send Disney an email to let them know of a problem, you can’t type in your problem and only get to select from four options in a drop down menu, and none of them are related to logging in, or any of the issues people are having. And they ask you to select which device you are using, but Roku isn’t listed in the drop down menu and if I select Apple TV 4 as my device, why am I getting an option to say it’s connected via 3G, 4G and 5G? When you do send in a report of a technical issue, you get a pop up window telling you, “Unfortunately sometimes we just aren’t able to show or license a particular movie or show.” That has nothing to do with the subject of the message I sent in.
While it appears the service is now working for the majority of people and complaints on social media have died down from day one, there are a lot of little bugs that still need to be fixed. For instance, if I start playing a video and exit it, when I go back in, sometimes it won’t pick back up where I left off. Issues like this Disney is working to fix but since they haven’t published any kind of functionality spec of what the user experience supports, we also don’t know which features are having problems or simply are not supported in the current build. For instance, the second episode of the Mandalorian comes out on the 15th, but nowhere on the show page does it tell me that. How can that be missing? I see lots of things that need to be fixed across different platforms, for instance on my MacBook, every show I start is muted by default, but that’s not the case on my iPad, Amazon or other devices. If you are watching something episodic, it will start the next show in the series automatically, but it doesn’t tell you that something is coming “up next”. These are just a few of the user experience issues myself and others have documented.
Disney Streaming Services knows what needs to be fixed, what’s not working right and what they need to do to improve the user experience. I expect they will resolve all of these issues sooner than later and the next major build and release of Disney+ should include all kinds of new features and improvements. But what those will be and when they will come out we just don’t know. While Disney Streaming Services has been working on the Disney+ platform for a while leading up to launch, now their work really begins to make it a feature-rich and reliable service that can scale to tens of millions of users, with good quality and a great user experience. The hardest part is now over with the launch, but they still have a lot of work ahead of them.
For those on Wall Street that would like to get a better understanding of Akamai’s business, especially in the web performance and security markets, I’ll be hosting a private dinner on Thursday November 7th in NYC. I will be leading a discussion with Rick McConnell, President of Akamai and GM of the Web Division, along with Josh Shaul, VP Product, Web Security. The discussion will be centered around Akamai’s performance and security solutions detailing customer use cases, how these services are deployed from a technical standpoint, the value they offer to customers and short and long-term trends the company is seeing in the market.
