How Is CDNs Network Performance For Streaming Measured?
No matter how big or small a content delivery network is, they all talk about the performance of their network. Small regional service providers, global CDNs, P2P providers etc… all market the value in everything they do regarding how good their network performs. The problem is, the more customers I ask, the more of them say they
don’t know how to compare one network to another for video streaming – and neither do I.
Aside from the network or servers being down completely and not being able to get the content at all, how does a customer see and more importantly measure the performance amongst any network? I’ve asked a lot of the CDNs and they all speak to Gomez, which is considered the industry standard amongst the networks, but Gomez doesn’t support streaming. It seems that most use Gomez to show network performance for delivering small cache objects and the market just assumes that any content, static or streaming, must see the same kind of performance. But as any network knows, delivering and scaling the delivery of small static images via HTTP is very different than delivering and scaling a 300Kbps stream via RTSP or RTMP.
Keynote provides a service to check your files on a particular network, but that service does not compare it from one network to another and it’s not real-world in the sense that Keynote does not have thousands of agents checking the files as if you have many viewers from all over the world testing the content. Keynote has only a handful of locations that tests the content which is evident by the fact that every delivery network in the Windows Media Certification program for hosting has been given an A+ ranking from Keynote, no matter the size of their network or the number of servers they have. I’m not knocking Keynote’s service, but it’s just not a real-world test if you don’t have thousands of clients all over the world, on many different networks, accessing the content.
I’ve asked many of the CDNs about any tools on the market or any products they have built that customers have access to, but so far, those CDNs I have asked have agreed that there is no easy way to really compare streaming media network performance. If that is the case, what options do customers really have? What is the solution to solve this problem in the market?
If you are a customer of a delivery network, how do you judge performance and what does performance mean to you? For the CDN vendors, what data do you share with customers that gives them an insight into how your network performs? Is there anything you show them that is specific to streaming media delivery?