Being a largely bootstrapped indie studio, we've been using CloudFlare's reasonable Pro plan to cut costs. CloudFlare offers some great features, but is it ideally suited to be a high performance content delivery network (CDN) like our game requires? This is what I set out to find, and I wrote and open-sourced a utility to help me out:
Using this tool made it easy to setup a long-running benchmark to give us a good idea of performance -- say, over a 24-hour period. Our command looked something like:
node bench.js \ --files https://cloudflare.example.com/large,https://cloudfront.example.com/large \ --interval 120 --limit 720 \ --out results.csv
Oklahoma City Results
There isn't much that needs to be said after looking at those charts. CloudFlare performs admirably...for some of the day. The other half renders our game almost unusable on a fresh cache hit. I actually ran the test 3 different days because I thought we might have hit an anomaly, but the pattern is pretty consistent every day.
The large file was 428% faster on CloudFront, medium 131% and small 49%. The standard deviation is even more telling, at 1,285% higher on CloudFlare for the large, 167% for medium and 63% for small.
Amsterdam shows a significantly different story than stateside, but still swings in the favor of CloudFront. 435% faster for the large file, 488% medium and a whopping 561% on the small. The standard deviations were much less troubling in this region, but still noticeable at a 44% higher rate on the large with CloudFlare, 63% medium and 5,389% on the small (somewhat skewed by a few abnormally long load times).
This is by no means an exhaustive test, but the results have made it pretty clear that if we want raw speed and overall consistency, we need to go with CloudFront. CloudFlare certainly has its benefits, but just like with any part of your stack, you need to select the tool that is best suited for the task at hand.
We only ran the test comparing CloudFlare to CloudFront as we already have a lot of resources in S3, but we'd love to see others use the utility to benchmark other networks (available on NPM as well) and share the results.