Speaking of Bandwidth
About a dozen EP podcast listeners wrote to me yesterday asking about slow downloads. Well, here's the scoop. It seems that EP is right on the edge of bandwidth constraints from BlueHost. Yes, they say "unlimited" file transfer, but to be realistic you can't expect that for a few bucks a month. There are a couple of interim fixes with Bluehost, then a much more complicated and expensive long-term fix of switching providers to one that offers dedicated servers, something I hope to put off for a while.
EP has been with BlueHost since the beginning, almost four years. Initially, as EP grew the quantity of bandwidth BlueHost allowed also grew. I'd been hoping that they might eventually get into the dedicated server business so that I wouldn't ever have to worry about switching to a new provider, but this morning I was told that they almost certainly won't — it's a very different market, with different margins, much more labor intensive. They like doing what they're doing and they're extremely good at it. I've been completely happy with BlueHost and, to be honest, I dread moving EP somewhere else.
One thing I can do with BlueHost, something that I don't think was available the last time I talked with them about expansion options, is to move EP to a high CPU server. That will allow more connections and should relieve some, but not all, of the constraints we're experiencing. I've put in the request and we've scheduled the move for this Sunday night. It will take the site down for most of the night, hopefully less. The high CPU server costs more, but $19 per month is not that much more. Very affordable.
Another thing I will do (once I locate the code) is to remove the streaming option for podcasts. Streaming creates bottlenecks such that, for example, a larger number of visitors to EP will be limited to streaming a smaller overall quantity of bandwidth than that which a smaller number of visitors would be able to use when the latter, instead of streaming, are downloading directly. Indeed, this is what appears to have happened yesterday. It seems a bit counter-intuitive but it makes sense, because the higher number of visitors chokes the CPU.
A third thing is to drop my bit-rate encoding back down to 96 from the 160 kbs I'd been testing the last two Fridays. Indeed, boosting the bit-rate unintentionally helped reveal the underlying incipient bandwidth constraint. Reluctantly, very reluctantly, I'll drop bit-rates back down — I've already swapped out yesterday's show for the same thing in the lighter 96 kbs flavor.
A further possibility, which would be extremely difficult and would create its own set of brand new problems, is to open a couple of additional BlueHost accounts, each on a different high CPU server, and spread podcast downloads between them. BlueHost tells me that's doable and they'll help, if asked.
In short, I expect we'll still have sporadic problems but some combination of the above should allow us to continue in reasonable shape while we investigate possible solutions.
The big question is what then. Ultimately we need to switch to a different provider that offers dedicated servers with bandwidth options that can handle high bursts of demand. It's both a logistical problem and an economic one. Logistically it's something of a nightmare, but I assume that I can hire someone, or some company, that can make the switch smoothly. In cost terms, there's the cost of the actual switch and then the expensive cost of dedicated server service. Based on some research today, the latter could run approximately ten to twenty five times what it costs for an account at BlueHost. Ouch!
[Update: I've identified what looks like a truly excellent hosting company, WiredTree, in Chicago, that offers a couple different options appropriate for EP in the $200-250 per month range. Also, WiredTree will move EP for free!]
To be honest, I'm not sure I can afford that. And at current levels the very kind donations to EP would not cover it either.
We seem to be somewhat stuck, but we have some time to think about what to do. If, in fact, it turns out to be easier to migrate EP than I've feared, I may even try to do it before New Year's.
This definitely would mean one thing, though: I'll be bugging you all for more donations. ☺
Posted by George Kenney on October 17, 2009 4:23 AM 
« How Fast Is That? | Main | EP Downtime »
Comments
I'd like to suggest that EP listeners take this opportunity to try a different browser. I recently started experimenting with Google Chrome and have been amazed at the improvement in speed. (I promise that I have no commercial tie to this product or company).
Using Chrome, EP podcasts begin playing in around 15 seconds as opposed to 15 minutes with IE8. I believe this is because Chrome starts playing before the download is complete, so it may be closer to a download/streaming hybrid than a solution to those that want to load the podcast to a portable device. For those listening on a computer, it is definitely worth a try.
Thanks for everything, George!
Posted by: TJ | October 17, 2009 11:43 AM
I have found, on a few occasions, that the podcast downloads slowly. I use iTunes to download so I can listen while I work at my convenience.
My consistently successful solution is to pause the process and try later. No big deal. I see that you have all this figured way past my ability to help. Your suggested adjustments (including the lack of a streaming option) sound appropriate.
It's great to hear your work, George and keep it up.
And thank you.
Posted by: Peter | October 17, 2009 4:37 PM
George, I enjoy your interviews and commentary a great deal and have recommended your podcast to a number of friends.
The lower bit-rate for the podcasts is just fine. For this sort of material, high fidelity is not necessary.
Thanks for your good work.
Posted by: Kevin Jones | October 17, 2009 5:46 PM
I cannot think of any solution other than Bittorrent. It would provide EP with truly unlimited bandwidth for free.
Posted by: Jean-Yves Coupal | October 17, 2009 10:50 PM
Seconded on the BitTorrent.
The torrent goes fast and it can be then used to provide the content to someone not using DSL or high speed transfer. Also it can be posted to a wider audience.
I regularly use One Big Torrent (which is politically oriented) for its content: new documentaries and other content not available any other way. The site was originally dedicated to writings and speeches of Noam Chomsky. You will see an illustration of him on the masthead:
https://onebigtorrent.org/
[No, sorry, no BitTorrent. g.]
Posted by: Lon C Ponschock | October 18, 2009 5:34 PM
"No, sorry, no BitTorrent. g."
Why not? Please tell me it's not because you think it's illegal.
[No, that's not the reason. It's because I want to be able to better guarantee podcast delivery. g.]
Posted by: Michael | October 20, 2009 3:02 PM
Bittorrent isn't a bad idea but inconveniences users (torrent software required). You could look into hexagon.cc — that is a new and different take on torrents.
A dedicated server is most likely overkill (go for a VPS first, I'd say, without seeing your stats) but both, even if managed, requires at least a basic familiarity of linux to keep everything just so — this means not only more monthly expense for the server, but also more time for server maintenance (or more money to hire an admin).
I'd recommend staying with bluehost for your webhosting, but serve the actual podcasts either from Amazon S3 or a similar service (content delivery network). This keeps your actual workload to a minimum and should work out to be less expensive than even a VPS.
Finally, if you want to try to eke out more time with bluehost, I would suggest compressing podcasts a bit more heavily to reduce the size — yes, might not sound quite as good on a high-end system, but for the average listener the difference would probably be barely noticeable. I actually think you could lose half the size (conservatively) and keep audio quality almost identical.
Perhaps test listener reception on an upcoming podcast by offering a low bandwidth (high compression) as well as a high bandwidth (low compression) version and get some feedback as to which listeners prefer.
Good luck — keep up the good work, George!
[Thanks very much for your comments and suggestions, John. You're right that Amazon S3 would be an inexpensive solution. I hesitate, however, to join the cloud computing world for philosophical reasons and it seems moreover, Googling S3, that there's a practical question also insofar as S3's downtime is not enterprise class, at least not yet. As to the possibility of greater compression — listeners have been suggesting that to me for years but I prefer the higher bit rate where possible. g.]
Posted by: John N. | October 23, 2009 11:53 PM