ETech 2007 SmugMug Amazon Slides are Up!
My slides from ETech 2007 about Amazon’s Web Services, especially S3, are up in PDF form.
Holler if something isn’t clear, but hopefully this’ll give anyone who couldn’t make it some good insight into what works and what doesn’t with S3 here at SmugMug.
Enjoy!
March 30th, 2007 at 5:14 pm
[...] ETech is next week, and I’m doing a session called Scalability: Set Amazon’s Servers on Fire, Not Yours. I’m not paid by Amazon or anything (quite the contrary – I’m paying them a lot of money), so this will be a real look at them from someone in-the-trenches. It’ll mostly focus on S3, both business and technical aspects, but I’ll touch on EC2 and some of the others, too. (I had hoped to be using EC2 large-scale by this talk, but ironically, a hardware vendor supply issue is preventing me from rolling out that software). I’ll leave plenty of time for questions, and you can give me some in advance. I will post slides here. [...]
March 30th, 2007 at 5:15 pm
[...] UPDATE: Slides from ETech 2007 are up. tags: etech, presentation, amazon s3 [...]
March 31st, 2007 at 1:14 pm
Thanks so much for posting those. It’s great to have info like that shared around.
April 1st, 2007 at 11:22 am
Hi Don,
Great presentation, I enjoyed your slides. I wish I could have seen itylive. Are you planning on using EC2 for I2E image processing, or do you have something else in mind?
Thanks
rufus
http://www.rufusshaw.smugmug.com
April 1st, 2007 at 3:37 pm
Hi, like the presentation!
But one point: the big drawback of proxying, instead of redirects, is that you pay for the data transfer 3 times, instead of 1, right? (out of S3, into your datacenter, and out of it). Isn’t that a strong reason to use redirects? Why still use the proxying?
Another question: Do you still have all your own data as well, or is there data that’s only in S3? (do you trust them that much?).
3th question: what cURL library do you use? The PHP internal one?
Thnx!
April 1st, 2007 at 4:06 pm
@PanMan:
Yes, that’s a big drawback to proxying – if you’re actually proxying lots of your data.
But our “hot” data, the data that’s accessed most of the time, is stored locally at SmugMug, so we’re only doing proxy reads in case we don’t have the data locally or we have an internal failure somewhere.
So for the vast majority of data, we write it once to S3 and then we’re done.
Make sense?
So the answer to your other question is yes, 90% of our data lives at S3, but not in our own datacenter now. We do trust them that much.
Finally, we use libcurl.
April 2nd, 2007 at 12:32 am
Thanks for the explanation: that makes sense.
April 2nd, 2007 at 1:19 pm
Don, interesting presentation. How are you abusing libcurl to get added speed?
April 30th, 2007 at 11:18 am
[...] UPDATE 4/30/07: I’ve updated these numbers since this post was written in November 2006. It’s now been 12 months and we’ve saved almost exactly $1M. You can see the numbers in my ETech slides. [...]
June 21st, 2007 at 2:55 pm
[...] Amazon Web Services and the lack of a SLA Interesting comments from Greg Linden and Doug Kaye where they take differing views and I think they are both right. Those SLAs suck if you are trying to run a business that is succesful – +1 to Greg. The economics of EC2 make it a very cheap way of boot strapping an idea. The probability is that the idea will fail any way. +1 to Doug. SmugMug pdf from Greg [...]
September 13th, 2007 at 8:58 am
[...] But the money part is important too. SmugMug CEO Don MacAskill reiterated he’s saved $692,000 using AWS for his photo-sharing startup. He said that figure was as of April of this year and would be higher if he recalculated. [...]
February 2nd, 2008 at 5:04 pm
Your slides won’t open for me. It says it’s unsupported or damaged. I tried downloading it a few times to make sure it wasn’t just a corruption during download. Could you email it to me. Thanks.
~ mel
February 20th, 2008 at 7:50 pm
[...] AWS outage comes a link to a great presentation on how to successfully use AWS from SmugMug founder Don MacAskill given at ETech last [...]
March 5th, 2008 at 6:39 am
[...] a successful photography community, saved an estimated $2 $1 Million dollars (edit per this link) in 2007 by using Amazon S3 cloud storage. Organizations and individuals can benefit from mass [...]
December 12th, 2008 at 11:25 am
[...] multiple machines with redundancy, and so on. Looking at that solution lead me to smugFS. And to Don MacAskill’s presentation on Scalability. Crap! I said. I don’t need to host the flatfile’s at our [...]
January 18th, 2009 at 7:38 pm
[...] económico y confiable es Sumgmug, un servicio pago de alojamiento de fotografías tipo Flickr que ha reportado ahorros de un millón de dólares al delegar su almacenamiento a Amazon [...]