Comments on: Amazon S3: What would you like to know?

By: Duvar Kagidi

Duvar Kagidi — Sun, 02 Mar 2008 09:33:41 +0000

est il availible en Francais, my English not good

By: SmugBlog: Don MacAskill » Blog Archive » Amazon S3: Conferences (ETech, Web 2, MIX07)

Fri, 23 Mar 2007 02:37:34 +0000

[...] ETech is next week, and I’m doing a session called Scalability: Set Amazon’s Servers on Fire, Not Yours. I’m not paid by Amazon or anything (quite the contrary - I’m paying them a lot of money), so this will be a real look at them from someone in-the-trenches. It’ll mostly focus on S3, both business and technical aspects, but I’ll touch on EC2 and some of the others, too. (I had hoped to be using EC2 large-scale by this talk, but ironically, a hardware vendor supply issue is preventing me from rolling out that software). I’ll leave plenty of time for questions, and you can give me some in advance. I will post slides here. [...]

By: Sphinx

Sphinx — Tue, 06 Feb 2007 18:54:18 +0000

You tease! Will you mail them!? I’ll be here imagining a what a universe finally balanced in it’s principles will be like…

By: Peter

Peter — Tue, 06 Feb 2007 10:32:45 +0000

Hi Don,

Are you using the REST or SOAP interface (or both)?

If both, which one do you find faster and which more reliable (if available)?

If one particular interface, how did you decide? Any information would be great.

Have you found latency effects the way you consume the service (compared to local storage)? (I know its a vague question, but being in Australia Amazon S3 is not as luring when there is a 220ms+ overhead attached– damn you physics!)

Do you employ any smarts to guess where and what your clients are going to do in regards to prefetching/caching of S3 data?

How do you handle a failure when fetching non-cached S3 data? (this is after trying more than once and at different data centers etc.)

I believe you mentioned in a previous article/post that you are planning on peering (directly) with Amazon to reduce bandwidth costs. Assuming this is true, more questions!

On peering; Are Amazon going to reduce or eliminate the bandwidth cost component of S3 (as seen with EC2)? Has this been discussed?

On peering; Are you planning on peering directly with Amazon in their data center or at the closest peering point (an internet exchange for example)? Some other location?

On peering; Will you be peering at each of Amazon’s significant S3 data center locations?

On peering; Will you pay anything, to Amazon, for peer directly with them?

On peering; Are you currently peered with Amazon? (Had to ask!)

On peering; Would I be correct in assuming that the cost and usage benefit of S3 has been significant enough (both current and predicted) to warrant the extra overhead of a peering agreement with Amazon?

Unfortunately I won’t be attending ETech so if you could include some of the answers as a reply or in the slides I would be appreciative. Thanks!

By: cabbey

cabbey — Sat, 03 Feb 2007 17:46:43 +0000

I was thinking something realted to Julian’s question… clearly you’re pulling the data from S3 to SmugMug, then pushing it back out to the viewer… but why not have S3 push it directly to the viewer? It would seem to be a straight forward thing to toggle in the html generation, could even get fancy with a bit of js to figure out which of the S3 (and smugmug) locations is fastest for a given viewer to pull from. Was it purely a brand preservation thing? that you want smugmug.com in the url?

By: Julian

Julian — Sat, 03 Feb 2007 15:35:06 +0000

What I wanna know is if everything that is on S3 always has to be transferred to your servers and is then send to the users or if one happens to load images directly from amazon servers. And if it’s the latter, than how do you hide that there’s amazon behind.
A direct answer here would be nice.

By: onethumb

onethumb — Sat, 03 Feb 2007 05:42:49 +0000

@Anonymous:

I’m not getting a volume discount, or any special treatment, from Amazon. I’m billed the same rate everyone else is, straight through the AWS billing system.

If another company were to offer a similar service, I’d be extremely interested. Assuming the price and performance were comparable or better, it’d probably be a no brainer for us to use this as well.

A few other large companies may or may not have talked to me about things like this already.

By: Anonymous

Anonymous — Sat, 03 Feb 2007 05:39:58 +0000

Are you getting a volume discount from Amazon?

If another company were to offer a similar service with an S3-compatible API, perhaps positioned as being complimentary to S3 (extra redundancy, extra location(s), different peering relationships), how interested would you be?

Sorry for posting anonymously, but this is a potential future offering from my employer and they don’t like it when I post about unannounced products.

By: Jake

Jake — Sat, 03 Feb 2007 01:08:12 +0000

I’d love to know more about how you determine what to serve locally and what to serve out of Amazon S3, and what generalizations you can make from that.

By: onethumb

onethumb — Fri, 02 Feb 2007 23:54:04 +0000

@Sphinx:

I have one right here for you. Might even be talked into a signed copy of Wages of Sin, too.