Amazon AWS: Two Pizza Team Engineering Delivers Indigestion to Lots of People
October 20, 2025
No smart software. Just a dumb and quite old dinobaby.
Years ago an investment bank asked me to write a report about Amazon’s technical infrastructure. I had visited Amazon as part of a US government entity. Along with four colleagues from different agencies, I had an opportunity to ask about how Amazon’s infrastructure could be used as an online services platform. I did not get an answer, just marketing talk. One of the phrases stuck with me; to wit, “We use two pizza teams.”
The idea is that no technical project can involve more developers than two pizzas can feed. I was not sure if this was brilliant, smart assery, or an admission that Amazon was a “good enough” engineering organization.
I had a couple of other Amazon projects after that big tech study. One was to analyze Amazon’s patents for blockchain. Let me tell you. Those Amazon engineers were into cross chain methods and a number of dizzying engineering innovations. Where did that blockchain stuff go? To tell the truth, I don’t have many Amazon blockchain items lighting up my radar. Then I did a report for a law enforcement group interested in Amazon’s drone demonstration in Australia. The idea was that Amazon’s drone had image recognition. The demo showed the drone spotting a shark heading toward swimmers. The alert was sounded and the shark had to go find another lunch spot. What happened to that? I have no idea. Then … oh, well, you get the idea.
Amazon does technology which seems to be okay with leasing Kindle books and allowing third party resellers to push polo shirts. The Ring thing, the Alexa gizmo, and other Amazon initiatives like its mobile phone were not hitting home runs.
I read “Widespread Internet Outage Reported As Amazon Web Services Works on Issue.” [This is a Microsoft link. If it goes dead, don’t call me. Give Copilot a whirl.] Okay, order those pizzas. The write up reports:
The Amazon cloud computing company, which supports wide swaths of the publicly available internet, issued an update Monday just after 3 p.m. ET saying that the company continues to “observe recovery across all AWS services.” “We are in the process of validating a fix,” AWS added, referring to a specific problem set off by the connectivity issue announced shortly after 3 a.m. Eastern Time.
Okay, that’s 12 hours and counting.
I want to point out that the two-pizza approach to engineering is cute. The reality is that AWS is vulnerable. The outage may be a result of engineering flubs. You are familiar with those. The company says, “An intern entered a invalid command.” The outage may be a result of Amazon’s giant and almost unmanageable archipelago of servers, services, software, and systems was hacked by a bad actor. Maybe it was one of those 1,000 bad actors who took out Microsoft a couple of years ago? Maybe it was a customer who grew frustrated with inexplicable fees and charges? Maybe it was a problem caused by an upstream or downstream vendor? One thing is sure: It will take more than a two pizza team to remediate and prevent the failure from happening again.
In that first report for the California money guys, I made one point: The AWS system will fail and no one will know exactly what went wrong.
Two pizza engineering is a Groucho Marx type of quip. Now we know what one gets: Digital food poisoning.
Stephen E Arnold, October 20, 2025 at 530 pm US Eastern
Comments
Got something to say?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
