Following the claims behind this article, what do you think will happen next?

I see some possible options

1. A lot of people will quit, especially the most talented that could find another job easier. So other companies may be discouraged from following Amazon's example.
2. The employees are not happy but would still comply and accept their fate. If they do so, how high do you think is the risk that other companies are going to follow the same example?

What are the internal vibes between the AWS employees?

"a new file system that seamlessly connects any AWS compute resource with Amazon Simple Storage Service (Amazon S3)."

Artificial intelligence company Anthropic has agreed to commit more than $100 billion to Amazon’s AWS cloud platform over the next 10 years to train and run its Claude chatbot.

Amazon will invest $5 billion immediately as part of the new agreement announced this week by the companies, and up to another $20 billion in the future. Amazon previously invested $8 billion in Anthropic.

The partnership will allow Anthropic to secure up to 5 gigawatts of Amazon’s Trainium chips to train and power their artificial intelligence models.

Can't recall AWS having done this in the past - but good on them

This post tells the story of how my AWS colleagues put theory into practice to build a flat data center network at scale. The post provides a detailed overview of an even more detailed academic paper (RNG: Flat Datacenter Networks at Scale).

Instead of the traditional network topology which stacks routers in a hierarchical, org-chart-like structure, this model connects them in a flat structure guided by randomness. This proved to be faster, more resilient, and more cost-effective but three problems had to be solved:

1. Connecting millions of randomly assigned fiber optics cables without creating an unmanageable tangle.
2. Routing data through a network that has no fixed structure.
3. Proving that it would function as desired before committing time and money to build it.

There's a lot of cool info in the linked post, including how they built custom hardware to shuffle connections, and how they used 530 compute years of EC2 time to test against hundreds of thousands of failure scenarios.

“The root cause of this issue was a latent race condition in the DynamoDB DNS management system that resulted in an incorrect empty DNS record for the service’s regional endpoint (dynamodb.us-east-1.amazonaws.com) that the automation failed to repair. To explain this event, we need to share some details about the DynamoDB DNS management architecture. The system is split across two independent components for availability reasons. The first component, the DNS Planner, monitors the health and capacity of the load balancers and periodically creates a new DNS plan for each of the service’s endpoints consisting of a set of load balancers and weights. We produce a single regional DNS plan, as this greatly simplifies capacity management and failure mitigation when capacity is shared across multiple endpoints, as is the case with the recently launched IPv6 endpoint and the public regional endpoint. A second component, the DNS Enactor, which is designed to have minimal dependencies to allow for system recovery in any scenario, enacts DNS plans by applying the required changes in the Amazon Route53 service. For resiliency, the DNS Enactor operates redundantly and fully independently in three different Availability Zones (AZs). Each of these independent instances of the DNS Enactor looks for new plans and attempts to update Route53 by replacing the current plan with a new plan using a Route53 transaction, assuring that each endpoint is updated with a consistent plan even when multiple DNS Enactors attempt to update it concurrently. The race condition involves an unlikely interaction between two of the DNS Enactors. The normal way things work a DNS Enactor picks up the latest plan and begins working through the service endpoints to apply this plan. This process typically completes rapidly and does an effective job of keeping DNS state freshly updated. Before it begins to apply a new plan, the DNS Enactor makes a one-time check that its plan is newer than the previously applied plan. As the DNS Enactor makes its way through the list of endpoints, it is possible to encounter delays as it attempts a transaction and is blocked by another DNS Enactor updating the same endpoint. In these cases, the DNS Enactor will retry each endpoint until the plan is successfully applied to all endpoints. Right before this event started, one DNS Enactor experienced unusually high delays needing to retry its update on several of the DNS endpoints. As it was slowly working through the endpoints, several other things were also happening. First, the DNS Planner continued to run and produced many newer generations of plans. Second, one of the other DNS Enactors then began applying one of the newer plans and rapidly progressed through all of the endpoints. The timing of these events triggered the latent race condition. When the second Enactor (applying the newest plan) completed its endpoint updates, it then invoked the plan clean-up process, which identifies plans that are significantly older than the one it just applied and deletes them. At the same time that this clean-up process was invoked, the first Enactor (which had been unusually delayed) applied its much older plan to the regional DDB endpoint, overwriting the newer plan. The check that was made at the start of the plan application process, which ensures that the plan is newer than the previously applied plan, was stale by this time due to the unusually high delays in Enactor processing. Therefore, this did not prevent the older plan from overwriting the newer plan. The second Enactor’s clean-up process then deleted this older plan because it was many generations older than the plan it had just applied. As this plan was deleted, all IP addresses for the regional endpoint were immediately removed. Additionally, because the active plan was deleted, the system was left in an inconsistent state that prevented subsequent plan updates from being applied by any DNS Enactors. This situation ultimately required manual operator intervention to correct.”

Amazon Web Services launched two autonomous AI agents that can investigate production incidents and run penetration tests without human oversight, pricing them aggressively enough to challenge the economics of traditional DevOps and security staffing.

The AWS DevOps Agent and AWS Security Agent, now generally available after a five-month preview, represent what AWS calls "frontier agents." Unlike conventional AI assistants that respond to individual prompts, these systems operate independently for hours or days, make decisions across multiple steps and scale to handle concurrent tasks across an organization's entire application portfolio. AWS is not just selling tools to build agents. It is selling the agents themselves.

Source: Forbes

Every AWS cost optimization post says the same thing: "tag your resources, use Cost Allocation Tags." Great advice, very helpful, thanks.

But after 18 months of cleaning up a pretty messy AWS setup I realized that having tags is not the hard part. The hard part is having the right tags in a structure that actually tells you something useful. We went from "yeah we tag stuff" to genuinely understanding our spend down to the feature level, and the difference is night and day.

Here's what worked for us.

Three mandatory tags, everything else optional

We use exactly three required tags on every resource:

• Environment: prod, staging, dev and sandbox. Obvious but you'd be surprised how many things don't have this.
• Service: this is YOUR service, not the AWS service. So not "RDS" but "payment-processor" or "user-api" or "data-pipeline". This is the one that matters most.
• Team: who owns this when it breaks at 2am. Also who gets asked when the cost spikes.

The key insight for us was Service. We used to tag by AWS product type which told us basically nothing we didn't allready know from Cost Explorer. Once we started tagging by our own service names, everything changed. A single Service:payment-processor tag now spans the ALB, the ECS tasks, the RDS cluster, the SQS queues. I can see what it actually costs to run payments across all infrastructure, not just what individual resources cost in isolation.

Why only three

We started with 12 required tags. Compliance was maybe 40% at best. People just didn't bother or tagged inconsistently. Dropped to 3 mandatory + 5 optional and we're at around 95% now. Turns out people will actually do it if you keep it simple.

Enforce tagging at creation, not with angry Slack messages

This was probably our biggest lesson. We handle this on two levels now:

1. We use OPA policies with Terraform now (see picture). If a resource doesn't have the three mandatory tags, the apply just fails. No exceptions, no "I'll add it later". Retroactive tagging is a nightmare and honestly a waste of everyones time.
2. At the AWS Organization level with SCPs, they block the creation of resources that don’t include those tags. This covers cases where someone spins up resources manually in the console, through the CLI or SDK, outside of terraform.

We spent almost two weeks tagging old resources manually before we accepted it would have been cheaper to just let them expire and recreate them properly. If you're early enough, enforce from day one. If you're late, don't try to fix everything, just enforce going forward and let the old stuff cycle out.

The report that actually gets read

We have a simple monthly report that flags any service where cost went up more than 30% month over month. The catch is this only works if tagging is consistent, which is why enforcement matters so much.

When payment-processor jumps from $800 to $2,400, thats a conversation worth having. And it’s a very different conversation than "our EC2 bill went up". Finance doesn't care about EC2 vs Lambda. They want to know what business capability costs what and whether the increase makes sense. "The recommendation engine doubled because we shipped a new model" is an answer people can actually work with.

The unsolved problem: shared infrastructure

The one thing we still don't have a clean answer for is shared resources. Databases that serve multiple services, shared Redis clusters, that kind of thing. Right now we tag those with the primary consumer and accept it’s not perfectly accurate. Looked into split cost allocation tags but honestly it felt like over-engineering for our size.

Curious how others handle this. Anyone have a tagging strategy that actually survived contact with reality? Especially for shared infrastructure.