Neither, they're talking about the culture of writing documents as a form of sharing ideas. Where other companies might use powerpoint presentations or unstructured meetings to brainstorm on ideas, Amazon instead encourages people to write a document summarizing their thoughts, and then there is a meeting where people silently read and comment on the document, and then afterwards discuss it.
That's an extremely sensible idea in multiple dimensions. It prioritises clarity of thought over rambling discussion in conference calls. I wonder if there's a feasible path to gradually steer an existing organisational structure in that direction.
> if there's a feasible path to gradually steer an existing organisational structure in that direction
The path I took was to just start doing it and expecting it for critical topics within my team (which was around 200-250 people when I started it). It takes several iterations to get good at it and the first few feel like it’s quite foreign and even wasteful. (It puts a lot more work on the author, by design.)
Eventually, it escaped just my group and (with support from others, including the CEO, who liked the process after seeing some of the documents that I or others shared with them) and is now fairly common in the corporate center for our standardized processes, though not nearly as standardized as I perceive Amazon to be in its use.
Basically: start doing it and stick to it for at least 5 complete cycles. Make sure that influential people (not necessarily org leaders) are public in their praise of well-constructed and effective documents.
From experience, yes / kind of; for the project I'm on right now, I introduced ADRs (https://adr.github.io/madr/) as a not-too-formal, but still formal way to talk about technology. This was after ten years of working in more hype-driven culture, where the choice was made by one person, or it was the tech du jour, or some guy spent a weekend playing with it so obviously we should integrate it.
It's a simple shift where people have to do their homework instead of just yeet something over the fence. You want to solve X? Present us with three options and we'll talk about it. You want to introduce Y? But we already use X to solve that, present us with new insights and the justification to spend the time on it.
It's far from perfect and it requires buy-in and (self) discipline, but it's still better than what happened before. Because some companies are still paying the debt of hype-driven development even though the people that introduced it are long gone.
- People leave 2 bullshit comments like "justification needed" for participation points and then go drink coffee instead of actually reading the doc or googling for said justification
- People waste inordinate amounts of time writing docs for things that could be discussed in 10 minutes
People taking calls from cars (and thinking it's OK in the first place) is exactly why we're going back to 5 days in office. People are simply taking too much advantage. You're being paid to work those 40 hours a week, not do whatever the hell you want to during the day and try to cram in work while you're driving from one errand to the next.
Why does any comment always assume the average American is commuting ludicrous distances each morning. People do this, but its very, very rare. Hyperbole is getting in the way of discussion.
1-hour one way commutes are NOT rare at all in Amazon's hub locations. Housing near the office is supremely expensive, school districts are also not the best, and most people have partners and need to live in a location that balances 2 peoples' commutes. On top of that, Amazon is such a big employer that they single-handedly make the traffic worse in at least Seattle.
Also, keep in mind that it isn't just the commute time. For a 1-hour commute I also need to prepare for the 1-hour commute, which includes making and packing up a lunch (because many offices have no cafeteria or no options that I'm able to eat), packing up my electric toothbrush and water flosser (because I need to brush 3 times a day for my braces and they won't let me leave shit at the office). Others need to deal with feeding pets, blah blah. For people with train commutes they need to deal with uncertainty in traffic just getting to the train station, so they have to leave an extra 15-20 minutes early and kill time waiting for the train on the platform because the next train won't come for a fucking hour (this is America), and leave time to line up to buy the goddamn parking ticket for the train station.
For the past 2 years people were able to roll out of bed and into a meeting, grabbing something from the fridge on the way. That's why stuff was efficient. Now we're going back to an inefficient world with the same high expectations of an effecient world.
Again, if we are talking about FANG HQs, anything in the Bay Area, Austin, or NYC then yea sure. I'm just suggesting that rhetorically this is a losing strategy because it's such an outlier. Most Americans don't work at the headquarter(s) of the most successful company on Earth.
Counting the time to get dressed and brush your teeth towards the office is comically absurd, in my opinion, even if I take your point. THAT SAID good luck with the braces I don't miss the constant brushing. That would make me want to WFH. Cheers
I haven’t seen a single instance of someone taking a call from their car since working remotely, likely because no one was commuting. It was fairly common prior to the pandemic and I was in office.
but why be so controlling? imagine someone is so busy they have to commute at 45 min commute in the 15 minute gap between meetings. so you’re refusal to be more understanding is blocking progress. likely it’ll be you removed from the meeting and going forward you’ll be included less. have fun being a low level IC!
lol you have too many meetings. I don't care if the manager with back to back meetings listens in from transit or something. im literally suggesting being more flexible
> People waste inordinate amounts of time writing docs for things that could be discussed in 10 minutes
I’m very much a writing kind of person when it comes to organizing my thoughts and I worked at a place where we did a ton of written documentation kind of like this. Then I left there and worked remote with a different company whose CEO, when I sent him an email, would pick up the phone and call me. We would then have the 10 minute conversation you’re talking about here. I came to love it because it’s true, a short focused conversation can be a huge timesaver. Since then, I’m often really frustrated by vendors and partners who steadfastly refuse to get on a call, when it’s very obvious to me that a quick call would be a far better use of time than endlessly going back and forth on email.
These are informal quick chats work nicely, until it doesn't, then it's a disaster and docs are needed.
With certain people I can work like that. With others that I don't trust, have seen don't shoddy work, don't communicate well, then write a doc and we'll discuss it.
The doc culture is great and I prefer meetings having the time upfront to get everyone up to speed. However, I regularly saw six page docs written for 25 line lambda functions.
Was it an important lambda function though? How often would it be called, would it still be around in 10 years, etc? If the six page document justified the existence of a separate deployable, then clearly it was important enough to warrant it. If you claim six pages was excessive, did the lambda even need to exist in the first place?
> - People waste inordinate amounts of time writing docs for things that could be discussed in 10 minutes
What about if someone wants to know what was discussed? You end up telling and retelling the same thing over and over again to get people in the loop, vs pointing to the doc and its remarks. (Depends on the subject of course)
Technical things (algorithms, locations of docker images, code, data, checkpoints, things that were tried but failed) should be documented.
Unfortunately most of the doc culture is people trying to convince each other within the same team that we need XYZ even when everyone with half a brain already knows it. Like "we need more GPUs to get shit done" should be a 10 minute call, not a PRFAQ.
Beyond the initial publication of the doc, the peer review process is much more sane than trying to review a bunch of power point slides. Similarly, it's much much easier to refer to a well written document when it comes time to implement or reevaluate an idea than going over some power point slides and maybe an associated recording, to say nothing about searchability, discoverability, and maintainability of an actual written document vs PowerPoint slides.
Also, idle side speculation: I wonder how much (if any) one of the underappreciated early employees @ Amazon had a hand in proselytizing this, given she (MacKenzie) is an author.
Oof, must suck to work for a company who doesn't use technical design docs well. I quite like Oxide's RFD model, based off Joyent's I assume, given who their CTO is. https://rfd.shared.oxide.computer/rfd/1
In my experience these documents were actually never good. I’ve never seen anyone ask for estimates either. Surely it’s better than some other companies but if it were good they wouldn’t need this absolutely horrific oncall
As a current Amazonian (and one that, as mentioned in my other comment, enjoys working at AWS largely because of interactions with brilliant tech minds and projects), I agree with most of your comment.
However...
>choose your team and manager very carefully and care a bit less about the company you join
I love my team, and even my organization that I work with. Multiple people on my team have stated that ours is the best team they've ever been part of in their career. But I don't love my company. I'm still at Amazon because even though my company is actively pushing me away, the love and enjoyment working with my team has been enough to get me to stay. So your advice here really strikes a chord with me, and I wish I could echo it.
Unfortunately, this advice isn't actually tenable, because no matter how great your team is, it's only one company leadership decision away from being ripped apart. I've watched this happen multiple times now, and this announcement is going to make it happen again. Caring less about your company just doesn't work when your company has shown multiple times that they are willing to throw away your team like that.
The problem isn't that you shouldn't care about your company, but that caring about your company is going to be far less important in your day to day.
And yes, your team is one decisions away from being ripped apart, or you are one manager change away from being very sad. I'm sure many of us have been there before: From top of a stack rank to bottom due to a manager change, with minimal in-team changes.
So you can try to care about your manager as little as you want, but the changes will happen to you eventually. Embrace that you are going to have to change teams or quit companies, because no love for your company is going to help.
If anything, what this should teach is to aim for a specific level of company growth: Grow too fast, and you might as well be at a different company in 8 months. Grow too slow (or shrink!) and there's no advancement, and it's all internal politics, as the L7 who has been here for 10 years is probably not leaving, because they know that nobody else would hire them at that level.
Everything is ephemeral though. Not just your team at work that you enjoy, or a team at any workplace that you enjoy, but everything. So don't worry about crossing that bridge until you come to it. There's no good situation that is a sure thing to continue indefinitely, so enjoy them while they're there and then be prepared to make moves if they end.
But also: working at AWS is genuinely really interesting at a technical level. Very few companies operate at the scale that AWS does, and being able to have technical documentation about the underlying workings of EC2 or IAM at your fingertips, or even just listen in on root cause discussions or technical analysis of incidents, or read the technical details of a new design in a service that saves hundreds of millions of dollars per month or day, is something that really scratches my engineer itch.
Amazon and AWS really have the potential to be a great place to work, but leadership just squanders it. That's what makes announcements like this even more painful.
I'll second this. You will learn a lot about operating at scale at Amazon. You'll learn many of the same things at another FAANG/hyperscale company, mind you, but they've all got their problems.
Both Amazon and Google are in their post-founder CEO phases. Microsoft went through the Balmer phase and found footing with Satya Nadella. Balmer probably set the stage for Nadella to succeed but sometimes changing the face is enough to reset culture a bit. Right now, Jassy and Pichai will never be seen in the same light as Bezos or Page & Brin. It might take another CEO at both Amazon and Google to unlock potential.
> We want to operate like the world’s largest startup
It's always amusing when a multi-decade-old, multi-hundred-billion-dollar company says stuff like this. You're not a startup. You never will be.
And if you were, you probably would actually offer perks in your offices that might actually encourage people to be there. Instead, the only perk that Amazon has is that you get one free coffee per day, and even that they have tried to remove multiple times.
I've never seen a company where it seemed more like the leadership of the company actively despises the employees that worked for them. Between stuff like this and the incessant pushing of Amazon Q against everyone's will, it's really apparent that Amazon execs think that having to employ humans as SDE is a defect they're trying to get rid of ASAP.
I like working there, but Amazon definitely has the worst in-office accomodations. No snacks, no free coffee, for-profit (like 4-5$ for a chocolate bar) vending machines on every second floor, no cafeterias in most building and when they have one it's a hole-in-the-wall that microwaves stuff (except Seattle).
In the original RTO email they even pointed the importance of employee spending money in the surrounding restaurants to support the downtime economy as if I should feel personally invested in spending 30$/meal on an overpriced burger for lunch.
Super tangential please accept my apology. Do you have any insight on why one of the biggest companies in the world can’t create a dark mode on their app after 10 years?
Haha no I do not, I work on an AWS AI service. If I could get some pull on other teams it would probably be the kindle one to fix the scribe syncing feature or the fire tv one to remove the god-awful ads.
They have these big dinner-style machines that you can use to brew a large batch of coffee and then put it in large thermos, but honestly the taste is pretty bad (but it is free!) and it requires you to babysit the process.
We also have free tea and hot cocoa.
In most places employees will also bring a Nespresso machine so you can bring your own pods which is somewhat better.
Writing all that I feel like it will come off as extremely entitled. I just want to stress that I personally don't mind much, but having worked for other tech companies, it's definitely at the bottom in terms of "free stuff".
I was an intern at Amazon in Seattle in 2018 and full-time from 2019-2021. Coffee was definitely free back then although someone had to brew it for the office.
I did it pretty often as it was a nice thing to do and a good way of meeting people who were waiting for the coffee.
They have repeatedly tried to remove the free coffee perk (usually by claiming that it was only intended as a temporary thing and will be removed at the end of the year) and the only reason it has been retained this far is because for multiple years running now there was an internal uproar about it.
I suspect at the end of this year they will fully remove it, uproar be damned.
That's surprising -- I would think it is obvious Amazon doesn't care about internal uproars, and there is nothing people can do about it. Otherwise the current 3 day RTO wouldn't have happened.
Not that we get great coffee here. It's one of this pod/packet machines. But, it does various teas and some cold drinks as well (fruit/herb infused stuff). So, it works for my second coffee, or a hot cocoa when it's cold outside.
We also get free lunch on T/Th, but that's a new thing since COVID (to entice people back), so I'm not counting on it being around forever. But for now, it is a nice perk and encourages some of that chit-chat the executive class tells us is so critical to making them more money.
Texas has a lot of incentives for residential solar. I'm not sure where you live, but in my DFW suburb, my neighborhood _is_ peppered in rooftop solar.
A lot of the incentives are from local power companies like Oncor, but one notable state-wide incentive is that solar installations are exempt from property tax by state law.
I dunno why people act like Texas is staunchly anti-renewable. TX state politicians have said some goofy stuff about "windmills freezing over", but overall Texans are extremely pro-wind and pro-solar. It's a huge economic driver for a large part of the state, and it's seen as an overall part of Texas's strong energy industry, complimentary to oil rather than as a competitor to it. George Bush and Rick Perry were both Republican governors but both were _very_ pro-renewable and oversaw massive booms in wind energy especially. In 2005 Texas (including Perry at the time) passed a law to invest billions of state dollars into building transmission lines specifically to make it feasible for renewable energy generation in west Texas to bring power to the populated areas in the east, which is attributed to the massive wind boom. Abbot, on the other hand, has sadly not been very pro-renewable, but much of the state still is.
Here just north of Austin, PEC implemented one of the most regressive solar programs in the country. Their argument in their study was that they make less money off solar since they can't sell as much power.
Regressive in what manner? Historically a lot of solar buy-back programs were incredibly inflated. Residential solar can be great for the resident but is usually not great for the grid. Paying resi. solar producers greater than market rates always felt foolish to me.
PEC never paid greater than market rates, they simply gave a net credit since solar was only reducing load at the service drop for a neighborhood. The cost to them was swapping the meter for a bidirectional one. Now at some point if enough people started using solar, they have the option of either curtailment (which would be automatically reflected in the existing meters), which at that point would justify an increase in fees, but their study was specific that they lowered solar compensation due to being able to sell less power to solar users (not for infrastructure reasons), which makes no sense why they are singling out solar since they offer incentives for other efficiency measures such as more efficient AC units to lower power consumption.
Regressive in that solar programs are not inflated, but do require distribution upgrades to realize their efficiency advantages over centralized power transmission. These distribution upgrades are costly to IOUs because they cut into their margins when the efficiency of distributed generation is considered.
Paying distributed generation export at retail rates or higher (DR, etc) makes plenty of sense because there are significant load, resiliency, and efficiency advantages to homeowners who are supposed to be the ones to benefit most from the grid.
The only change needed for solar users is a different meter swapped at the house that supports bidirectional metering. Solar power at the residential level only lowers overall demand in the neighborhood and on the grid, and in the very rare case where the net solar production exceeds the entire neighborhood's demand, PEC could choose to simply not use that excess (curtailment) and the meter at each person's house would accurately reflect that with no upgrades needed (Texan power utilities are not required to buy back excess solar). So the added cost to PEC is entirely optional. At its worst PEC was only crediting almost half of the power they were reselling from solar users (from originally a simple net credit), although they've thankfully been starting to backpeddle on that.
Honestly, I would prefer they simply charged the cost of swapping meters and adjusted the flat infrastructure fee for solar users (when necessary) for cases where upgrades are needed in neighborhoods with excess solar generation. Instead, PEC is able to resell solar power for a very significant profit with their current rates.
No. The 130 MW remaining capacity was the amount available "in SCED within 5 minutes", which in super simplified terms means "the amount of energy available quickly and economically".
The grid actually had ~4 GW spare capacity (according to the graph) if it was needed, but it wasn't part of SCED.
My understanding is that it's the latter. "in SCED" basically means they have pre-planned availability that is cheap.
The "Physical Response Capacity" in that graph is the amount of capacity actually available, but it's not part of SCED. However it doesn't say anything about the timeframe it would be available in. Given that ERCOT didn't call for conservation, I would have to assume it was capacity that was "quickly available, but not cheap" rather than "not quickly available", but I don't know for sure.
Being in SCED just means that the resource bid into the real time energy market (which clears every 5 mins), it does not necessarily mean that the resource is cheap to dispatch. The confusion here might be caused by the differences of the ancillary market (PRC) and the energy market (SCED).
> At the time we did not auto-scale any of our containerized services and were spending a lot of unnecessary money to keep services provisioned such that they could always handle peak load, even on nights and weekends when our traffic is much lower.
Huh? You've been running on AWS for how long and haven't been using auto scaling AT ALL? How was this not priority number one for the company to fix? You're just intentionally burning money at that point!
> While there is some support for auto-scaling on ECS, the Kubernetes ecosystem has robust open source offerings such as Keda for auto-scaling. In addition to simple triggers like CPU utilization, Keda supports scaling on the length of an AWS Simple Queue Service (SQS) queue as well as any custom metrics from Datadog.
ECS autoscaling is easy, and supports these things. Fair play if you just really wanted to use CNCF projects, but this just seems like you didn't really utilize your previous infrastructure very well.
The article brings up the two new LPs that were added, one being "Strive to be the world's best employer".
The biggest slap in the face about this LP is that the word "strive" used to be a bad word at Amazon. If you ever said you were going to "strive" to do something, you would be told that's not good enough. If you put it in a doc, you would be told to remove it. "Strive" was a weasel word. We never accepted "striving" as part of our tenets or our goals. Our goals never were to "strive to build a good service", our goal was just "build a good service"!
And then they went and made the new LP "Strive to be the world's best employer". Why isn't it just "Be the world's best employer"?
It may seem like a small thing, but it's an example of the ethos that has been chipped away inside Amazon. That particular LP has always been the butt of jokes because it's been clear over the last 5 years that leadership isn't actually serious about it, but the use of the weasel word "strive" really just put the cherry on top.
I mean, everybody knows LPs are BS when they put "Strive to be the world's best employer". Anyone who either works/worked at Amazon or knows someone working there understand what the work culture is like there.
And of course, everybody understands there is no way you can force people back into office -- many people commute over an hour a day -- and also be "best employer" at the same time. That's just obvious.
