Key takeaway is that people don't consider it ( the three R's) an ordered list in terms of preference. What made this most clear to me was a full list of 5 R's.
Warning this is rando internet detective work so YMMV JustJake at twitter would likely know the actual answer to this question.
It was mentioned that the service was artifact registery[1]. They were 5000/10min and approved at 18k/min [2] The ui appears to show a limit of 100 but not what of [3].
So reading the docs on artifact registery quota docs the only things that stands out is the "1 repository creation or deletion operation every 2 seconds."[4] so doing the math that is a rate limit of 30 per minute or 300 per 10 min so aroudn the ballpark of what they are. Looking at my own project this matches up and I can add request new quotas[4] but it needs to go through a human on googles end to approve.
Anyhoo if they were approved for 18k/min that is nearly 600x the normal quota. So they were likely (pure speculation) caught up in some kind of dragnet of quotas that were well beyond the norm. Skimming the site[6] it appears to be a rapid cloud based ide / deployment thing and I can't imagine how they are creating so many repos unless for each customer build they pass along to gcp's artifact registery 1-1. I use artifactory repo as well for managing docker images but mostly just make new tags off a handful of repos and am not typically creating them at such scale.
Sucks that they had the quota lowered without warning but "if" they were operating at the speculated quota that is flying pretty close to the sun.
I couldn't find it anywhere on the docs or in a few google searches. What is a porter? and how does it differ from a developer?
I understand the general concept of porting (ie https://en.wikipedia.org/wiki/Porting) but never heard of it as a specialization. Also what might people be porting from? If this intended as a migration guide.
On "how we got here" you use "bulleted list" rather than "numbered list". This is important as "If you rearrange the items in a bulleted list, the list's meaning does not change. "
Looking at the status page vs the history. They list the Messaging as being the affected service out of 10 other services. Seems like a pretty lynchpin service and also the one affected the most just eyeballing the last few incidences in history.
Background: Someone who has worked in the old Microsoft Analysis Services stack 8 years ago and has moved to more columnar data formats like parquet or row based MPP's.
Transition plan I would say is find the dataset that is exploding in size or complexity and start your POC there. I did customer service datasets on OLAP so it only grew at a pretty small scale and the data model didn't change that much. So OLAP was fine except for the fact that nobody else knew how to maintain it.
The main growing pain is find what your front end developer flow will be. It will be the same governance as the OLAP but more democratized so be ready to make your back end more front end. For MSAS it was excel but more modern systems are also more wide open. The article suggests just SQL but that can get out of control. How do you reduce reinventing the wheel etc? How do you prevent a lineage mess of derivative on top of derivative if you give users write access. Etc. IMO Tableau is a great product that allows the OLAP like exploration but can use SQL as an input. Just make sure people get the sql behind under some kind of source control and governance.
From the data model perspective it I think the main difference is make the tables wider and "pre join" in your immutable dimensions with higher carnality (ie customer). Just be careful of highly mutable data and keep those in separate tables because it is very painful to rewrite columnar data. Ie if you partition by date to update a single record you rewrite the entire date.
(About governance) I mean more passive governance not gatekeeping. Pretend each end user and dataset costs you money. How do you track them passively with some thin yet easily trackable logging? Business Unit and unique Job are bare minimums.
