Let’s Encrypt does operate CT logs. I wrote a blog post about our current-generation logs at https://letsencrypt.org/2024/03/14/introducing-sunlight

Let’s Encrypt currently has a single primary with a handful of replicas, split across a primary and backup DC.

We’re in progress of adopting Vitess to shard into a handful of smaller instances, as our single big database is getting unwieldy.

Let’s Encrypt is an incredible project and the internet is better off for it. If you ever have questions about vitess or need help please let me know.

Thanks. Would love to see a tech blog post once you get Vitess implemented.

We’ve already started drafting it :)

I'm the technical lead for Let's Encrypt SRE.

Publishing more about our resilience engineering sounds like a great idea!

I'll get that on our blogging schedule for next year

Versity Gateway looks like a reasonable option here. I haven't personally used it, but I know some folks who say it performs pretty great as a "ZFS-backed S3" alternative.

https://github.com/versity/versitygw

Unlike other options like Garage or Minio, it doesn't have any clustering, replication, erasure coding, ...

Your S3 objects are just files on disk, and Versity exposes it. I gather it exists to provide an S3 interface on top of their other project (ScoutFS), but it seems like it should work on any old filesystem.

Versity is really promising. I got a chance to meet with Ben recently at the Super Computing conference in St. Louis and he was super chill about stuff. Big shout out to him.

He also mentioned that the minio-to-versity migration is a straight forward process. Apparently, you just read the data from mino's shadow filesystem and set it as an extended attribute in your file.

I really like what I've (just now) read about Versity. I like that they are thinking about large scale deployments with tape as the explicit cold-storage option. It really makes sense to me coming from an HPC background.

Thanks for posting this, as it's the first I've come across their work.

Garage also decide to not implement erasure coding.

I don't expect we'll ever remove the other validation methods, and certainly have no plans to do so.

There are pros and cons of various approaches.

Note that renewing certificates is generally exempt from rate limits: https://letsencrypt.org/docs/rate-limits/#limit-exemptions-f...

We are working on further improvements to our rate limits, including adding more automation to how we adjust them. We're not ready to announce that yet.

We wanted to get this post out as soon as we'd decided on a timeline so everyone's on the same page here.

We had (optional) safety training when I worked at Square, which included CPR training, going over office evacuation procedures, etc. One of the most fun parts was they had a virtual fire extinguisher training, which involved basically a video game fire extinguisher controller to put out a fire on a tv. It’s definitely not as exciting as using a real fire extinguisher training, but can also be done in an office meeting room.

I’m still upset they didn’t let us start fires in the meeting rooms to practice using real extinguishers.

Miss working with you!

Virtual training is better than nothing. But I don't think it really compares doing IRL.

The Register is adding very little on top of https://blog.cloudflare.com/detecting-cgn-to-reduce-collater...

Previously discussed (a bit) at https://news.ycombinator.com/item?id=45746509

I wonder if that's not very visible in Cloudflare's data because those mobile devices will likely use IPv6 to connect to Cloudflare-hosted sites.

That’s what I was thinking. Anyone coming from Cloudflare will end up getting there via IPv6.

I use cloudflare to make my weather station available over T-Mobile. They don’t filter inbound ipv6 on regular phone lines (they do for TMHI) so you can host a simple page on ipv6, only set the AAAA record in cloudflare, and they will proxy it for ipv4 users so I can ignore being CGNAT’d for ipv4. Make sure if you do this setup with a tool like ddclient to keep the record current as T-mobile rotates ipv6 frequently

Well, what I mean to say was “anyone coming from Verizon will get to Cloudflare via IPv6”.

My current endpoint lacks IPv6, so I use Cloudflare so IPv6 clients can get to it. Verizon’s IPv6 is noticeably faster than their IPv4 CG-NAT.

mobile devices dont get ip6 do they? last i looked my cheapo gateway only provided v4 cgnat

Phones have been on IPv6 for years.

https://blog.nelhage.com/post/regex-crosswords-z3/ was posted a few days ago here and is an interesting way to solve these

Yeah, that's how I found regexle, which eventually led me to this!