IP reputation is a gamble, and there is no recourse. If you're lucky, awesome. But if you're unlucky and switching host isn't an option, you pretty much have to involve a large third party to act your behalf - there is zero appetite in the industry for interacting with individuals.
The best solution I've been able to find is to self-host /almost/ everything, but route outgoing mail through Amazon SES.
The pricing for vanity email volumes is negligible (a few cents a year), and they have people whose full time job is wrangling IP reputation / Office 365 / etc.
This setup has survived several ISP/hosting switches; at times when I am lucky with IP reputation I route only mail going to Office 365 recipients via SES and deliver the rest directly; at times when I am less lucky, everything goes via SES.
If there was a much larger list of problem destinations I'd maybe do something nicer involving separate routers and a domainlist, but those cover all the cases that are broken right now.
Interestingly, I've not had a problem delivering directly to those (except the time I switched to an IP block with a bad rep and couldn't deliver anything anywhere directly at all); it's just the ones on the list above that don't like me.
Mysterious and ineffable are the ways of Microsoft.
(note that their MX record is usually a *.protection.outlook.com entry regardless of the custom domain, so I'd use that to bootstrap a rule if I had a more general problem with Microsoft)
Yes, you do need to include:amazonses.com in your SPF. Amazon aren't too bad at kicking spammers off SES promptly. More importantly, Amazon doesn't sign for DKIM - your server still does that; so no-one else gets to DKIM for you; and you can set the DMARC policy to require both.
SES currently charges $0.10 per 1000 outbound emails. The first 3000 mails are free. I received my first official bill for $0.02 after around two years of use.
Do investigate other relay services. I only stopped at SES because I was in a mad rush and it was the first one I tried that did everything I needed, without bouncing or getting filed to trash on any services I cared about. I have done nothing like a full survey of the market, and there may well be a better option. It is the general approach I am suggesting, not trying to shill SES specifically despite what it may look like.
i didn't assume that. obviously you can only talk about the one that you are using, and while the general setup applies to other such services, i can now file SES as an option that works. and with that price point i am probably going to be to lazy to look for alternatives. (although i should check if the email service i am already paying can do that too without requiring me to send all emails through them)
You can usually switch host. Some have better IP reputations than others.
There are quite a few other providers of email forwarding services, although I might look at SES myself if its that cheap as I have issues with hotmail (I seem to be OK with most mail to email on MS hosted email on other domains, oddly enough).
...it took OP 8 months of "rolling the gacha" and waiting to get a clean IP; no mention of costs. Not really a solution in my book. If you're willing to wait 8 months for working email, I put it to you you're actually using some other provider for your life and the thing you are playing with is a toy.
I've been self-hosting my email for a pretty long time. I first started down the reputation rabbit hole when a provider decided to shut up shop after a decade of operation, causing me to lose my lovely fixed IP block with its decade-old clean rep. Waiting/playing around isn't really an option when your email is broken and you need it working /today/ because it's not a throwaway toy - your digital life is tied to it.
Still, as I said at the start, if you get lucky, awesome for you.
If cost is not an issue one can run standby servers in multiple locations and have backups to all of them. Just as MX records allow for multiple inbound servers one can have multiple outbound servers as well. Park a few unused or vanity domains on them and have cronjobs send automated emails to yourself. I reply to those emails so the likes of Gmail see interaction between them. With time all IP addresses get good reputation.
An IP laundering service certainly sounds like a potential startup opportunity. Certainly I'd have paid for a proven good IP in the past before I developed my current solution.
...as disconnected from "email marketing services" as possible, please, because IME gmail is wise to those and files email associated with them directly in the trash regardless of all other concerns.
I suspect the reason SES is an exception is because it is very widely used for things like e-tickets, transaction confirmations and so on, and also goes to a nonzero amount of trouble to dissuade marketers rather than having them as the main customers.
> ..it took OP 8 months of "rolling the gacha" and waiting to get a clean IP; no mention of costs.
I dont see anything about it taking the OP 8 months to get a clean IP? They were on Hetzner, and can presumably keep making new VM's for a while until they get a clean one. Hetzner bills based on hours used, so I imagine that total cost would be quite low.
Nope. The biggest impact on gmail was making sure I had DMARC, DKIM and SPF all set up correctly.
(I tried several other relay services like mailgun and those /did/ have noticeable impact - SES was the first one I tried that didn't, so I stuck with it).
The best solution I've been able to find is to self-host /almost/ everything, but route outgoing mail through Amazon SES.
The pricing for vanity email volumes is negligible (a few cents a year), and they have people whose full time job is wrangling IP reputation / Office 365 / etc.
This setup has survived several ISP/hosting switches; at times when I am lucky with IP reputation I route only mail going to Office 365 recipients via SES and deliver the rest directly; at times when I am less lucky, everything goes via SES.