Hey folks - I wrote the blog post & the unicorn patch. If you've got questions, I'm watching the thread.
This isn't a magic bullet that'll be good for everyone. Since we're processing money, we prefer to make people wait instead of returning them an error when busy, so we opt towards larger queue lengths instead of erroring early.
That increases the chance a user's been around long enough to get disgruntled and hit refresh, which is where this patch shines.
Really interesting (although really unique) issue, and solid solution. Obviously this greatly differs for various use cases. In the news industry we see similar behavior at peak traffic, although not at this degree, and for the most part caching deals with this issue.
Makes sense, and you're right that it's a unique issue.
In flash sales our page caching solution becomes less than useful, because every sale invalidates the page cache (in case their shop template changes to reflect quantity). We cache individual records as well with our open-sourced https://github.com/Shopify/identity_cache gem but still have to render the liquid template.
I'd have to imagine that you're doing more granular caching... perhaps of queries or widgets? I don't mean to say it's uniqueness makes it irrelevant for the rest of us. Nice writeup that can teach us all something either way.
This is quite clever. But why wouldn't you want this to be the default behavior and have a configuration option to disable it? Are there more situations in which you really want to process a request even when the client can't get the response than the opposite?
Unicorn processes millions of requests per minutes for some very large companies. Changing the default actions of a core part of the web stack in a version bump could have terrible consequences for some use cases.
You're probably right that this is ideal default configuration, and maybe in the future it will be defaulted to on.
Right now though the best thing would get people who experience this type of problem to try the configuration change and report their findings.
I don't disagree at all with the decision to keep the standard behavior as default for now. For something as critical as an HTTP application server you definitely don't want any surprises.
I just wanted to get your thoughts on the tradeoffs and what you think is appropriate for the majority of use-cases.
If, under load, you can gracefully return an error and your user will just come back later (like the twitter fail whale), then you might as well keep your queue lengths short which close the window where a waiting user will hit refresh.
If you have some requirement that prevents the above, and have large queue lengths, then this may be very useful for you.
As well, some server configurations might not like getting the first part of the HTTP response much before the rest of it. Nginx will just buffer it all before returning the entire response to the client so it doesn't affect packetization of the response sent back to the end-user.
This isn't a magic bullet that'll be good for everyone. Since we're processing money, we prefer to make people wait instead of returning them an error when busy, so we opt towards larger queue lengths instead of erroring early.
That increases the chance a user's been around long enough to get disgruntled and hit refresh, which is where this patch shines.