If you're building complex things... most likely you should be building simple things.
Most software in the world implements business rules defined by people who are way less smart than your average engineer. And yet our software is more complicated than ever.
^ THIS, 100%. Some SW engineers truly believe that complexity is the enemy, and that it can always be eliminated. Sometimes it can, but other times, you can watch developers push complexity around on their plate to avoid eating it.
They move logic to a different library, break it into 10 libraries, introduce layers of abstraction, try increasingly complicated View/Model architectures… And ultimately the code is no easier to understand or manage than when they started.
But complexity is the enemy. Where it can't be eliminated, it can be contained. That's what software engineering is about.
It is nearly impossible to find a complex domain that doesn't naturally break into smaller subdomains.
That's because "domain" is human concept. The ones that are too complex and don't have natural boundaries within them just don't stick.
There are two main reasons software becomes complicated:
1. It is over-engineered at the beginning of it's lifecycle.
2. It is under-engineered in the middle of it's lifecycle.
Interestingly, unit tests contribute to both of these.
For an engineer it's always hard to admit that your system is complex because you failed at engineering. It's much easier to claim that you're working in a complex domain: you get to shift blame and boost your ego at the same time.
I get it, and this is why a lot of SW engineers think they could solve any systemic issue in the world by breaking it down into sub-problems. Unfortunately it isn't true.
I'll give you a concrete example: AML (anti money laundering). If you build a fintech with zero dependencies, you will need to deal with this. It's not manufactured complexity—it's an authentically complex problem which stems from 1) the actual law and 2) the risk profile the business is willing to assume. Now, you can come up with clever ways to make the rules of your AML policy composable and easy to change, but the reality is that you will immediately hit an ocean of edge cases that require ever more nuanced mitigations. You can make keep subdividing and decomposing the problem, but you will end up with an architecture diagram which is (wait for it) complex. So yeah, you shouldn't over-engineer before you launch, but eventually, your AML system will take on a certain amount of unavoidable complexity.
Now try to do all of the above WITHOUT unit tests, and see how far you get.
I fail to see how unit tests are going to help. If rules are independent, overall complexity is low, even if there are thousands of rules. If rules are dependent, unit tests are useless.
> introduce layers of abstraction, try increasingly complicated View/Model architectures
> you can come up with clever ways to make the rules of your AML policy composable and easy to change
These are the exact design decisions that are typically accompanied by unit tests.
Those are the design decisions accompanied by developers who think complexity is the enemy and seek to "contain" it at all costs—even when it adds hierarchical/organizational complexity.
And yes, if we're being pedantic, an AML system would need unit and integration tests—but the question is whether or not tests as a whole are useful. So unless you're arguing that an AML system can be made so simple that no tests are required and it "fits in your head," let's agree that testing has its place.
> Those are the design decisions accompanied by developers who think complexity is the enemy and seek to "contain" it at all costs—even when it adds hierarchical/organizational complexity.
Not in my experience. Overengineering and unit-testing always come hand in hand, likely because both are the "enterprise" way of doing things.
> the question is whether or not tests as a whole are useful
Huh? Nobody ever asked this question.
Questions I'm always debating are:
1. Whether unit tests are useful (no in 99% of cases)
2. Whether more tests = better (no)
3. Whether you should invest in automated testing before manual becomes painful (no)
> unless you're arguing that an AML system can be made so simple that no tests are required and it "fits in your head,"
Some tests will be required. But yes, it should fit in your head, otherwise you cannot reason about it. Building AML system that you can't reason about is a road to failure.
It doesn't mean that you need to load implementation of every single rule in your head before you make any change. But you need to fit, for example, the idea of rule engine and enforce constraints on rules working together. If you build rules that can do anything with no constraints (depend on each other, override each other), then you're screwed. You're going to be playing whack-a-mole with your tests, and while it will feel productive, it's a death sentence.
If you're building simple things, tests can be overkill.
If you're building complex things, tests pay for themselves rather quickly.