We don't need to switch everyone over at the same time. For example we could start with young (more likely to be drunk and inexperienced?) or known-bad (traffic offenses) drivers where perhaps even sub-average autonomous vehicles could make a difference.
You’re right, I shouldn’t have said “At the same time” but the point still stands: your other comment was talking past the OP, not addressing their point.
You’re talking about it as a macro optimization problem while the OP was explaining a rational decision at the level of the individual.
