You've got a point, the only problem is that we know when humans are more likely to do errors : stress , night time, long hours, high traffic, etc.

With AI we could enter a world where you'll have 0.0001% of having an accident, but this could happen anywhere anytime in any random situation (such as a specially shaped cloud in the sky).

This is what makes it unacceptable, IMHO.