Over time this is likely true, but the reality of the top-end of sport is that tiny differences in tracks/environments have a big impact. So if your goal is to find the best out of a group of people at a point in time, I’d suggest the only fair way to do it is to pit them against each other at the same event.

For an example of this, see how the Paris Olympic pool was considered a slow pool due to its depth. Or how wind impacts running times, and can’t always be fairly accounted for (e.g. gusts).

There’s also the phenomenon where athletes perform better when competing against someone vs competing by themselves.