I don't buy that you're actually examining compiled programs. Very few people do. Theoretically you could, but the whole point of the compiler is to find optimizations that you wouldn't think of yourself.
The point of an optimizing compiler is to find optimizations which, crucially, are semantics-preserving. This is the contract that we have with compilers, is the reason that we trust them to transform our code, and is the reason why people get up in arms every time some C compiler starts leveraging undefined behavior in new and exciting ways.
We have no such contract with LLMs. The comparison to compilers is highly mistaken, and feels like how the cryptocurrency folks used to compare cryptocurrency to gestures vaguely "the internet" in an attempt to appropriate legitimacy.
A big feature of compilers is to find optimizations you wouldn't think of. I tried to make the point that compiled output is typically not read by humans
> I don't buy that you're actually examining compiled programs. Very few people do
I take it you don't write C, C++, or any language at that level? It is very common to examine compiled programs to ensure the compiler made critical optimizations. I have done that many times, there are plenty of tools to help you do that.
