> It's beyond frustrating that any "i+=1" change requires relinking a 50mb binary from scratch
It’s especially hard to solve this with a language like rust, but I agree!
I’ve long wanted to experiment with a compiler architecture which could do fully incremental compilation, maybe down the function in granularity. In the linked (debug) executable, use a malloc style library to manage disk space. When a function changes, recompile it, free the old copy in the binary, allocate space for the new function and update jump addresses. You’d need to cache a whole lot of the compiler’s context between invocations - but honestly that should be doable with a little database like LMDB. Or alternately, we could run our compiler in “interactive mode”, and leave all the type information and everything else resident in memory between compilation runs. When the compiler notices some functions are changed, it flushes the old function definitions, compiles the new functions and updates everything just like when the DOM updates and needs to recompute layout and styles.
A well optimized incremental compiler should be able to do a “i += 1” line change faster than my monitor’s refresh rate. It’s crazy we still design compilers to do a mountain of processing work, generate a huge amount of state and then when they’re done throw all that work out. Next time we run the compiler, we redo all of that work again. And the work is all almost identical.
Unfortunately this would be a particularly difficult change to make in the rust compiler. Might want to experiment with a simpler language first to figure out the architecture and the fully incremental linker. It would be a super fun project though!
It’s especially hard to solve this with a language like rust, but I agree!
I’ve long wanted to experiment with a compiler architecture which could do fully incremental compilation, maybe down the function in granularity. In the linked (debug) executable, use a malloc style library to manage disk space. When a function changes, recompile it, free the old copy in the binary, allocate space for the new function and update jump addresses. You’d need to cache a whole lot of the compiler’s context between invocations - but honestly that should be doable with a little database like LMDB. Or alternately, we could run our compiler in “interactive mode”, and leave all the type information and everything else resident in memory between compilation runs. When the compiler notices some functions are changed, it flushes the old function definitions, compiles the new functions and updates everything just like when the DOM updates and needs to recompute layout and styles.
A well optimized incremental compiler should be able to do a “i += 1” line change faster than my monitor’s refresh rate. It’s crazy we still design compilers to do a mountain of processing work, generate a huge amount of state and then when they’re done throw all that work out. Next time we run the compiler, we redo all of that work again. And the work is all almost identical.
Unfortunately this would be a particularly difficult change to make in the rust compiler. Might want to experiment with a simpler language first to figure out the architecture and the fully incremental linker. It would be a super fun project though!