I do have to try mamba sometime but I feel like there is something more than python slowness going on.

I use Gentoo and its package manager is written in python. Even though it is more complex (IMO) it doesn’t have nearly the same slowness when it comes to dependency resolution and conflict detection.