> I'm not sure there even is a good place where programs can store their internal system files without requiring root other than mixed in with the user Home.
You are right. I see a lot of applications putting stuff into /var/run/user/<uid>/, but this is mostly ephemeral runtime data, notably sockets. And of course systemd has service units there, which don't seem temporary at all, but it's systemd, so whatever... .
The correct way for what I intend would be /var/lib/user/<uid>/, but yes this does not exist on my host. But I honestly don't see why non-ephemeral per user data shouldn't be put into /home. It is definitely user-specific, under the control of the user, instead of the OS, and when you want to move your user to another host (or sync it), you definitely want to include that data as well, so putting it across the OS filesystem sounds kind of dirty to me.
I think the systemd units in /run are indeed ephemeral, and indicate units that are active. If they have permanent counterparts (some units, like transient timer units, don't have them), you'll find them under /etc/systemd or /lib/systemd for system units and ~/.local/share/systemd or ~/.config/systemd for user units.
> But I honestly don't see why non-ephemeral per user data shouldn't be put into /home
I think I agree— as long as the conventions are clear, I think it's reasonable to have some hidden dirs under $HOME set aside for configuration and cache and so on.
Maybe there's value in exposing a single directory as the root of a sandbox for user files, so users have to go farther out of their way to screw things up, especially depending on your audience. Maybe a decade from now Linux desktops will have something like this, because most apps will run sandboxed in Flatpak, unable to write to the root of $HOME. (Idrk how that's organized, though— maybe apps are just allowed to edit "their" dotfiles without modifying their location.)
But I'm not sure that obscuring configuration data's place in the filesystem in that way is really desirable or necessary. I doubt most users inspect or think about hidden directories on Unix-likes unless they're looking for them anyway.
The average user does not see things like app config files, cache data, etc as their own files. The current /home has so much absolute junk that it's easy to not know where your stuff is. "Real" files to a normal user are .docx, photos, downloaded files, etc. Not auto generated configs.
There ideally should be some separation between your actual documents and system utility stuff. I guess this has mostly happen already with real files sitting in cloud storage. With local storage just being replacable.
> The average user does not see things like app config files, cache data, etc as their own files.
All the more reason to put this stuff into the user directory, so that it is automatically included in backups and syncs done by the unaware user.
> The current /home has so much absolute junk that it's easy to not know where your stuff is.
Not my experience. Application data of application XYZ is either in ~/.xyz or in ~/.{config,local,cache}/xyz, depending on whether the application is FHS compliant or not.
/var/run/user/<uid>/ ?
Or what do you mean?