I don't know a lot about this corner of the OS. Why are syscalls in libc, instead of something like a libsyscall? I could see why a language might want not to depend on what's at least notionally the C runtime. Is the fact that the kernel interface is in libc an accident of Unix being written in C, or is there something more fundamental there?
Libc is really a conflation of at least three different notional libraries. The first library is what its name suggests it is, a standard library for C. Another part of the library is in providing the userspace portion of system services--things like handling static initializers, dynamic loading, or the userspace side of things like creating new threads (not to mention, the actual raw functions you call to get the kernel to do something). The final part of the library is a collection of userspace services which are language agnostic, you might choose different implementations, but you'll always assume are somehow present--libm and malloc are the goto examples here.
As for why the userspace system service library is part of libc instead of being a separate libsyscall or libkernel, that probably is due to Unix being a C operating system--written at a time when most operating systems also came with their own system language. It's definitely not the case for all OS's that the C runtime library is the same as the libsyscall/libkernel--most notably, on Windows, the former is MSVCRT*.dll and the latter is kernel32.dll (or ntdll.dll if you're looking very specifically at syscalls themselves, but ntdll.dll is largely an unstable interface).
That's the best explanation I have ever read! I even knew these things beforehand, nodding along, but now I can explain it also to someone else in a coherent way instead of rambling up implementation details like a madman.
In pretty much every OS of the unix tradition. libc is the API, with a stable ABI. Although Windows has a something pretty much identical in — I believe — ntdll. The name and API differ, but the intent is the same.
No, Windows has stable API and ABI named Win32 API. This comes from times of 16bit Windows and works also in x64 / ARM / ARM64, previously it worked in Alpha / MIPS / IA64. This API is implemented in kernel.dll user.dll gdi.dll advapi.dll and similar, it does some stuff, but mostly forwards to the NT API. Beware, kernel.dll is user space component despite its name (historical reasons). NT API is undocumented and not meant to be used by user programs, it is not stable, it lives in ntdll.dll, it does syscalls to the kernel: ntoskrnl.exe. Windows doesn't have a libc (for user programs, it has private one for its own programs), Visual Studio has a libc. Each version of Visual Studio (roughly) has its own libc named msvcrt.dll msvrt100.dll msvcrt140.dll and similar, it hosts the C and C++ libc, it could be linked statically for various benefits and drawbacks.
A big motivation for it was security posture; it means that Microsoft can now ship security updates to UCRT that everyone can rely on rather than a ton of extra surface area through various multiple versions of runtime libraries.
It had (has) an unsupported, crippled, unversioned msvcrt.dll which if you used it very carefully with a subset of functions, you could write programs which worked fine on Windows NT and up.
FWIW, large swaths of ntdll are documented and supported these days, for performing work that can't be expressed via the win32 API like raw disk manipulation.
glibc actually goes through quite a bit of effort to remain backwards compatible. There have been times when mistakes were made here, but the extent to which glibc does not have a stable ABI is overstated. The only annoying thing about glibc and ABI stability is that the way it achieves that is via symbol versioning which makes it annoying to target an old glibc on a system with a new er glibc.
Another stable ABI boundary is anything produced by glibc that can be conceivably placed into memory shared by different unrelated processes (ie. Posix IPC primitives). The requirement for backwards compatibility and architecture variant compatibility (32b/64b…) is the largest reason why things like pthread_mutex have somewhat large overhead and why it is worthwhile to invent various iterations of “futex in userspace”.
It is not different on mac (although there libc is just a subset of the wider libSystem).
It is different, uniquely so, on linux: on most unices the kernel and libc are developed as two sides of an entire system, both being updated in lockstep when the system is updated. As such there is no real concern about keeping the syscalls stable, if you need to change it on the kernel side you update it to match on the libc side and you're done, everybody is supposed to use the libc.
Not so on linux, the kernel and the libc (most commonly glibc) are developed by entirely different groups which don't necessarily like or communicate with one another. As a result, on linux syscalls are a stable API, and direct syscalls are an officially supported method of interaction. In fact its sometimes necessary as the libc might decide not to expose a syscall.
> It is different, uniquely so, on linux: on most unices the kernel and libc are developed as two sides of an entire system, both being updated in lockstep when the system is updated.
Do FreeBSD [0] and NetBSD [1] not fall under "most unices"? They similarly retain backward compatibility in their syscall interface, so that old binaries with old libcs can run on newer kernels. Linux does not stand alone here.
Only on systems where the syscall ABI is stable, like on Linux. Others, like macOS and Windows[0], can and will change theirs between releases. OpenBSD even goes one step further and actively prevents code other than libc from performing syscalls[1]
[0] I seem to remember that this changed in a recent Windows version, but I couldn't immediately find a source.
