And then you lose functionality, for example `git blame` depends on the history being available locally. If you want a working repository with all the source control features you need a regular clone. That's where "being able to get the rest via API calls..." kicks in :)
OK so `git clone --filter=blob:none`, then. That downloads the tip and commit history, but no historic blobs. `git blame` then works by downloading missing blobs on demand, which doesn't sound too different to making an API call.