Super interesting, thanks! I didn't think about how this issue could easily point to a major improvement that could be made on the package manager level. Unfortunate that in Python land only Conda supports it though. It looks from the docs that uv may also support it? https://docs.astral.sh/uv/concepts/cache/#cache-directory
> It looks from the docs that uv may also support it?
I was just repeating what chatgpt told me - that uv does the same thing as npm. The documentation wasn't clear about exactly what it does either. So I had to confirm it using strace. You are indeed correct. UV does hardlink the dependencies from the global cache to the project venv. So that's good!
There's just one caveat. Hardlinking is not possible if the project and the cache are on different file systems. Even btrfs subvolume mounts from a single physical volume is considered as different filesystems. I think pnpm quite happily symlinks it instead of hardlinking it when such a situation arises (I'm not sure though. Need to check that as well.) UV doesn't do that. It complains (prints a warning) and just copies the entire thing over to the venv. So you won't get that advantage with uv if the venv and the cache are on different filesystems (like in my system). I don't know how many developers actually deal with this problem.
Hey, I'm Conor, a software engineer specializing in the backend, but capable of using whatever technologies required to get a problem solved. I have experience with large multinationals, but much prefer my time working with startups. I was the first hire in my current company, and am now looking for a new opportunity having helped the company find product/market fit, scale the solution, and reach a position of stability. Most of my details can be found on my LinkedIn, including links to my Github, Blog, and other technical works.
I am a software engineer specializing in Python backends, but have demonstrated ability using whatever technology necessary to best solve the problem at hand. I have experience working with large multinationals, and startups, both in person and on a remote-only basis. My current role is fully remote, and involves software architecture/design decisions, PoC prototyping, DevOps, improving developer tooling, and general purpose programming.
The best place to see my professional experience is on LinkedIn, but feel free to reach out to me by any means you prefer so we can discuss opportunities.
Another interesting thing in this space is Traindown[1]. It's simply Markdown for your exercises. As an example for those who need convincing to go to a link, this could be a simple workout:
It's much more free-form than this, which is a bonus for me and allows me to track metadata such as weight, time of day I'm exercising at, or general mood/feeling about the workout. I can ultimately just take these plain markdown files from the app (I use a basic Android app that visualizes this Markdown[2]), import them and do whatever processing I like in Python.
Interesting. I wrote a small program some time ago that would have needed it... well i am needing the workout... so i should go and refactor the app instead of refactoring the body...
There it is! I've been looking for something like this!
I've been using my own homebrewed toml spec, but since I am more experienced with code than training, I was concerned if it would still work well as I become more experienced. Not sure if I'll use this, but good to see another interpretation of this kind of data!
I expressed myself too succinctly and without context, sorry.
I meant we need a new DSL better suited for prompt engg, and a UI that better supports longer strings. Actualy this UI can be something compatible with Python.
But overall a reimagination of the dev experience is what I am getting at (like Jupyter for LLMs).
1) Can you get the actual code output or will this end up calling OpenAI each function call?
2) What latency does it add? What about token usage?
3) Is the functionality deterministic?
1) The OpenAI API will be queried each time a "prompt-function" is called in python code. If you provide the `functions` argument in order to use function-calling then magentic will not execute the function the LLM has chosen, instead it returns a `FunctionCall` instance which you can validate before calling.
2) I haven't measured additional latency but it should be negligible in comparison to the speed of generation of the LLM. And since it makes it easy to use streaming and async functions you might be able to achieve much faster generation speeds overall - see the Async section in the README. Token usage should also be a negligible change from calling the OpenAI API directly - the only "prompting" magentic does currently is in naming the functions sent to OpenAI, all other input tokens are written by the user. A user switching from explicitly defining the output schema in the prompt to using function-calling via magentic might actually save a few tokens.
3) Functionality is not deterministic, even with `temperature=0`, but since we're working with python functions one option is to just add the `@cache` decorator. This would save you tokens and time when calling the same prompt-function with the same inputs.
- Getting familiar with new APIs
- Bouncing general knowledge questions off it
- Having "discussions" to interact with it in a Socratic style
- Giving some "personality" to automated services by calling it as an API
2) Thanks for the pointers! Will check them out today!
I don't have much interest in playing with the internals for now, but I generally like keeping my data personal and the services I use self-maintainable as much as reasonably possible! I also feel like I could find the token limits and price limiting with ChatGPT.
