Hacker Newsnew | past | comments | ask | show | jobs | submit | nravic's commentslogin

I think even easier in fact - what's happening behind the scenes w/ an LLM is far more opaque


This is super interesting! We do something similar I think by taking a checkpoint after model initialization. I'm curious what you think about our approach, here's some benchmarks: https://docs.cedana.ai/articles/performance-of-cedanas-gpu-i...

We do some on-the-fly optimizations as well (like compiling into CUDA graphs or fusing together calls) which ends up resulting (for some inference engines) faster token throughput too.


How were you handling GPU state w/ pytorch? We added some custom code around CRIU to enable GPU checkpointing fwiw: https://docs.cedana.ai/setup/gpu-checkpointing/


Not at all. I forked before I used anything with CUDA. I didn't need it but I guessed this could cause all kind of weird problems.


We do this using CRIU right now! https://github.com/cedana/cedana

In fact one of our customer's use cases is exactly what you describe, allowing users to "hibernate" container workspaces.


I'm probably the cofounder of the guy you spoke with! Here's our repo: https://github.com/cedana/cedana


Indeed, that was it. All the best with your startup!


what kind of pannier bags do you use to haul groceries? All the ones I've used have been too small for my purposes.


The bike in question has a front basket with a bag designed to fit it:

https://www.radpowerbikes.com/products/large-basket

https://www.radpowerbikes.com/products/large-basket-roll-top...

So I don't have panniers. I might get some in the future to have a bit of extra capacity.


Used to do groceries with a regular bike and two ortlieb panniers (you can leave the top open to use their full capacity)

For extra space a front "Porteur style" rack is nice (you can carry a pizza or takeout), or a rear rack with a set of lower mounting rails allows carrying stuff ontop of your rack without interfering as much with the panniers.

nice front racks: https://www.passandstowracks.com

nice front and back racks: https://www.tubus.com/en/products/

some people also really like the topeak rail system, which has for example a little wheely cart you can pull around the store with you, the "Topeak TrolleyTote"

Over the summer I bought a used (not electric) bakfiets on craigslist (and only recently did a mid-drive electric conversion) it's been really great for doing more with a bike because it doesn't take as much "how am I going to carry this home" planning. I've picked up groceries, dog food, filing cabinets, my wife, my dog, friends, lumber, tools, etc. It's quite an amazing bike format.


Not OP but I have essentially the same bike (Packa) with kid bars on the back. I throw a big Home Depot plastic storage crate into the bars, then 4 bags of groceries go into the crate. 6+ if you stack and bungee them. Plus another bag in the front basket.

If the kids want to go to the store then the crate goes on the bike trailer.


> big Home Depot plastic storage crate into the bars

Nice! That's such a simple solution. Thanks for the tip.


I don't have a cargo bike, but you can fit a surprising amount of crap in a milk crate strapped to a bike rack. Especially if you have a pannier on the side (they even make some grocery tote bag style panniers)


Not OP, but I use Arkel RT-60. They are huge and strong. I've been able to haul two 24-cases of beer, plus half a dozen mixers.


ortlieb has some super sturdy ones, I grocery shop for 2 with this pair on a road bike. https://www.ortlieb.com/en_us/back-roller-city+F5003


IIRC (and in my experience) KL divergence doesn't account for double counting. Wrote a paper where I ended up having to use a custom metric instead: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=4...


...what does this mean?


This is so cool! Out of curiosity, could you walk through your decisionmaking around using Nix and how it fits in with Rivet?


how often are these batch jobs run? I'm curious to know what the absolute maximum sync frequency can be.


We don't run any batch jobs - Retake streams changes in real time via CDC (change data capture). The only batch job you would need to run is to populate an index when it's first created.


Self plug: run llama.cpp as an inference server on a spot instance anywhere: https://cedana.readthedocs.io/en/latest/examples.html#runnin...


Looks cool, joined the waitlist.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: