introducing moderation, steerage and censorship in your LLM is a great way to not even show up to the table with a competitive product. builders have woken up to this reality and are demanding local models
the real insight here isn't that sizing is broken. Everyone knows that. It's that fixing it would require brands to admit their current customers don't match the label they've been selling them. "You're not a size 6, but size 10" is bad for business
No phd, no funding committee, no peer review anxiety. Just curoisity and paper. Sometimes the breakthrough comes from not knowing the problem was supposed to be difficult. This is what happens when you don't tell a kid something is too hard
This is why Ctrl+C is 0x03 and Ctrl+G is the bell. The columns aren't arbitrary. They're the control codes with bit 6 flipped. Once you see it, you can't unsee it. Best ASCII explainer I've read.
The moat here is local inference. Whisper.cpp + Metal gives you <500ms latency on M1 with the small model. no API costs + no privacy concerns. Ship that and you've got something the paid tools can't match. The UI is already solid, the edge is in going fully offline.
Side projects don't die from lack of time. They die from success anxiety. Shipping means facing judgment. An eternal WIP stays safe in the "potential" zone where it can't disappoint anyone including yourself
I'm going through this now. Once I start selling (hardware product), the cat is out of the bag, and further development becomes more difficult because there are paying customers. And we don't have the capacity to scale up if we "go viral". But AI has me pushing to do this somewhat prematurely because there's no guarantee I'll still be employed at the end of the year (due to "AI"), so the side project has to become a real thing right now.
Yep, that feels likely. When tech people don't ship, they overarchitect. Add all the stuff 'the big guys' have to Be Ready for the big time when you launch. And then never launch after spending 1000s on infrastructure, tools etc. I have done it, I see many people do it around me.
Debugging an LLM integration without seeing the reasoning is like debugging a microservice with no logs. You end up cargo-culting prompt changes until something works, with no idea why.
Yes. His work is really inspiring, eg. the text editor less than 1000 lines, tiny language in less than 1000 lines etc. When I read this on HN, I converted his text editor to my hobby programming language [1] (330 lines), and then wrote a chess engine with terminal UI [2] (400 lines), Tetris clone [3] in 140 lines. I also have a QR code generator and parser [4] (700 lines, still in Java only), a PDF generator (200 lines), and now also a tiny programming language [5].
reply