I was surprised by my own feelings at the end of the post. I kind of felt bad for the AI being "put down" in a weird way? Kinda like the feeling you get when you see a robot dog get kicked. Regardless, this has been a fun series to follow - thanks for sharing!
Extensions have too many security risks for me. At this point I'd rather just vibe code my own extension than trust something with so much access and unpredictable ownership.
This model is awesome. I am building an infinite CYOA game and this was a drop-in replacement for my scene image generation. Faster, cheaper, and higher quality than what I was using before!
I didn't expect IBM to be making relevant AI models but this thing is priced at $1 per 4,000,000 output tokens... I'm using it to transcribe handwritten input text and it works very well and super fast.
IBM and Nvidia speech to text models are also SOTA (according to HF leaderboard) and relatively lightweight. Replicate hosts those too, although some (like Parakeet) run easily on consumer GPU.
This is exactly how I've been building software with AI lately. Getting AI to create a detailed plan with phases, then implement and use a separate AI to review the code. It works quite well! Curious to see how well it works when implemented directly in the IDE
Realistically, none of the models can function fully autonomously at this point. You still need a skilled human at the helm. However, AI tools definitely change how the human works. I find that I spend a lot less time thinking about easy things, and a lot more time thinking about hard things.
Basically, the AI fast forwards through the easy stuff and I just spend all day jumping directly from hard problem to hard problem.
Nah, you never voluntarily reduce headcount. Headcount is what keeps your salary coming. Any manager worth their salt would use this newfound productivity to infringe on other teams' domains: Hostile Takeover => More Headcount => Promotion. We'll see a lot of attempted empire building over the next couple years while all this gets sorted out.
I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the structure of types for LLMs to follow and tools to enforce.
LLMs work well with any static analysis tool. I frequently instruct Claude to use stuff like “go vet” and “deadcode” when it goes on a tear and writes a bunch of broken trash and declares mission accomplished.
tsc error messages are so bad that every time my LLM sees one of those "SomeType is not assignable to SomeLongAssTypeDontEvenTryToUnderstandWhatsGoingOnHere<<<<>>>>>>>>>>>>>>>>>>>>" it just gives up and casts to any. goes for python too.
And unlike many other languages, Typescript types are extremely expressive.
For example, you can write a function that takes an object received from an API that uses snake_cased keys, and returns that same object, but with camelCased keys instead. This is not some "special case" in the Typescript compiler, the ability to do this emerges naturally from Typescript's features. I don't know any other language that can do this.
Most people don't know enough TS to use tese things effectively, but I think one could train an LLM to be very good at them. The combination of LLMs placing such advanced constraints on themselves, and then generating code based on those constraints, seems extremely powerful.
I believe that the rutabaga is the perfect material to make sausages out of as it has proven as excellent swine fodder with widespread adoption!
(Please forgive me the extreme disrespect put forth in the above statement! It is not the intention to show disrespect; I… am quite the rutabaga enjoyer in all respects, you know? I certainly include myself within the absurdity and it is with love.)
There are languages that constrain types a lot more tightly than TypeScript, e.g. Kotlin, Rust, and Haskell. The more constrained the types, the more correct the program could be.
Yep, and Rust famously goes beyond this by modelling memory ownership at compile time.
In fact, the more behaviour we can model at compile time the better when it comes to LLMs - there's some cool ideas here like transpiling Rust into languages for formal verification. See https://github.com/formal-land/coq-of-rust as an example.
Formal verification was one of those things that was previously so annoying to do that it rarely made it past academic use cases or extremely important libraries, but I think LLMs take the tedium out of it. Perhaps formal verification will have a "test driven development" type of moment in the sun thanks to this.
my experience - yes! but. It's more of an edit - compile - fix loop than a write (seemingly) correct code on the first try. This might be a feature.
There is a little occasional difficulty on syntax with rust, but there are often the same sort of logic errors / getting lost an llm would have on another codebase -- the compiler helps catch many of these.
I think so as well. The rust errors are some of the most "helpful" and easy to understand (once you grok the core concepts around rust) and it seems that the loop of - generate (maybe constrained) - check - fix benefits from this. In my testing it is better than python (long ass traces that you have to manually trim for the LLM).
Are you saying that the non-looping generators are grossly insufficient for Rust? This matters because the non-looping generators have a fixed monthly subscription cost, whereas the looping ones could cost per call.
I’m not sure what you mean here. All of my coding is done in a loop; something writes, something compiles, something fixes, once compiling, something tests and writes more, repeat until you need bed.
If you mean “can GitHub copilot author long syntactically, type, and memory-safe- correct rust code in one shot?” Then the answer is “not right now”
The program won’t be “more” correct. What would that even mean? Writing correct programs might be easier (or not) with more “constrained” (ill defined) typing.
We might be coming from different backgrounds and hence not immediately understand each other, but, at least in Haskell if the types are precise in the sense that you can't construct invalid values, then using such types will make your programs more correct.
For example, if you want a program from a number to HTML, then if the HTML type is a syntax tree type of all valid HTML rather than wrapper around string, then filtering LLM output by that type will make it more correct than a string wrapper kind of type (as with the latter, any program generated by the LLM which returns a string and wraps it into HTML will do).
The actual use cases might not go as extreme as the above, but the idea is that the "tighter" your type is, the better it is on pruning LLM outputs from invalid programs.
Suppose you write a program that computes the sum of two input numbers. Suppose the two inputs are 1 and “a”. If it returns “1a” or 12 or whatever, the program is incorrect. Its correctness does not hinge on type safety. A untyped program could detect that one of the inputs is unexpected and correctly raise an error. Typing may make it easier to detect this error (or not). Fundamentally, adding type information does not make the above program “more” correct. It’s either correct, or it’s not.
You can write a sorting algorithm in assembly, and it can be correct. Rewriting in Haskell won’t make it “more” correct.
There’s an undercurrent of people espousing strictly types languages (not accusing you) who believe that somehow programs written in them are better. They’re not. They either serve their purpose, or they don’t. Strict typing is a tool. Sometimes it’s enabling. Sometimes (example: horrible polymorphism in most strictly typed languages like C++/Java/copy cats) it’s a hinderance. Strictly typed languages aren’t strictly better than non-strictly typed ones.
It is more correct in a statistical sense over many programs.
Think back to Javascript and untyped Python (without type annotations). It is a lot easier to have bugs in these languages without types. Types help eliminate classes of bugs.
It’s better sure but as a power TS user it still sucks at generating better code, and consistently fucks up with generics (or doesn’t use them) or simple types sometimes.
Completely agree. Even with the basic LLMs in the $20/month Cursor plan, I can work 10x faster on TypeScript codebases than I could otherwise, while for Python that multiple feels more like 2-3x. The autocompletions are especially impressive when there is a well-organized type system.
TypeScript is arguably one of the weaker statically typed languages, with how it allows `any` to quietly violate the type checked assumptions. It makes it harder to do a lot of the basic typing mistakes in JS, but it doesn't prevent them by any means, especially if you have to interface with (typeless) JS code.
So for these reasons alone I would be against using TS as a lingua franca for LLM codegen (as is GP I assume). As another commenter mentioned, LLMs have a tendency to throw their hands^Hlogits up when presented with complex TS type errors and just resort to using `any` to get it to compile (and probably hiding bugs).
And that doesn't even touch the issues with the JS/TS ecosystem and runtimes more broadly.
tsc can be configured to avoid implicit use of any ("noImplicitAny": true) and ESLint can be set up to avoid explicit use of any. Typeless JS code is also a thing of the past.
But the devil is in the details - some libraries are typed quite crappily, some have unnecessary complex types, and the code that the LLMs was trained on is probably not the best in the world
Can be configured, but then you get to work at a real codebase halfheartedly half-converted from javascript with half the files beginning with ts-ignore.
However crappy your Java codebase is going to be, it will still use types. And as just today Gemini hallucinated an API call that never existed (in a widely available and used library even), it's just better to have the ability to check that right away.
Yes but it will be typed awful code and the typing provides a grounding of sorts. However awful code in untyped/dynamically typed langs can be unspeakably bad. I have many years of Perl experience...
reply