We should be able to pin to a version of training data history like we can pin to software package versions. Release new updates w/ SemVer and let the people decide if it’s worth upgrading to
I’m sure it will get there as this space matures, but it feels like model updates are very force-fed to users
If you talk to people who deal with inference using large fungible datasets, this is an extremely difficult governance problem. semver is incredibly insufficient and you don't have a well defined meaning of what "upgrade" even means let alone "major", "minor", and "patch".
It's a major disservice to the problem to act like it's new and solved or even solvable using code revision language.
I think the models are so big that they can’t keep many old versions around because they would take away from the available GPUs they use to serve the latest models, and thereby reduce overall throughput. So they phase out older models over time. However, the major providers usually provide a time snapshot for each model, and keep the latest 2-3 available.
This is done so that application developers whose systems depend upon specific model snapshots don't have to worry about unexpected changes in behaviour.
You can access these snapshots through OpenRouter too, I believe.
Every model update would be a breaking change, an honest application of SemVer has no place in AI model versions.
Not saying using major.minor depending on architecture is a bad thing, but it wouldn’t be SemVer, and that doesn’t even cover all the different fine tuning / flavors that are done off those models, which generally have no way to order them.
there's figurative and literal though. Figurative semver (this is a system prompt update vs a model train) would actually work ok... at least build numbers.
I think you could actually pretty cleanly map semver onto more structured prompt systems ala modern agent harnesses.
Same here, I thought it was going to be an analogy for a political story.
Now that I know it is literal leeches and that the options are scraping them off or waiting for them to finish, avoiding areas with leeches feels like the move.
When I was about 10 I picked my first ever CD at a music shop, and it was a recording of the Gershwin piano rolls, because the cover photo caught my eye [1]. I didn't really understand what I was listening to, I assumed "piano roll" was a musical genre, like "rock'n'roll", until years later when my English became good enough to read the CD's booklet.
It was also a time when all these midi files started being available, like the 6000 rolls from Terry Smythe [2], and I figured out transcribing these could be a good way to learn old-school Jazz, which is otherwise difficult to find as sheet music.
Does a piano roll sound different (I assume it does)? Ie, is or was there a specific market for a CD of a piano roll specifically, not, of someone playing the piano?
In terms of the music being played, piano rolls can be different from "normal piano music" because it's not played live by a real human, so it can have complex parts with full chords, additional voices, all with perfect rhythm and no wrong notes. This can be very compelling when well executed on the right songs (and it can also sound "mechanical" on others).
There isn't a huge market for piano roll recordings, and these recordings are rare. It's a niche topic that can attract
- Older people who have known the time piano rolls (say, until the 1950s)
- People nostagic of old times in general (in particular the 1910s-1940s), the age of early jazz with stride piano and early Broadway.
- Music scholars, because some of these rolls are of historical/musical importance, in particular those "recorded" by George Gershwin or Fats Waller and other big names. A lot of material exists only as piano rolls.
For the example of the Gershwin CD I posted above, it was produced by musicologist Artis Wodehouse [1] in parnership with the yamaha disklavier pianos iirc [2], so my guess is this was a passion project above all, with a bit of Yamaha marketing.
Excellent article, but the typesetting / justification in that pdf is horrendous.
The text columns look like the side of that hallway rubber mat that my dog keeps chewing on.
spend a lot of time trying
edge. However, as someone
lieve that true progress is
mes, and for the chickens
y zombies, and the polite
to eat your brain to acquire
be prepared; thus, in the
e scientific breakthroughs,
ast inevitably becomes
he main thing that I ponder is
post-apocalyptic survival
ag-tag group of associates.
cruit: a locksmith (to open
ith has run out of ideas);
row snakes at my enemies
g is a reasonable way to
ble in my ultimate success
I’m sure it will get there as this space matures, but it feels like model updates are very force-fed to users