My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger.
As a dev I very much subscribe to this line of thought, but I also have to admit most of the business class people would disagree.
“Fill less than 1% of its space” becomes a very counter intuitive statement in any case when discussing high dimensions. If you consider a unit n-sphere bounded by a unit cube, the fraction occupied by the sphere vanishes for high n. (Aside: Strangely, the relationship is non monotonic and is actually maximal for n=6). For n=100 the volume of the unit 100-sphere is around 10^-40 (and you certainly cannot fit a second sphere in this cube…) so its not surprising that the gains to be made in improving packing can be so large.
I'm a big fan of killing time on long drives with friendly word games. One of my favorites is a mix between rhyming and square theory. Here's how it works: one player picks two words that rhyme perfectly. Then, for each of those words, they choose a clue word, usually a synonym, but any kind of related word is fair game. They say those two clue words out loud, and the other players have to guess the original rhyming pair.
What makes it fun is trying to reverse-engineer the original rhyme from the clues. It's like solving a little logic puzzle. It's easy to come up with new puzzles, but cracking them can be surprisingly tricky. Still, the structure gives just enough to keep it solvable most of the time.
> The time to halt this trading of assets
for consumables is now, and I have a plan
to suggest for getting it done. My remedy
may sound gimmicky, and in truth it is a tariff called by another
name. But this is a tariff that retains most free-market virtues,
neither protecting specific industries nor punishing specific countries nor encouraging trade wars. This plan would increase our
exports and might well lead to increased overall world trade. And
it would balance our books without there being a significant decline in the value of the dollar, which I believe is otherwise almost certain to occur.
> We would achieve this balance by issuing what
I will call Import Certificates (ICs) to all U.S.
exporters in an amount equal to the dollar
value of their exports. Each exporter would,
in turn, sell the ICs to parties—either exporters abroad or importers here—wanting to get goods into the
U.S. To import $1 million of goods, for example, an importer
would need ICs that were the byproduct of $1 million of exports. The inevitable result: trade balance.
Easy way to get a fair result from an unfair coin toss: Flip the coin twice in a row, in this case starting with the same side facing up both times, so it's equally unfair for both tosses. If you get heads-heads or tails-tails, discard and start over until you get either heads-tails or tails-heads, which have equal probabilities (so you can say something like HT = "heads" and TH = "tails").
This works even if the coin lands heads 99% of the time, as long as it's consistent (but you'll probably have to flip a bunch of times in that case).
Made me think of this old joke that's been on HackerNews, Reddit, etc for years:
The European Commission has just announced an agreement whereby English will be the official language of the European Union rather than German, which was the other possibility.
As part of the negotiations, the British Government conceded that English spelling had some room for improvement and has accepted a 5- year phase-in plan that would become known as "Euro-English".
In the first year, "s" will replace the soft "c". Sertainly, this will make the sivil servants jump with joy. The hard "c" will be dropped in favour of "k". This should klear up konfusion, and keyboards kan have one less letter.
There will be growing publik enthusiasm in the sekond year when the troublesome "ph" will be replaced with "f". This will make words like fotograf 20% shorter.
In the 3rd year, publik akseptanse of the new spelling kan be expekted to reach the stage where more komplikated changes are possible.
Governments will enkourage the removal of double letters which have always ben a deterent to akurate speling.
Also, al wil agre that the horibl mes of the silent "e" in the languag is disgrasful and it should go away.
By the 4th yer peopl wil be reseptiv to steps such as replasing "th" with "z" and "w" with "v".
During ze fifz yer, ze unesesary "o" kan be dropd from vords kontaining "ou" and after ziz fifz yer, ve vil hav a reil sensi bl riten styl.
Zer vil be no mor trubl or difikultis and evrivun vil find it ezi TU understand ech oza. Ze drem of a united urop vil finali kum tru.
Und efter ze fifz yer, ve vil al be speking German like zey vunted in ze forst plas.
A long discredited intervention where a "facilitator" guides the hand of a non-verbal human to help them write down their thoughts and experiences. Experiments that blinded the facilitator to the observations of the subject, where the written message matched the facilitator's, rather than the subject's, observations, have convincingly proved that it was so much bunkum. It's the Clever Hans Effect all by another name, and with non-verbal humans rather than horses.
Chain of Thought works like that: without hand-holding by a human who understands how to answer a question, the LLM's performance drops, or drops off a cliff even. Of course this is much harder to prove for LLMs than it was for facilitated communication because LLMs don't really do anything without a prompt in the first place. Which should be a very big hint of what's really going on with CoT.
I think the "uncle that doesn't know anything but claims to be an expert and will tell you his opinion" is the best description of LLMs that I can think of. They will say anything confidently without knowing anything.
> “As long as you keep thinking about the problem, even if its in short bursts every few years, you’re still making progress. And if you never finish? If all you find are side-paths and obstacles, and it turns out the entire mission was doomed from the outset? That’s okay too. Projects like this nourish us, because there’s a part of the human mind that wants nothing more than to climb the mountain, rappel into the cave, explore the unknown and grapple with it.”
In addition to the impressive technical details, this is some really beautiful writing
I think evolution hardwired our primate brains against inequality. We go out of our way to punish those who don't play nice and share with the broader group. Otherwise complex societies would have never emerged, because assholes exploiting and taking advantage of peers would ruin and doom any group larger than a few individuals.
Robert Sapolsky found the the most obvious predictor of stress in primate societies (measured via cortisol levels) was inequality. Speaking as a member of a highly inequal primate society, it makes sense to me.
The concept of intellectual property is antithetical to how humans have progressed socially and technologically for millenia.
It's a wasteful aberration that strangles innovation with red tape and hands control of our culture over to those with the financial mean to claim to own it.
The day we wrest back that control will be a good day.
eloquent words aren’t true.
Tao Te Ching – Verse 81