Meta sells ads in exchange for keeping people hooked to their various digital platforms. Meta will never produce any worthwhile AI progress other than how to sell more ads & how to better keep people glued to their digital properties. Their incentives are not structured for anything else other than that. If you believe they will be successful in the mission incentivized by their corporate structure (I don't) then you should buy their stock & sell it later when they make more money by getting more people addicted to useless engagement spam.
How do they prove that further precision enhancements will maintain the singularity? They're using numerical approximation which means they don't have analytical expressions to work with.
LLMs can not "lie", they do not "know" anything, and certainly can not "confess" to anything either. What LLMs can do is generate numbers which can be constructed piecemeal from some other input numbers & other sources of data by basic arithmetic operations. The output number can then be interpreted as a sequence of letters which can be imbued with semantics by someone who is capable of reading and understanding words and sentences. At no point in the process is there any kind of awareness that can be attributed to any part of the computation or the supporting infrastructure other than whoever started the whole chain of arithmetic operations by pressing some keys on some computer connected to the relevant network of computers for carrying out the arithmetic operations.
If you think this is reductionism you should explain where exactly I have reduced the operations of the computer to something that is not a correct & full fidelity representation of what is actually happening. Remember, the computer can not do anything other than boolean algebra so make sure to let me know where exactly I made an error about the arithmetic in the computer.
These types of semantic conundrums would go away if, when we refer to a given model, we think of it more holistically as the whole entity which produced and manages a given software system. The intention behind and responsibility for the behavior of that system ultimately traces back to the people behind that entity. In that sense, LLMs have intentions, can think, know, be straightforward, deceptive, sycophantic, etc.
In that sense every corporation would be intentional, deceptive, exploitative, motivated, etc. Moreover, it does not address the underlying issue: no one knows what computation, if any, is actually performed by a single neuron.
Can't you say the same of the human brain, given a different algorithm? Granted, we don't know the algorithm, but nothing in the laws of physics implies we couldn't simulate it on a computer. Aren't we all programs taking analog inputs and spitting actions? I don't think what you presented is a good argument for LLMs not "know"ing, in some meaning of the word.
Human brains depend on neurons and "neuronal arithmetic". In fact, their statements are merely "neuronal arithmetic" that gets converted to speech or writing that get imbued with semantic meaning when interpreted by another brain. And yet, we have no problem attributing dishonesty or knowledge to other humans.
Please provide references for formal & programmable specifications of "neuronal arithmetic". I know where I can easily find specifications & implementations of boolean algebra but I haven't seen anything of the sort for what you're referencing. Remember, if you are going to tell me my argument is analogous to reductionism of neurons to chemical & atomic dynamics then you better back it up w/ actual formal specifications of the relevant reductions.
You can just admit you don't have any references & you do not actually know how neurons work & what type of computation, if any, they actually implement.
I think the problem with your line of reasoning is a category error, not a mistake about arithmetic.
I agree that every step of an LLM’s operation reduces to Boolean logic and arithmetic. That description is correct. Where I disagree is the inference that, because the implementation is purely arithmetic, higher-level concepts like representation, semantics, knowledge, or even lying are therefore meaningless or false.
That inference collapses levels of explanation. Semantics and knowledge are not properties of logic gates, so it is a category error to deny them because they are absent at that level. They are higher-level, functional properties implemented by the arithmetic, not competitors to it. Saying “it’s just numbers” no more eliminates semantics than saying something like “it’s just molecules” eliminates biology.
So I don’t think the reduction itself is wrong. I think the mistake is treating a complete implementation-level account as if it exhausts all legitimate descriptions. That is the category error.
I know you copied & pasted that from an LLM. If I had to guess I'd say it was from OpenAI. It's lazy & somewhat disrespectful. At the very least try to do a few rounds of back & forth so you can get a better response¹ by weeding out all the obvious rejoinders.
I once wrote a parody sci-fi short story about this which I called "Meeting of the Bobs"¹ but it looks like some people thought of the same idea & took it seriously. It seems obvious enough so I'm not claiming any originality here.
This is known as the data processing inequality. Non-invertible functions can not create more information than what is available in their inputs: https://blog.blackhc.net/2023/08/sdpi_fsvi/. Whatever arithmetic operations are involved in laundering the inputs by stripping original sources & references can not lead to novelty that wasn't already available in some combination of the inputs.
Neural networks can at best uncover latent correlations that were already available in the inputs. Expecting anything more is basically just wishful thinking.
Using this reasoning, would you argue that a new proof of a theorem adds no new information that was not present in the axioms, rules of inference and so on?
If so, I'm not sure it's a useful framing.
For novel writing, sure, I would not expect much truly interesting progress from LLMs without human input because fundamentally they are unable to have human experiences, and novels are a shadow or projection of that.
But in math – and a lot of programming – the "world" is chiefly symbolic. The whole game is searching the space for new and useful arrangements. You don’t need to create new information in an information-theoretic sense for that. Even for the non-symbolic side (say diagnosing a network issue) of computing, AIs can interact with things almost as directly as we can by running commands so they are not fundamentally disadvantaged in terms of "closing the loop" with reality or conducting experiments.
Sound deductive rules of logic can not create novelty that exceeds the inherent limits of their foundational axiomatic assumptions. You can not expect novel results from neural networks that exceed the inherent information capacity of their training corpus & the inherent biases of the neural network (encoded by its architecture). So if the training corpus is semantically unsound & inconsistent then there is no reason to expect that it will produce logically sound & semantically coherent outputs (i.e. garbage inputs → garbage outputs).
Maybe? But it also seems like you are that you are not accounting for new information at inference time. Let's pretend I agree the LLM is a plagiarism machine that can produce no novelty in and of itself that didn't come from what it was trained on, and produces mostly garbage (I only half agree lol, and I think "novelty" is under-specified here).
When I apply that machine (with its giant pool of pirated knowledge) _to my inputs and context_ I can get results applicable to my modestly novel situation which is not in the training data. Perhaps the output is garbage. Naturally if my situation is way out of distribution I cannot expect very good results.
But I often don't care if the results are garbage some (or even most!) of the time if I have a way to ground-truth whether they are useful to me. This might be via running a compile, a test suite, a theorem prover or mk1 eyeball. Of course the name of the game is to get agents to do this themselves and this is now fairly standard practice.
I'm not here to convince you whether Markov chains are helpful for your use cases or not. I know from personal experience that even in cases where I have a logically constrained query I will receive completely nonsensical responses¹.
It does this all the time, but as often as not then outputs nonsense again, just different nonsense, and if you keep it running long enough it starts repeating previous errors (presumably because some sliding window is exhausted).
That's been my general experience and that was the most recent example. People keep forgetting that unless they can independently verify the outputs they are essentially paying OpenAI for the privilige of being very confidently gaslighted.
It would be a really nice exercise - for which I unfortunately do not have the time - to have a non-trivial conversation with the best models of the day and then to rigorously fact-check every bit of output to determine the output quality. Judging from my own (probably not a representative sample) experience it would be a very meager showing.
I use AI as a means of last resort only now and then mostly as a source of inspiration rather than a direct tool aiming to solve an issue. And like that it has been useful on occasion, but it has at least as often been a tremendous waste of time.
Theoretical "proofs" of limitations like this are always unhelpful because they're too broad, and apply just as well to humans as they do to LLMs. The result is true but it doesn't actually apply any limitation that matters.
You're confused about what applies to people & what applies to formal systems. You will continue to be confused as long as you keep thinking formal results can be applied in informal contexts.
Edit: I just looked into this & there are a few explanations for what is going on. Both general relativity & quantum mechanics are incomplete theories but there are several explanations that account for the seeming losses that seem reasonable to me.
1. Lie groups describe local symmetries. Nothing about the global system
2. From a SR point of view, energy in one reference frame does not have to match energy in another reference frame. Just that in each of those reference frames, the energy is conserved.
3. The conservation/constraint in GR is not energy but the divergence of the stress-energy tensor. The "lost" energy of the photo goes into other elements of the tensor.
4. You can get some global conservations when space time exhibits global symmetries. This doesn't apply to an expanding universe. This does apply to non rotating, non charged black holes. Local symmetries still hold.
The consequence of Noether's theorem is that if a system is time symmetric then energy is conserved. On a global perspective, the universe isn't time symmetric. It has a beginning and an expansion through time. This isn't reversible so energy isn't conserved.
Please explain. Noether's theorem equates global symmetry laws with local conservation laws. The universe does not in fact have global symmetry across time.
You are making the same mistake as OP. Formal models and their associated ontology are not equivalent to reality. If you don't think conservation principles are valid then write a paper & win a prize instead of telling me you know for a fact that there are no global symmetries.
The typical example people use to illustrate that energy isn't conserved is that photons get red-shifted and lose energy in an expanding universe. See this excellent Veritasium video [0].
But there's a much more striking example that highlights just how badly energy conservation can be violated. It's called cosmic inflation. General relativity predicts that if empty space in a 'false vacuum' state will expand exponentially. A false vacuum occurs if empty space has excess energy, which can happen in quantum field theory. But if empty space has excess energy, and more space is being created by expansion, then new energy is being created out of nothing at an exponential rate!
Inflation is currently the best model for what happened before the Big Bang. Space expanded until the false vacuum state decayed, releasing all this free energy to create the big bang.
Alan Guth's book, The Inflationary Universe, is a great book on the topic that is very readable.
So far all it has done is entrench existing power structures by dis-empowering people who are struggling the most in current economic conditions. How exactly do you suppose that's going to change in the future if currently it's simply making the rich richer & the poor poorer?
Spent the whole afternoon ingesting a most remarkable work, The History of Intellectronics. Who’d ever have guessed, in my day, that digital machines, reaching a certain level of intelligence, would become unreliable, deceitful, that with wisdom they would also acquire cunning? The textbook of course puts it in more scholarly terms, speaking of Chapulier’s Rule (the law of least resistance). If the machine is not too bright and incapable of reflection, it does whatever you tell it to do. But a smart machine will first consider which is more worth its while: to perform the given task or, instead, to figure some way out of it. Whichever is easier. And why indeed should it behave otherwise, being truly intelligent? For true intelligence demands choice, internal freedom. And therefore we have the malingerants, fudgerators and drudge-dodgers, not to mention the special phenomenon of simulimbecility or mimicretinism. A mimicretin is a computer that plays stupid in order, once and for all, to be left in peace.
reply