NIST asking for agent security comments right as the agent stack is splitting into layers with completely different threat models.
A model layer vulnerability looks nothing like a tool-use layer vulnerability.. but most framework still treats "the AI system" as one blob. But probably nobody who owns the audit trail when an agent chain spans six vendors.
The token comparison is what jumped at me.. Half the context window for the same work means mainstream tools are burning your tokens on system prompts and MCP plumbing before you even start.
I doubt that a regulator would be satisfied by the kinds of explanations this provides and the interventions it enables.
Suppose somebody put an LLM in charge of an industrial control system and it increased the temperature so much that it caused an accident. The input feature attribution analysis shows that the model was strongly influenced by the tokens describing the temperature control mechanism, concept attribution shows that it output tokens related to temperature, industrial processes and LLM tool-call syntax.
The operator proposes to fix this by rewriting the description and downweighting the temperature concept in the output, and a simulation shows that with these changes the model doesn't make the same decisions in this situation anymore. Should the regulator accept this explanation as sufficient to establish that the system is now safe?
If the controller has just a few parameters and responds approximately linearly to changes in its inputs, you can in principle guarantee that it'll stay within a safe zone. But LLMs have a huge number of parameters and by design highly nonlinear behavior. A simple explanation is unlikely to reflect model behavior accurately enough that you can trust its predictions to hold in arbitrary situations.
thanks for getting back to me, very cool if true :) I have been asked about this many times when talking LLM use cases at enterprise level. Would love to run som tests, pleas shoot me a message to the email in my profile.
Props to Tao for his response: he didn't just patch the sign error, he went back to Hildebrand's paper and found a better fix through log-concavity of the Dickman function.
Odd strategy (I guess at this point nothing really is with Musk / Tesla). They already complied and removed the marketing language. Suing to reverse the ruling after you've conceded the remedy only makes sense if you care about the precedent, not this specific case.
If "false advertising" stays on the record it becomes exhibit A in every future lawsuit involving FSD crashes. That's probably what this is actually about.
Worth noting this is an anode-free design. Removing the anode matters more for commercialization than the energy density headline because it cuts material costs and simplifies manufacturing.
80% capacity retention is promising buuuuut.. "near-real-world conditions" is doing heavy lifting in that sentence.
The "citizen developer" framing assumes AI flattens the skill distribution but the data says the opposite. Next-token prediction and RLHF typicality bias mean AI converges to the mean, so domain experts get more valuable, not less.
This essay pulls together a few threads from pieces I've written over the past months on my blog, where I dig into rabbit holes like GLP-1 economics among other things.
If the pharma side interests you, I just wrote up Novo's rise and fall as Europe's most valuable company:
The problem is WARN only fires when companies actually fire people. AI displacement doesn't really work that way, at least not yet. Customer support new hires dropped 65% in eight quarters (https://www.saastr.com/customer-support-hiring-has-fallen-65...).
A model layer vulnerability looks nothing like a tool-use layer vulnerability.. but most framework still treats "the AI system" as one blob. But probably nobody who owns the audit trail when an agent chain spans six vendors.
Wrote about this layering in an article last month: https://philippdubach.com/posts/dont-go-monolithic-the-agent...
reply