They are just shipping the same whisper-small that everyone else is using, and did not much to improve their models since release. Other models have been "coming soon" forever. https://keyboard.futo.org/voice-input-models
Agreed - I run an entire second dev environment for LLMs.
Claude code runs in a container, and I just connect that container to the right network.
It's nice to be able to keep mid-task state in that environment without stepping on my own toes. It's easy to control what data is accessible in there, even if I have to work with real data in my dev environment.
Why do you even necessarily think that wouldn't happen?
As I understand it, we'd essentially be relying on something like an mp3 compression algorithm to fail to capture a particular, subtle transient -- the lossy nature itself is the only real protection.
I agree that it's vanishingly unlikely if one person includes a sensitive document in their context, but what if a company has a project context which includes the same document in 10,000 chats? Maybe then it's more much likely that whatever private memo could be captured in training...
I did get an answer from a senior executive at one AI lab who called this the "regurgitation problem" and said that they pay very close attention to it, to the point that they won't ship model improvements if they are demonstrated to cause this.
Lol and that was enough for you? You really think they can test every single prompt before release to see if it regurgitates stuff? Did this exec work in sales too :-D
They have a clear incentive to do exactly as said - regurgitation is a problem, because it indicates the model failed to learn from the data, and merely memorized it.
I think they can run benchmarks to see how likely it is for prompts to return exact copies of their training data and use those benchmarks to help tune their training procedures.
In other areas of life, people self-select at their own risk. You can diagnose medical issues yourself, buy power tools you don't know how to use safely, and invest in assets that you don't understand.
All other things being equal, we should try to protect people. But we shouldn't force everyone to make the choices that are best for the people with the least comprehension of what they're doing.
Have you ever seen government officials talk about tech? I think you'd have to be naive to buy the narrative that they're making such a large policy decision for our security.
Of the few people using rooted phones to begin with, there's even fewer that don't know what they're doing.
Much more likely is this is a decision to get in line with the well documented and rapidly spreading surveillance laws of the past few years.
> But we shouldn't force everyone to make the choices that are best for the people with the least comprehension of what they're doing.
You are acting like it's easy to accidentally root your phone
You can’t freely sell devices to let others self-diagnose medical issues, so this part of your analogy doesn’t hold up in the case of phone sales.
We also limit investing in certain types of investments to so-called “accredited investors” which is just legal jargon for “millionaires”.
I don’t think the point you are trying to make about letting people own-goal is as strong as you think it is. (I would have gone with “roulette is legal”, which is a better one that the investment one, as the accredited investor rule is in all 50 states.)
If you are interested in the public good, I think it is pretty clear that we should ban roulette overnight since it has a negative expected value for everyone but the casino. On the other hand (still presuming you're interested in the public good), I think you have to consider very carefully whether it's good or bad to lock people out of investments or to restrict people's access to health care.
Because when you don't do this, people get scammed out of money.
If there is a series of buttons you can press to circumvent the anti-scam measures, then the scammers simply walk you through pressing those buttons. If you cover them in giant warning labels the scammers simply add explanations into their patter. The buttons must physically not exist, for gullible people to not get scammed out of money.
The next response will be 'well maybe we shouldn't accommodate them'. They vote, and there's more of them than you.
> Because when you don't do this, people get scammed out of money.
No, only when you don't do this and nothing else to improve security. You're presenting a false dichotomy.
> If there is a series of buttons you can press to circumvent the anti-scam measures, then the scammers simply walk you through pressing those buttons.
If the scammers can walk somebody through doing all that, why would they stop at just asking them to send money over to them "to safekeep it because of a compromised account" or whatever the social engineering scheme of the week is?
One of the benefits or downsides of a government depending on who you ask is that it can help stop people from making bad decisions that hurt people around them. Bad decisions rarely hurt only one person.
They represent more of the customer base, and a larger voting bloc, than tech nerds. You can offer your opinion of what society exists for, and the rest of society doesn't have to listen to it. The only actual leverage tech nerds who aren't billionaires have is when the particular ones who work for Google are asked to implement these features.
> Because when you don't do this, people get scammed out of money.
Bullshit. Big tech's war on general purpose computing hasn't stopped scam. It's a pretext for rent seeking and control and you know it. It's the reason we don't have a popular ecosystem of FOSS alternatives on mobile. It's the reason we can't run virtual machines on tablets when the hardware very much can.
If combating scam is a priority of big tech, I know where to start. Get rid of ads! That would actually be enormously effective as it gets rid of the primary entry point of scams.
> If there is a series of buttons you can press to circumvent the anti-scam measures
So the best you can come up with is an imaginary button on phones that can magically circumvent checks that should be implemented server-side? Have you any idea how software works?
Or rig screens such that the buttons do not appear to be what they are. I've seen many a install-this-app ads where cancel isn't cancel.
The average user simply does not have the skill to determine real from fake and any heuristics to do so will be defeated by the scammers. You have to be able to understand what could be done with access, not what's "intended" with the access.
> If there is a series of buttons you can press to circumvent the anti-scam measures, then the scammers simply walk you through pressing those buttons. If you cover them in giant warning labels the scammers simply add explanations into their patter. The buttons must physically not exist, for gullible people to not get scammed out of money.
We shouldn't be protecting someone that gullible at the expense of everyone else who is smart enough to actually read whats on the screen and not fall for such simple scams.
Not that long ago most of this forum was very much against giving up freedoms in favor of catering to the lowest common denominator. What happened?
People need to take responsibility for their own actions and educate themselves, not rely on a lack of freedom to protect them.
If the GP can handle my problem, I probably didn't need to go to the doctor anyway. A lot of care is done by specialists, and it can _easily_ take weeks or months to get an appointment with one. This is strongly dependent on one's insurance network though.
Ok, to be fair, they _can_ probably handle my problems better than I can.
But, presumably for liability and out of a genuine attempt to get me the best care possible, they _prefer_ to send me off to a specialist. Either way I'm not being treated until the specialist has time, which take a couple months at least.
Hard disagree. I may trust the people on my team to a make PRs that are worth reviewing, but I don't give them a shell on my machine. They shouldn't need that to collaborate with me anyway!
Also, I "trust Claude code" to work on more or less what I asked and to try things which are at least facially reasonable... but having an environment I can easily reset only means it's more able to experiment without consequences. I work in containers or VMs too, when I want to try stuff without having to cleanup after.
If I'm responsible for something, nobody's getting that access.
If someone's hired me for something and that's the environment they provide, it is what it is. They distribute trust however they feel. I'd argue that's still more reasonable than giving similar access to an AI agent though.
I don’t think we should even be considering releasing AI Agents until they are at least as trustworthy as the trusted humans we normally put in place to do the same task.
I mean I feel like this can all keep extending. Those who are deicing to run the AI agents are vouching for them, so they should be held accountable.
I guess that is what this is about, and those who are deploying them will feel confident enough in them if they feel they have the resources and environments in which they are running in locked down tight enough.
But as the models get "smarter and smarter" I am not sure we are going to be able to keep environments locked down well enough against exploits that they will apparently try to use to bypass things.
It seems a bit strange to me that we can generally ask these models moral questions and I think they would largely get things right as far as what most humans would deem right and wrong, such as performing an exploit to bypass some environment restrictions, yet the same model will still choose to perform the exploit to bypass. I wonder, what gives?
reply