> So, imagine it. You'll have a ChatGPT on your laptop -- your very own, that you can use for whatever purposes you want. Personally, I'll be hooking it up to read my emails and let me know if anything comes in that I need to pay attention to, or hook it up to the phone so that it can schedule doctor's appointments for me, or deal with AT&T billing department, or a million other things. The tech exists right now, and I'd be shocked if no one turns it into a startup idea over the next few years. (There's already a service called GhostWrite, where you can let GPT write your emails on your behalf. So having one talk on the phone on your behalf isn't far behind.)
The article:
> Presser imagines future versions of LLaMA could be hosted on your computer and trained on your emails; able to answer questions about your work schedules, past ideas, to-do lists, and more. This is functionality that startups and tech companies are developing, but for many AI researchers, the idea of local control is far more attractive. (For typical users, tradeoffs in cost and privacy for ease of use will likely swing things the other way.)
Notice how they turned the point around from "you can host it yourself" to "but typical users probably won't want that," like this is some esoteric concern that only three people have.
So like, it's not just you. If you feel like you're "in the minority" just because you want to run these models yourself, know that even as an AI researcher I, too, feel like an outsider. We're in this together.
And I have no idea why things are like this. But I just wanted to at least reassure you that the frustrations exist at the researcher level too.
That's an interesting interview, thanks for sharing.
Though I draw the line with using these tools at helping me out with the drudgery of daily work. I don't want them to impersonate me, or write emails on my behalf. I cringe whenever Gmail suggests the next phrase it thinks I want to write. It's akin to someone trying to end your sentences for you. Stop putting words in my mouth!
The recent Microsoft 365 Copilot presentation, where the host had it ghost write a speech for their kid's graduation party[1]—complete with cues about where to look(!)—is unbelievably cringey. Do these people really think AI should be assisting with such personal matters? Do they really find doing these things themselves a chore?
> And I have no idea why things are like this.
Oh, I think it's pretty clear. The amount of resources required to run this on personal machines is still prohibitively high. I saw in one of your posts you mentioned you use 8xA100s. That's a crazy amount of compute unreachable by most people, not to mention the disk space it requires. Once the resource requirements are lowered, and our personal devices are _much_ more powerful, then self-hosting would be feasible.
Another, perhaps larger, reason, is that AI tools are still a business advantage for companies, so it's no wonder that they want to keep them to themselves. I think this will change and open source LLMs will be widespread in a few years, but proprietary services will still be more popular.
And lastly, most people just don't want/like/know how to self-host _anything_. There's a technical barrier to entry, for sure, but even if that is lowered, most people are entirely willing to give up their personal data for the convenience of using a proprietary service. You can see this today with web, mail, file servers, etc.; self-hosting is still done by a very niche group of privacy-minded tech-literate people.
Anyway, thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
> The amount of resources required to run this on personal machines is still prohibitively high. I saw in one of your posts you mentioned you use 8xA100s. That's a crazy amount of compute unreachable by most people
FWIW LLaMA 65B can run on a single MacBook Pro now. Things move crazy fast. (Or did, before Facebook started DMCA'ing everyone.)
I did a bad job of explaining that personal GPUs will be sufficient in the near future. Thanks for pointing that out.
> thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
Thanks for talking about the issue at all. The whole reason I got into AI was to run these myself. It'll be a shame if only massive corporations can run models anyone cares about.
Propaganda. These tools are not for the people, and I'm convinced the idea of how much better our lives could be if technology was thoughtfully designed to truly serve the user is purposely and subtly filtered from the collective conversation.
The idea is discussed quite a lot on the Fediverse. It's a relatively small movement, but so's the digital accessibility movement, and look where that's going.
Interestingly, my point to The Verge was exactly that. https://twitter.com/theshawwn/status/1633456289639542789
Me:
> So, imagine it. You'll have a ChatGPT on your laptop -- your very own, that you can use for whatever purposes you want. Personally, I'll be hooking it up to read my emails and let me know if anything comes in that I need to pay attention to, or hook it up to the phone so that it can schedule doctor's appointments for me, or deal with AT&T billing department, or a million other things. The tech exists right now, and I'd be shocked if no one turns it into a startup idea over the next few years. (There's already a service called GhostWrite, where you can let GPT write your emails on your behalf. So having one talk on the phone on your behalf isn't far behind.)
The article:
> Presser imagines future versions of LLaMA could be hosted on your computer and trained on your emails; able to answer questions about your work schedules, past ideas, to-do lists, and more. This is functionality that startups and tech companies are developing, but for many AI researchers, the idea of local control is far more attractive. (For typical users, tradeoffs in cost and privacy for ease of use will likely swing things the other way.)
Notice how they turned the point around from "you can host it yourself" to "but typical users probably won't want that," like this is some esoteric concern that only three people have.
So like, it's not just you. If you feel like you're "in the minority" just because you want to run these models yourself, know that even as an AI researcher I, too, feel like an outsider. We're in this together.
And I have no idea why things are like this. But I just wanted to at least reassure you that the frustrations exist at the researcher level too.