> To run Llama 3.1 8B locally, you would need a GPU with a minimum of 16 GB of VRAM, such as an NVIDIA RTX 3090
In full precision, yes. But this talaas chip uses a heavily quantized version (the article calls it "3/6 bit quant", probably similar to Q4_K_M). You dont even need a GPU to run that with reasonable performance, a CPU is fine.
This is what always confused me about VC AI enthusiasm. Their moat is the capital. As AI improves, it destroys their moat. And yet, they are stoked to invest in it, the architects of their own demise.
> If AI gets so good that it can replace all human labor, will capital like money and data centers be the only moat left?
If AI gets good enough to replace all human labor then actual physical moats to keep the hungry, rioting replaced humans away will be the most important moats.
Money is useful mostly for hiring human labor to outcompete others, e.g. Satya Nadella has 100K employees under his command, you don't, so you can't realistically compete with MS today - this is their main moat.
If AI renders human labor a cheap commodity (say you can orchestrate a bunch of agents to develop + market a Windows competitor for $1000 of compute), what used to be "Satya + his army vs. you" now becomes mostly a 1:1 fair fight, which favors the startup.
The incompetent have always pantomimed the competent. It never works. Although the incompetent will always pay a huge amount to try to achieve this fantasy.
The featured blog post where one of their senior engineering PMs presented an allegedly "production grade" Matrix implementation, in which authentication was stubbed out as a TODO, says it all really. I'm glad a quarter of the internet is in such responsible hands.
Management thinks AI tools should make everyone 10x as productive, so they're all trying to run lean teams and load up the remaining engineers with all the work. This will end about as well as the great offshoring of the early 2000s.
Wait till you get AI to write unit tests and tell it the test must pass. After a few rounds it will make the test “assert(true)” when the code cant get the test to pass
No joke. In my company we "sabotaged" the AI initiative led by the CTO. We used LLMs to deliver features as requested by the CTO, but we introduced a couple of bugs here and there (intentionally). As a result, the quarter ended up with more time allocated to fix bugs and tons of customer claims. The CTO is now undoing his initiative. We all have now some time more to keep our jobs.
Thats actively malicious. I understand not going out of your way to catch the LLMs' bugs so as to show the folly of the initiative, but actively sabotaging it is legitimately dangerous behavior. Its acting in bad faith. And i say this as someone who would mostly oppose such an initiative myself
I would go so far as to say that you shouldnt be employed in the industry. Malicious actors like you will contribute to an erosion of trust thatll make everything worse
Might be but sometimes you don’t have another choice when employers are enforcing AIs which have no „feeling“ for context of all business processes involved created by human workers in the years before. Those who spent a lot of love and energy for them mostly. And who are now forced to work against an inferior but overpowered workforce.
I dont like it either but its not malicious. The LLM isnt accessing your homeserver, its accessing corporate information. Your employer can order you to be reckless with their information, thats not malicious, its not your information. You should CYA and not do anything illegal even if your asked. But using LLMs isnt illegal. This is bad faith argument
You're talking about legality again. I'm talking about ethics.
Using LLMs for software development is a safety hazard. It also has a societal risk, because it centralizes more data, more power, more money to tech oligarchs.
It's ethical to fight this. Still not commenting on legality.
You're not forced to work there and use those tools. If you don't like it, then leave the job. Intentionally breaking things is unethical especially when you're receiving a paycheck to do the opposite.
Again, no one is forcing him to be there. He's breaking something on purpose. I think you should read up on ethics because this take "I don't like it therefore whatever I do is ethical" is juvenile.
That's quite the strawman. The reason it's ethical is not that LLM's are unpopular or someone dislikes them. It's ethical because LLMs introduce safety hazards, i.e. they cause harm.
That's extremely unethical. You're being paid to do something and you deliberately broke it which not only cost your employer additional time and money, but it also cost your customers time and money. If I were you, I'd probably just quit and find another profession.
That's not "sabotaged", that's sabotaged, if you intentionally introduced the bugs. Be very careful admitting something like that publicly unless you're absolutely completely sure nobody could map your HN username to your real identity.
They've got capital, but I'd argue that they're long way from model capitalism since some time. There's both over- (regulatory capture) and under regulation (consumer and environment protection) that goes against this and companies have enough sway to influence the law and consequently the market. Free market in the original sense of "free from all forms of economic privilege, monopolies and artificial scarcities" is not even a goal anymore.
You seem to be implying that to fail to enforce the right to sustenance nets negative freedom. It's not clear whether you've weighed the loss in freedom required to enforce this right. Can I presume you believe this right is worth forcing people to give, forcing by threat of armed expropriation or incarceration?
Microsoft fell into this trap in the 90s -- they believed that they could hide the DOS prompt, and make everything "easier" with wizards where you just go through a series of screens clicking "next", "next", "finish".
Yes, it was easier. But it dumbed down a generation of developers.
It took them two decades to try to come up with Powershell, but it was too late.
> Personally, I get creeped out by how many things CC is doing and tokens it's burning in the background. It has a strong "trust me bro" vibe that I dislike.
100% this.
It might be convenient to hide information from non-technical users; but software engineers need to know what is happening. If it is not visible by default, it should be configurable via dotfiles.
There are a lot of Claude Code users who aren't software developers. Maybe they've decided that group is the one they want to cater to? I recognize that won't be a popular decision with the HN crowd, but that doesn't mean it's the wrong one.
I fully agree with you on almost everything you wrote in this thread, but I’m not sure this is the right answer. I myself currently spend a lot of time with CC and belong to that group of developers who don’t care about this problem. It’s likely that I’m not alone. So it doesn’t have to be the least professional audience they serve with this update. It’s possible that Anthropic knows what are they doing (e.g. reducing level of detail to simplify task of finding something more important in the output) and it’s also possible that they are simply making stupid product decisions because they have a cowboy PM who attacks some OKR screaming yahoo. We don’t know. In the end having multiple verbosity levels configured with granularity similar to java loggers would be nice.
Oh totally - I'm definitely not saying that they made the decision to cater to non-dev users, just that it's a possibility. Totally agree with you that at the end of the day, we haven't the foggiest idea.
Yeah, I made a similar point about the tone of ChatGPT responses; to me, I can't imagine why someone would want less information when working and tuning an AI model. However, something tells me they actually have hard evidence that users respond better with less information regardless of what the loud minority say online, and are following that.
100%. Metrics don't lie. I've A/B tested this a lot. Attention is a rare commodity and users will zone out and leave your product. I really dislike this fact
Metrics definitely lie, but generally in a different way to users/others. It's important to not let the metric become the goal, which is what often happens in a metric-heavy environment (certainly Google & FB, not sure about the rest of big tech).
Product managers are fooling themselves if they think they can "improve the user experience" for developers -- developers can't agree on the simplest things such as key bindings (vim, emacs) or identation (tabs, spaces).
Make the application configurable. Developers like to tinker with their tools.
Talas promises a 10x higher throughtput, being 10x cheaper and using 10x less electricity.
Looks like a good value proposition.
reply