I have a maxed out M3 Ultra. It runs quantized large open Chinese models pretty well. It's slow-ish, but since I don't use them very frequently, most of the time is waiting to the model to load from disk to RAM.
There are benchmarks on token generation speed out there for some of the large models. You can probably guess the speed for models you're interested in by comparing the sizes (mostly look at the active params).
Currently the main issue for M1-M4 is the prompt "preprocessing" speed. In practical terms, if you have a very long prompt, it's going to take a much long time to process it. IIRC it's due to lack of efficient matrix multiplication operations in the hardware, which I hear is rectified in the M5 architecture. So if you need to process long prompts, don't count on the Mac Studio, at least not with large models.
So in short, if your prompts are relatively short (eg. a couple thousand tokens at most), you need/want a large model, you don't need too much scale/speed, and you need to run inference locally, then Macs are a reasonable option.
For me personally, I got my M3 Ultra somewhat due to geopolitical issues. I'm barred from accessing some of the SOTA models from the US due to where I live, and sometimes the Chinese models are not conveniently accessible either. With the hardware, they can pry DeepSeek R1, Kimi-K2, etc. from my cold dead hands lol.
> Heck look at /r/locallama/ There is a reason its entirely Nvidia.
That's simply not true. NVidia may be relatively popular, but people use all sorts of hardware there. Just a random couple of recent self-reported hardware from comments:
You have a point that at scale everybody except maybe Google is using Nvidia. But r/locallama is not your evidence of that, unless you apply your priors, filter out all the hardware that don't fit your so called "hypotheticals and 'testing grade'" criteria, and engage in circular logic.
PS: In fact locallamma does not even cover your "real world use". Most mentions of Nvidia are people who have older GPUs eg. 3090s lying around, or are looking at the Chinese VRAM mods to allow them run larger models. Nobody is discussing how to run a cluster of H200s there.
There's a dissonance between the claim that it's "independent of the political bodies" and "held to account by respective legislative bodies". Are the legislative bodies not political? In the sense of aren't they elected through a political process?
This is kind of a genuine question from me since I have no idea how these authorities are set up in France or the UK...
We don't consider warehouses & stores to be a "slippery slope" away from toll roads, so no I really don't see any good faith slippery slope argument that connects enforcing the law against X to be the same as government censorship of ISPs.
I mean even just calling it censorship is already trying to shove a particular bias into the picture. Is it government censorship that you aren't allowed to shout "fire!" in a crowded theater? Yes. Is that also a useful feature of a functional society? Also yes. Was that a "slippery slope"? Nope. Turns out people can handle that nuance just fine.
I think I understand what the author is trying to say.
We miss thinking "hard" about the small details. Maybe "hard" isn't the right adjective, but we all know the process of coding isn't just typing stuff while the mind wanders. We keep thinking about the code we're typing and the interactions between the new code and the existing stuff, and keep thinking about potential bugs and issues. (This may or may not be "hard".)
And this kind of thinking is totally different from what Linus Torvalds has to think about when reviewing a huge patch from a fellow maintainer. Linus' work is probably "harder", but it's a different kind of thinking.
You're totally right it's just tools improving. When compilers improved most people were happy, but some people who loved hand crafting asm kept doing it as a hobby. But in 99+% cases hand crafting asm is a detriment to the project even if it's fun, so if you love writing asm yourself you're either out of work, or you grudgingly accept that you might have to write Java to get paid. I think there's a place for lamenting this kind of situation.
Spot on. It’s the lumberjack mourning the axe while holding a chainsaw. The work is still hard. it’s just different.
The friction comes from developers who prioritize the 'craft' of syntax over delivering value. It results in massive motivated reasoning. We see people suddenly becoming activists about energy usage or copyright solely to justify not using a tool they dislike. They will hunt for a single AI syntax error while ignoring the history of bugs caused by human fatigue. It's not about the tech. it's about the loss of the old way of working.
And it's also somewhat egotistical it seems to me. I sense a pattern that many developers care more about doing what they want instead of providing value to others.
I disagree. It's like the lumberjack working from home watching an enormous robotic forestry machine cut trees on a set of tv-screens. If he enjoyed producing lumber, then what he sees on those screens will fill him with joy. He's producing lots of lumber. He's much more efficient than with both axe and chainsaw.
But if he enjoyed being in the forest, and _doesn't really care about lumber at all_ (Because it turns out, he never used or liked lumber, he merely produced it for his employer) then these screens won't give him any joy at all.
That's how I feel. I don't care about code, but I also don't really care about products. I mostly care about the craft. It's like solving sudokus. I don't collect solved sudokus. Once solved I don't care about them. Having a robot solve sudokus for me would be completely pointless.
> I sense a pattern that many developers care more about doing what they want instead of providing value to others.
And you'd be 100% right. I do this work because my employer provides me with enough sudokus. And I provide value back which is more than I'm compensated with. That is: I'm compensated with two things: intellectual challenge, and money. That's the relationship I have with my employer. If I could produce 10x more but I don't get the intellectual challenge? The employer isn't giving me what I want - and I'd stop doing the work.
I think "You do what the employer wants, produce what needs to be produced, and in return you get money" is a simplification that misses the literal forest for all the forestry.
But now you are conflating solving problems with a personal preference of how the problem should be solved. This never bodes well (unless you always prefer picking the method best suited to solve the problem.)
Well as I said, I consider myself compensated with intellectual challenge/stimulus as part of my compensation. It's _why_ I do the work to begin with. Or to put it another way: it's either done in a way I like, or it's probably not done at all.
I'm replaceable after all. If there is someone who is better and more effective at solving problems in some objectively good way - they should have my job. The only reason I still have it is because it seems this is hard to find. Employers are stuck with people who solve problems in the way they like for varying personal reasons and not the objectively best way of solving problems.
The hard part in keeping employees happy is that you can't just throw more money at them to make them effective. Keeping them stimulated is the difficult part. Some times you must accept that you must perhaps solve a problem that isn't the most critical one to address, or perhaps a bad call business wise, to keep employees happy, or keep them at all. I think a lot of the "Big rewrites" are in this category, for example. Not really a good idea compared to maintenance/improvement, but if the alternative is maintaining the old one _and_ lose the staff who could do that?
> And it's also somewhat egotistical it seems to me. I sense a pattern that many developers care more about doing what they want instead of providing value to others.
I use LLMs a lot. They're ridiculously cool and useful.
But I don't think it's fair to categorize anybody as "egotistical". I enjoy programming for the fun puzzley bits. The big puzzles, and even often the small tedious puzzles. I like wiring all the chunks up together. I like thinking about the best way to expose a component's API with the perfect generic types. That's the part I like.
I don't always like "delivering value" because usually that value is "achieve 1.5% higher SMM (silly marketing metric) by the end of the quarter, because the private equity firm that owns our company is selling it next year and they want to get a good return".
Egotistical would be to reject the new tools in principle and be a less efficient developer.
But really, most of us who personally feel sad about the work being replaced by LLMs can still act reasonable, use the new tooling at work like a good employee, and lament about it privately in a blog or something.
> We see people suddenly becoming activists about energy usage or copyright solely to justify not using a tool they dislike.
Maybe you don’t care about the environment (which includes yourself and the people you like), or income inequality, or the continued consolidation of power in the hands of a few deranged rich people, or how your favourite artists (do you have any?) are exploited by the industry, but some of us have been banging the drum about those issues for decades. Just because you’re only noticing it now or don’t care it doesn’t mean it’s a new thing or that everyone else is being duplicitous. It’s a good thing more people are waking up and talking about those.
I agree. I think some of us would rather deal with small, incremental problems than address the big, high-level roadmap. High-level things are much more uncertain than isolated things that can be unit-tested. This can create feelings of inconvenience and unease.
I work with a lot of artists, and selling them on (not totally rejecting) AI has largely been unsuccessful until they both understand the analogies and the specifics of what different tools do.
AI makes you the manager. The models are like GRAs or contract workers, maybe new to their fields but with tireless energy, and you need to be able to instruct them correctly and evaluate their outputs. None of them can do everything, and you'll need to carefully hire the ones you want based on the work you need, which means breaking workflows into batchable parts. If you've managed projects before, you've done this.
Right now, my focus is improving pipelines in composition and arrangement based on an artist's corpus. A lot of them just want to be more productive, and it's a slog to write, then break into parts, etc using modern notation software.
1. tones, and generally the gatekeeping of some Cantonese communities towards people who haven't gotten the tones completely right
2. the lack of learning materials relative to the number of speakers, the confusion between written Chinese and written Cantonese (and also the general lack of the latter)
As they say, "a language is a dialect with an army and navy"... I'll leave it at that.
You seem to be confusing/overgeneralizing the understandable resentment of "some Cantonese" who likely had bad experiences of postcolonialism and/or authoritarian-revanchist state policies. If Hong Kong diaspora has a poor reception towards newcomers to their local microculture, maybe it's because the people attempting to engage are not treading lightly with those actual historical legacies in mind.
"Taiwanese lectures Hong Konger about Hong Kong" is a recurring meme on Threads among local Hong Kongers. I didn't expect I'd experience one here.
I mean, I know I am supposed to refute your point with rational points, but I really don't know what to say except that you're wrong, and you're confusing the cultural divide between Cantonese speakers and non-Cantonese speakers, and the political tensions between Hong Kong and mainland China.
Note that I never said "Hong Kong" in my comment because the majority of Cantonese speakers are actually in mainland China.
Given that linguistics does not have a concept of what makes a language «hard» or not, the language hardness classification is highly subjective and perceptional.
I have already commented on why I do not think that Cantonese tones are hard, so I will leave it at that – it is the first, oft repeated myth that is not based on facts.
> 2. the lack of learning materials relative to the number of speakers […]
On the subject of the availability of learning materials, there would have been a strong case for, e.g. Wu (Shanghainese), Min (Hokkien), Hakka etc – for all of which the learning materials virtually do not exist, and that is true.
With Cantonese, it is a remarkably different situation. My local bookshop has two large shelves stacked with Cantonese textbooks and dictionaries that suit a range of people from vastly different age groups – from toddlers starting to babble to serious advanced learners and anyone in between. More is available online, e.g. Virginia Yip's Routledge series, which includes a comprehensive book on the Cantonese grammar of rarely seen quality and coverage, Robert Bauer and Victor Mair's «ABC Cantonese-English Comprehensive Dictionary», and many more. There are online resources, an open-source, cross-platform «Jyut Dictionary», Google and Apple support the Cantonese keyboard etc.
If their printed versions are not easily locally available, they can be purchased as Kindle books as well.
Granted, Mandarin surpasses Cantonese in terms of the quantity of learning materials, and that is a dry fact.
> […] the confusion between written Chinese and written Cantonese […]
Many languages have quirks or come with a wealth of idiosyncrasies when it comes to how the language is spoken and written down. Burmese, Thai, and Tibetan, for example, are written according to extremely archaic pronunciation rules to the point that spoken and written languages have to be learned separately.
Written Cantonese has existed since at least the Ming dynasty[0][1], but the reasons why there are two distinct forms are entirely different as they go back to replacing Classical Chinese, which had become incomprehensible to anyone in the late 19th century without years of dedicated study, with a modern standard written standard based on northern Chinese varieties.
> […] (and also the general lack of the latter).
This is the second often repeated myth. Many Cantonese speakers believe that Cantonese can only be spoken but not written down, which is patently false – if a language has a writing system, it can be written down with it. When pressed with question «why do you think so», there is typically no answer or «because we have been told so». 口語粵語好容易用漢字寫低，就好似書面粵語咁。 There is a real issue of some native Cantonese words not having dedicated Chinese characters for them, but it is more of a philosophical disgreement between the academics rather than an insurmountable problem.
So, in reality – at least in Hong Kong – since formal literacy has long meant competence in Standard Written Chinese, not in a full Cantonese-written system, schools and institutions tend to penalise written vernacular Cantonese forms in formal contexts – entirely for non-linguistic reasons as explained in [2].
To sum it up, I do not find any of the counterarguments to be compelling, persuasive or supported by linguistic facts which would make Cantonese a «hard» language.
You both are in violent agreement and it is amusing to see in the wild.
As an 外國人 who learned Cantonese as an adult (I moved to HK) I'm jealous of the quantity and quality of materials that exist for learning (not Cantonese). That being said, there are _enough_ materials so it's nowhere near as rough as e.g. Shanghainese.
My opinions on hard language reduces to "is this the first language you're learning from a particular language family?" If so, it's hard to learn. But "is ontologically hard" isn't something that I think is really worth ranking. Any four year old can speak their mother tongue just fine.
But the perception of "hard to learn" did work in my favor for learning Cantonese: as a 鬼佬 who speaks Cantonese I was given lots of latitude to be bad while learning because of that perception. And now I could go back and learn Mandarin now and it would be _much_ simpler than the task that I had in learning Cantonese.
That being said I still write in 口語. Slowly learning 書面語 as I read more and more of it.
(Not sure if you remember or recognize me from this handle. I was with Chaak on the words.hk project . Also Jon spoke highly of you for helping with the tough problems on the fonts :D )
You're absolutely right in theory and in linguistics.
The issue is that among the more common languages that people (outside of language nerds) tend to learn, what I said still holds true for the average learner who's there to learn and whom face the practical difficulties of learning a language, and none of your totally correct linguistic facts really make them less real.
> > […] (and also the general lack of the latter).
> This is the second often repeated myth
The size of written Cantonese corpora was abysmally small up until recent (<10?) years, and much of the content was interwoven with Standard Written Chinese. You still generally can't find written Cantonese on printed materials. Until recent months, LLMs couldn't even write a proper children's story in Cantonese without inadvertently code switching to SWC.
Trust me when I say I'm one of the many people who worked hard to make this "myth" not true (not in linguistic theory but in practice). I never knew it would be thrown back to me like this as a lecture on a random forum lol.
There's a lot more to be done to make Cantonese an assessable language for learners compared with the other major languages. You can compare the linguistic properties of languages all you want, and you'd be absolutely right, but that doesn't make a difference to the prospective learner at all.
Pretty sure more than 3.5% of the people in Hong Kong was protesting a couple weeks after the article came out. It took the CCP about two years and a COVID lockdown to get things under control.
Developing in Hong Kong has been much harder and expensive than before. The high speed rail that connected Hong Kong to the mainland system was (IIRC) the most expensive rail project per kilometer. (They did it anyway since it was a national objective from the central authorities.) And, given the recent tragic fire in Tai Po, there has been a lot more worry about people not being able to afford to renew aging infrastructure (as in residential buildings).
There are benchmarks on token generation speed out there for some of the large models. You can probably guess the speed for models you're interested in by comparing the sizes (mostly look at the active params).
Currently the main issue for M1-M4 is the prompt "preprocessing" speed. In practical terms, if you have a very long prompt, it's going to take a much long time to process it. IIRC it's due to lack of efficient matrix multiplication operations in the hardware, which I hear is rectified in the M5 architecture. So if you need to process long prompts, don't count on the Mac Studio, at least not with large models.
So in short, if your prompts are relatively short (eg. a couple thousand tokens at most), you need/want a large model, you don't need too much scale/speed, and you need to run inference locally, then Macs are a reasonable option.
For me personally, I got my M3 Ultra somewhat due to geopolitical issues. I'm barred from accessing some of the SOTA models from the US due to where I live, and sometimes the Chinese models are not conveniently accessible either. With the hardware, they can pry DeepSeek R1, Kimi-K2, etc. from my cold dead hands lol.
reply