Perhaps for now, but I wouldn't count on there always being a Meta spending $$$ to train enormous models and then giving them away for free. What's the long-term game plan for open-source models when the corporate charity inevitably dries up?
If your product is an AI model (OpenAI, Anthropic, etc) you can't give it away for free.
If your product is a social graph w/ ads (Meta), you can.
It's hardly corporate charity:
* Meta releasing these models creates an improvement and tuning ecosystem around it, giving them access to tons of free developer time.
* It's also a strong recruiting tool, for engineers and researchers frustrated by, e.g., Google and OpenAI becoming increasingly closed. They know they can publish at Meta.
* The cost is insignificant. Meta had 30B in revenue just in Q2 2023.
It's great PR. I hear people refer to Mark Zuckerberg as a "real engineer" and other such platitudes and it's doing wonders to reduce the stigma around Meta amongst the tech savvy minority.
Despite Meta being very similar to Google in terms of incentives, and Google nowadays being decidedly uncool. Doesn't hurt that Mark shipped something worth a damn whereas Google has been floundering for ages.
With credit to Google, they were basically neck-and-neck with Facebook AI Research in a lot of ways. Both were publishing text transformers (BERT vs FastText), both were maintaining SOTA inference libraries (Tensorflow vs Pytorch) and both were investing heavily into researching the field further. I'd even argue that Google was the largest contributor to making open-source AI more like Linux and less like a shitty proprietary product.
There's a whole history of recent machine-learning development where both Google and Facebook have worked together and against each other to push things forward. I think it's entirely mistaken to characterize Google as the understudy when in many ways it's the other way around.
I mean I have been watching what Google has been doing in AI with wonder for ages and do agree that they seem to be rather underrated merely because they've recently been caught on the back foot.
At the end of the day though I use Llama and I use GPT4 and I don't use Bard. Google has an amazing legacy around AI but it really hasn't been performing in the last couple of years. I can imagine they'll have a comeback, but one does wonder if Google has lost their mojo.
How is it that Zuckerberg is suddenly an "engineer"? He's a CEO, but just because you run a business, does not mean you actually do any of the underlying technical work. Are they blind to the emperor's clothing?
He's obviously not closing Jira tickets at Meta, but that doesn't mean he's not an engineer. As an example of the positive impact that the Llama releases have had, this post from him has been doing the rounds lately in response to criticsm like yours:
Building open models is a very strong approach to cornering the market on top tier AI researchers. And as other commenters have mentioned, the raw models are not the product - the vast majority of the value is in how they are integrated into useful products.
The corporate charity will not dry up. AI makes it easier to generate content, and Meta's in the business of facilitating the sharing of that content. Content is surface area for ads. AI will also make the virtual realities of the "metaverse", as defined by Mark, easier to reify. It's also a giant marketing and recruiting strategy.
AI does make it easier to generate content, but the type of content it lends itself towards the most is spam. Whether a Facebook where the majority of the content is neural net slurry is something that people will want to engage with once they realise what's going on is an open question I think.
Anecdotally it seems like older demographics are the prime target of the current wave of AI engagement farming on Facebook, because they just don't understand that this technology exists now and assume that all of the "photos" they're seeing are real.
> What’s the long-term game plan for open-source models when the corporate charity inevitably dries up?
Once open models reach and stay at near parity for a while, it’ll make sense for commercial downstream users to support open source community efforts rather than building their own, same as has happened in many other categories of key infrastructure software.
Unless Meta’s bet is that, going forward, models themselves won’t be the competitive differentiator, that it will be about integration. They can give away Llama3,4,5 for free, because no one else can put them Whatsapp or whatever.
I guess many here can think a lot 5D chess business strategies. For me it is just Zuck trying to reach greatness, down in history his name will prob pop up when people think about who brought LLM/AI/AGI? to the masses.
Maybe, but at some point vague strategic moves like that will need to actually justify themselves at the earnings call. Throwing endless millions of dollars down the drain to what, depreciate the value of another companies product, which doesn't even directly compete with Metas main breadwinners? What do they actually get out of this?
I suppose if there's any consolation for the open-source AI community it's that Meta has demonstrated a willingness to burn a lot of money for unclear benefit, they're still single-handedly keeping the VR industry on life support at the expense of about $4 billion per quarter. A decade on from acquiring Oculus and no closer to making it profitable, if Metas AI efforts get the same treatment then the free models will probably keep coming for a while.
Zuckerberg's reasoning on AI/ML seemed very justifiable to me in the Dwarkesh podcast
IIRC,
- Having a better model is a competitive advantage in fighting against spam
- Better models enables facebook itself to understand their code vulnerabilities, better employee productivity etc.
- Being in the frontier of open source, keeps them at an advantage in terms of updates from community
Also Meta is in the business of sharing content. Instagram, Facebook, Threads, and WhatsApp are all about sharing content.
They’re highly incentivized to expand the options for generating content. OpenAI and Anthropic have to make money from the generation, which raises the barrier to creation. Meta can say fuck it, let everyone create and well sell ads against what they create.
