I'm not the one building out datacenters but I believe the power consumption is the reason for the devaluation. It's the same reasons we saw bitcoin miners throw all their ASICs in the bin every 6 months. At some point it becomes cheaper to buy new hardware than to keep running the old inefficient chips, when the power savings of new chips exceed the purchase price of the new hardware.
These AI data centers are chewing up unimaginable amounts of power, so if nvidia releases a new chip that does the same work in half the power consumption. That whole datacenter of GPUs is massively devalued.
The whole AI industry is looking like there won't be a first movers advantage, and if anything there will be a late mover advantage when you can buy the better chips and skip burning money on the old generations.
The rule of thumb i heard was that over the "useful lifetime" of a chip the cost of energy+maintenance/infrastructure is about the same as the cost of the chip. IIRC the energy was some like 15-20% of the overall cost over the "useful lifetime". I'm putting it in quotes because it's kind of a cyclical definition.
That makes me wonder if it's more a performance thing than an energy cost thing. I guess you also have to factor the finite supply of reliable energy hookups for these things, so if you're constrained on total kwH consumption your only way to more TOPs is upgrade. Probably ties in with real estate/permitting difficulty too. I guess what I'm picturing is if energy availability(not cost) and real estate availability/permitting timelines weren't issues, that %20 of cost probably wouldn't look too bad. So it's probably those factors combined. Market pricing dynamics are hard :/
I didn't know the recycle time of the Asics was that fast! That's an interesting point about the first-mover. I would counter that a large part of the first move value in this case is ai engineer experience and grabbing exception talent early. But with Facebook and these guys paying for them like athletes, I'd guess that experience build up and talent retention aren't as robust. Openai lost most of it's best, but maybe that's an exceptional example..
All of that to say, yeah, first mover advantage seems dulled in this situation.
Who knows, maybe apple is playing the long game and betting on just that.
These AI data centers are chewing up unimaginable amounts of power, so if nvidia releases a new chip that does the same work in half the power consumption. That whole datacenter of GPUs is massively devalued.
The whole AI industry is looking like there won't be a first movers advantage, and if anything there will be a late mover advantage when you can buy the better chips and skip burning money on the old generations.