I think the consensus is that the general purpose transformer-based pertaining models like gpt4 are roughly as good as they’ll be. o1 seems like it will be slightly better in general. So I think it’s fair to say the capability is not there, and even if it was the reliability is not going to be there either, in this generation of models.
It might be true that pretraining scaling is out of juice - I'm rooting for that outcome to be honest - but I don't think it's "consensus". There's a lot of money being bet the other way.
It is the consensus. I can’t think of a single good researcher I know who doesn’t think this. The last holdout might have been Sutskever, and at NeurIPS he said pretaining as we know it is ending because we ran out of data and synthetics can’t save it. If you have an alternative proposal for how it avoids death I’d love to hear it but currently there are 0 articulated plans that seem plausible and I will bet money that this is true of most researchers.
I wrote more about the implications of this elsewhere: https://news.ycombinator.com/item?id=42465598