If the bar is for LLMs to replace domain experts about four years after introduction then yes, they are failing miserably.
But if you were to go back to 2020 and ask if your take a random human over a the state of the art AI to answer a text question you’d take the random human every time except for arithmetic (and you’d have to write it in math notation and not plain English).
And if you were to ask AI experts when would you chose an AI they’d say at least not for a decade or two, if ever.
I wasn't talking about how impressive AI systems are, or how far they've come. I was talking about the fact that any random human with any experience in a specific field -- even though they are not a domain expert -- is going to do better than an LLM. Or, human common sense >>>> what LLMs are doing.
But if you were to go back to 2020 and ask if your take a random human over a the state of the art AI to answer a text question you’d take the random human every time except for arithmetic (and you’d have to write it in math notation and not plain English).
And if you were to ask AI experts when would you chose an AI they’d say at least not for a decade or two, if ever.