Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think this benchmark could be slightly misleading to assess coding model. But still very good result.

Yes, SVG is code, but not in a sense of executable with verifiable inputs and outputs.





I love that we are earnestly contemplating the merits of the pelican benchmark. What a timeline.

It's not even halfway up the list of inane things of the AI hype cycle.

But it does have a verifiable output, no more or less than HTML+CSS. Not sure what you mean by "input" -- it's not a function that takes in parameters if that's what you're getting at, but not every app does.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: