I'm just not talking about trying to fish an accurate answer. I think you can do that. I'm talking about getting an answer about something that requires an interaction that gets the model to "understand" the problem (like you would a person) - which GPT can't do.
Something akin to: Here is a niche mathematical problem that does not fit any existing solutions/publications, we need a working but not necessarily optimal solution to do this. ?
Sure, or even an existing algo or method that is applied to something very niche that hasn't been done much (at least in the data GPT is trained on, I guess).
In my experience these things need a little nuance communicated as a part of the problem, but GPT can't get there. It just starts looping over incorrect solutions (rather than modifying them slightly to get it right like a person would).
These aren't crazy advanced things, either. I'm not a genius. I solved the problems when GPT couldn't. I was just trying out GPT4.