> Wouldn't this also mean that there's an inherent limit to that sort of model? ...

moelf · 2025-12-09T04:59:41 1765256381

you can always make a new vector that's orthogonal to all the ones currently used and see if the inclusion improves performance on your tasks

scotty79 · 2025-12-09T08:00:14 1765267214

> see if the inclusion improves performance on your tasks

Apparently it doesn't at least not in our models with our training applied to our tasks.

So if we expand one of those 3 things and notice that 17-th vector makes a difference then we are having progress.