Yes, you are entirely correct. Making a language that’s fully ‘culturally neutral’ is likely impossible, but even so, Esperanto could have done much better in this regard (and as far as I know, some other constructed languages do much better in this regard).
Even then, it's a very big jump from ‘people in some countries already know this and don’t need to learn at all’ to ‘people in some countries will have an easier time learning since they’ll recognise a lot of the words’; making this jump would already be a great improvement upon the status quo. Out of the languages which make the jump, Esperanto is by far the most widely spoken and otherwise used in practice.
Even then, it's a very big jump from ‘people in some countries already know this and don’t need to learn at all’ to ‘people in some countries will have an easier time learning since they’ll recognise a lot of the words’; making this jump would already be a great improvement upon the status quo. Out of the languages which make the jump, Esperanto is by far the most widely spoken and otherwise used in practice.