Neurons aren't allowed to vary independently of each other, and neither are pixels; A grayscale image with random pixels is static, not even recognizable as an image. The mind cannot decode those pixels in a seven-dimensional indexing scheme, it can't even decode them in the given two dimensions if you have an array size error and store the same data in an array 87 columns wide. In your analogy, if you put a stop sign into the upper right side of the image, that is always going to be recalled associativity with the green caterpillar you put in the lower left side of the image. These properties don't work so well for memories & imperfect/error-prone but statistically correct biological systems.
The average neuron has 1000 synapses, and for geometric reasons (Synaptic connections take up space) most of those are to other neurons that aren't very far away in 3D space.
You’re not contradicting my point, you’re just using the word “image” to refer to a different concept than I did. That “100x100 pictures of the real world” won’t reflect all the 10,000 dimensions available in “a 100x100 bitmap” is certainly the case—and that is what is meant by saying they lie on a lower-dimensional manifold within that space.
Similarly: yes, physics limits neuronal connectivity. The actual space of neuronal connections lies on a manifold inside the full “n squared, divided by 2” dimensions of connectivity of any old set of n points. That still doesn’t mean neurons can’t represent high-dimensional concepts, because your treatment of physical dimensions as the same thing as concept space is still mistaken. Taking your 1000 synapses number for granted, the input to a given neuron would be 1000-dimensional, not three. If you’re not arguing the concept space is 3d, and merely arguing against those who’d say neuronal connectivity isn’t limited by physical constraints, then I’d advise a reread of the ancestor comments; none of them are saying that.
The average neuron has 1000 synapses, and for geometric reasons (Synaptic connections take up space) most of those are to other neurons that aren't very far away in 3D space.