Because without the secondary market there’d be no incentive for anyone to give up a domain. At least now the domains actually end up with parties willing to pay a fair price for them.
Nobody is proposing we get rid of the secondary market, so you can stop proposing that straw man argument.
I'm not saying we should get rid of the secondary market. I'm saying the secondary market should consist of entities who are making a good-faith effort to make use of the domains, not middlemen who actively suck value out of the system while contributing nothing.
Please explain why you think this.
Also, what could be more silly than doing nothing with it, except wait for a sufficiently-funded buyer?