Maybe there's a middle ground: a site that wants to work as well as possible for agents could present a stripped-down standardized page depending on the user agent string, while the agent tries to work well even for pages that haven't implemented that interface?

(or, perhaps, agents could use web accessibility tools if they're set up, incentivizing developers to make better use of them)