At the same time you're executing fewer instructions, because 64b instruction set is a bit more powerful. There's a balance between many things here.

If they use the X32 ABI, that won't be a problem.

Ah yes of course. But if so, wouldn't they have issues interacting with the driver, since they have to be x64?

AMD64 would bring more cache pressure on the CPU, but also a lot more memory for caching the world, plus more registers, a better instruction set… For a game, I expect memory would make a large difference.