You are describing something that is practically more like a computer algebra system than a number system. To go infinite without infinite storage, you need to store the information required to compute the trailing digits of the number. That is possible with things like pi, which have recursive formulas to compute, but it's not easy for arbitrary numbers.
> That is possible with things like pi, which have recursive formulas to compute, but it's not easy for arbitrary numbers.
It is possible for pretty much all the numbers you could care about. I'm not claiming it is possible for all real numbers though (notice my wording with "express" and "represent"). In fact since this creates an equivalence between real numbers and functions on natural numbers, and not all functions are computable, it follows that some real numbers are not representable because they correspond to non-computable functions. Those that are representable are instead called computable numbers.
How would you get those numbers into the computer anyway? It seems like this would be a practical system to deal with numbers that can be represented exactly in that way, and numbers you can get at from there.
The way every other weird number gets into a computer: through math operations. For example, sqrt(7) is irrational. If you subtract something very close to sqrt(7) from it, then you need to keep making digits.
