Alamaral basically has it right. It really has nothing to do with Quantum or Relativistic effects but everything to do with the angle of light rays coming from the object entering your eye.
This diagram shows how the light from different points of an object travel through the lens of an observers eye and reaches the back of the retina where one "perceives" the object. The more space on the retina the object covers the "larger" that object appears to the observer.
Now due to the inverse square law as Alamaral mentioned previously, the further an object is away from the observer the shallower the angle will be for the light entering the eye. A lot of the light that was traveling from the object that entered the eye when up close will now miss the eye completely because of the increased distance between them. Only light with much shallower angles or close to parallel will now enter the eye.

.
Here's a rough sketch I tried to make to illustrate this.
Since the refracted image on the retina covers a smaller area as compared to when the object was close up, we perceive it as "smaller". No modern physics needed

Bookmarks