Many people use phrases like "seeing into the past" when talking about looking at distant objects, without recognizing that such language implies the existence of an absolute and unique time scale that we have known for a century does not exist. A key distinction to make to understand this is the difference between time as a coordinate, and time as an actual physical thing or process. The latter is known in relativity to be the concept of proper time, which is the time registered on a clock. "Looking back in time" when seeing some distant star is not that kind of time, it is actually just coordinate time-- which has a very nonunique and arbitrary contextual convenience associated with it. In my opinion, recognizing the difference between proper time and coordinate time should come well before anyone is even exposed to more intricate details like time dilation and the relativity of simultaneity, and the fact that this rarely actually occurs is why so few people really understand those latter topics.
But before we get into all that, let me give you two examples of how commonly used coordinate times can give two rather different meanings to how "far back in the past" we are seeing distant objects:
1) The Einstein simultaneity convention in special relativity: the simplest example of this is when you observe a distant object with no relative velocity to you, and then you associate your "now" with a time at that distant object that would be reached by a time signal from the central point between you and it, at the same time that you yourself receive that same signal.
2) The cosmological principle: here we associate our "now" with distant points that have the same age as us (13.7 billion years after the Big Bang).
Questions to the reader to get you started in understanding the arbitrariness of coordinate time:
A) Do #1 and #2 associate our "now" the same way to distant events?
B) Which would be relevant to a distant star in our galaxy? To a different galaxy? Is there a smooth transition of some kind, and why?
C) If the universe did not obey the cosmological principle, which it easily could not have, then what would we use to associate our "now" with cosmologically distant events?
Give special attention to C-- for those of you who think that "seeing into the past" really means something when applied to other objects that do not share the past of your own "world line" through spacetime, what is your answer to C?