Location isn't really a thing right? It's just a chain of "over here" and "over there" tied inexorably with space. There is only ever "here", the place you occupy while things move around you. Behind is places you have been, forward is places you might be.

Reading that formed an interesting question in my (also non-physicist) mind:

If we can, at most, take advantage of relativity to slow down our own time frame, then time could just be our way to describe the pace of how space changes around us following simple causality.

But if, on the other hand, it is possible to move backwards through time, wouldn't the universe have to necessarily exist not only as a giant block of eternally changing 3D space, but as a giant block of 4D spacetime one could move around in? And would that mean predetermined past and future, or would that 4D block of spacetime change, too, advancing through meta-time, continually changing future and past of the universe?

ScienceClic has a cool video, stipulating that we live in 4D spacetime and are bound to always move forward at light speed. If we stand still in 3D space, we move forwards in time at light speed. If we accelerate in 3D space, we change out motion vector from only pointing forwards in time to pointing slightly sideways (up to completely sideways, i.e. time stops, if we were able to move at light speed). But there may be now way to do a 180 involving the time axis like we could do involving the other 3 axes.