Optical Flow with Horn-Schunck, current frame and n-th frame back

I am currently working on my master thesis with the topic of "Concept for Optical Flow application in lunar navigation." Using images from this site, I created a video (160x120, 20 fps) that simulates a flight over the moon's surface (assuming the camera is mounted on the bottom side of a spacecraft with constant height above the moon, the camera is pointing down). The video "moves" with a velocity of 1 pixel per frame, meaning there is one vertical line of new pixels entering the video from one side while another vertical line of old pixels leaves the video via the other edge. For optical flow I use the Horn-Schunck method.

My problem is an unexpected change in output values with change of the N-value (which determines the number of frames between the reference frame and the current frame). To my understanding increasing this value should also increase the output of the optical flow block, since the distance each pixel travels increases with the two frames being more apart from each other. But instead the output value decreases with increasing N. Setting N to 2 gives a mean velocity of 0.08, increasing N by 1 constantly decreases the output by 50%. If I'm not mistaken the output unit of optical flow is [pixel / n-th frame back], to convert it into [pixel / frames] you have to divide the output by the value of N [frames / n-th frame back], which also means the output should increase with increasing N to keep the [pixel / frames]-value constant.

Since the video (and all objects in the video) moves with a velocity of 1 pixel per frame, converting the output of the optical flow block into [pixel / frames] should give a roughly constant value of 1 regardless of N. But the biggest result the optical flow block gives me is 0.08, which isn't even close to 1. Am I making any wrong assumptions here or could there be a problem because of the video I use (it doesn't have any color information, it's just "Intensity")?