Opencv, Python: Eliminating Eventual Narrowing When Stitching Images

Thanks in large part to some great answers on stackoverflow (here, here, and here) I've been having some pretty good success in aligning images. There is one issue, though, as you

Solution 1:

My theory on why this is going on is that the camera wasn't exactly perpendicular to the ground.

This is a good intuition. If the camera is angled, then as it moves towards an object, that object becomes larger in the frame. So if you're stitching that to the previous frame, the current frame needs to shrink to fit to the object in the previous frame.

Full 3x3 homographies include distortions in the x and y directions, but 2x3 affine transformations do not. To stick with your current pipeline, you can try finding an affine or Euclidean (rigid) transformation instead. The difference between them is an affine warp allows for shearing and stretching separately in the x and y directions, Euclidean transforms only do translation, rotation, and uniform scaling. Both preserve parallel lines, whereas a full homography does not, so you could end up with a square image becoming more trapezoidal, and repeating that will shrink your image. An affine warp can still shrink in just one direction, turning a square into a rectangle so it still might shrink. Euclidean transformations can only scale the whole square, so it still might shrink.

Of course, they won't be as perfect matches as findHomography either, but they should be able to get you to close matches without distorting the size as much. There are two options to find Euclidean or affine transformations with OpenCV:

1. estimateRigidTransform() instead of warpPerspective() to get either a rigid warp with the parameter fullAffine=False or an affine warp with fullAffine=True.

2. findTransformECC() with optional parameter motionType=cv2.MOTION_EUCLIDEAN or motionType=cv2.MOTION_AFFINE (but affine is the default so it's not necessary to specify).

You can check out the difference between the algorithms on their documentation pages, or try both to see what works best for you.

If this doesn't work out as well, you can try estimating the homography which warps a frame to be completely perpendicular to the ground. If you do that, you can try applying it to all frames first, and then matching the images. Otherwise, you'll probably want to move to more advanced methods than finding just an homography between each frame.