The Mapping-Adaptive Convolution: A Fundamental Theory for Homography or Perspective Invariant Matching Methods

If the local area of a three-dimensional object surface can be considered as a plane, the deformation between its two images captured from different camera placements is modelled by a homography. By tuning the parameters in a homographic mapping, all possible deformations caused by the change of camera placement can be simulated for the local feature matching method. Since aliasing may happen when resampling the original image to the geometry of the simulated image, an antialiasing convolution must be applied before resampling. However, the antialiasing convolution itself must also be homography-adaptive. In the scale invariant feature transform (SIFT) or affine-SIFT (ASIFT) method, the similitude or affine rectification scheme of the convolution is applied to solve this problem under similitude or affine mapping. However, these schemes will not work under homographic or perspective mapping. Although the perspective invariant matching method (perspective-SIFT or PSIFT) has been proposed in some references, the antialiasing scheme with perspective-adaption has not been proposed. This paper will show that the standard convolution is not adaptive to the change of planar mapping, and the simulated images under the same simulated camera placement will not be identical if they are resampled from different original images captured from different camera placements. To solve this issue, a natural extension of the standard convolution, the mapping-adaptive convolution (MA-convolution), is proposed, and its mapping-adaption is proved mathematically in this paper. Based on this novel convolution, the homography invariant simulation scheme can be modelled. We have applied the MA-convolution to the antialiasing scheme in the PSIFT method, and the effectiveness of the MA-convolution has been verified experimentally.