Abstract
In this paper, a novel interpolation-based subpixel mapping (ISPM) for hyperspectral image by using pansharpening (PAN-ISPM) is proposed. In the proposed method, a novel processing path is added into the existing processing path of ISPM. Firstly, the original coarse hyperspectral image is improved by pansharpening technique in the novel processing path, and the novel fine fraction images are derived by unmixing the improved image. Secondly, the novel fine fraction images from the novel path and the existing fine fraction images from the existing path are integrated to produce the finer fraction images with more spatial-spectral information. Finally, according to the predicted values from the finer fraction images, class labels are allocated into subpixel to obtain the final mapping result. Experimental results show that the proposed method produces the higher mapping accuracy than the existing ISPM methods.
Due to limitation of hardware and complexity of environment, hyperspectral image always contains lots of mixed pixels, resulting in the inaccurate land cover class mapping informatio
There are two main SPM types: initialization then optimization type and soft then hard typ
In addition, the interpolation-based subpixel mapping (ISPM) has been an important method of soft then hard type due to its simple physical meaning. The existing ISPM method basically contains two processing step
In this paper, using pansharpening technique improves interpolation-based subpixel mapping (PAN-ISPM) is proposed. The original coarse remote sensing image is first fused with the high resolution panchromatic image from the same area by pansharpening techniqu
Suppose is the zoom factor, the spectral unmixing results of the original coarse remote sensing image are ( is the number of land cover classes) coarse fraction images (=1, 2,…,), and each mixed pixel is divided into subpixels. Suppose is the fraction of the kth class for pixel (=1, 2,…, , is the number of pixels) and is the predicted value for the th class at subpixel (=1, 2,…, , is the number of subpixels).
As shown in
, | (1) |
where is the number of subpixels for the th class, is a function that takes the integer nearest to .

Fig.1 The flowchart of ISPM
图1 ISPM流程图
Finally, class allocation method is utilized to allocate the class labels to all subpixels according to the predicted values.
As shown in

Fig.2 The flowchart of proposed PAN-ISPM
图2 提出的PAN-ISPM流程图
Firstly, the resolution of the original image is improved by pansharpening technique in a novel processing path. The main purpose of this paper is to improve the existing ISPM model by the new processing path. Pansharpening technique is just a tool to get new processing path. Therefore, we only consider the role of the new processing path. Due to effectively rendering spatial details and fast implementation, principal component analysis (PCA) is selected as the pansharpening method here. Other more effective pansharpening methods can also be used in the new path, but it is beyond the scope of this article. The novel fine fraction images with predicted values are derived by unmixing the improved image.
Secondly, the finer fraction images with the predicted values are obtained by integrating the novel fine fraction images from the novel processing path and the existing fine fraction images from the existing processing path by the appropriate parameter . Due to its simple physical meaning, bilinear interpolation
The formula of integrating is given as:
, | (2) |
Finally, class allocation method is utilized to obtain the mapping result according to the predicted values from the finer fraction images. Linear optimizatio
Since the resolution of the original coarse image is improved by pansharpening technique, the more spatial-spectral information is supplied to improve the final mapping result.
Five ISPM methods are tested and compared: bilinear interpolation (BI
The original fine remote sensing image is downsampled by low pass filter to produce the simulated coarse image for quantitative assessment. Since the land cover classes at the subpixel level are known in the downsampled case, we can facilitate direct evaluation of the impact of image registration error on the technique. The original fine hyperspectral image performed on an urban site of the airborne HYDICE is from the mall in Washington DC. As shown in
To avoid the effect of errors caused by the acquisition of the panchromatic image, only considering the effect of pansharpening technique, the spectral response of the IKONOS satellite is utilized in the original remote sensing image to create appropriate synthetic panchromatic image

(a)

(b)

(c)

(d)
Fig.3 (a) False color image of Washington DC (bands 65, 52, and 36 for red, green, and blue, respectively). (b) Coarse image (). (c) Panchromatic image. (d) Pansharpening result.
图3 (a) 华盛顿DC数据集的假彩色图像(波段65,52,和36对应红,绿和蓝)(b) 粗糙图像(S=2) (c) 全色图像. (d) 全色锐化结果.
As shown in

(a)

(b)

(c)

(d)

(e)

(f)

Fig.4 (a) Reference image, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM
图4 (a) 参考图像, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM
Five ISPM methods are quantitatively evaluated by the classification accuracy of each class, PCC and Kappa. Checking the
To evaluate the effect of the zoom factor on the performance of the results, the five methods are tested for the two other zoom factors of 4 and 6. The PCC and Kappa of the five methods for all three zoom factors are shown in

(a)

(b)
Fig.5 (a) PCC (%) of the five methods in relation to zoom factor , (b) Kappa of the five methods in relation to zoom factor .
图5 (a) 五种方法中与缩放因子S相关的PCC (%), (b) 五种方法中与缩放因子S相关的Kappa.
To better demonstrate the effectiveness of the proposed PAN-ISPM, a real data set is used in experiment 2. A 30-m hyperspectral image is captured by the Hyperion satellite over Rome, Italy. As shown in

(a)

(b)

(c)
Fig.6 (a) False color image of Rome (bands 150, 10, and 24 for red, green, and blue, respectively), (b) Panchromatic image, (c) Pansharpening result.
图6 (a) 罗马数据集的假彩色图像(波段150,10,和24对应红,绿和蓝), (b) 全色图像, (c) 全色锐化结果.
As shown in

(a)

(b)

(c)

(d)

(e)

(f)

Fig.7 (a) Reference image, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM
图7 (a) 参考图像, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM
First, the weight parameter is introduced to balance the influence of and on the PAN-ISPM. Here we choose the appropriate parameter value through multiple tests. Experiment 1 and 2 are repeated to evaluate the PCC (%) for ten combinations of θ in the range of [0, 0.9] at an interval of 0.1 in order to determine the most suitable value of . As shown in

Fig. 8 PCC (%) of the two experiments in relation to weight parameter .
图8 两个实验的PCC(%)与权重参数θ的关系.
Second, the computing time is an important index to estimate the performance of ISPM methods. The computing time of five ISPM methods in experiment 1 and 2 is shown in

Fig. 9 Computing time of the five ISPM methods in the two experiments
图9 两个实验中五种ISPM方法的计算时间
Finally, the performance of PAN-ISPM depends on pansharpening technique. Therefore, it is necessary to test the effects of different pansharpening methods on the performance of the proposed method. The band-dependent spatial detail (BDSD

Fig. 10 PCC (%) of PAN-ISPM result in relation to BDSD and PCA in the two experiments
图10 两个实验中PAN-ISPM的PCC(%)与BDSD和PCA的关系
In this paper, the PAN-ISPM is proposed to improve the mapping result. First of all, the original coarse hyperspectral image is utilized to obtain the improved image by pansharpening in the novel processing path, and the improved image is unmixed to produce the novel fine fraction images. The finer fraction images with more spatial-spectral information e then obtained by integrating the novel fine fraction images from the novel path and the existing fine fraction images from the existing path. Finally, the final mapping result is derived by class allocation method according to the predicted values from the finer fraction images. Because the coarse resolution of the original image is improved by pansharpening in the novel processing path, the more spatial-spectral information of the original image could be fully supplied to ISPM, and the final mapping result is improved. The visual and quantitative comparison with the existing ISPM methods shows the result of the PAN-ISPM is better.
The appropriate parameter is selected by multiple tests in this paper. Therefore, an adaptive method for selecting is worth studying in future work. In addition, the PAN-ISPM includes more processing steps than the other four ISPM methods. Therefore, it is necessary to optimize the structure of the proposed method and speed up its operation in the future.
References
LING Feng, WU Shen-Jun, XIAO Fei, et al. Sub-pixel mapping of remotely sensed imagery: a review[J]. Journal of Image and Graphics. (凌峰, 吴胜军, 肖飞, 等. 遥感影像亚像元定位研究综述. 中国图象图形学报) 2011, 16(8): 1335-1345. [百度学术]
Ling F , Foody G M. Super-resolution land cover mapping by deep learning[J]. Remote Sensing Letters. 2019. 10(6): 598-606. [百度学术]
Wang Q , Zhang C , and Atkinson P M. Sub-pixel mapping with point constraints[J]. Remote Sensing of Environment. 2020, 224: 111817. [百度学术]
Wang Q M, Shi W, and Wang L. Allocating classes for soft-then-hard sub-pixel mapping algorithms in units of class[J]. IEEE Transactions on Geoscience and Remote Sensing. 2014, 52(5): 2940-2959. [百度学术]
Atkinson P M. Sub-pixel target mapping from soft-classified remotely sensed imagery[J]. Photogrammetric Engineering and Remote Sensing. 2005, 71(7): 839-846. [百度学术]
Villa A, Chanussot J, Benediktsson J A, et al. Spectral unmixing for the classification of hyperspectral images at a finer spatial resolution[J]. IEEE Journal of Selected Topics in Signal Processing. 2011, 5(3): 521–533. [百度学术]
Ling F, Li W, Du Y, et al. Land cover change mapping at the subpixel scale with different spatial-resolution remotely sensed imagery[J]. IEEE Geoscience and Remote Sensing Letters. 2011, 8(1): 182-186. [百度学术]
Makido Y, Shortridge A, Messina J P. Assessing alternatives for modeling the spatial distribution of multiple land-cover classes at subpixel scales[J]. Photogrammetric Engineering and Remote Sensing. 2007, 73: 935-943. [百度学术]
He D, Zhong Y, Feng R, et al. Spatial-temporal sub-pixel mapping based on swarm intelligence theory. Remote Sensing. 2016, 8: 894. [百度学术]
Tong X, Xu X, Plaza A. A new genetic method for subpixel mapping using hyperspectral images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2016, 9(9): 4480-4491. [百度学术]
Nigussie D, Zurita-Milla R, and Clevers J G P W. Possibilities and limitations of artificial neural networks for subpixel mapping of land cover[J]. International Journal of Remote Sensing. 2011, 32:7203-7226. [百度学术]
XU Xiong, ZHONG Yan-Fei, ZHANG Liang-Pei, et al. A sub-pixel mapping algorithm based on BP neural network with spatial autocorrelation function for remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica许雄, 钟燕飞, 张良培, 等. 基于空间自相关BP神经网络的遥感影像亚像元定位. 测绘学报, 2011, 40(3): 307-311. [百度学术]
Ling F, Li X, Du Y, et al. Sub-pixel mapping of remotely sensed imagery with hybrid intra- and inter-pixel dependence. International Journal of Remote Sensing. 2013, 34(1): 341-357. [百度学术]
Wang P, Wang L. Soft-then-hard super-resolution mapping based on a spatial attraction model with multiscale sub-pixel shifted images[J]. International Journal of Remote Sensing. 2017, 38(15): 4303-4326. [百度学术]
Lu L, Hang Y, Di L. A new spatial attraction model for improving subpixel land cover classification. Remote Sensing. 2017, 9: 360. [百度学术]
Wang P, Wang L, Leung H, et al. Super-resolution mapping based on spatial-spectral correlation for spectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 2020, online. [百度学术]
Jin H, Mountrakis G, Li P. A super-resolution mapping method using local indicator variograms. International Journal of Remote Sensing. 2012, 33(24): 7747-7773. [百度学术]
Wang Q, Atkinson P M, Shi W. Indicator cokriging-based subpixel mapping without prior spatial structure information[J]. IEEE Transactions on Geoscience and Remote Sensing. 2015, 53(1): 309-323. [百度学术]
Li X, Du Y, Ling F, et al. Superresolution mapping of remotely sensed image based on hopfield neural network with anisotropic spatial dependence model. IEEE Geoscience and Remote Sensing Letters. 2014, 11(7): 1265-1269. [百度学术]
Wang P, Wang L, Leung H, et al. Subpixel mapping based on hopfield neural network with more prior information. IEEE Geoscience and Remote Sensing Letters. 2019,16(8): 1284-1288. [百度学术]
Wang P, Wang L and Chanussot J. Soft-then-hard subpixel land cover mapping based on spatial-spectral interpolation[J]. IEEE Geoscience and Remote Sensing Letters. 2016, 13(12): 1851-1854. [百度学术]
Wang L, Wang Z, Dou Z, et al. Edge-directed interpolation-based sub-pixel mapping[J]. Remote Sensing Letters. 2013, 12(4): 1195-1203. [百度学术]
Wang Q and Shi W. Utilizing multiple subpixel shifted images in subpixel mapping with image interpolation[J]. IEEE Geoscience and Remote Sensing Letters. 2014, 11(4): 798-802. [百度学术]
Wang P, Wang L, Mura M D, et al. Using multiple subpixel shifted images with spatial-spectral information in soft-then-hard subpixel mapping. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2017, 10(6): 2950-2959. [百度学术]
Wang Q, Shi W, and Atkinson P M. Sub-pixel mapping of remote sensing images based on radial basis function interpolation[J]. ISPRS Journal of Photogrammetry and Remote Sensing. 2014, 92: 1-15. [百度学术]
Y. Chen, Y. Ge, and D. Song. Superresolution land-cover mapping based on high-accuracy surface modeling. IEEE Transactions on Geoscience and Remote Sensing. 2015, 12(12): 2516-2520. [百度学术]
Wang P, Zhang G, Kong Y, et al. Superresolution mapping based on hybrid interpolation by parallel paths[J]. Remote Sensing Letters. 10(2): 149-157. [百度学术]
Vivone G, Alparone L, Chanussot J, et al. A critical comparison among pansharpening algorithms[J]. IEEE Transactions on Geoscience and Remote Sensing. 2015, 53: 2565-2585. [百度学术]
J. Verhoeye and R. De Wulf. Land-cover mapping at sub-pixel scales using linear optimization techniques[J]. Remote Sensing of Environment. 2002, 79(1): 96-104. [百度学术]
T. M. Tu, P. S. Huang, C. L. Hung, C. P. Chang. A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery[J]. IEEE Geoscience and Remote Sensing Letters. 2004, 1: 309-312. [百度学术]
Garzelli A, Nencini F, Capobianco L. Optimal MMSE pan sharpening of very high resolution multispectral images[J]. IEEE Transactions on Geoscience and Remote Sensing. 2008, 46(1): 228-236. [百度学术]