网刊加载中。。。

使用Chrome浏览器效果最佳,继续浏览,你可能不会看到最佳的展示效果,

确定继续浏览么?

复制成功,请在其他浏览器进行阅读

A novel interpolation-based subpixel mapping for hyperspectral image by using pansharpening  PDF

  • WANG Peng 1,2,3
  • YAO Hong-Yu 1
  • ZHANG Gong 1
1. College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China; 2. Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China; 3. Hubei Key Laboratory of Regional Development and Environment Response, Hubei University, Wuhan 430062, China

CLC: O43

Updated:2021-02-20

DOI:10.11972/j.issn.1001-9014.2020.01.010

  • Full Text
  • Figs & Tabs
  • References
  • Authors
  • About
OUTLINE

Abstract

In this paper, a novel interpolation-based subpixel mapping (ISPM) for hyperspectral image by using pansharpening (PAN-ISPM) is proposed. In the proposed method, a novel processing path is added into the existing processing path of ISPM. Firstly, the original coarse hyperspectral image is improved by pansharpening technique in the novel processing path, and the novel fine fraction images are derived by unmixing the improved image. Secondly, the novel fine fraction images from the novel path and the existing fine fraction images from the existing path are integrated to produce the finer fraction images with more spatial-spectral information. Finally, according to the predicted values from the finer fraction images, class labels are allocated into subpixel to obtain the final mapping result. Experimental results show that the proposed method produces the higher mapping accuracy than the existing ISPM methods.

Introduction

Due to limitation of hardware and complexity of environment, hyperspectral image always contains lots of mixed pixels, resulting in the inaccurate land cover class mapping information

1. Subpixel mapping (SPM2 can explore the spatial distribution information of the land cover class at subpixel scale. In SPM, a mixed pixel is changed into S×S subpixels by the zoom factor S, class labels are assigned to subpixel to achieve the mapping results. SPM transforms the fraction images (spectral unmixing results of hyperspectral image) to a hard classification image3.

There are two main SPM types: initialization then optimization type and soft then hard type

4. SPM based on pixel swapping5, perimeter minimization6, value of neighbouring7 and Moran’s index8 all belong to the initialization then optimization type. In this type, the class labels are first randomly assigned to subpixel. The location of subpixel is then gradually transformed by optimization algorithm. Simulating annealing, particle swarm, and genetic algorithm910 are selected as optimization algorithm for this type. However, this type often requires a lot of computation time due to its complex physical structure. The more popular type is soft then hard type. Soft then hard type mainly contains back propagation neural network1112, spatial attraction model13-16, indicator cokriging1718, Hopfield neural network1920.

In addition, the interpolation-based subpixel mapping (ISPM) has been an important method of soft then hard type due to its simple physical meaning. The existing ISPM method basically contains two processing steps

21: 1) interpolation and 2) class allocation. Firstly, the coarse fraction images are utilized to produce the fine fraction images with the predicted values (between 0 and 1) of all land cover classes for all subpixels by interpolation method. Then class allocation method is used to assign class labels to subpixel, producing the final mapping result. In particular, when selecting the bilinear interpolation (BI) or bicubic interpolation (BIC) as interpolation method, the mapping result can be very quickly achieved21]-[28. However, due to the coarse resolution of the original image, the fraction images cannot pick up the full spatial-spectral information of the original image, the mapping result will be affected.

In this paper, using pansharpening technique improves interpolation-based subpixel mapping (PAN-ISPM) is proposed. The original coarse remote sensing image is first fused with the high resolution panchromatic image from the same area by pansharpening technique

29 in a novel processing path, and the improved image is unmixed to obtain the novel fine fraction images. The finer fraction images with more spatial-spectral information are then derived by integrating the two kinds of the fine fraction images from the novel path and the existing path. Finally, the predicted values from the finer fraction images are utilized to allocate hard class labels to all subpixels to achieve mapping result. The proposed method realizes that the more spatial-spectral information of the original image is supplied. Experimental results show that the proposed PAN-ISPM can obtain the better mapping result than the existing ISPM methods.

1 Method

1.1 ISPM model

Suppose S is the zoom factor, the spectral unmixing results of the original coarse remote sensing image are KK is the number of land cover classes) coarse fraction images Lk k=1, 2,…,K), and each mixed pixel is divided into S×S subpixels. Suppose LkPJ is the fraction of the kth class for pixel PJJ=1, 2,…, MM is the number of pixels) and Hk1pj is the predicted value for the kth class at subpixel pjj=1, 2,…, MS2MS2 is the number of subpixels).

As shown in Fig. 1, taking the fraction images Lk as inputs, the interpolation method is utilized to produce the fine fraction images, each of which is composed of MS2 predicted values Hk1pj. Constraints from class fractions should meet the formula as follow:

Nk(PJ)=Round(Lk(PJ)×S2) (1)

where Nk(PJ) is the number of subpixels for the kth class, Round() is a function that takes the integer nearest to Lk(PJ)×S2.

Fig.1 The flowchart of ISPM

图1 ISPM流程图

Finally, class allocation method is utilized to allocate the class labels to all subpixels according to the predicted values.

1.2 PAN-ISPM model

As shown in Fig. 1, we can find that the existing ISPM method is implemented in the coarse fraction images derived from the spectral unmixing. However, due to the coarse resolution of the original image, the spatial-spectral information of the original image is not fully utilized. To solve this problem, the PAN-ISPM is proposed. The flowchart of PAN-ISPM is shown in Fig.2.

Fig.2 The flowchart of proposed PAN-ISPM

图2 提出的PAN-ISPM流程图

Firstly, the resolution of the original image is improved by pansharpening technique in a novel processing path. The main purpose of this paper is to improve the existing ISPM model by the new processing path. Pansharpening technique is just a tool to get new processing path. Therefore, we only consider the role of the new processing path. Due to effectively rendering spatial details and fast implementation, principal component analysis (PCA) is selected as the pansharpening method here. Other more effective pansharpening methods can also be used in the new path, but it is beyond the scope of this article. The novel fine fraction images with predicted values Hk2pjare derived by unmixing the improved image.

Secondly, the finer fraction images with the predicted values Fkpj are obtained by integrating the novel fine fraction images from the novel processing path and the existing fine fraction images from the existing processing path by the appropriate parameter θ. Due to its simple physical meaning, bilinear interpolation

23 is selected as the interpolation method in the existing processing path.

The formula of integrating is given as:

Fkpj=1-θHk1pj+θHk2pj (2)

Finally, class allocation method is utilized to obtain the mapping result according to the predicted values Fkpjfrom the finer fraction images. Linear optimization

30 is employed as class allocation method here.

Since the resolution of the original coarse image is improved by pansharpening technique, the more spatial-spectral information is supplied to improve the final mapping result.

2 Experiment

Five ISPM methods are tested and compared: bilinear interpolation (BI

22, bicubic interpolation (BIC23, spatial-spectral bilinear interpolation (SS-BI21, hybrid interpolation based on parallel paths (HIPP27, and the proposed PAN-ISPM. The accuracy of mapping result is evaluated quantitatively by the percentage of correctly classified pixels (PCC) and Kappa coefficient (Kappa5. All experiments are tested on a Pentium(R) Dual-core Processor (2.20 GHz) with MATLAB R2018 version.

2.1 Simulated Data

The original fine remote sensing image is downsampled by S×S low pass filter to produce the simulated coarse image for quantitative assessment. Since the land cover classes at the subpixel level are known in the downsampled case, we can facilitate direct evaluation of the impact of image registration error on the technique. The original fine hyperspectral image performed on an urban site of the airborne HYDICE is from the mall in Washington DC. As shown in Fig.3(a), the tested region is with 240×240 pixels and 191 spectral bands. As shown in Fig.3(b), the coarse image is generated by degrading the fine data by S=2.

To avoid the effect of errors caused by the acquisition of the panchromatic image, only considering the effect of pansharpening technique, the spectral response of the IKONOS satellite is utilized in the original remote sensing image to create appropriate synthetic panchromatic image

31. The panchromatic image is shown in Fig.3(c). As shown in Fig.3(d), the pansharpening result is close to the original fine image. The appropriate parameter θ is selected as 0.5.

(a)

(b)

(c)

(d)

Fig.3 (a) False color image of Washington DC (bands 65, 52, and 36 for red, green, and blue, respectively). (b) Coarse image (S=2). (c) Panchromatic image. (d) Pansharpening result.

图3 (a) 华盛顿DC数据集的假彩色图像(波段65,52,和36对应红,绿和蓝)(b) 粗糙图像(S=2) (c) 全色图像. (d) 全色锐化结果.

As shown in Fig.4(a), the reference image is obtained from the Fig.3(a) by classification method, including shadow, water, road, tree, grass, roof and trail. The mapping results of the five methods are shown in Figs.4(b)-(f). A visual comparison of the results suggests that the proposed PAN-ISPM model is closer to the reference map due to utilizing more spatial-spectral information.

(a)

(b)

(c)

(d)

(e)

(f)

Fig.4 (a) Reference image, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM

图4 (a) 参考图像, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM

Five ISPM methods are quantitatively evaluated by the classification accuracy of each class, PCC and Kappa. Checking the Table 1, the accuracy of PAN-ISPM is superior to the existing ISPM methods. With respect to the overall accuracy, PAN-ISPM increases the PCC by around 5.6% when compared with HIPP. PAN-ISPM obtains the highest Kappa of 0.842 6.

Table 1 Accuracy evaluation of the five methods.
表1 五种方法的精度评价
ClassBIBICSS-BIHIPPPAN-ISPM
Shadow 73.44% 75.03% 81.62% 82.83% 86.28%
Water 85.56% 88.97% 94.45% 94.73% 95.03%
Road 70.55% 72.74% 76.29% 79.10% 81.16%
Tree 72.45% 75.45% 76.61% 78.64% 79.14%
Grass 74.70% 78.60% 82.74% 83.93% 86.60%
Roof 70.67% 72.98% 77.18% 78.56% 80.02%
Trail 73.88% 75.58% 79.08% 82.53% 84.01%
PCC 76.82% 77.47% 80.72% 81.54% 87.18%
Kappa 0.7356 0.7429 0.7772 0.8055 0.8426

To evaluate the effect of the zoom factor S on the performance of the results, the five methods are tested for the two other zoom factors of 4 and 6. The PCC and Kappa of the five methods for all three zoom factors are shown in Figs.5(a)-(b). We can note that as S increases, the PCC and the Kappa of the five methods decrease. But consistent with the result presented in Table 1, the proposed PAN-ISPM produces the higher PCC and Kappa than the other four methods.

(a)

(b)

Fig.5 (a) PCC (%) of the five methods in relation to zoom factor S, (b) Kappa of the five methods in relation to zoom factor S.

图5 (a) 五种方法中与缩放因子S相关的PCC (%), (b) 五种方法中与缩放因子S相关的Kappa.

2.2 Real data

To better demonstrate the effectiveness of the proposed PAN-ISPM, a real data set is used in experiment 2. A 30-m hyperspectral image is captured by the Hyperion satellite over Rome, Italy. As shown in Fig. 6(a), the tested region is with 300×160 pixels and 198 bands. As shown in Fig. 6(b), the 15-m panchromatic image is obtained from the Landsat 8 panchromatic band over the same area. This corresponds to the scale S=2 between the original coarse remote sensing and the panchromatic image. The pansharpening result is shown in Fig. 6(c). The appropriate parameter θ is selected as 0.6.

(a)

(b)

(c)

Fig.6 (a) False color image of Rome (bands 150, 10, and 24 for red, green, and blue, respectively), (b) Panchromatic image, (c) Pansharpening result.

图6 (a) 罗马数据集的假彩色图像(波段150,10,和24对应红,绿和蓝), (b) 全色图像, (c) 全色锐化结果.

As shown in Fig. 7(a), the reference image includes vegetation, soil, built-up, and water. Figs. 7(b)-(f) show the ISPM results for the five methods. In Figs. 7(b)-(e), there are many speckle artifacts and disconnected holes in the land cover classes. The proposed PAN-ISPM produces more continuous and smoother land cover classes in Fig. 7(f). Because more spatial-spectral information is utilized, the mapping result of PAN-ISPM is closer to the reference image. Table 2 lists the accuracy of each class (%), PCC (%), and Kappa of the five methods. Similar to the previous experimental results in experiment 1, the PAN-ISPM outperforms the other four ISPM methods.

Table 2 Accuracy evaluation of the five methods.
表2 五种方法的精度评价.
ClassBIBICSS-BIHIPPPAN-ISPM
Vegetation 60.08% 61.96% 66.56% 74.50% 75.93%
Soil 60.22% 61.43% 64.93% 65.80% 71.78%
Built-up 81.32% 82.42% 83.97% 84.99% 87.09%
Water 37.18% 44.10% 49.49% 54.36% 61.03%
PCC 70.62% 72.06% 74.89% 77.55% 80.03%
Kappa 0.587 7 0.598 5 0.616 4 0.639 9 0.673 6

(a)

(b)

(c)

(d)

(e)

(f)

Fig.7 (a) Reference image, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM

图7 (a) 参考图像, (b) BI, (c) BIC, (d) SS-BI, (e) HIPP, (f) PAN-ISPM

2.3 Discussion

First, the weight parameter θ is introduced to balance the influence of Hk1pj and Hk2pj on the PAN-ISPM. Here we choose the appropriate parameter value through multiple tests. Experiment 1 and 2 are repeated to evaluate the PCC (%) for ten combinations of θ in the range of [0, 0.9] at an interval of 0.1 in order to determine the most suitable value of θ. As shown in Fig. 8, it can be seen that the most appropriate θ value of Experiments 1 and 2 are 0.5 and 0.6, respectively, the most appropriate value is the one at which the PCC (%) is the highest.

Fig. 8 PCC (%) of the two experiments in relation to weight parameter θ.

图8 两个实验的PCC(%)与权重参数θ的关系.

Second, the computing time is an important index to estimate the performance of ISPM methods. The computing time of five ISPM methods in experiment 1 and 2 is shown in Fig. 9. Because the PAN-ISPM includes processing step of pansharpening, the proposed method is complex and time consuming compared to the existing ISPM methods. This is the cost of obtaining higher mapping precision.

Fig. 9 Computing time of the five ISPM methods in the two experiments

图9 两个实验中五种ISPM方法的计算时间

Finally, the performance of PAN-ISPM depends on pansharpening technique. Therefore, it is necessary to test the effects of different pansharpening methods on the performance of the proposed method. The band-dependent spatial detail (BDSD

31 is selected as another pansharpening method to compare the previous PCA in the experiment 1 and 2. Fig. 10 show the PCC (%) of PAN-ISPM result in relation to two pansharpening methods. As shown in Fig. 10, since BDSD is more effective than PCA, the PCC (%) in BDSD-based PAN-ISPM is higher than that in PCA-based PAN-ISPM. Hence, the more effective pansharpening method can obtain the better mapping result.

Fig. 10 PCC (%) of PAN-ISPM result in relation to BDSD and PCA in the two experiments

图10 两个实验中PAN-ISPM的PCC(%)与BDSD和PCA的关系

3 Conclusion

In this paper, the PAN-ISPM is proposed to improve the mapping result. First of all, the original coarse hyperspectral image is utilized to obtain the improved image by pansharpening in the novel processing path, and the improved image is unmixed to produce the novel fine fraction images. The finer fraction images with more spatial-spectral information e then obtained by integrating the novel fine fraction images from the novel path and the existing fine fraction images from the existing path. Finally, the final mapping result is derived by class allocation method according to the predicted values from the finer fraction images. Because the coarse resolution of the original image is improved by pansharpening in the novel processing path, the more spatial-spectral information of the original image could be fully supplied to ISPM, and the final mapping result is improved. The visual and quantitative comparison with the existing ISPM methods shows the result of the PAN-ISPM is better.

The appropriate parameter θ is selected by multiple tests in this paper. Therefore, an adaptive method for selecting θ is worth studying in future work. In addition, the PAN-ISPM includes more processing steps than the other four ISPM methods. Therefore, it is necessary to optimize the structure of the proposed method and speed up its operation in the future.

References

1

LING FengWU Shen-JunXIAO Feiet al. Sub-pixel mapping of remotely sensed imagery: a review[J]. Journal of Image and Graphics. (凌峰吴胜军肖飞. 遥感影像亚像元定位研究综述. 中国图象图形学报) 2011168): 1335-1345. [百度学术

2

Ling FFoody G M. Super-resolution land cover mapping by deep learning[J]. Remote Sensing Letters. 2019. 106): 598-606. [百度学术

3

Wang QZhang C , and Atkinson P M. Sub-pixel mapping with point constraints[J]. Remote Sensing of Environment. 2020224111817. [百度学术

4

Wang Q MShi W, and Wang L. Allocating classes for soft-then-hard sub-pixel mapping algorithms in units of class[J]. IEEE Transactions on Geoscience and Remote Sensing. 2014525): 2940-2959. [百度学术

5

Atkinson P M. Sub-pixel target mapping from soft-classified remotely sensed imagery[J]. Photogrammetric Engineering and Remote Sensing. 2005717): 839-846. [百度学术

6

Villa AChanussot JBenediktsson J Aet al. Spectral unmixing for the classification of hyperspectral images at a finer spatial resolution[J]. IEEE Journal of Selected Topics in Signal Processing. 201153): 521533. [百度学术

7

Ling FLi WDu Yet al. Land cover change mapping at the subpixel scale with different spatial-resolution remotely sensed imagery[J]. IEEE Geoscience and Remote Sensing Letters. 201181): 182-186. [百度学术

8

Makido YShortridge AMessina J P. Assessing alternatives for modeling the spatial distribution of multiple land-cover classes at subpixel scales[J]. Photogrammetric Engineering and Remote Sensing. 200773935-943. [百度学术

9

He DZhong YFeng Ret al. Spatial-temporal sub-pixel mapping based on swarm intelligence theory. Remote Sensing. 20168894. [百度学术

10

Tong XXu XPlaza A. A new genetic method for subpixel mapping using hyperspectral images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 201699): 4480-4491. [百度学术

11

Nigussie DZurita-Milla R, and Clevers J G P W. Possibilities and limitations of artificial neural networks for subpixel mapping of land cover[J]. International Journal of Remote Sensing. 2011327203-7226. [百度学术

12

XU XiongZHONG Yan-FeiZHANG Liang-Peiet al. A sub-pixel mapping algorithm based on BP neural network with spatial autocorrelation function for remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica许雄钟燕飞张良培. 基于空间自相关BP神经网络的遥感影像亚像元定位. 测绘学报2011403): 307-311. [百度学术

13

Ling FLi XDu Yet al. Sub-pixel mapping of remotely sensed imagery with hybrid intra- and inter-pixel dependence. International Journal of Remote Sensing. 2013341): 341-357. [百度学术

14

Wang PWang L. Soft-then-hard super-resolution mapping based on a spatial attraction model with multiscale sub-pixel shifted images[J]. International Journal of Remote Sensing. 20173815): 4303-4326. [百度学术

15

Lu LHang YDi L. A new spatial attraction model for improving subpixel land cover classification. Remote Sensing. 20179360. [百度学术

16

Wang PWang LLeung Het al. Super-resolution mapping based on spatial-spectral correlation for spectral imagery. IEEE Transactions on Geoscience and Remote Sensing2020online. [百度学术

17

Jin HMountrakis GLi P. A super-resolution mapping method using local indicator variograms. International Journal of Remote Sensing. 20123324): 7747-7773. [百度学术

18

Wang QAtkinson P MShi W. Indicator cokriging-based subpixel mapping without prior spatial structure information[J]. IEEE Transactions on Geoscience and Remote Sensing. 2015531): 309-323. [百度学术

19

Li XDu YLing Fet al. Superresolution mapping of remotely sensed image based on hopfield neural network with anisotropic spatial dependence model. IEEE Geoscience and Remote Sensing Letters. 2014117): 1265-1269. [百度学术

20

Wang PWang LLeung Het al. Subpixel mapping based on hopfield neural network with more prior information. IEEE Geoscience and Remote Sensing Letters. 2019168): 1284-1288. [百度学术

21

Wang PWang L and Chanussot J. Soft-then-hard subpixel land cover mapping based on spatial-spectral interpolation[J]. IEEE Geoscience and Remote Sensing Letters. 20161312): 1851-1854. [百度学术

22

Wang LWang ZDou Zet al. Edge-directed interpolation-based sub-pixel mapping[J]. Remote Sensing Letters. 2013124): 1195-1203. [百度学术

23

Wang Q and Shi W. Utilizing multiple subpixel shifted images in subpixel mapping with image interpolation[J]. IEEE Geoscience and Remote Sensing Letters. 2014114): 798-802. [百度学术

24

Wang PWang LMura M Det al. Using multiple subpixel shifted images with spatial-spectral information in soft-then-hard subpixel mapping. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2017106): 2950-2959. [百度学术

25

Wang QShi W, and Atkinson P M. Sub-pixel mapping of remote sensing images based on radial basis function interpolation[J]. ISPRS Journal of Photogrammetry and Remote Sensing. 2014921-15. [百度学术

26

Y. ChenY. Ge, and D. Song. Superresolution land-cover mapping based on high-accuracy surface modeling. IEEE Transactions on Geoscience and Remote Sensing. 20151212): 2516-2520. [百度学术

27

Wang PZhang GKong Yet al. Superresolution mapping based on hybrid interpolation by parallel paths[J]. Remote Sensing Letters. 102): 149-157. [百度学术

28

Vivone GAlparone LChanussot Jet al. A critical comparison among pansharpening algorithms[J]. IEEE Transactions on Geoscience and Remote Sensing. 2015532565-2585. [百度学术

29

J. Verhoeye and R. De Wulf. Land-cover mapping at sub-pixel scales using linear optimization techniques[J]. Remote Sensing of Environment. 2002791): 96-104. [百度学术

30

T. M. TuP. S. HuangC. L. HungC. P. Chang. A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery[J]. IEEE Geoscience and Remote Sensing Letters. 20041309-312. [百度学术

31

Garzelli ANencini FCapobianco L. Optimal MMSE pan sharpening of very high resolution multispectral images[J]. IEEE Transactions on Geoscience and Remote Sensing. 2008461): 228-236. [百度学术