The Positional Effect in Soft Classification Accuracy Assessment

Jianyu Gu; Russell G. Congalton

doi:doi:10.11648/j.ajrs.20190702.13

| Peer-Reviewed

The Positional Effect in Soft Classification Accuracy Assessment

Jianyu Gu, Russell G. Congalton

Published in American Journal of Remote Sensing (Volume 7, Issue 2)

Received: 26 August 2019 Accepted: 15 October 2019 Published: 24 October 2019

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

Recent research has included the rapid development of soft classification algorithms and soft classification accuracy assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification accuracy assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.

Published in	American Journal of Remote Sensing (Volume 7, Issue 2)
DOI	10.11648/j.ajrs.20190702.13
Page(s)	50-61
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2019. Published by Science Publishing Group

Keywords

Positional Error, Soft Classification, Accuracy Assessment, Spatial Resolution, Spatial Characteristics

References

[1]	Giri, C., et al., Next generation of global land cover characterization, mapping, and monitoring. International Journal of Applied Earth Observation and Geoinformation, 2013. 25 (1): p. 30-37.
[2]	Sahle, M. and K. Yeshitela, Dynamics of land use land cover and their drivers study for management of ecosystems in the socio-ecological landscape of Gurage Mountains, Ethiopia. Remote Sensing Applications: Society and Environment, 2018. 12: p. 48-56.
[3]	Wang, Y., et al., Remote sensing of land-cover change and landscape context of the National Parks: A case study of the Northeast Temperate Network. Remote Sensing of Environment, 2009. 113 (7): p. 1453-1461.
[4]	Yin, H., et al., Mapping Annual Land Use and Land Cover Changes Using MODIS Time Series. Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014. 7 (8): p. 3421-3427.
[5]	Bartholomé, E. and A. Belward, GLC2000: a new approach to global land cover mapping from Earth observation data. International Journal of Remote Sensing, 2005. 26 (9): p. 1959-1977.
[6]	Bontemps, S., et al., GlobCover 2009: products description and validation report. URL: http://ionia1. esrin.esa.int/docs/GLOBCOVER2009_Validation_Report_2, 2011. 2.
[7]	Congalton, R. G., A review of assessing the accuracy of classifications of remotely sensed data. Remote Sensing of Environment, 1991. 37 (1): p. 35-46.
[8]	Congalton, R. G. and K. Green, Assessing the accuracy of remotely sensed data: principles and practices. 2019: CRC press.
[9]	Story, M. and R. G. Congalton, ACCURACY ASSESSMENT - A USERS PERSPECTIVE. Photogrammetric Engineering and Remote Sensing, 1986. 52 (3): p. 397-399.
[10]	Husak, G. J., B. C. Hadley, and K. C. McGwire, Landsat thematic mapper registration accuracy and its effects on the IGBP validation. Photogrammetric Engineering and Remote Sensing, 1999. 65 (9): p. 1033-1039.
[11]	Hansen, M. C. and B. Reed, A comparison of the IGBP DISCover and University of Maryland 1 km global land cover products. International Journal of Remote Sensing, 2000. 21 (6-7): p. 1365-1373.
[12]	McCallum, I., et al., A spatial comparison of four satellite derived 1 km global land cover datasets. International Journal of Applied Earth Observation and Geoinformation, 2006. 8 (4): p. 246-255.
[13]	Friedl, M. A., et al., Global land cover mapping from MODIS: algorithms and early results. Remote Sensing of Environment, 2002. 83 (1–2): p. 287-302.
[14]	Defourny, P., et al. Accuracy assessment of global land cover maps: lessons learnt from the GlobCover and GlobCorine experiences. in 2010 European Space Agency Living Planet Symposium. 2010.
[15]	Powell, R. L., et al., Sources of error in accuracy assessment of thematic land-cover maps in the Brazilian Amazon. Remote Sensing Of Environment, 2004. 90 (2): p. 221-234.
[16]	Brown, K. M., G. M. Foody, and P. M. Atkinson, Modelling geometric and misregistration error in airborne sensor data to enhance change detection. International Journal of Remote Sensing, 2007. 28 (12): p. 2857-2879.
[17]	Eastman, R. D., et al., Research issues in image registration for remote sensing, in 2007 Ieee Conference on Computer Vision and Pattern Recognition, Vols 1-8. 2007. p. 3233-3240.
[18]	Verbyla, D. L. and T. O. Hammond, Conservative bias in classification accuracy assessment due to pixel-by-pixel comparison of classified images with reference grids. International Journal of Remote Sensing, 1995. 16 (3): p. 581-587.
[19]	Stehman, S. V. and J. D. Wickham, Pixels, blocks of pixels, and polygons: Choosing a spatial unit for thematic accuracy assessment. Remote Sensing of Environment, 2011. 115 (12): p. 3044-3055.
[20]	Gu, J. Y., R. G. Congalton, and Y. Z. Pan, The Impact of Positional Errors on Soft Classification Accuracy Assessment: A Simulation Analysis. Remote Sensing, 2015. 7 (1): p. 579-599.
[21]	Mayaux, P., et al., Validation of the global land cover 2000 map. Ieee Transactions on Geoscience and Remote Sensing, 2006. 44 (7): p. 1728-1739.
[22]	Scepan, J., Thematic validation of high-resolution global land-cover data sets. Photogrammetric Engineering and Remote Sensing, 1999. 65 (9): p. 1051-1060.
[23]	Zhao, Y. Y., et al., Towards a common validation sample set for global land-cover mapping. International Journal of Remote Sensing, 2014. 35 (13): p. 4795-4814.
[24]	Uuemaa, E., et al., Landscape metrics and indices: an overview of their use in landscape research. Living reviews in landscape research, 2009. 3 (1): p. 1-28.
[25]	Leitão, A. B., et al., Measuring landscapes: A planner's handbook. 2012: Island press.
[26]	McGarigal, K. and B. J. Marks, FRAGSTATS: spatial pattern analysis program for quantifying landscape structure. Gen. Tech. Rep. PNW-GTR-351. Portland, OR: US Department of Agriculture, Forest Service, Pacific Northwest Research Station. 122 p, 1995. 351.
[27]	Wickham, J., et al., Thematic accuracy assessment of the 2011 National Land Cover Database (NLCD). Remote Sensing of Environment, 2017. 191: p. 328-341.
[28]	Doan, H. T. X. and G. M. Foody, Increasing soft classification accuracy through the use of an ensemble of classifiers. International Journal of Remote Sensing, 2007. 28 (20): p. 4609-4623.
[29]	Mackin, K. J., et al., Land Surface Cover Classification by Soft Computing Methods using MODIS Satellite Data. Information-an International Interdisciplinary Journal, 2010. 13 (3B): p. 1013-1018.
[30]	Pontius, R. G. and M. L. Cheuk, A generalized cross-tabulation matrix to compare soft-classified maps at multiple resolutions. International Journal of Geographical Information Science, 2006. 20 (1): p. 1-30.
[31]	Patidar, N. and A. K. Keshari, A multi-model ensemble approach for quantifying sub-pixel land cover fractions in the urban environments. International Journal of Remote Sensing, 2018. 39 (12): p. 3939-3962.
[32]	Van Niel, T. G., et al., The impact of misregistration on SRTM and DEM image differences. Remote Sensing of Environment, 2008. 112 (5): p. 2430-2442.
[33]	Dai, X. L. and S. Khorram, The effects of image misregistration on the accuracy of remotely sensed change detection. Ieee Transactions on Geoscience and Remote Sensing, 1998. 36 (5): p. 1566-1577.
[34]	Chen, G., K. Zhao, and R. Powers, Assessment of the image misregistration effects on object-based change detection. ISPRS Journal of Photogrammetry and Remote Sensing, 2014. 87 (0): p. 19-27.
[35]	Foody, G. M., Sample size determination for image classification accuracy assessment and comparison. International Journal of Remote Sensing, 2009. 30 (20): p. 5273-5291.
[36]	Plourde, L. and R. G. Congalton, Sampling method and sample placement: How do they affect the accuracy of remotely sensed maps? Photogrammetric Engineering and Remote Sensing, 2003. 69 (3): p. 289-297.
[37]	Pontius, R. G., Jr. and M. Millones, Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment. International Journal of Remote Sensing, 2011. 32 (15): p. 4407-4429.
[38]	Chen, J., et al., Global land cover mapping at 30m resolution: A POK-based operational approach. ISPRS Journal of Photogrammetry and Remote Sensing, 2015. 103: p. 7-27.
[39]	Liu, X., et al., High-resolution multi-temporal mapping of global urban land using Landsat images based on the Google Earth Engine Platform. Remote Sensing of Environment, 2018. 209: p. 227-239.
[40]	Wang, J., et al., Mapping global land cover in 2001 and 2010 with spatial-temporal consistency at 250m resolution. ISPRS Journal of Photogrammetry and Remote Sensing, 2015. 103: p. 38-47.
[41]	Loveland, T., et al., Development of a global land cover characteristics database and IGBP DISCover from 1 km AVHRR data. International Journal of Remote Sensing, 2000. 21 (6-7): p. 1303-1330.
[42]	Hansen, M. C., et al., Global land cover classification at 1 km spatial resolution using a classification tree approach. International Journal of Remote Sensing, 2000. 21 (6-7): p. 1331-1364.
[43]	Mora, B., et al., Global land cover mapping: Current status and future trends, in Land Use and Land Cover Mapping in Europe. 2014, Springer. p. 11-30.
[44]	Friedl, M. A., et al., MODIS Collection 5 global land cover: Algorithm refinements and characterization of new datasets. Remote Sensing of Environment, 2010. 114 (1): p. 168-182.
[45]	Mayaux, P., et al., A new land-cover map of Africa for the year 2000. Journal of Biogeography, 2004. 31 (6): p. 861-877.
[46]	Zhang, B., et al., Land use and land cover classification for rural residential areas in China using soft-probability cascading of multifeatures. Journal of Applied Remote Sensing, 2017. 11: P. 1-17.
[47]	Chen, Y. H., et al., Subpixel Land Cover Mapping Using Multiscale Spatial Dependence. Ieee Transactions on Geoscience and Remote Sensing, 2018. 56 (9): p. 5097-5106.
[48]	Ma, A. L., et al., Multiobjective Subpixel Land-Cover Mapping. Ieee Transactions on Geoscience and Remote Sensing, 2018. 56 (1): p. 422-435.
[49]	Suresh, M. and K. Jain, Subpixel level mapping of remotely sensed image using colorimetry. Egyptian Journal of Remote Sensing and Space Sciences, 2018. 21 (1): p. 65-72.
[50]	Carmel, Y., Controlling Data Uncertainty via Aggregation in Remotely Sensed Data. Ieee Geoscience And Remote Sensing Letters, 2004. 1 (2): p. 39-41.
[51]	Zhang, M., et al., Impacts of Plot Location Errors on Accuracy of Mapping and Scaling Up Aboveground Forest Carbon Using Sample Plot and Landsat TM Data. IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2013. 10 (6): p. 1483.
[52]	Tan, B., et al., The impact of gridding artifacts on the local spatial properties of MODIS data: Implications for validation, compositing, and band-to-band registration across resolutions. Remote Sensing of Environment, 2006. 105 (2): p. 98-114.
[53]	Congalton, R. G. Thematic and positional accuracy assessment of digital remotely sensed data. in Proceedings of the 7th annual forest inventory and analysis symposium. 2005.
[54]	Huang, Z. and B. G. Lees, Assessing a single classification accuracy measure to deal with the imprecision of location and class: Fuzzy weighted Kappa versus Kappa. Journal of Spatial Science, 2007. 52 (1): p. 1-12.
[55]	Hagen, A., Fuzzy set approach to assessing similarity of categorical maps. International Journal of Geographical Information Science, 2003. 17 (3): p. 235-249.

Cite This Article

Plain Text BibTeX RIS

APA Style

Jianyu Gu, Russell G. Congalton. (2019). The Positional Effect in Soft Classification Accuracy Assessment. American Journal of Remote Sensing, 7(2), 50-61. https://doi.org/10.11648/j.ajrs.20190702.13

Copy | Download

ACS Style

Jianyu Gu; Russell G. Congalton. The Positional Effect in Soft Classification Accuracy Assessment. Am. J. Remote Sens. 2019, 7(2), 50-61. doi: 10.11648/j.ajrs.20190702.13

Copy | Download

AMA Style

Jianyu Gu, Russell G. Congalton. The Positional Effect in Soft Classification Accuracy Assessment. Am J Remote Sens. 2019;7(2):50-61. doi: 10.11648/j.ajrs.20190702.13

Copy | Download

@article{10.11648/j.ajrs.20190702.13,
  author = {Jianyu Gu and Russell G. Congalton},
  title = {The Positional Effect in Soft Classification Accuracy Assessment},
  journal = {American Journal of Remote Sensing},
  volume = {7},
  number = {2},
  pages = {50-61},
  doi = {10.11648/j.ajrs.20190702.13},
  url = {https://doi.org/10.11648/j.ajrs.20190702.13},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajrs.20190702.13},
  abstract = {Recent research has included the rapid development of soft classification algorithms and soft classification accuracy assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification accuracy assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.},
 year = {2019}
}

Copy | Download

TY  - JOUR
T1  - The Positional Effect in Soft Classification Accuracy Assessment
AU  - Jianyu Gu
AU  - Russell G. Congalton
Y1  - 2019/10/24
PY  - 2019
N1  - https://doi.org/10.11648/j.ajrs.20190702.13
DO  - 10.11648/j.ajrs.20190702.13
T2  - American Journal of Remote Sensing
JF  - American Journal of Remote Sensing
JO  - American Journal of Remote Sensing
SP  - 50
EP  - 61
PB  - Science Publishing Group
SN  - 2328-580X
UR  - https://doi.org/10.11648/j.ajrs.20190702.13
AB  - Recent research has included the rapid development of soft classification algorithms and soft classification accuracy assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification accuracy assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.
VL  - 7
IS  - 2
ER  -

Copy | Download

Author Information

Jianyu Gu

Department of Natural Resources and the Environment, University of New Hampshire, Durham, USA
Russell G. Congalton

Department of Natural Resources and the Environment, University of New Hampshire, Durham, USA

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Jianyu Gu, Russell G. Congalton. (2019). The Positional Effect in Soft Classification Accuracy Assessment. American Journal of Remote Sensing, 7(2), 50-61. https://doi.org/10.11648/j.ajrs.20190702.13

Copy | Download

ACS Style

Jianyu Gu; Russell G. Congalton. The Positional Effect in Soft Classification Accuracy Assessment. Am. J. Remote Sens. 2019, 7(2), 50-61. doi: 10.11648/j.ajrs.20190702.13

Copy | Download

AMA Style

Jianyu Gu, Russell G. Congalton. The Positional Effect in Soft Classification Accuracy Assessment. Am J Remote Sens. 2019;7(2):50-61. doi: 10.11648/j.ajrs.20190702.13

Copy | Download

@article{10.11648/j.ajrs.20190702.13,
  author = {Jianyu Gu and Russell G. Congalton},
  title = {The Positional Effect in Soft Classification Accuracy Assessment},
  journal = {American Journal of Remote Sensing},
  volume = {7},
  number = {2},
  pages = {50-61},
  doi = {10.11648/j.ajrs.20190702.13},
  url = {https://doi.org/10.11648/j.ajrs.20190702.13},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajrs.20190702.13},
  abstract = {Recent research has included the rapid development of soft classification algorithms and soft classification accuracy assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification accuracy assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.},
 year = {2019}
}

Copy | Download

TY  - JOUR
T1  - The Positional Effect in Soft Classification Accuracy Assessment
AU  - Jianyu Gu
AU  - Russell G. Congalton
Y1  - 2019/10/24
PY  - 2019
N1  - https://doi.org/10.11648/j.ajrs.20190702.13
DO  - 10.11648/j.ajrs.20190702.13
T2  - American Journal of Remote Sensing
JF  - American Journal of Remote Sensing
JO  - American Journal of Remote Sensing
SP  - 50
EP  - 61
PB  - Science Publishing Group
SN  - 2328-580X
UR  - https://doi.org/10.11648/j.ajrs.20190702.13
AB  - Recent research has included the rapid development of soft classification algorithms and soft classification accuracy assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification accuracy assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.
VL  - 7
IS  - 2
ER  -

Copy | Download