|
| |
Image classification Comparison
White paper on comparison of image classification using pixel based and
object oriented image analysis processes.
Glenn H. Lawson
USDA, NRCS, NRI&A Institute
817-509-3524
Abstract
Current accurate land cover mapping is needed by various governmental and
private segments of our society for the use, management and evaluation of many
natural resources. This paper compares object oriented image analysis and pixel
based classification of a satellite image to generate a land cover thematic map.
Pixel based classification is based on only the sensor values selected by the
person doing the classification. The technician may supply the number of land
cover classes to be produced by the classification process and allow the
classification software to develop signature areas for each cover type
(unsupervised classification). The technician may also choose to supply specific
areas in the unclassified data that represent specific cover types and develop
signature areas for each type (supervised classification). In either case, the
same mathematical algorithm is used to classify the raw data as selected by the
technician. The classification process evaluates each pixel by the information
provided by the signature information. Each pixel will be placed into a group
represented by the signature information until the process has satisfied a
technician-defined number if grouping iterations at a specified confidence
level.
The object oriented image analysis used for comparison mainly differed from the
pixel based classification because of a pre-classification process of
"Multi-resolution Segmentation" The concept behind segmentation is that
important semantic information, necessary to interpret an image, is not
represented in single pixels but in meaningful image objects and their mutual
relationships (Martin Baatz et. al, 2001). An automatic segmentation process was
formed prior to classification of the raw imagery. This segmentation process
results in the condensing of information in the raw data and an extraction of
image objects. The formation of the objects is carried out in a way that an
overall homogeneous resolution is kept. The segmentation algorithm does not only
rely on the single pixel value, but also on pixel spatial continuity (texture,
topology). The formatted objects have now not only the value and statistic
information of the pixels that they consist. They carry also texture, form
(spatial features) and topology information in a common attribute table. (Ioannis
Manakos, 2001) The organized image objects carry not only the value and
statistical information of the pixels of which they consist, but also
information on texture and shape as well as their position within the
hierarchical network (Ambiente Humano, 2000). The basic difference, especially
when compared to pixel-based procedures, is that object oriented analysis does
not classify single pixels, but rather image objects which are extracted in a
previous image segmentation step.
Project
The project evaluated satellite image classification using ERDAS Imagine and
Definiens Imaging eCognition software. The satellite data used was Land sat TM
for Johnson County, Iowa. The USDA/NRCS NRI point data was used in the
classification process and classified image accuracy assessment. Signatures were
developed from the NRI data and a supervised nearest neighbor classification was
completed using Imagine software. An object oriented image analysis and a
nearest neighbor classification was completed using eCognition. More about the
software used can be found at ERDAS Imagine
and Definiens Imaging eCognition
Several methods were used to evaluate the resulting classifications. The first
evaluation described here is a system used by most image classification
processes. An error matrix, accuracy report and Kappa Statistics report for 1992
and 1997 using each classification process follows. NRI data was used as
Reference Data.
|
Table 1: Error Matrix for 1997 TM Classification using supervised pixel base
|
| Classified Data |
Reference Data |
| |
Corn |
Soybean |
Grass |
Tree |
Water |
Total Row |
| Corn |
141 |
13 |
14 |
14 |
1 |
183 |
| Soybean |
22 |
128 |
20 |
4 |
0 |
175 |
| Grass |
22 |
16 |
91 |
5 |
1 |
135 |
| Tree |
24 |
9 |
17 |
54 |
0 |
104 |
| Water |
0 |
0 |
0 |
0 |
13 |
13 |
| Total Column |
77 |
142 |
15 |
209 |
166 |
610 |
|
Table 2: Accuracy Report for 1997 TM Classification using supervised
pixel base |
| Class Name |
Reference Totals |
Classified Totals |
Number Correct |
Producers Accuracy |
Users Accuracy |
| Corn |
209 |
183 |
141 |
67.46% |
77.05% |
| Soybean |
166 |
175 |
128 |
77.11% |
73.14% |
| Grass |
142 |
135 |
91 |
64.08% |
67.41% |
| Tree |
77 |
104 |
54 |
70.13% |
51.92% |
| Water |
15 |
13 |
13 |
86.67% |
100.00% |
| Totals |
610 |
610 |
427 |
|
|
| Overall Classification Accuracy = 70.00% |
|
Table 3: Kappa (K^) Statistics for 1997 TM Classification using
supervised pixel base |
| Class Name |
Kappa |
| Corn |
0.6509 |
| Soybean |
0.6310 |
| Grass |
0.5752 |
| Tree |
0.4498 |
| Water |
1.0000 |
| Overall Kappa Statistics = 0.5976 |
|
Table 4: Error Matrix for 1997 TM Classification using "Multi-resolution
Segmentation" process |
| Classified Data |
Reference Data |
| |
Corn |
Soybean |
Grass |
Tree |
Water |
Total Row |
| Corn |
188 |
23 |
22 |
17 |
1 |
251 |
| Soybean |
6 |
118 |
6 |
0 |
0 |
131 |
| Grass |
2 |
3 |
2 |
49 |
0 |
56 |
| Tree |
2 |
3 |
2 |
49 |
0 |
56 |
| Water |
0 |
0 |
0 |
0 |
13 |
13 |
| Total Column |
210 |
166 |
144 |
77 |
16 |
614 |
|
Table 5: Accuracy Totals for 1997 TM Classification using
"Multi-resolution Segmentation" process |
| Class Name |
Reference |
Classified Totals |
Number Correct |
Producers Accuracy |
Users Accuracy |
| Corn |
210 |
251 |
188 |
89.52% |
74.90% |
| Soybean |
166 |
131 |
118 |
71.08% |
90.08% |
| Grass |
144 |
163 |
114 |
79.17% |
69.94% |
| Tree |
77 |
56 |
49 |
63.64% |
87.50% |
| Water |
15 |
13 |
13 |
71.08% |
90.08% |
| Totals |
614 |
614 |
482 |
|
|
|
Table 6: Kappa (K^) Statistics for 1997 TM Classification using
"Multi-resolution Segmentation" process |
| Class Name |
Kappa |
| Corn |
0.6185 |
| Soybean |
0.8640 |
| Grass |
0.6073 |
| Tree |
0.8571 |
| Water |
1.0000 |
|
Table 7: Error Matrix for 1992 TM Classification using supervised pixel
base |
| Classified Data |
Reference Data |
| |
Corn |
Soybean |
Grass |
Tree |
Water |
Total Row |
| Corn |
157 |
16 |
39 |
11 |
1 |
224 |
| Soybean |
86 |
104 |
36 |
8 |
1 |
235 |
| Grass |
8 |
1 |
61 |
4 |
0 |
74 |
| Tree |
8 |
0 |
9 |
59 |
0 |
76 |
| Water |
0 |
0 |
0 |
1 |
14 |
15 |
| Total Column |
259 |
121 |
145 |
83 |
16 |
624 |
|
Table 8: Accuracy Report for 1992 TM Classification using supervised
pixel base |
| Class Name |
Reference Totals |
Classified Totals |
Number Correct |
Producers Accuracy |
Users Accuracy |
| Corn |
259 |
224 |
157 |
60.62% |
70.09% |
| Soybean |
121 |
235 |
104 |
85.95% |
44.26% |
| Grass |
145 |
74 |
61 |
42.07% |
82.43% |
| Tree |
83 |
76 |
59 |
71.08% |
77.63% |
| Water |
16 |
15 |
14 |
87.50% |
93.33% |
| Totals |
624 |
624 |
395 |
|
|
|
Table 9: Kappa (K^) Statistics for 1992 TM Classification using
supervised pixel base |
| Class Name |
Kappa |
| Corn |
0.4886 |
| Soybean |
0.3085 |
| Grass |
0.7711 |
| Tree |
0.7420 |
| Water |
1.0000 |
|
Table 10: Error Matrix for 1992 TM Classification using
"Multi-resolution Segmentation" process |
| Classified Data |
Reference Data |
| |
Corn |
Soybean |
Grass |
Tree |
Water |
Total Row |
| Corn |
203 |
23 |
17 |
5 |
0 |
248 |
| Soybean |
17 |
88 |
3 |
1 |
0 |
109 |
| Grass |
38 |
10 |
120 |
21 |
2 |
191 |
| Tree |
3 |
0 |
7 |
56 |
1 |
67 |
| Water |
0 |
1 |
0 |
0 |
13 |
14 |
| Total Column |
261 |
122 |
147 |
83 |
16 |
629 |
|
Table 11: Accuracy Totals for 1992 TM Classification using
"Multi-resolution Segmentation" process |
| Class Name |
Reference Totals |
Classified Totals |
Number Correct |
Producers Accuracy |
Users Accuracy |
| Corn |
261 |
248 |
203 |
77.78% |
81.85% |
| Soybean |
122 |
109 |
88 |
72.13% |
80.73% |
| Grass |
147 |
191 |
120 |
81.63% |
62.83% |
| Tree |
83 |
67 |
56 |
67.47% |
83.58% |
| Water |
16 |
14 |
13 |
81.25% |
92.86% |
| Totals |
629 |
629 |
480 |
|
|
|
Table 12: Kappa (K^) Statistics for 1992 TM Classification using
"Multi-resolution Segmentation" process |
| Class Name |
Kappa |
| Corn |
0.6899 |
| Soybean |
0.7610 |
| Grass |
0.5149 |
| Tree |
0.8109 |
| Water |
0.9267 |
Here is an example of terms used in this paper for accuracy assessment of
classified imagery. The level of accuracy associated with class assignments
allows the confidence of classification to be assessed. Terms commonly used are:
Overall, Producers, and Users accuracy. A mathematical error matrix is developed
of classified cell values in correlation with field measurements.
These terms are best defined by using a simplified example. An image was
classified using 3 classes: Water, Tree, and Grass and 17 cells in the image
were validated using some method of field evaluation. A mathematical matrix was
developed as follows:
|
Table 13: Error Matrix of Classified Image of simplified example |
| Cell Value Classified Image |
Reference Field Data |
| |
Water |
Tree |
Grass |
Row Total |
| Water |
2 |
1 |
1 |
4 |
| Tree |
0 |
8 |
1 |
9 |
| Grass |
0 |
0 |
4 |
4 |
| Column Total |
2 |
9 |
6 |
17 |
|
Table 14: Accuracy Assessment Report of simplified example |
| Class Value |
Reference Total |
Classified Total |
Number Correct |
Producers Accuracy |
| Water |
2 |
4 |
2 |
100.00% |
| Tree |
9 |
9 |
8 |
88.89% |
| Grass |
6 |
4 |
4 |
66.67% |
| Total |
17 |
17 |
14 |
|
Producers Accuracy = Number Correct / Reference Total * 100 Water: 2 / 2 * 100 =
100.00% It is a measure of how accurate a class can be classified in an image.
Users Accuracy = Number Correct / Classified Total * 100 Water: 2 / 4 * 100 =
50.00% It measures the confidence of a class in a classified image.
Overall Accuracy = Total Number Correct / (Total Reference or Total Classified *
100 = 14 / 17 * 100 = 82.35% It is a combination of Producers Accuracy and Users
Accuracy for entire image.
Additional information about accuracy assessment of classified imagery can be
found at: Congalton, R.G. and Green, K. 1999. Assessing the accuracy of remotely
sensed data: Principles and practices. Lewis Publishers, New York: 137 p.
Comparisons were made of Various Statistical Data Bases with the land cover
Classified Images of Johnson County, Iowa. A note should be made that the
databases contain many classes that could not be expressed in the classified
image. Therefore, the values for corn and soybeans would be expected to be
higher because areas of other cropland that were included in them. The value for
grass in the classified image would be expected to be larger because it contains
hay land, pastureland, urban grassland, farmstead grassland, road ditches, and
any area of cover that was dominated by grasses or forbs. Urban land is always
difficult to compare manly because of definition within each database. The tree
area of the classified image may differ from the statistical values because it
consists of all tree areas in forest, grazed forest, forested wetland, urban
areas, farm stead, etc. The NASS and Ag Census data had no information on water,
size or county, or artificial cover. The NASS data for grass only contained
Hayland and no data for trees or forested areas. Below is comparison of values
in acres.
|
Table 15: Comparison of Classified Land cover maps for 1992 and 1997 |
| |
Imagine 1997 |
eCognition 1997 |
Imagine 1992 |
eCognition 1992 |
| County Total |
399414 |
399414 |
399415 |
399434 |
| Corn |
116361 |
128710 |
133476 |
129599 |
| Soybean |
101114 |
78663 |
136284 |
57867 |
| Water |
7855 |
8861 |
20787 |
8550 |
| Tree |
62117 |
37260 |
49258 |
47469 |
| Grass |
101440 |
140068 |
42740 |
137616 |
| Artificial |
10527 |
5844 |
26870 |
18010 |
|
Table 16: Comparison of NRI, NASS, and Ag. Census databases for 1997 |
| |
NRI 1992 |
NASS 1992 |
Ag Census 1992 |
| County Total |
399000 |
0 |
0 |
| Corn |
113600 |
126000 |
116475 |
| Soybean |
64800 |
60000 |
57277 |
| Water |
7855 |
0 |
0 |
| Tree |
38500 |
0 |
22654 |
| Grass |
85300 |
53600 |
112969 |
| Artificial |
22600 |
0 |
0 |
Conclusion
Even though the classification accuracy did not appear to be significantly
different between the two methods, the segmentation product showed some
advantages over the pixel product. The object oriented image analysis greatly
reduced the salt-and-pepper classification effect in the classified image
without adversely affecting the classified image accuracy. This greatly improves
the visual effect of the classified image. The reduced variability in the land
use layer produced with the segmentation process may present advantages in
sampling for inventories.
It could be used to collect land use data or as a boundary frame for stratified
random surveys base on land use. It also makes a better choice as an ancillary
layer in a multi-level GIS application because of the much better defined areas.
Recommendations
After reviewing the accuracy assessment and visually inspecting the results
of the two methods of imagery classification, I conclude the Object oriented
classification using the segmentation process software produced a more useable
earth cover image from the classification of the Landsat TM imagery using the
NRS NRI data as training and assessment points.
I recommend that a larger area, such as watershed or state, should be tested
using the object oriented classification method to produce an earth cover image
of the area.
For additional information about this project contact:
Dr. Emil Horvath
Director NRI&A Institute
817-509-3221
These documents require
Adobe Acrobat.
 1997 Landcover Classified Image, Johnson County, Iowa, Classification using
"Multi-resolution Segmentation" process (1,103 KB)
1997 Landcover Classified Image, Johnson County, Iowa, Classification using
supervised pixel base (1,324 KB)
1992 Landcover Classified Image, Johnson County, Iowa, Classification using
"Multi-resolution Segmentation" process (1,092 KB)
1992 Landcover Classified Image, Johnson County, Iowa, Classification using
supervised pixel base (1,610 KB)
 Back to Top
|
About Us
Back to Top