Please download to get full document.

View again

of 28
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report



Views: 10 | Pages: 28

Extension: PDF | Download: 0

Related documents
GEOSPATIAL EXPLOITATION PRODUCTS (GXP ) AUTOMATIC SPATIAL MODELER (ASM): ELEVATION BY INNOVATION Dr. Bingcai Zhang, Engineering Fellow BAE Systems, Geospatial exploitation Products Contents Executive summary 1 Introduction 1 Using ASM 2 Case study 1: EuroSDR project 4 Case Study 2: UAV images 15 Case Study 3: ADS40 images 19 Case Study 4: Satellite (WorldView-1) images 22 ASM Accuracy Enhancement 24 Summary 26 Acknowledgements 26 References 26 ii Executive summary BAE Systems has been active in the use of digital photogrammetry for elevation extraction since the 1980s. The release of Next-Generation Automatic Terrain Extraction (NGATE) in 2006 pioneered the matching of every image pixel in a commercial software product. Subsequently, there have been significant advances in dense stereo image matching. Semi-Global Matching (Hirschmüller, 2005, 2008) enforces a smoothness constraint in a semi-global way, achieving close-to-global minimum cost to solve this non-deterministic polynomial-time hard (NP-hard) optimization problem. BAE Systems takes a new approach by enhancing algorithms from NGATE with adaptive pixel aggregation, enforcing a local smoothness constraint, and utilizing massive parallel computing power from graphics processing units (GPUs). The Automatic Spatial Modeler (ASM) is designed to generate 3-D point clouds with accuracy similar to LiDAR, such as BAE Systems Automatic Feature Extraction (AFE) functionality, which can extract 3-D objects from stereo images. ASM can extract dense 3-D point clouds from stereo images, and extract accurate building edges and corners from stereo images with high resolution, large overlaps, and high dynamic range. Tests indicate that ASM is much faster than NGATE and more accurate, especially for building edges and corners. Introduction BAE Systems has been involved in the automatic extraction of elevation data from overlapping imagery since its development of digital photogrammetric workstations for the Defense Mapping Agency in the 1980s. Early work focused on area correlation in harness with a range of empirical constraints and quality control metrics, which resulted in Automatic Terrain Extraction (ATE) and Adaptive Automatic Terrain Extraction (AATE). Both were included in the SOCET SET commercialoff-the-shelf product in 1990 and 1997, respectively. Like all BAE Systems photogrammetric products, these were able to process imagery from frame cameras and a wide range of other airborne and orbital sources; government and commercial. The ATE and AATE modules were complemented by the Interactive Terrain Editing (ITE) module that included the manual measurement of elevations, as well as a host of tools for editing automatically generated elevations. In 2006, the company established the use of matching on every pixel with NGATE, for both SOCET SET and the newer SOCET GXP product (Zhang, 2006; Zhang et al., 2006a, 2006b). A crucial improvement over ATE and AATE was the lessening of the assumption that terrain within the search area was a horizontal plane. The computational effort of matching on every pixel was improved by offering options to forego some precision in cases where maximum speed was essential. A further benefit was that the software could compute digital surface models (DSM) as well as digital elevation models (DEM), from which buildings and trees had been removed. As NGATE developed, more tools were added to ITE. Cue cards with graphics and simple instructions were included in SOCET GXP to provide a visual representation of the use of the tool. The product range, which included Automatic Terrain Generation (ATG), a combination of ATE, AATE, NGATE and ITE, satisfied customers from numerous countries and market segments. In 2012, BAE Systems introduced the associated product AFE for automatic identification and extraction of buildings and trees from both photogrammetrically derived and LiDAR point clouds (Smith et al., 2011). BAE Systems enhanced NGATE, especially where high buildings caused particular difficulties. Considerable investments were made that resulted in new functionality, ASM. The crux of the new algorithms is the notion that the real purpose is not just matching every pixel 1 individually, but generating the most accurate possible DSM, and maximizing the success rate, i.e., the number of pixels correctly matched as a proportion of the total. Various techniques have been developed, for example, changing the direction of search according to the orientation of a building. A new approach has been taken to smoothing, based on a variant of the semi-global matching algorithm, which allows for smaller matching windows, yet increases robustness and reliability even within this smaller window. BAE Systems acknowledges that the introduction of these complex algorithms sacrifices speed. Progress has been made to optimize computational effort. A version of the code is being written to run on GPUs, the low-cost computing engines that are present on modern high-performance graphics cards and offer remarkable power at minimal cost. ASM runs several times faster on GPUs than on CPUs alone. Using ASM The user interface for ASM is essentially the same as the one for NGATE both are invoked from the ATG window. Figure 1 displays AMS as one of three options in ATG. New options are available for ASM as well as a number of strategy files starting with asm_. Figure 1 ASM GUI. 2 Two sets of ASM strategy files display: asm_urban.strategy; asm_urban_ CPU.strategy. The _CPU component of the strategy name indicates its use when running ASM without a GPU. Generally, the use of a GPU is faster, depending on the capacity of GPU cards. ASM can run on all GPU cards that Accelerated Massive Parallelism with Microsoft Visual C++ (Microsoft C++ AMP) supports. The latest and most capable GPU card that Microsoft C++ AMP supports is recommend. Unsupported GPU cards can use the advanced algorithms in ASM on a multi-core CPU by selecting one of the _CPU strategy files. Custom strategy files can be created as well. The strategy files provide sufficient information for experienced users to modify and generate new strategy files for their particular images and terrain. The Number of Processes or Number of Sections field in ASM is different from NGATE. NGATE is a computationally intensive, single-threaded CPU application that runs four or even more processes for the sake of speed. ASM uses GPUs and multi-threaded CPUs, which does not require four or more processes. For a single workstation, the Number of Processes is defined as 1 or 2; for powerful workstations, a definition of 4 is preferred. ASM may encounter unknown issues when utilizing the GPU. Microsoft C++ AMP is still in its early stages of development and less reliable than the rest of Visual Studio C++. ASM has logic to recover, switch to CPU mode, and complete the job; however, much more slowly. For example, for the accurate urban modeling of houses and buildings, the Maximum Number of Image Pairs field should be defined as 4 or more if there are significant overlaps both in the flight direction and cross flight. A large value definition for the Maximum Number of Image Pairs field will slow down ASM significantly. ASM will do almost twice as many computations when the value increases from 2 to 4. And, ASM may generate very sharp building edges and corners, which are important for AFE. Post spacing also affects accurate modeling of buildings. A post spacing as small as the image GSD may be used in urban areas to achieve maximum accuracy. It is very important that stereo images are free of Y-parallax before running ASM or NGATE. Images successfully triangulated with SOCET GXP display little or no residual Y-parallax and are ready for ASM. Images imported from other packages without triangulation in SOCET GXP should be checked for significant Y-parallax using Split Screen Stereo. Triangulation and sensor modeling in SOCET GXP can be quite different from other packages. Triangulated stereo images from other packages are not necessarily free of Y-parallax in SOCET GXP. Generally, when ASM or NGATE do not generate an accurate DTM, it is due to Y-parallax. Images must be free of Y-parallax before running ASM. Areas with significant elevation differences, such as mountainous areas, require the use of a Seed DTM. The SOCET GXP functionality, Use Auto DTED, provides a beneficial initial elevation to start ASM. The Seed Point option Automatic works well in most cases. Process ASM 1. Select the radio button ASM 2. In the text field, Strategy, enter the name of the strategy file 3. In the text field, Maximum Number of Image Pairs field, enter the parameters 4. In the text field, Number of Processes, enter the parameters 5. Select Start Now to immediately process the job Select Start Later to delay the process of the job 3 Case study 1: EuroSDR project The data set in this case study was used for the European Spatial Data Research Organization (EuroSDR) project Benchmarking of Image Matching Approaches for DSM Computation ( It is used to compare the speed and accuracy of ASM with similar technologies from other commercial vendors as well as universities. Since the release of NGATE in 2006, with the matching of every pixel in a commercial product, dense matching has become the norm for DSM generation from stereo images. SOCET GXP has a unique method of photogrammetric processing (sensor modeling, epipolar resampling, etc.). The majority of SOCET GXP customers use satellite images and images from other government sensors, which are very different from well known, aerial frame cameras. To make SOCET GXP terrain generation independent of different sensors, a generic algorithm is used for epipolar resampling, which may not work correctly with orientation parameters from third-party triangulation software. Most complaints about poor DSM quality from NGATE are due to significant Y-parallax, which may be a result of third-party orientation parameters. As recommended, always check Y-parallax and run triangulation to eliminate any issues before running NGATE. Unfortunately, the participant who used NGATE in the EuroSDR case study may have overlooked the difference between SOCET GXP s photogrammetric processing and the standard photogrammetric processing found in competitors offerings. As a result, the DSM from NGATE is poor. After triangulating the same images with SOCET SET, SOCET SET v5.6 was used to run NGATE and the DSM accuracy is comparable to the rest of the technologies evaluated by EuroSDR, as shown in Figure 2. Figure 2 Terrain-shaded relief (TSR) image of DSM generated by NGATE after the triangulation of images. 4 Figure 3 The München dataset from the EuroSDR project. The test data set consisted of 15 panchromatic images from an Intergraph DMC II 230 digital airborne frame camera with 15,552 x 14,144 pixels per image, 16 bits per pixel, 10 cm GSD, as shown in Figure 3. The 15 images consist of three strips and five images per strip covering the central part of the city of München, Germany. There is 80% overlap in both directions, which results in 15 stereo pairs being available for certain areas. The terrain is fairly flat, with trees, rivers, bridges, moving vehicles, streets, buildings, shadows, grass areas, etc. The area is densely covered by buildings with heights of up to 50 m. The buildings result in occlusions, especially for surface areas close to building facades. Visibility can be limited for such regions, which will potentially affect the matching processes during DSM generation. 5 Figure 4 TSR image of DSM generated by ASM. ASM processed on a desktop workstation with an Intel Xeon CPU running at 3.2 GHz (two processors), 24 GB RAM and a Geforce GTX Titan GPU. It generated a DSM with 10 cm post spacing and 569 million posts. The DSM covers the entire area of stereo image coverage as shown in Figure 4. The Maximum Number of Image Pairs field was defined with a value of 12 to generate sharp, accurate building edges and corners from different perspectives. Some of the posts were matched 12 times. ASM performs blunder detection on the 12 different Z values, and selects the most accurate and reliable Z value. The Number of Processes field was defined with a value of 2, which generated in 6 hours and 27 minutes. ASM used the GPU for extra speed. 6 Figure 5 Accuracy evaluation site of the EuroSDR project. Figure 5 displays a TSR image of the DSM generated by ASM at the benchmarking site. The highlighted building was used to compare accuracy among the different DSMs from participants using different digital photogrammetric software. ASM accurately generated building edges and corners, which use different strategy files for different types of terrain. Selecting the appropriate strategy file enables ASM to apply different algorithms, or different parameters for the same algorithms, to generate a DSM reliably and accurately based on the underlying terrain. For example, a close to vertical elevation discontinuity is considered correct in urban areas, but it is likely to be a blunder in rural areas. The strategy file asm_urban.strategy was used in this case study. 7 Figure 6 Contours (0.5 meter) of the test area of the EuroSDR project. Vertical building edges, image shadows and rooftops with repeated patterns and featureless man-made areas such as streets are challenging problems in stereo image matching in built-up areas. Figure 6 displays the repeated patterns highlighted in red. ASM has a shadow detection algorithm that allows it to use shadow-specific image matching logic. With 16-bit images, a value of 12 defined for the Maximum Number of Pairs, and the shadow-specific image matching algorithm, shadows become less of a problem as long as they are not in featureless areas. To match vertical building edges, there must be at least one stereo pair in which the vertical building edges are visible. ASM used up to 12 stereo pairs to match every pixel. ASM selected the most reliable matches from 12 of them based on their visibility, figure of merit, precision, and consistency. ASM generated an accurate DSM on repeated patterns using its advanced new algorithms. ASM has the capability to use images with multiple bands, which also improves dynamic range since pixel values are additive. For example, a threeband, eight-bits-per-band color images can have 9.5 bits dynamic range when the bands are added together. ASM generated an accurate DSM in shadows as shown in areas highlighted in white. 8 Figure 7 Vertical wall on the left side of a building. To generate sharp building edges on the left side of a building, use stereo pairs looking from the left perspective. It is important to have large overlaps in both directions. It is also important to define a large value for the Maximum Number of Pairs field for ASM. Figure 7 shows a vertical wall on the left side of a building. Highlighted in red is a shadow in a featureless area, which is still a difficult problem for ASM. The DSM in this shadow may have elevations up to 1.5 m above ground at the intersection between the vertical building façade and the horizontal street. In built-up areas with large overlaps, define a value of 4 or higher for the Maximum Number of Pairs field, 12 was defined for this example. ASM DSM generation speed is directly related to the Maximum Number of Pairs. For example, ASM may take up to twice the amount of time if the Maximum Number of Pairs is defined as 8 rather than 4. 9 Figure 8 Trees without dense canopies. Trees and shadows can be challenging for stereo image matching. Trees with dense canopies are not problematical as long as they do not move (windy day vs. calm day). Trees without leaves can be difficult since there is no continuous surface for stereo image matching. Shadows can be a challenge for stereo image matching depending on what is in the shadow. When shadows cover featureless areas, they can be difficult to match since the signal-to-noise ratio becomes very poor. In general, shadows are more of a problem for 8-bit images than for 16-bit images. Figure 8 displays trees without dense canopies highlighted in red, which have no unique Z value. One DSM cannot model these trees. We are working on a true 3-D point cloud instead of one DSM and one DEM highlighted in white, dark shadows in featureless areas are still a problem for ASM. 10 University of Stuttgart dense matching benchmarking site Figure 9 Light poles, traffic posts and moving vehicles. Small but tall man-made features, such as light poles, traffic posts, flag poles or moving vehicles are commonplace in urban areas as shown in Figure 9. These features are not part of terrain, but they may be part of the DSM generated by ASM. Since they are so small and there are not enough pixels to work with, ASM may miss them. And, since they are so different from their surroundings, ASM may actually match them. It is debatable whether these small made-made features should be part of a DSM, which depends on the applications that use DSM. Moving vehicles (highlighted in white) are abundant in urban areas and are a challenging problem. ASM has a special logic to detect them and remove them (Zhang and Walter, 2009). SOCET GXP has terrain tracking functionality. A profile from the EuroSDR case study was used to assess the sharpness of building edges in the DSM generated by ASM. 11 Figure 10 Sharpness of building edges in ASM generated DSM. The DSM was loaded into the SOCET GXP Multiport and terrain tracking turned on. A point was measured at the top edge (X=25.13, Y=25.88, Z=98.33) and a point at the base (X=24.58, Y=25.91, Z=65.51) as shown in Figure 10. The XY coordinates are from the images and the Z coordinate is from the DSM. The measuring cursor was offset to measure a point at street level. Using the XY offset and Z difference, the slope angle of the building edge was computed in the DSM. The result is 89, which is very close to vertical; 90 cannot be reached exactly because the DSM model can have only one Z per XY. 12 Figure 11 Used 316 point to assess DSM accuracy generated by ASM. A rigorous method of assessing DSM accuracy is to compare a DSM against ground-truth posts. In Figure 11, 316 points spaced 100 meters apart were used. The points cover the entire DSM and are a good representation of the different types of terrain (trees, buildings, streets, grasses, rivers, shadows, etc.). The 316 points were not located with ground GPS. Instead, stereo images were used and all 316 points manually measured four times the zoom to achieve high accuracy. Several points were excluded and do not display because they are on the tops of trees that do not have dense canopies. For these points, there are multiple Z values that cannot be used for DSM accuracy assessment. A few points in occluded areas were also excluded because they could not be measure in the stereo images. 13 The SOCET GXP Quality Statistics module was used to assess the accuracy of the DSM. For every manually measured point in the Master DTM, consisting of the stereoscopically measured points, the Z value was interpolated at the same XY coordinates from the Slave DTM, which was the DSM generated by ASM. The differences in Z were used to generate accuracy statistics. For the 316 points, the root-mean-square error (RMSE) was 0.23 m, and the standard deviation 0.22 m, as shown in Figure 12. Comparison of two DTM files Master DTM File:\EDIT_MASTER2.dth Slave DTM File: ASM_10cm_12pairs_2sections_3.dth num_pts = 316, rms =.2308, std =.2248, bias = percent blunders =.0000 After two outlier removal num_pts = 313, rms =.1745, std =.1699, bias = percent blunders =.9585 After three outlier removal num_pts = 312, rms =.1707, std =.1665, bias = percent blunders = NOTE: bias = (Z from Master) - (Z interpolated from Slave) Figure 12 Output from SOCET GXP Quality Statistics module for the 316 points. Figure 13 displays
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!