Merge pull request #432 from ASFHyP3/develop

Release 0.8.9 - Water masking documentation

.github/dictionary.txt

@@ -43,6 +43,7 @@ area.tif
 ASF
 ASF's
 ASF-specific
+asf-tools
 asf_search
 asf_tools
 ASFHyP3
@@ -88,6 +89,7 @@ DockerizedTopsApp
 docstring
 DOI
 DOIs
+Downloader
 downsampled
 DV
 DynamoDB
@@ -188,6 +190,8 @@ L5
 L7
 L8
 L9
+landfast
+landuse
 lifecycle
 Log10
 ls_map.tif
@@ -206,6 +210,8 @@ Miniconda
 MintPy
 mintpy
 MkDocs
+mosaicked
+mosaicking
 MTI
 multiband
 Multilook
@@ -222,8 +228,9 @@ NoData
 non-geocoded
 NoSQL
 OpenAPI
-OpenStreetMaps
+OpenStreetMap
 orthometric
+OSM
 Ottinger
 par_s1_slc
 Pavia
@@ -340,13 +347,15 @@ VV
 waterbodies
 waterbody
 Watermap
+watermasking
 webpage
 Werner
 Werner's
 WGS84
 whitespace
 Woodhouse
 WorldCover
+wr
 XML
 xx
 Yukon-Kuskokwim

CHANGELOG.md

@@ -7,6 +7,14 @@ and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/)
 and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 
+## [0.8.9]
+
+### Added
+* [Water Mask page](docs/water_masking.md) describing the water mask used for InSAR processing, and linking to code and resources for accessing and generating the new reference water mask
+
+### Changed
+* Updated ASF Tools for Python to [v0.7.2](https://github.com/ASFHyP3/asf-tools/blob/main/CHANGELOG.md#072)
+
 ## [0.8.8]
 
 ### Changed
@@ -23,7 +31,7 @@ and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ## [0.8.6]
 
 ### Changed
-* Updated water mask language to reflect switch to OpenStreetMaps/ESA WorldCover based water map.
+* Updated water mask language to reflect switch to OpenStreetMap/ESA WorldCover based water map.
 
 ## [0.8.5]
 

docs/guides/burst_insar_product_guide.md

@@ -133,14 +133,16 @@ The ISCE2 InSAR processing this product uses follows the workflow in [topsApp.py
 1. Geocode the output products.
 
 #### Apply Water Mask
-There is an option to apply a **water mask**. This mask includes coastal waters and most inland waterbodies. Masking waterbodies can have a significant impact during the phase unwrapping, as water can sometimes exhibit enough coherence between acquisitions to allow for unwrapping to occur over waterbodies, which is invalid. 
+There is an option to apply a **water mask**. This mask includes coastal waters and most inland waterbodies. Masking waterbodies can have a significant impact during phase unwrapping, as water can sometimes exhibit enough coherence between acquisitions to allow for unwrapping to occur over waterbodies, which is invalid. 
 
 A GeoTIFF of the water mask is always included with the InSAR product package, but when this option is selected, the conditional water mask will be applied along with coherence and intensity thresholds during the phase unwrapping process. Water masking is turned off by default. 
 
-The water mask is generated by the HyP3 team using data from [OpenStreetMaps](https://www.openstreetmap.org/about){target=_blank} and/or [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} depending on location. Areas within Canada, Alaska, and Russia are primarily covered by ESA WorldCover data, while the rest of the world is covered by OpenStreetMaps data. This water mask is available for all longitudes, but data is only available from -85 to 85 degrees latitude. All areas between 85 and 90 degrees north latitude are treated as water, and all areas between 85 and 90 degrees south latitude are treated as land for the purposes of the water mask.
+The water mask is generated by ASF using data from [OpenStreetMap](https://www.openstreetmap.org/about){target=_blank} and/or [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} depending on location. Areas within Canada, Alaska, and Russia are primarily covered by ESA WorldCover data, while the rest of the world is covered by OpenStreetMap data. Refer to the [Water Masking](../water_masking.md "Water Masking Documentation" ){target=_blank} documentation page for more details.
+
+This water mask is available for all longitudes, but data is only available from -85 to 85 degrees latitude. All areas between 85 and 90 degrees north latitude are treated as water, and all areas between 85 and 90 degrees south latitude are treated as land for the purposes of the water mask.
 
 Water masks were previously generated from the [Global Self-consistent,
-Hierarchical, High-resolution Geography Database (GSHHG)](http://www.soest.hawaii.edu/wessel/gshhg "http://www.soest.hawaii.edu/wessel/gshhg" ){target=_blank} dataset, but we transitioned to using the OpenStreetMaps/ESA WorldCover datasets in February 2024 to improve performance. In addition to being a more recent and accurate dataset, this also allows us to mask most inland waterbodies. When using the GSHHG dataset, we only masked large inland waterbodies; with the new mask, all but the smallest inland waterbodies are masked. 
+Hierarchical, High-resolution Geography Database (GSHHG)](http://www.soest.hawaii.edu/wessel/gshhg "http://www.soest.hawaii.edu/wessel/gshhg" ){target=_blank} dataset, but we transitioned to using the OpenStreetMap/ESA WorldCover datasets in February 2024 to improve performance. In addition to being a more recent and accurate dataset, this also allows us to mask most inland waterbodies. When using the GSHHG dataset, we only masked large inland waterbodies; with the new mask, all but the smallest inland waterbodies are masked. 
 
 We originally applied a 3 km buffer on coastlines and a 5 km buffer on the shorelines of inland waterbodies in the water mask dataset before using it to mask the interferograms, in an effort to reduce the chance that valid land pixels would be excluded from phase unwrapping. It appears, however, that the inclusion of more water pixels is more detrimental to phase unwrapping than the exclusion of some land pixels, so as of September 27, 2022, the water mask used for this option is no longer buffered. 
 
@@ -181,23 +183,29 @@ The Burst InSAR product names are packed with information pertaining to the proc
 
 ### Image Files
 
-All of the main InSAR product files are 32-bit floating-point single-band GeoTIFFs. The exceptions to this are the connected components and the water mask files, which are both 8-bit unsigned-integer single-band GeoTIFFs.
+All of the main InSAR product files are 32-bit floating-point single-band GeoTIFFs. The exceptions to this are the connected components and the water mask files, which are both 8-bit unsigned-integer single-band GeoTIFFs. 
+
+The following image files are geocoded to the appropriate UTM Zone map projection, based on the location of the output product:
 
-- The *coherence* file pixel values range from 0.0 to 1.0, with 0.0 being completely non-coherent and 1.0 being perfectly coherent. 
-- The *unwrapped phase* file shows the results of the phase unwrapping process. Negative values indicate movement towards the sensor, and positive values indicate movement away from the sensor. This is the main interferogram output.
-- The *wrapped phase* file indicates the interferogram phase after applying the adaptive filter immediately before unwrapping. Values range from negative pi to positive pi.
+- The *normalized coherence* file contains pixel values that range from 0.0 to 1.0, with 0.0 being completely non-coherent and 1.0 being perfectly coherent. 
+- The *unwrapped geocoded interferogram* file shows the results of the phase unwrapping process. Negative values indicate movement towards the sensor, and positive values indicate movement away from the sensor. This is the main interferogram output.
+- The *wrapped geocoded interferogram* file indicates the interferogram phase after applying the adaptive filter immediately before unwrapping. Values range from negative pi to positive pi.
 - The *connected components* file delineates regions unwrapped as contiguous units by the SNAPHU unwrapping algorithm.
 - The *look vectors* theta (θ) and phi (φ) describe the elevation and orientation angles of the look vector in radians. The look vectors refer to the look direction back towards the sensor. 
     - The *lv_theta* (θ) file indicates the SAR look vector elevation angle (in radians) at each pixel, ranging from -π/2 (down) to π/2 (up). The look vector elevation angle is defined as the angle between the horizontal surface and the look vector with positive angles indicating sensor positions above the surface. 
     - The *lv_phi* (φ) file indicates the SAR look vector orientation angle (in radians) at each pixel. The look vector orientation angle is defined as the angle between the East direction and the projection of the look vector on the horizontal surface plane. The orientation angle increases towards north, with the North direction corresponding to π/2 (and south to -π/2). The orientation angle range is -π to π.
 - The *DEM* file gives the local terrain heights in meters, with a geoid correction applied.
 - The *water mask* file indicates coastal waters and large inland waterbodies. Pixel values of 1 indicate land and 0 indicate water. This file is in 8-bit unsigned integer format.
 
-If the **water mask** option is selected, the water mask is applied prior to phase unwrapping to exclude water pixels from the process. The water mask is generated using the [OpenStreetMaps](https://www.openstreetmap.org/about){target=_blank} and [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} datasets. Refer to the [Water Masking Processing Option section](#apply-water-mask) and our [InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information about water masking.
+If the **water mask** option is selected, the water mask is applied prior to phase unwrapping to exclude water pixels from the process. The water mask is generated using the [OpenStreetMap](https://www.openstreetmap.org/about){target=_blank} and [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} datasets. Refer to the [Water Masking Processing Option](#apply-water-mask) section and our [InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information about water masking.
+
+There are also four non-geocoded images that remain in their native range-doppler coordinates. These four images comprise the image data required to merge Burst InSAR products together, and include: 
 
-There are also four non-geocoded images that remain in their native range-doppler coordinates. These four images compose the image data needed to merge Burst InSAR products together. These images include a range-doppler version of the wrapped interferogram, a two-band range-doppler look vector image in the native ISCE2 format, and latitude/longitude images that provide the information necessary to map range-doppler images into the geocoded domain.
+- a *wrapped Range-Doppler interferogram*, which is a Range-Doppler version of the wrapped interferogram
+- a two-band *Range-Doppler look vectors* image in the native ISCE2 format
+- *Range-Doppler latitude coordinates* and *Range-Doppler longitude coordinates* images that provide the information necessary to map Range-Doppler images into the geocoded domain
 
-A **browse image** is included for the unwrapped (unw_phase) phase file, which is in PNG format and is 2048 pixels wide.
+An *unwrapped phase browse image* is included for the unwrapped (unw_phase) phase file, which is in PNG format and is 2048 pixels wide.
 
 The tags and extensions used and example file names for each raster are listed in Table 2 below. 
 

docs/guides/insar_product_guide.md

@@ -52,10 +52,12 @@ There is an option to apply a **water mask**. This mask includes coastal waters
 
 A GeoTIFF of the water mask is always included with the InSAR product package, but when this option is selected, the conditional water mask will be applied along with coherence and intensity thresholds during the phase unwrapping process. Water masking is turned off by default. 
 
-The water mask is generated by the HyP3 team using data from [OpenStreetMaps](https://www.openstreetmap.org/about){target=_blank} and/or [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} depending on location. Areas within Canada, Alaska, and Russia are primarily covered by ESA WorldCover data, while the rest of the world is covered by OpenStreetMaps data. This water mask is available for all longitudes, but data is only available from -85 to 85 degrees latitude. All areas between 85 and 90 degrees north latitude are treated as water, and all areas between 85 and 90 degrees south latitude are treated as land for the purposes of the water mask.
+The water mask is generated by ASF using data from [OpenStreetMap](https://www.openstreetmap.org/about){target=_blank} and/or [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} depending on location. Areas within Canada, Alaska, and Russia are primarily covered by ESA WorldCover data, while the rest of the world is covered by OpenStreetMap data. Refer to the [Water Masking](../water_masking.md "Water Masking Documentation" ){target=_blank} documentation page for more details. 
+
+This water mask is available for all longitudes, but data is only available from -85 to 85 degrees latitude. All areas between 85 and 90 degrees north latitude are treated as water, and all areas between 85 and 90 degrees south latitude are treated as land for the purposes of the water mask. 
 
 Water masks were previously generated from the [Global Self-consistent,
-Hierarchical, High-resolution Geography Database (GSHHG)](http://www.soest.hawaii.edu/wessel/gshhg "http://www.soest.hawaii.edu/wessel/gshhg" ){target=_blank} dataset, but we transitioned to using the OpenStreetMaps/ESA WorldCover datasets in February 2024 to improve performance. In addition to being a more recent and accurate dataset, this also allows us to mask most inland waterbodies. When using the GSHHG dataset, we only masked large inland waterbodies; with the new mask, all but the smallest inland waterbodies are masked.
+Hierarchical, High-resolution Geography Database (GSHHG)](http://www.soest.hawaii.edu/wessel/gshhg "http://www.soest.hawaii.edu/wessel/gshhg" ){target=_blank} dataset, but we transitioned to using the OpenStreetMap/ESA WorldCover datasets in February 2024 to improve performance. In addition to being a more recent and accurate dataset, this also allows us to mask most inland waterbodies. When using the GSHHG dataset, we only masked large inland waterbodies; with the new mask, all but the smallest inland waterbodies are masked.
 
 We originally applied a 3 km buffer on coastlines and a 5 km buffer on the shorelines of inland waterbodies in the water mask dataset before using it to mask the interferograms, in an effort to reduce the chance that valid land pixels would be excluded from phase unwrapping. It appears, however, that the inclusion of more water pixels is more detrimental to phase unwrapping than the exclusion of some land pixels, so as of September 27, 2022, the water mask used for this option is no longer buffered. 
 
@@ -161,7 +163,9 @@ Coherence is estimated from the normalized interferogram. The pixel values in th
 
     In the past, we also used an amplitude threshold of 0.2 (in power scale) when generating the validity mask. While this approach tends to mask out inland waters, providing a less noisy interferogram in some cases, it also masks arid regions that have low amplitude values but reasonably high coherence. As of March 2022, we have set the amplitude threshold to 0.0, so that coherence is the only driver of the validity mask.
 
-When the [water masking option](#apply-water-mask "Jump to the Apply Water Mask item in the Processing Options section of this document") is applied, the validity mask is further amended to apply 0 values to any pixels classified as water in the water mask. In some cases, pixels over water may still meet the coherence and amplitude threshold criteria for inclusion, even though they are not valid for use during phase unwrapping. When processing scenes with extensive coverage by coastal waters or large inland waterbodies, there can be erroneous deformation signals or phase jumps introduced if unwrapping proceeds over water as if it were land. In such cases, choosing the option to apply the water mask can improve the results. Visit our [InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information.
+In some cases, pixels over water may still meet the coherence threshold criteria for inclusion, even though they are not valid for use during phase unwrapping. When the [water masking option](#apply-water-mask "Jump to the Apply Water Mask item in the Processing Options section of this document") is applied, the validity mask is further amended to apply 0 values to any pixels classified as water in the water mask. 
+
+When processing scenes with extensive coverage by coastal waters or large inland waterbodies, there can be erroneous deformation signals or phase jumps introduced if unwrapping proceeds over water as if it were land. In such cases, choosing the option to apply the water mask can significantly improve the results. Refer to our [InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information.
 
 #### Reference point
 In order to perform phase unwrapping, a reference point must be selected. The unwrapping will proceed relative to this reference point; the 2π integer multiples will be applied to the wrapped phase using this pixel as the starting point. The unwrapped phase value is set to 0 at the reference point.
@@ -225,7 +229,7 @@ All of the main InSAR product files are 32-bit floating-point single-band GeoTIF
 - The *DEM* file gives the local terrain heights in meters, with a geoid correction applied. *(optional)*
 - The *water mask* file indicates coastal waters and most inland waterbodies. Pixel values of 1 indicate land and 0 indicate water. This file is in 8-bit unsigned integer format.
 
-If the **water mask** option is selected, the water mask is applied prior to phase unwrapping to exclude water pixels from the process. The water mask is generated using the [OpenStreetMaps](https://www.openstreetmap.org/about){target=_blank} and [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} datasets. Refer to the [Water Masking Processing Option section](#apply-water-mask) and our [InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information about water masking.
+If the **water mask** option is selected, the water mask is applied prior to phase unwrapping to exclude water pixels from the process. The water mask is generated using the [OpenStreetMap](https://www.openstreetmap.org/about){target=_blank} and [ESA WorldCover](https://esa-worldcover.org/en/about/about){target=_blank} datasets. Refer to the [Water Masking Processing Option section](#apply-water-mask) and our [InSAR Water Masking Tutorial](https://storymaps.arcgis.com/stories/485916be1b1d46889aa436794b5633cb "InSAR Water Masking StoryMap" ){target=_blank} for more information about water masking.
 
 **Browse images** are included for the wrapped (color_phase) and unwrapped (unw_phase) phase files, which are in PNG format and are each 2048 pixels wide. The browse images are displayed using a cyclic color ramp to generate fringes. 
 

docs/tools/asf_tools.md

@@ -1 +1 @@
-{{ get_content('https://raw.githubusercontent.com/ASFHyP3/asf-tools/v0.6.0/src/asf_tools/README.md') }}
+{{ get_content('https://raw.githubusercontent.com/ASFHyP3/asf-tools/v0.7.2/src/asf_tools/README.md') }}

docs/tools/asf_tools_api.md

@@ -1,4 +1,4 @@
-# `asf_tools` *v0.6.0* API Reference
+# `asf_tools` *v0.7.2* API Reference
 
 ::: asf_tools
     options: