Use of BurstIDs from ESA S1 Burst Database

[scottyhq]

Background

If generating many burst interferograms you run into the case where it's not clear what burst-number to use if the framing is different for different granules.

IPF>=3.4 has standard burstIDs in the metadata for every granule which would be nice to use
https://sentinels.copernicus.eu/web/sentinel/-/publication-of-brust-id-maps-for-copernicus-sentinel-1

For the Iran Earthquake example in the readme, here is a quick plot of all the ASF S1 frames and the burst 64_IW2_136231 that is processed

Describe the solution you'd like

I'm thinking as alternative to setting --reference-burst-number and --secondary-burst-number you could have --burst-id IW2_136231 and the software would figure out which burst within the SLC to extract

Additional context

For IPF>=3.4 the burst_ids should be in the metadata, but for IPF<3.4 this would require an additional lookup maybe based on the burstid database polygons (available via the above link). short example below for reading the burst_id info

gf = gpd.read_file('S1_burstid_20220530/IW/sqlite/burst_map_IW_000001_375887.sqlite3',
                   bbox=(54.1, 27.4, 54.2, 27.5)
                  )
myburst = gf[(gf.relative_orbit_number == 64) & (gf.subswath_name == 'IW2') & (gf.burst_id == 136231)]                 

burst_id                                                          136231
subswath_name                                                        IW2
relative_orbit_number                                                 64
time_from_anx_sec                                             2514.65709
orbit_pass                                                    DESCENDING
geometry               MULTIPOLYGON Z (((54.146844 27.677253 0, 53.68...

[asjohnston-asf]

Agreed, providing SLC + swath + burst index is cumbersome. It's difficult to know where a particular burst is spatially located within the SLC, and the burst index for a particular location may not be consistent from SLC to SLC. Our hope is to abstract the SLC away and deal with bursts directly.

ASF has already built an API to download the data for a particular burst from a particular SLC. Our insar_tops_burst workflow uses that API to download just the data needed to create the interferogram. For more information, see https://sentinel1-burst-docs.asf.alaska.edu/ .

ASF also intends to advertise these individual SLC burst products in our DAAC archive and make them accessible via Vertex, CMR, and Earthdata Search. You can explore our initial non-production efforts in Earthdata Search UAT or via the CMR UAT Search API.

With the ability to directly search for individual bursts, we plan to update the interface for insar_tops_burst to simply take the names of the reference and secondary bursts. For example:

--reference-scene S1A_IW_SLC__1SDV_20230523T170718_20230523T170745_048664_05DA5B_8850
--secondary-scene S1A_IW_SLC__1SDV_20230511T170718_20230511T170745_048489_05D52B_07DE
--swath-number 1
--polarization VV
--reference-burst-number 5
--secondary-burst-number 5

becomes:

--reference-scene S1_249434_IW1_20230523T170733_VV_8850-BURST
--secondary-scene S1_249434_IW1_20230511T170732_VV_07DE-BURST

[scottyhq]

@asjohnston-asf thanks for this roadmap, it sounds nicely designed and it looks like things are already moving forward for v0.4.0. Can you comment on how bursts are currently being indexed and the timeline for adding them? For example, my search for a point in Seattle currently only turns up 6 bursts:

https://search.uat.earthdata.nasa.gov/search/granules?p=C1257024016-ASF&pg[0][v]=f&pg[0][gsk]=-start_date&q=bursts&sp[0]=-122.37451%2C47.61444&tl=1684954770!3!!&lat=47.67619905654632&long=-124.16528320312499&zoom=7

[asjohnston-asf]

@scottyhq I'm not in a position to make commitments, but my best guess is that we'll begin indexing bursts in production for new SLCs as they arrive sometime this summer, and that we'll index bursts for the back-catalog of SLCs sometime in the next year.

The bursts currently indexed in CMR UAT are just to support development work. The bulk are from SLCs acquired over a few weeks in May 2023, so I've only been seeing 1-4 acquisitions available for any given area.

The change we have in-progress for tops_insar_burst does limit use to only bursts available in CMR. That will be limiting for time series until the back catalog of bursts is available. I'm curious to hear how ASF's timeline fits with any work you might have in mind.

[scottyhq]

The change we have in-progress for tops_insar_burst does limit use to only bursts available in CMR. That will be limiting for time series until the back catalog of bursts is available. I'm curious to hear how ASF's timeline fits with any work you might have in mind.

Thanks for this information @asjohnston-asf , if the timeline is a year for the back-catalog then it would be nice to retain an option for the existing syntax.

So far we've just started experimenting with this locally, but have in mind time series analysis for different areas running via HYP3 (possibly setting up our own deployment). This is the workflow we're using to calculate the relative burst numbering in any SLC based on TANX, seems to work so far but not extensively tested!

https://nbviewer.org/github/relativeorbit/hyp3bursts/blob/main/workflow-ascending.ipynb

[asjohnston-asf]

@scottyhq do you have specific temporal/spatial ranges of data you'd like to explore in the short term? ASF could weigh the cost of indexing those bursts vs the cost of maintaining the second interface. I suspect we'll be motivated to have a complete temporal coverage for at least some small area of interest available sooner than later.

[asjohnston-asf]

FYI, separately, the OPERA project at JPL has an implementation of computing burst IDs for IPF <3.4 SLCs at https://github.com/opera-adt/s1-reader/blob/main/src/s1reader/s1_burst_id.py . There's discussion at isce-framework/s1-reader#117 that the metadata from ESA is occasionally inaccurate causing the computed burst ID to be incorrect, however. ASF's seen similar occasional metadata issues with our burst extractor API.

[scottyhq]

@scottyhq do you have specific temporal/spatial ranges of data you'd like to explore in the short term?

Sure, here are two points of interest (-122.299, 47.657) and (-121.767, 44.133), and the full temporal range, but if it helps to restrict we could focus on 2019-2023.

the metadata from ESA is occasionally inaccurate causing the computed burst ID to be incorrect,

Thanks for the linked info, good to know!

[asjohnston-asf]

ASF made a provisional release of S1 burst products available in production last Friday! They're accessible through:

• Vertex via the S1 Bursts (beta) dataset, including Baseline and SBAS searches
• asf_search via processingLevel=asf_search.PRODUCT_TYPE.BURST
• Earthdata Search via the Sentinel-1 Single Look Complex (SLC) Bursts (BETA) collection
• CMR via short_name=SENTINEL-1_BURSTS

The collection currently includes:

• bursts from all SLCs intersecting your two points of interest since 2019
• bursts from all SLCs added to ASF's archive since Fri Jun 9
• descending path 174 over Mt Edgecumbe, AK
• ascending path 94 over Fairbanks, AK
• descending path 87 over Hawaii's big island
• descending path 21 over the 2023 Turkey-Syria earthquake

Thanks to @benbart for helping add all of the historical data!

We likely won't officially integrate tops_insar_burst with the production archive for another week or two, but you can integrate locally by setting host=cmr.earthdata.nasa.gov at https://github.com/ASFHyP3/hyp3-isce2/blob/develop/src/hyp3_isce2/burst.py#L346 . Integration with HyP3 and submitting burst InSAR jobs through Vertex are further off; you'll still need to export your scene names/pair lists from your search results and use them to run tops_insar_burst locally.

Eager to hear any feedback you might have!

[scottyhq]

Just noting that >=v0.4 uses preprocessed ASF bursts (https://github.com/ASFHyP3/hyp3-isce2/releases/tag/v0.4.0) that almost match ESA naming scheme. Just the track number must be looked up in as 'Full Burst Id' in the metadata:

S1_136231_IW2_20200604T022312_VV_7C85-BURST (vs 064_136231_IW2)