Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add time column to output #109

Open
chuckwondo opened this issue Feb 27, 2025 · 0 comments
Open

Add time column to output #109

chuckwondo opened this issue Feb 27, 2025 · 0 comments

Comments

@chuckwondo
Copy link
Collaborator

Users currently use the filename column (implicitly added to the output file) to determine the date/time for a row since the filenames include date/time information.

For example, the filename GEDI01_B_2019146134206_O02558_01_T05641_02_005_01_V002.h5 contains 2019146134206, which indicates the year (2019), day of year (146), and the time (24hr clock) down to the second (134206), although it's not obvious what the timestamp represents, given that the file contains many shots over a span of roughly 20 minutes (for this particular file).

Having to parse the filenames is cumbersome, so adding a time column should be much more convenient for users (no parsing required), but at the cost of producing a larger output file.

However, there is no "time" variable that indicates the absolute time. Rather, it must be computed, based upon the following (see the GEDI L1B Data Dictionary):

  1. master_time_epoch: Number of GPS seconds between the GPS epoch (1980-01-06T00:00:00Z) and the GEDI epoch (2018-01-01T00:00:00Z). Add this value to delta_time parameters to compute full GPS seconds (relative to the GPS epoch) for each data point.
  2. delta_time: Transmit time of the shot, measured in seconds from the master_time_epoch. By adding the offset contained within /BEAMXXXX/ancillary/master_time_epoch to delta_time, the time in GPS seconds relative to the GPS epoch can be computed.

Therefore, the absolute time is simply the GEDI epoch (2018-01-01T00:00:00Z) + delta_time (seconds). (We don't need to use master_time_epoch since we know delta_time is simply the number of seconds since the GEDI epoch.)

For example, one of the delta_time values in GEDI01_B_2019146134206_O02558_01_T05641_02_005_01_V002.h5 is 4.411330800229415E7, thus the absolute time is:

>>> import datetime as dt
>>> gedi_epoch = dt.datetime(2018, 1, 1, tzinfo=dt.UTC)
>>> time = gedi_epoch + dt.timedelta(seconds=4.411330800229415E7)
>>> time.isoformat()
'2019-05-26T13:41:48.002294+00:00'
>>> time.timetuple().tm_yday
146

To confirm the date matches what is specified in the filename, notice the day of year shown in the last line of output is 146, which corresponds to the date string in the filename (i.e., 2019146, day 146 of 2019).

Alternative approach

Given that adding a DATETIME (SQLite) type will increase the size of the output file, if we were to add a time column as computed above, we might instead want to simply provide examples of how the user can compute the value themselves, in either of the following ways:

  1. If the user wants only the date (no time), provide an example of extracting the date from the implicitly added filename column. This requires no additional column in the output file.
  2. If the user wants the full datetime value, they can include delta_time in the columns list and then compute the absolute time themselves, as shown above (simply add a timedelta of delta_time seconds to the gedi_epoch datetime value). While this will increase the size of the output file, it will be a smaller increase than a pre-computed time column because delta_time is a DOUBLE/REAL (SQLite) value of 8 bytes, whereas a DATETIME (SQLite) value is (likely) either 24 (UTF-8 text) or 48 bytes (UTF-16 text), given that an ISO 8601 date string is 24 characters (YYYY-MM-DDTHH:MM:SS.SSSZ) (see Table 1 at https://docs.ogc.org/is/12-128r19/12-128r19.html).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant