Missing data due to time Zone

[advinay]

When running an extraction from Stitch for customer database (for example), bookmark will be updated according to most recent "updated_at" value.
But this value is in EST time, and the bookmark will store it with a "Z" in the end, so in UTC

This leads to strong discrepencies ;

The only way to ensure data is correct, is to manually change start date to 2 days ago. And redo that every day ...

I think this old post was right, but nothing was made to resolve it ; can anyone help ?

From Recharge Support

our DB is in EST time zone and when you add Z at the end you are moving time zone to UTC so you open 4 hour gap I believe

Originally posted by @Brobin in #1 (comment)

[robindebois]

@advinay I'm not entirely sure if I understand where this fails. After processing all the pages the last updated_at is stored in the bookmarks. Where would this go wrong?

• imagine the last update from the API is 2021-04-15T15:00:00 (in EST, implicitly).
• because of lack of timezone awareness, it seems to me this is converted to UTC and so ends up as 2021-04-15T15:00:00Z.
• the code however doesn't use the time when it passes it, it takes [0:10] of the bookmark string, which is just the date, in my example would be 2021-04-15 :

  tap-recharge/tap_recharge/sync.py

  Line 142 in e5fbbee

  params[bookmark_query_field] = last_datetime[0:10] # last_datetime date

When I run it in stitch this is the log (Example):

2021-05-01 13:53:03,146Z    tap - INFO URL for addresses: https://api.rechargeapps.com/addresses?limit=100&updated_at_min=2021-05-01&page=1```

I don't see how it would "skip" updates. Am I missing something?