Export custom fields on Transaction

README.md

@@ -34,14 +34,19 @@ This tap:
     ```json
     {"merchant_id": "your-merchant-id",
      "public_key": "your-public-key",
-     "private_key": "your-private-key"}
+     "private_key": "your-private-key",
+     "start_date": "2017-01-17T20:32:05Z"}
     ```
+
+   If desired, you can also provide `"environment": "Sandbox"` (useful for running it 
+   locally during development).
 
 4. [Optional] Create the initial state file
 
     You can provide JSON file that contains a date for the API endpoints
     to force the application to only fetch data newer than those dates.
-    If you omit the file it will fetch all Braintree data.
+    If you omit the file it will fetch all Braintree data starting at the 
+    `start_date` above.
 
     ```json
     {"transactions": "2017-01-17T20:32:05Z"}
@@ -55,6 +60,23 @@ This tap:
     tap-braintree --config config.json [--state state.json]
     ```
 
+## Trailing days when syncing ransactions
+
+As of now, Braintree does not offer an option to fetch transactions through its API
+based on an `updated_at` field ([Transaction.search() documentation](https://developers.braintreepayments.com/reference/request/transaction/search/python)). 
+
+This tap uses `created_at` as a filter instead, but since transactions can have their 
+status or other fields updated after they have been created, this tap performs searches 
+based on the last start date (if available on the state) minus 30 trailing days 
+(currently fixed and not configurable) to check if any transaction created in the
+last 30 days had any update. 
+
+Note that if a transaction has indeed been updated, it will contain a `updated_at`
+timestamp on the response, and this field (among with more logic) is used
+to filter out transactions that have been fetched due to the "trailing days" logic
+but have not been updated since the last run, avoiding them to be exported again on
+every run. 
+
 ---
 
 Copyright © 2017 Stitch

tap_braintree/schemas/transactions.json

@@ -185,6 +185,7 @@
                     "type": "null"
                 }
             ]
-        }
+        },
+        "custom_fields": {}
     }
 }

tap_braintree/transform.py

@@ -66,6 +66,10 @@ def _transform_field(value, field_schema):
     if "anyOf" in field_schema:
         return _anyOf(value, field_schema["anyOf"])
 
+    if "type" not in field_schema:
+        # indicates no typing information so don't bother transforming it
+        return value
+
     if field_schema["type"] == "array":
         return _array(value, field_schema["items"])
 

tests/test_tap_braintree.py

@@ -5,6 +5,35 @@
 from datetime import datetime, timedelta
 
 
+# Converts a dict into a Python object
+class MockObject(object):
+    def __init__(self, d):
+        self.__dict__ = d
+
+
+class TestTransform(unittest.TestCase):
+
+    def test_transaction_custom_fields(self):
+        """
+        Transactions may contain custom fields, which are represented
+        as key-value pairs with unkown schema.
+        """
+
+        schema = tap_braintree.load_schema("transactions")
+
+        transaction_row = MockObject({
+            'id': 'id123',
+            'created_at': '2020-04-27T22:04:48.000000Z',
+            'custom_fields': {
+                'key1': 'value1',
+                'key2': 2
+            }
+        })
+
+        output = tap_braintree.transform_row(transaction_row, schema)
+        self.assertEqual(transaction_row.custom_fields, output['custom_fields'])
+
+
 class TestDateRangeUtility(unittest.TestCase):
 
     def test_daterange_normal(self):