This repository has been archived by the owner on Nov 19, 2024. It is now read-only.
Merge changes from origin #4
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add function to convert to camel case because the API wants it * Make dateRange included automatically * Update schemas to ad analytics V2 fields * Add new fields to campaigns.targeting_criteria.exclude * Add function selected_fields() * Send only v2 fields to the API * Add error if we break the API's rule * Add parent ID to child records * Add Circle context user * Pretty print the json file, add pivot, pivotValue back as valid fields * Revert this change * Remove unused code * pretty print json for all the jsons * Remove ad_analytics_by_creative from the set of testable streams
Corrects the field `cost_in_local_cy` to read `cost_in_local_currency`. This addresses a bug introduced in v 1.0.1.
Co-authored-by: Zach Harris <[email protected]>
* Remove formatting a string * Add new function to date window the ad analytics sync * Use new sync function * Reduce the number of times we write the schema to once per selected stream * Update windowing logic to work for X days * Make date_window_size configurable * Make pylint happy * Update and clarify a comment * Sync all fields (#18) * Remove restriction on the number of fields you can select * Add a new function that mirrors the functionality of sync_endpoint * Add function to chunk fields, add unit test * Add unit tests for get_next_url * Add shift_sync_window, add unit test for shift_sync_window * Add merge_responses, add unit tests for merge_responses * WIP: requesting <20 fields works as expected * WIP: get bookmarks working * Make pylint happy * Clean up comments * Refactor start_date to last_datetime for clairity * Finish refactor * Fix date time parsing bug from refactor * Make pivot and pivot_value automatic for ad_analytics_by_campaign * Add optional param as a test seam * Add some tests
* Move `ad_analytics_by_creative` from `sync_endpoint()` to `sync_ad_analytics()` * Update merge_responses to use the primary key of the known ad_analytics streams
… 5 minute ads reporting api throttling (#22) * Tune backoff to retry for up to 10 minutes with jitter to account for 5 minute ads reporting api throttling * Fix circle config for tap-tester-v4, add json validation and copy in sandbox dev env variables * setuptools Co-authored-by: Kyle Allan <[email protected]>
Co-authored-by: Kyle Speer <[email protected]>
* changed precision of numbers from e-8 to e-15 * added precision on a couple more * Increase precision to 1e-20 Co-authored-by: Collin Simon <[email protected]>
Co-authored-by: Zach Harris <[email protected]>
* TDL-13315:Added account number validation in discover mode * handled empty value in accounts field * Added unit test step in config * Changed exception message
* added discovery and sync tests * added discovery and sync tests * added pagination test * added start date test * updated comments * changed pagination test * updated unittest * updated the code based on the comments * modified start date * reverted the start date change * added stream for pagination * updated pagination test * updated some test for analytics stream * resolved error causing code * resolved error in sync test * set page size to 1 * Delete .gitignore * test: changed pagination test * test: testing pagination * test: set page size 1 in code * test: making pylint happy * added comment for pagination of analytics stream * test: start date test changes * test: start date test * test: start date test * test: start date test * test: start date test * test: start date test * test: start date test * test: updated tap-tester image * test: updated config file Co-authored-by: Kyle Allan <[email protected]>
* TDL-13285: Skipping video_Adds call if reference_organization_id is None and made reference_organization_id automatic in schema. * TDL-13285: Fix 400 for adDirectSponseredContents * TDL-13285: Make pylint happy * TDL-13285: Pylint changes * TDL-13285: Removed extra comment * TDL-13285: Keep reference_organization_id as is and taking it from account data befor it transformed. * TDL-13285: Updated messages Co-authored-by: dbshah1212 <[email protected]> Co-authored-by: savan-chovatiya <[email protected]>
* TDL-14513: made replication key automatic * TDL-14513: Make pylint happy * added replication key in automatic fields * updated comment in automatic replicated keys Co-authored-by: dbshah1212 <[email protected]> Co-authored-by: Harsh <[email protected]> Co-authored-by: harshpatel4_crest <[email protected]> Co-authored-by: Kyle Allan <[email protected]>
* TDL-14402: Added test_All_Fields * TDL-14402: expected some fields which won't be genarated by LinkedIn with comment * TDL-14402: Updated expected fields list * TDL-14402: Rmoved a field from except as we are getting now Co-authored-by: dbshah1212 <[email protected]> Co-authored-by: savan-chovatiya <[email protected]> Co-authored-by: Kyle Allan <[email protected]>
* TDL-14567: Add check for valid linkedin ads accounts in discover mode * TDL-14567: Resolved pylint failure * TDL-14567: Resolved unit test failure * Added valid number format check in API call handling for accou Co-authored-by: Harsh <[email protected]>
) * added code for handling 4xx error * Added unittest cases for error handling * updated error message * changed code according to the comments * updated code * updated the code according to the comments * removed elif condition * updated logger.error message * updated test cases Co-authored-by: Kyle Allan <[email protected]>
* added request timeout * updated readme and added coverage * updated the code * added comments * resolve pylint error * updated backoff location * resolve pylint error * updated backoff code * updated backoff location * resolved comments * resolve comments
Co-authored-by: KrishnanG <[email protected]>
* TDL-19157:Refresh Token for Linkedin ads * TDL-19157:Imported timeout in init to use it in requried config * Added access token in the get properties * Removed redundant code in pagination test and fixed base class * removed unnecessary code , modified code for retrieving the access code * removed the verified check for fetch_and_set_access_token Co-authored-by: RushT007 <[email protected]>
…onnections and generate new access token for new connections using refresh token (#43) * Added implementation to use user provided access token for existing connections and generate new access token for new connections using refresh token - Added tap-tester to test existing connection scenario i.e. no refresh token provided - Added the unittest to test access token creation - Updated existing unittests * Fixed the spelling and removed extra spaces * Removed the un-used import statement * Updated comments and fixed alignment issues * Updated comment and reformatted the unittest * Updated reformatted the unittest * Updated reformatted the unittest Co-authored-by: RushT007 <[email protected]>
Co-authored-by: RushT007 <[email protected]>
…sible (#50) * Add access token introspection to avoid new access token fetch if feasible * Add UnitTests and cleanup client.py * Add one more unit test, retry for unauthorized * Make Pylint happy * More Pylint happiness * Pull token check out of fetch_and_set * Add refresh_access_token function for clarity. Pull out comments * A little more refactoring / test function rename
Co-authored-by: Dan Mosora <[email protected]>
* Tdl 20157 Add backoff error handling for 5xx (#45) * Added backoff for all 5xx errors. * Skipped test case to check backoff. * Updated unit test case for exception handling. * Updated comment in client. * Updated error handling for 500 error. * TDL-20155 Add missing tap tester (#46) * Refactored tap tester test cases. * Updated pylint. * Added comments in the tap-tester. * Updated spell mistake in base.py * Updated base.py * Updated sync_without_resfresh_token test case. * Updated tap-tester test cases. * Removed sync_canary test case. * TDl-20154 dict base to class base implementation (#48) * Refactored dict base code to class base. * Updated parent child bookmark logic * Added detailed unit test cases. * Resolved pylint errors. * Updated Makefile. * Update Makefile to run pylint. * Updated unit test cases. * Added integration test case for parent child independent. * Updated streams.py * Updated sync to resolve pylint. * Added detailed code comments. * Added unit test to achive more coverage * Updateed get_streams_to_sync method. * Added unit test scenarios for split chunk method. * Updated Makefile to update pylint. * Remove version pins for dev requirements (#38) * Remove version pins for dev requirements * Making pylint happy * Actually make pylint happy * Add standard list of disables to the extras already discussed Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> Co-authored-by: prijendev <[email protected]> * Reverted back pylint changes in config.yml * Resolved pylint errors. * typecast date_window_size with int data type. * Updated sync_without_refresh_token test. * Added missing fields in the schemas and upgraded API version. (#47) * Added missing fields in the schemas and upgraded API version. * Updated schemas and tap tester. * Updated streams file. * Updated schema files. * Updated all_field test case. * Updated all fields test case. * Replaced header param `expires_in` with `expires_at`. * Resolved pylint errors. * Replaced current_time with time.time() * Skipped fields for analytics stream that are in beta. Co-authored-by: Dylan <[email protected]> Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> Co-authored-by: Dylan <[email protected]> Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> * Updated field name in campaigns. * TDL-20798 Write Access Token in Config (#52) * Initial commit. * Replaces expires_at with expires_in. * Updated unit tests. * Added comments to explain config rewrite. * Fixed all_fields test case. * Changes - Moved the unittest scenarios to test_client.py. Fixed test_all_fields by adding reference_person_id in the missing fields. * Fixed the arguments for Linkedin client in unit test case * Removed the functionality of generating new access token from the integration test * Fixed unit test case mock value * Reverted the unit test case name Co-authored-by: Sourabh Gandhi <[email protected]> Co-authored-by: Dylan <[email protected]> Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> Co-authored-by: Sourabh Gandhi <[email protected]>
* minor bump version to 1.3.0 * Realigned the changelog
* Tdl 20157 Add backoff error handling for 5xx (#45) * Added backoff for all 5xx errors. * Skipped test case to check backoff. * Updated unit test case for exception handling. * Updated comment in client. * Updated error handling for 500 error. * TDL-20155 Add missing tap tester (#46) * Refactored tap tester test cases. * Updated pylint. * Added comments in the tap-tester. * Updated spell mistake in base.py * Updated base.py * Updated sync_without_resfresh_token test case. * Updated tap-tester test cases. * Removed sync_canary test case. * TDl-20154 dict base to class base implementation (#48) * Refactored dict base code to class base. * Updated parent child bookmark logic * Added detailed unit test cases. * Resolved pylint errors. * Updated Makefile. * Update Makefile to run pylint. * Updated unit test cases. * Added integration test case for parent child independent. * Updated streams.py * Updated sync to resolve pylint. * Added detailed code comments. * Added unit test to achive more coverage * Updateed get_streams_to_sync method. * Added unit test scenarios for split chunk method. * Updated Makefile to update pylint. * Remove version pins for dev requirements (#38) * Remove version pins for dev requirements * Making pylint happy * Actually make pylint happy * Add standard list of disables to the extras already discussed Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> Co-authored-by: prijendev <[email protected]> * Reverted back pylint changes in config.yml * Resolved pylint errors. * typecast date_window_size with int data type. * Updated sync_without_refresh_token test. * Added missing fields in the schemas and upgraded API version. (#47) * Added missing fields in the schemas and upgraded API version. * Updated schemas and tap tester. * Updated streams file. * Updated schema files. * Updated all_field test case. * Updated all fields test case. * Replaced header param `expires_in` with `expires_at`. * Resolved pylint errors. * Replaced current_time with time.time() * Skipped fields for analytics stream that are in beta. Co-authored-by: Dylan <[email protected]> Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> Co-authored-by: Dylan <[email protected]> Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> * Updated field name in campaigns. * TDL-20798 Write Access Token in Config (#52) * Initial commit. * Replaces expires_at with expires_in. * Updated unit tests. * Added comments to explain config rewrite. * Fixed all_fields test case. * Changes - Moved the unittest scenarios to test_client.py. Fixed test_all_fields by adding reference_person_id in the missing fields. * Fixed the arguments for Linkedin client in unit test case * Removed the functionality of generating new access token from the integration test * Fixed unit test case mock value * Reverted the unit test case name Co-authored-by: Sourabh Gandhi <[email protected]> * Upgrade creatives api endpoint (#57) * modify code for the creatives API endpoint update * add specific headers for creatives endpoint and update the comments with the documentation link * make pylint happy. fix the integration tests. * update readme file * remove the older unit test case file * major bump version changes --------- Co-authored-by: Prijen Khokhani <[email protected]> Co-authored-by: Dylan <[email protected]> Co-authored-by: dylan-stitch <[email protected]> Co-authored-by: Dan Mosora <[email protected]> Co-authored-by: prijendev <[email protected]>
* Add descriptive error for 431 failures * use query tunneling to allow large URIs * handle null url * version bump and changelog update, removes unnecessary error msg --------- Co-authored-by: Bryant Gray <[email protected]>
* Update API Version; Move and Update FIELDS_UNACCEPTED_BY_API * Get Tests Happy * Version bump and changelog Co-authored-by: Leslie VanDeMark <[email protected]>
* upgrade API to 202304 * update setup and changelog
* Update pagination logic for streams - accounts,campaign_groups,campaigns,creatives * update API endpoint * update the APi endpoints to include Adaccount in the url * remove unsupported params for Accounts stream * updated query param format for accounts * handle pivot field for analytics API * comment video_ads for the time being * exclude missing fields from the all fields test * Upgrade api version 202403 video ads (#71) * added video-ads implementation * fixed pylint issues * added handler for missing permission, removed test for skip condition * updated message * fstring -> format for 3.6 compatibility * unit test fix * update setup.py and changelog.md * fix review comments --------- Co-authored-by: Vishal <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
(write a short description or paste a link to JIRA)
Manual QA steps
Risks
Rollback steps