Extremely slow data load

[thomas-spotnana]

I am using tap-freshdesk with Stitch to sync data from Freshdesk to Snowflake. The integration tool seems to load only around 250 rows per 10 minutes, which is extremely slow. This makes the tool non-usable. I suspect this is due to the way data is batched, but was hoping you could elaborate/fix this so that loads from Freshdesk can be as fast as other tools.

[shubham-124]

can anyone help or update here!
I am also facing similar issue.

[jaketho]

The issue with rate limiting is here

tap-freshdesk/tap_freshdesk/__init__.py

Line 43 in dc64f26

@utils.ratelimit(1, 2)

According to the documentation, rate limiting is "x requests per y seconds".
I think the default of (1, 2) assumes the growth plan with a limit of 50 requests per minute and allows some headroom for other API calls.
Depending on your plan and plan age, you can determine your actual rate limit and adjust accordingly. You can find Freshdesk API limits for current plans here > https://support.freshdesk.com/en/support/solutions/articles/225439-what-are-the-rate-limits-for-the-api-calls-to-freshdesk-
For older plans, reference the image at the bottom of this page > https://support.freshservice.com/en/support/solutions/articles/50000000293-what-is-the-rate-limit-for-apis-across-all-plans-

To avoid problems raised in other issues, instead of using the full amount of requests per time, like (50, 60), I use (1, 1.2).

A possible permanent solution for this issue would be to look at the x-requests-total and assume the plan and rate limit or add as a configuration parameter.