Incomplete data while fetching minute level historical data

[ankush1729]

Hi,

I was testing accessing the historical data using for 1, 5 and 30 minute interval. It seems to be missing the data for certain dates altogether, while the same is present in the 1day data fetch?

Has someone else noticed the same?

[Idirect-Tech]

Dear Ankush, Thank you for showing interest in Breeze API. Could you please share screenshots of the instances where you observed missing data? [E29CF836] Thanks & Regards, Akash Sapkal, Team Breeze API, ICICI Securities

…

________________________________ From: ankush1729 ***@***.***> Sent: 29 December 2024 21:13 To: Idirect-Tech/Breeze-Python-SDK ***@***.***> Cc: Subscribed ***@***.***> Subject: [Idirect-Tech/Breeze-Python-SDK] Incomplete data while fetching minute level historical data (Issue #178) CAUTION: This email originated from outside the organization! Do not click links, open attachments or reply, unless you recognize the sender's email address! Use 'Report suspicious Email' button in the toolbar to report Phishing mails. Beware! Hi, I was testing accessing the historical data using for 1, 5 and 30 minute interval. It seems to be missing the data for certain dates altogether, while the same is present in the 1day data fetch? Has someone else noticed the same? — Reply to this email directly, view it on GitHub<#178>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3TQZ43OLEZP3TOILSNONFL2IAKBLAVCNFSM6AAAAABUK5BSFKVHI2DSMVQWIX3LMV43ASLTON2WKOZSG43DEMZQGI4TENA>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> Print this mail only if absolutely necessary. Save Paper. Save Trees." "The information contained in this e-mail and any attachments to this message are intended for the exclusive use of the intended recipient and may contain proprietary, confidential or legally privileged information. If you are not the intended recipient, please note that you are not authorised to disseminate, distribute or copy this e-mail or any parts of it or act upon/rely on the contents of this e-mail in any manner. Please notify the sender immediately by e-mail and destroy all copies of this e-mail and any attachments. Please also note that ICICI Bank or its subsidiaries and associated companies, (collectively "ICICI Group"), are unable to exercise control or ensure the integrity of/over the contents of the information contained in e-mail transmissions and that any views expressed in this e-mail are not endorsed by/binding on the ICICI Group unless the sender does so expressly with due authority of ICICI Group. Before opening any attachments please check them for viruses and defects and please note that ICICI Group accepts no liability or responsibility for any damage caused by any virus that may be transmitted by this email. Any unauthorized use of this email or any attachment(s) may be treated as infringement of right(s) including right of privacy under the applicable laws of India including Information and Technology Act, 2000. Thank you for your cooperation. Regulatory Disclosures are available on https://www.icicisecurities.com (for institutional business) and www.icicidirect.com (for retail business).

[ankush1729]

This is how the complete data look like for a day when I am doing a pull for 30 minute interval. It'll have 14 rows.

There are few days when less than 14 records are present. example:

Additionally, below are the dates for which daily data is available, but not even a single record for minute level data is coming:

2024-01-03
2024-08-23
2024-01-20
2024-09-18
2023-11-12
2023-12-18
2024-05-18
2024-03-01
2024-09-16
2022-10-24
2024-02-16
2024-02-09
2024-03-12
2023-12-12
2024-07-04
2024-03-02
2021-11-04
2024-09-30
2024-10-01
2024-09-17

Above analysis was done on just 1 script 'ICICI BANK' for period between "2021-01-01T07:00:00.000Z" and "2024-12-17T07:00:00.000Z"

I also observed that some of the above dates are falling on Saturday, in this case no data should be available even in the daily pull.

[Idirect-Tech]

Dear Ankush, Kindly share the payload to check the issue. [E29CF836] Thanks & Regards, Hiten Solanki, Team Breeze API, ICICI Securities

…

________________________________ From: ankush1729 ***@***.***> Sent: Tuesday, December 31, 2024 4:01 PM To: Idirect-Tech/Breeze-Python-SDK ***@***.***> Cc: BREEZEAPI ***@***.***>; Comment ***@***.***> Subject: Re: [Idirect-Tech/Breeze-Python-SDK] Incomplete data while fetching minute level historical data (Issue #178) CAUTION: This email originated from outside the organization! Do not click links, open attachments or reply, unless you recognize the sender's email address! Use 'Report suspicious Email' button in the toolbar to report Phishing mails. Beware! This is how the complete data look like for a day when I am doing a pull for 30 minute interval. It'll have 14 rows. image.png (view on web)<https://github.com/user-attachments/assets/a69fb232-4c9a-4966-bae4-6e45a7d241fb> There are few days when less than 14 records are present. example: image.png (view on web)<https://github.com/user-attachments/assets/75e3928c-458f-4d8a-93e5-7181d71f3580> image.png (view on web)<https://github.com/user-attachments/assets/298247ce-cfa4-45c8-a515-6ff026705cb8> Additionally, below are the dates for which daily data is available, but not even a single record for minute level data is coming: 2024-01-03 2024-08-23 2024-01-20 2024-09-18 2023-11-12 2023-12-18 2024-05-18 2024-03-01 2024-09-16 2022-10-24 2024-02-16 2024-02-09 2024-03-12 2023-12-12 2024-07-04 2024-03-02 2021-11-04 2024-09-30 2024-10-01 2024-09-17 Above analysis was done on just 1 script 'ICICI BANK' for period between "2021-01-01T07:00:00.000Z" and "2024-12-17T07:00:00.000Z" — Reply to this email directly, view it on GitHub<#178 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3TQZ42623P5B7UXWTNYULD2IJW7TAVCNFSM6AAAAABUK5BSFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRWGMZDMOJZGU>. You are receiving this because you commented.Message ID: ***@***.***> Print this mail only if absolutely necessary. Save Paper. Save Trees." "The information contained in this e-mail and any attachments to this message are intended for the exclusive use of the intended recipient and may contain proprietary, confidential or legally privileged information. If you are not the intended recipient, please note that you are not authorised to disseminate, distribute or copy this e-mail or any parts of it or act upon/rely on the contents of this e-mail in any manner. Please notify the sender immediately by e-mail and destroy all copies of this e-mail and any attachments. Please also note that ICICI Bank or its subsidiaries and associated companies, (collectively "ICICI Group"), are unable to exercise control or ensure the integrity of/over the contents of the information contained in e-mail transmissions and that any views expressed in this e-mail are not endorsed by/binding on the ICICI Group unless the sender does so expressly with due authority of ICICI Group. Before opening any attachments please check them for viruses and defects and please note that ICICI Group accepts no liability or responsibility for any damage caused by any virus that may be transmitted by this email. Any unauthorized use of this email or any attachment(s) may be treated as infringement of right(s) including right of privacy under the applicable laws of India including Information and Technology Act, 2000. Thank you for your cooperation. Regulatory Disclosures are available on https://www.icicisecurities.com (for institutional business) and www.icicidirect.com (for retail business).

[ankush1729]

I've attached both 30minute and daily level pull payloads.

Thanks,
Ankush
30_min_data.json
daily_data.json

[Idirect-Tech]

Dear Ankush, Greetings for the day! Please share your contact details to connect with you. [E29CF836] Thanks & Regards, Akash Sapkal, Team Breeze API, ICICI Securities

…

________________________________ From: ankush1729 ***@***.***> Sent: 03 January 2025 11:34 To: Idirect-Tech/Breeze-Python-SDK ***@***.***> Cc: BREEZEAPI ***@***.***>; Comment ***@***.***> Subject: Re: [Idirect-Tech/Breeze-Python-SDK] Incomplete data while fetching minute level historical data (Issue #178) CAUTION: This email originated from outside the organization! Do not click links, open attachments or reply, unless you recognize the sender's email address! Use 'Report suspicious Email' button in the toolbar to report Phishing mails. Beware! I've attached both 30minute and daily level pull payloads. Thanks, Ankush 30_min_data.json<https://github.com/user-attachments/files/18296869/30_min_data.json> daily_data.json<https://github.com/user-attachments/files/18296871/daily_data.json> — Reply to this email directly, view it on GitHub<#178 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3TQZ42XADZFQKPANAV36Q32IYR7TAVCNFSM6AAAAABUK5BSFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRYG4ZTCOBYG4>. You are receiving this because you commented.Message ID: ***@***.***> Print this mail only if absolutely necessary. Save Paper. Save Trees." "The information contained in this e-mail and any attachments to this message are intended for the exclusive use of the intended recipient and may contain proprietary, confidential or legally privileged information. If you are not the intended recipient, please note that you are not authorised to disseminate, distribute or copy this e-mail or any parts of it or act upon/rely on the contents of this e-mail in any manner. Please notify the sender immediately by e-mail and destroy all copies of this e-mail and any attachments. Please also note that ICICI Bank or its subsidiaries and associated companies, (collectively "ICICI Group"), are unable to exercise control or ensure the integrity of/over the contents of the information contained in e-mail transmissions and that any views expressed in this e-mail are not endorsed by/binding on the ICICI Group unless the sender does so expressly with due authority of ICICI Group. Before opening any attachments please check them for viruses and defects and please note that ICICI Group accepts no liability or responsibility for any damage caused by any virus that may be transmitted by this email. Any unauthorized use of this email or any attachment(s) may be treated as infringement of right(s) including right of privacy under the applicable laws of India including Information and Technology Act, 2000. Thank you for your cooperation. Regulatory Disclosures are available on https://www.icicisecurities.com (for institutional business) and www.icicidirect.com (for retail business).

[Idirect-Tech]

Dear Ankush, Please call get_historical_data_v2() endpoint for fetching historical data instead of get_historical_ data(). If you still face any issue, please let us know. [E29CF836] Thanks & Regards, Akash Sapkal, Team Breeze API, ICICI Securities

…

________________________________ From: Akash Sapkal /EPG/ISEC/NAVI MUMBAI ***@***.***> Sent: 03 January 2025 15:31 To: Idirect-Tech/Breeze-Python-SDK ***@***.***>; Idirect-Tech/Breeze-Python-SDK ***@***.***> Cc: BREEZEAPI ***@***.***>; Comment ***@***.***> Subject: Re: [Idirect-Tech/Breeze-Python-SDK] Incomplete data while fetching minute level historical data (Issue #178) Dear Ankush, Greetings for the day! Please share your contact details to connect with you. [E29CF836] Thanks & Regards, Akash Sapkal, Team Breeze API, ICICI Securities

________________________________ From: ankush1729 ***@***.***> Sent: 03 January 2025 11:34 To: Idirect-Tech/Breeze-Python-SDK ***@***.***> Cc: BREEZEAPI ***@***.***>; Comment ***@***.***> Subject: Re: [Idirect-Tech/Breeze-Python-SDK] Incomplete data while fetching minute level historical data (Issue #178) CAUTION: This email originated from outside the organization! Do not click links, open attachments or reply, unless you recognize the sender's email address! Use 'Report suspicious Email' button in the toolbar to report Phishing mails. Beware! I've attached both 30minute and daily level pull payloads. Thanks, Ankush 30_min_data.json<https://github.com/user-attachments/files/18296869/30_min_data.json> daily_data.json<https://github.com/user-attachments/files/18296871/daily_data.json> — Reply to this email directly, view it on GitHub<#178 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3TQZ42XADZFQKPANAV36Q32IYR7TAVCNFSM6AAAAABUK5BSFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRYG4ZTCOBYG4>. You are receiving this because you commented.Message ID: ***@***.***> Print this mail only if absolutely necessary. Save Paper. Save Trees." "The information contained in this e-mail and any attachments to this message are intended for the exclusive use of the intended recipient and may contain proprietary, confidential or legally privileged information. If you are not the intended recipient, please note that you are not authorised to disseminate, distribute or copy this e-mail or any parts of it or act upon/rely on the contents of this e-mail in any manner. Please notify the sender immediately by e-mail and destroy all copies of this e-mail and any attachments. Please also note that ICICI Bank or its subsidiaries and associated companies, (collectively "ICICI Group"), are unable to exercise control or ensure the integrity of/over the contents of the information contained in e-mail transmissions and that any views expressed in this e-mail are not endorsed by/binding on the ICICI Group unless the sender does so expressly with due authority of ICICI Group. Before opening any attachments please check them for viruses and defects and please note that ICICI Group accepts no liability or responsibility for any damage caused by any virus that may be transmitted by this email. Any unauthorized use of this email or any attachment(s) may be treated as infringement of right(s) including right of privacy under the applicable laws of India including Information and Technology Act, 2000. Thank you for your cooperation. Regulatory Disclosures are available on https://www.icicisecurities.com (for institutional business) and www.icicidirect.com (for retail business).

[ankush1729]

Missing day issue remains with get_historical_data_v2() method too. Although the number of missing days are less but issue is still there.

These are the days that are missing entirely from daily data:

2022-06-14 Tuesday
2022-03-29 Tuesday
2021-04-23 Friday
2021-07-30 Friday
2024-12-17 Tuesday

And this particular day had only 6 record for the day for 30 minute pull, instead of 13 records.

Moreover, get_historical_data_v2() method has 2 other major problems.

• It only returns 1000 rows at a time. The 1000 rows that are returned contain data starting from to_date to going back 1000 records. This makes data pull a little tricky, and sharing the payload tricky as well.
• Daily level data returned by _v2 doesn't match to official exchange (NSE) data. This makes correctness of the whole data doubtful.

I can provide with the code that I've written for 3 year worth of 30_min interval data pull and daily data pull if required so that you can recreate the issue at your end.

You can give me your email ID and I can drop a mail if you need my contact details.

Thanks,