Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Fix Azure mail reports #20

Merged
merged 3 commits into from
Sep 11, 2024
Merged

[BUG] Fix Azure mail reports #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a370be8
fix azure mail reports bug
8naama Sep 11, 2024
6037d48
update readme
8naama Sep 11, 2024
c3d3866
fix tests
8naama Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 12 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,15 +174,16 @@ For Azure Mail Reports, use type `azure_mail_reports` with the below parameters.
For structuring custom general Azure API calls use type `azure_general` API with the parameters below.

## Configuration Options
| Parameter Name | Description | Required/Optional | Default |
|-----------------------|-------------------------------------------------------------------------------------------------------------|-------------------|-------------|
| name | Name of the API (custom name) | Optional | `azure api` |
| azure_ad_tenant_id | The Azure AD Tenant id | Required | - |
| azure_ad_client_id | The Azure AD Client id | Required | - |
| azure_ad_secret_value | The Azure AD Secret value | Required | - |
| data_request | Nest here any detail relevant to the data request. (Options in [General API](./src/apis/general/README.md)) | Required | - |
| days_back_fetch | The amount of days to fetch back in the first request | Optional | 1 (day) |
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |
| Parameter Name | Description | Required/Optional | Default |
|-----------------------|-------------------------------------------------------------------------------------------------------------|-------------------|----------------------------------------------------------|
| name | Name of the API (custom name) | Optional | `azure api` |
| azure_ad_tenant_id | The Azure AD Tenant id | Required | - |
| azure_ad_client_id | The Azure AD Client id | Required | - |
| azure_ad_secret_value | The Azure AD Secret value | Required | - |
| data_request | Nest here any detail relevant to the data request. (Options in [General API](./src/apis/general/README.md)) | Required | - |
| scope | The specific permissions an application is requesting during a token request. | Optional | Azure graph scope `https://graph.microsoft.com/.default` |
| days_back_fetch | The amount of days to fetch back in the first request | Optional | 1 (day) |
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |

</details>
<details>
Expand Down Expand Up @@ -284,6 +285,8 @@ docker stop -t 30 logzio-api-fetcher
```

## Changelog:
- **0.2.2**:
- Resolve Azure mail reports bug
- **0.2.1**:
- Add 1Password Support
- Add `next_body` support to allow more customization in general settings
Expand Down
4 changes: 3 additions & 1 deletion src/apis/azure/AzureApi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ class AzureApi(OAuthApi):
date_filter_key: str = Field(default="createdDateTime")

def __init__(self, **data):
scope = data.pop('scope', "https://graph.microsoft.com/.default")

token_request = ApiFetcher(
url=f"https://login.microsoftonline.com/{data.get('azure_ad_tenant_id')}/oauth2/v2.0/token",
body=f"""client_id={data.get('azure_ad_client_id')}
&scope=https://graph.microsoft.com/.default
&scope={scope}
&client_secret={data.get('azure_ad_secret_value')}
&grant_type=client_credentials
""",
Expand Down
7 changes: 4 additions & 3 deletions src/apis/azure/AzureMailReports.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ def __init__(self, **data):
stop_indication=StopPaginationSettings(field="d.results",
condition="empty")),
response_data_path="d.results")
data["scope"] = "https://outlook.office365.com/.default"
super().__init__(data_request=data_request, **data)

# Structure the next URL format, to automatically update start date.
Expand All @@ -44,15 +45,15 @@ def _initialize_url_date(self, fetch_start_date, fetch_end_date):
https://url/from/input?$filter=StartDate eq datetime '2024-05-28T13:08:54Z' and EndDate eq datetime '2024-05-29T13:08:54Z'
"""
self.data_request.url = (self.data_request.next_url
.replace(f"{{res.value.[0].{self.end_date_filter_key}}}", fetch_start_date)
.replace(f"{{res.d.results.[0].{self.end_date_filter_key}}}", fetch_start_date)
.replace("NOW_DATE", fetch_end_date))

def _initialize_next_url(self):
"""
initializing the data request next url to be in format:
https://url/from/input?$filter=StartDate eq datetime '{res.d.results.[0].EndDate}' and EndDate eq datetime 'NOW_DATE'
https://url/from/input?$filter=StartDate eq datetime'{res.d.results.[0].EndDate}' and EndDate eq datetime'NOW_DATE'
"""
new_next_url = self.data_request.url + f"?$filter={self.date_filter_key} eq datetime '{{res.d.results.[0].{self.end_date_filter_key}}}' and {self.end_date_filter_key} eq datetime 'NOW_DATE'"
new_next_url = self.data_request.url + f"?$filter={self.date_filter_key} eq datetime'{{res.d.results.[0].{self.end_date_filter_key}}}' and {self.end_date_filter_key} eq datetime'NOW_DATE'&$format=json"
self.data_request.update_next_url(new_next_url)

@staticmethod
Expand Down
19 changes: 10 additions & 9 deletions src/apis/azure/README.MD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,16 @@ Configuration [example here](#example).
## Azure General
Below fields are relevant for **all Azure API types**

| Parameter Name | Description | Required/Optional | Default |
|-----------------------|-----------------------------------------------------------------------------------------------------|-------------------|-------------|
| name | Name of the API (custom name) | Optional | `azure api` |
| azure_ad_tenant_id | The Azure AD Tenant id | Required | - |
| azure_ad_client_id | The Azure AD Client id | Required | - |
| azure_ad_secret_value | The Azure AD Secret value | Required | - |
| data_request | Nest here any detail relevant to the data request. (Options in [General API](../general/README.md)) | Required | - |
| days_back_fetch | The amount of days to fetch back in the first request | Optional | 1 (day) |
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |
| Parameter Name | Description | Required/Optional | Default |
|-----------------------|-----------------------------------------------------------------------------------------------------|-------------------|----------------------------------------------------------|
| name | Name of the API (custom name) | Optional | `azure api` |
| azure_ad_tenant_id | The Azure AD Tenant id | Required | - |
| azure_ad_client_id | The Azure AD Client id | Required | - |
| azure_ad_secret_value | The Azure AD Secret value | Required | - |
| data_request | Nest here any detail relevant to the data request. (Options in [General API](../general/README.md)) | Required | - |
| scope | The specific permissions an application is requesting during a token request. | Optional | Azure graph scope `https://graph.microsoft.com/.default` |
| days_back_fetch | The amount of days to fetch back in the first request | Optional | 1 (day) |
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |

## Azure Graph
By default `azure_graph` API type has built in pagination settings and sets the `response_data_path` to `value` field.
Expand Down
12 changes: 9 additions & 3 deletions tests/UnitTests/test_azure_api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,17 @@ def test_valid_setup(self):
&grant_type=client_credentials
"""

expected_token_body2 = """client_id=some-client
&scope=https://outlook.office365.com/.default
&client_secret=some-secret
&grant_type=client_credentials
"""

# Validate the token request
self.assertEqual(ag.token_request.url, "https://login.microsoftonline.com/some-tenant/oauth2/v2.0/token")
self.assertEqual(am.token_request.url, "https://login.microsoftonline.com/some-tenant/oauth2/v2.0/token")
self.assertEqual(ag.token_request.body, expected_token_body)
self.assertEqual(am.token_request.body, expected_token_body)
self.assertEqual(am.token_request.body, expected_token_body2)

# Validate the data request URL was updated
self.assertIn("https://azure-graph?$filter=createdDateTime gt", ag.data_request.url)
Expand All @@ -64,7 +70,7 @@ def test_valid_setup(self):
self.assertEqual("https://azure-graph?$filter=createdDateTime gt {res.value.[0].createdDateTime}",
ag.data_request.next_url)
self.assertEqual(
"https://azure-mail?$filter=StartDate eq datetime '{res.d.results.[0].EndDate}' and EndDate eq datetime 'NOW_DATE'",
"https://azure-mail?$filter=StartDate eq datetime'{res.d.results.[0].EndDate}' and EndDate eq datetime'NOW_DATE'&$format=json",
am.data_request.next_url)

def test_start_date_generator(self):
Expand Down Expand Up @@ -159,4 +165,4 @@ def test_azure_mail_send_request(self):

self.assertEqual(result, data_res_body.get("d").get("results"))
self.assertEqual(a.data_request.url,
"https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime '2024-05-30T13:08:54Z' and EndDate eq datetime 'NOW_DATE'")
"https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate eq datetime'2024-05-30T13:08:54Z' and EndDate eq datetime'NOW_DATE'&$format=json")