Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Target Server Validator #698

Merged
merged 15 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/in-solidarity.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@

ignore:
- "tools/hybrid-quickstart/steps.sh" # because the GKE cli uses 'master'
- "tools/target-server-validator/callout/build_setup.sh" # because https://github.com/apigee/api-platform-samples uses 'master' branch
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions .github/workflows/devrel-static-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ jobs:
LINTER_RULES_PATH: "."
GROOVY_NPM_GROOVY_LINT_FILTER_REGEX_EXCLUDE: "Jenkinsfile"
MARKDOWN_MARKDOWN_LINK_CHECK_DISABLE_ERRORS: true
PYTHON_MYPY_DISABLE_ERRORS: true
anaik91 marked this conversation as resolved.
Show resolved Hide resolved

commit-messages:
name: Conventional Commits Lint
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@
.vscode
.DS_Store
*.jar
!edge-custom-policy-java-hello.jar
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@
/tools/pipeline-runner @seymen @danistrebel
/tools/sf-dependency-list @yuriylesyuk
/tools/proxy-endpoint-unifier @anaik91
/tools/target-server-validator @anaik91
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ Apigee products.
A tool to set up the sample deployments of Apigee Envoy.
- [Apigee API Proxy Endpoint Unifier](tools/proxy-endpoint-unifier) -
A tool to unify/split proxy endpoints based on API basepath.
- [Apigee Target Server Validator](tools/target-server-validator) -
A tool to validate all targets in Target Servers & Apigee API Proxy Bundles.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Evaluate and adapt, if you like:

A tool to validate connectivity to targets, extracted from API Proxy configurations and references to Target Servers.


## Labs

Expand Down
4 changes: 2 additions & 2 deletions tools/proxy-endpoint-unifier/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@

import os
import sys
from apigee import Apigee
import utils
from apigee import Apigee # pylint: disable=import-error
import utils # pylint: disable=import-error
anaik91 marked this conversation as resolved.
Show resolved Hide resolved


def main():
Expand Down
90 changes: 90 additions & 0 deletions tools/target-server-validator/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Apigee Target Server Validator

The objective of this tool to validate targets in Target Servers & Apigee API Proxy Bundles exported from Apigee OPDK/X/Hybrid.
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
Validation is done by deploying a sample proxy which check if HOST & PORT is open from Apigee OPDK/X/Hybrid.
anaik91 marked this conversation as resolved.
Show resolved Hide resolved

> **NOTE**: Discovery of Targets in API Proxy & Sharedflows is limited to only parsing URL from `TargetEndpoint` & `ServiceCallout` Policy.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just parsing URL or also parsing TargetEndpoint/HTTPTargetConnection/LoadBalancer/Server[ ]?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if multiple TargetEndpoints exist?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about 'ServiceCallout' policies in Shared Flows?


> **NOTE**: Dynamic targets are **NOT** supported, Ex : `https://host.{request.formparam.region}.example.com}`

## Disclaimer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy from your other PR/project

This is not an Officially Supported Google Product!

## Pre-Requisites
* python3.x
* Please Install required Python Libs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy from your other PR/project


```
python3 -m pip install requirements.txt
```
* Please fill in `input.properties`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If referring to all Apigee products I think you can just say "Apigee"


```
[source]
baseurl=http://34.131.144.184:8080/v1 # Apigee OPDK/Edge/X/Hybrid Base URL
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make the variable name and/or description clarify that this is for the Management API.

Using the Edge cloud management API URL as the example instead of a public IP address might help achieve that. e.g. http://management-api.apigee-opdk.corp:8080. In any case, please remove the IP.

org=xxx-xxxx-xxx-xxxxx # Apigee OPDK/Edge/X/Hybrid Org
auth_type=basic # API Auth type basic | oauth
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beware that auth_type=oauth does not necessarily mean that it's an X/hybrid org, because OAuth can be enabled and used to access Management APIs in Edge: https://docs.apigee.com/api-platform/system-administration/using-oauth2


[target]
baseurl=https://apigee.googleapis.com/v1 # Apigee OPDK/Edge/X/Hybrid Base URL
org=xxx-xxxx-xxx-xxxxx # Apigee OPDK/Edge/X/Hybrid Org Id
auth_type=oauth # API Auth type basic | oauth

[csv]
file=input.csv # Path to input CSV. Note: CSV needs HOST & PORT columns
default_port=443 # default port if port is not provided in CSV

[validation]
check_csv=true # 'true' to validate Targets in input csv
check_proxies=true # 'true' to validate Proxy Targets else 'false'
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
skip_proxy_list=mock1,stream # Comma sperated list of proxies to skip validation;
proxy_export_dir=export # Export directory needed when check_proxies='true'
api_env=dev # Target Environment to deploy Validation API Proxy
api_name=target_server_validator # Target API Name of Validation API Proxy
vhost_domain_name=devgroup # Target VHost or EnvGroup
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
vhost_ip=<IP> # IP address corresponding to vhost_domain_name. Use if DNS record doesnt exist
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
report_format=csv # Report Format. Choose csv or md (Markdown)
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
```

* Sample input CSV with target servers
> **NOTE:** You need to set `check_csv=true` in the `validation` section of `input.properties`

> **NOTE:** You need to set `file=<CSV Name>` in the `csv` section of `input.properties`

anaik91 marked this conversation as resolved.
Show resolved Hide resolved
```
HOST,PORT
httpbin.org
mocktarget.apigee.net,80
smtp.gmail.com,465
```


* Please run below command to authenticate against Apigee X/Hybrid APIS
anaik91 marked this conversation as resolved.
Show resolved Hide resolved

```
export APIGEE_OPDK_ACCESS_TOKEN=$(echo -n "<user>:<password>" | base64) # Access token for Apigee OPDK
export APIGEE_ACCESS_TOKEN=$(gcloud auth print-access-token) # Access token for Apigee X
```

## Highlevel Working
* Export Target Server Details
* Export Proxy Bundle
* Parse Each Proxy Bundle for Target
* Run Validate API against each Target
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
* Generate CSV Report
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSV


## Usage

Run the Script as below
anaik91 marked this conversation as resolved.
Show resolved Hide resolved
```
python3 main.py
```

## Report
Validation Report : `report.md` OR `report.csv` can be accessed in same localtion as script.
anaik91 marked this conversation as resolved.
Show resolved Hide resolved

Please check a [Sample report](report.md)

## Copyright
anaik91 marked this conversation as resolved.
Show resolved Hide resolved

Copyright 2023 Google LLC. This software is provided as-is, without warranty or representation for any use or purpose. Your use of it is subject to your agreement with Google.
Loading