Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow PagerDuty migrator script to migrate data while ignoring users #5375

Merged
merged 1 commit into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 69 additions & 6 deletions tools/migrators/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,68 @@ oncall-migrator
Consider modifying [alert templates](https://grafana.com/docs/oncall/latest/alert-behavior/alert-templates/) of the created
webhook integrations to adjust them for incoming payloads.

### Migrate your PagerDuty data while ignoring Grafana users

This scenario may be relevant where you are unable to import your list of Grafana users, but would like to experiment
with Grafana OnCall, using your existing PagerDuty setup as a starting point for experimentation.

If this is relevant to you, you can migrate as such:

```bash
# Step 1: run a plan of what will be migrated, ignoring users for now
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<ONCALL_API_URL>" \
-e ONCALL_API_TOKEN="<ONCALL_API_TOKEN>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
-e MIGRATE_USERS="false" \
oncall-migrator

# Step 2. Actually migrate your PagerDuty data, again ignoring users
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="migrate" \
-e ONCALL_API_URL="<ONCALL_API_URL>" \
-e ONCALL_API_TOKEN="<ONCALL_API_TOKEN>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
-e MIGRATE_USERS="false" \
oncall-migrator

# Step 3. Optional; import your users from PagerDuty into your Grafana stack using our provided script
# For more information on our script, see "Migrating Users" section below for some more information on
# how users are migrated.
#
# Alternatively this can be done with other Grafana IAM methods.
# See Grafana's "Plan your IAM integration strategy" docs for more information on this.
# https://grafana.com/docs/grafana/latest/setup-grafana/configure-security/planning-iam-strategy/
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e GRAFANA_URL="<GRAFANA_API_URL>" \
-e GRAFANA_USERNAME="<GRAFANA_USERNAME>" \
-e GRAFANA_PASSWORD="<GRAFANA_PASSWORD>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
oncall-migrator python /app/add_users_to_grafana.py

# Step 4: When ready, run a plan of what will be migrated, including users this time
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="plan" \
-e ONCALL_API_URL="<ONCALL_API_URL>" \
-e ONCALL_API_TOKEN="<ONCALL_API_TOKEN>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
oncall-migrator

# Step 4: And finally, when ready, actually migrate your PagerDuty data, again including users
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e MODE="migrate" \
-e ONCALL_API_URL="<ONCALL_API_URL>" \
-e ONCALL_API_TOKEN="<ONCALL_API_TOKEN>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
oncall-migrator
```

### Configuration

Configuration is done via environment variables passed to the docker container.
Expand All @@ -165,6 +227,7 @@ Configuration is done via environment variables passed to the docker container.
| `UNSUPPORTED_INTEGRATION_TO_WEBHOOKS` | When set to `true`, integrations with unsupported type will be migrated to Grafana OnCall integrations with type "webhook". When set to `false`, integrations with unsupported type won't be migrated. | Boolean | `false` |
| `EXPERIMENTAL_MIGRATE_EVENT_RULES` | Migrate global event rulesets to Grafana OnCall integrations. | Boolean | `false` |
| `EXPERIMENTAL_MIGRATE_EVENT_RULES_LONG_NAMES` | Include service & integrations names from PD in migrated integrations (only effective when `EXPERIMENTAL_MIGRATE_EVENT_RULES` is `true`). | Boolean | `false` |
| `MIGRATE_USERS` | If `false`, will allow you to important all objects, while ignoring user references in schedules and escalation policies. In addition, if `false`, will also skip importing User notification rules. This may be helpful in cases where you are unable to import your list of Grafana users, but would like to experiment with OnCall using your existing PagerDuty setup as a starting point for experimentation. | Boolean | `true` |

### Resources

Expand Down Expand Up @@ -340,9 +403,9 @@ Grafana users via the Grafana HTTP API.
```bash
docker run --rm \
-e MIGRATING_FROM="pagerduty" \
-e GRAFANA_URL="http://localhost:3000" \
-e GRAFANA_USERNAME="admin" \
-e GRAFANA_PASSWORD="admin" \
-e GRAFANA_URL="<GRAFANA_API_URL>" \
-e GRAFANA_USERNAME="<GRAFANA_USERNAME>" \
-e GRAFANA_PASSWORD="<GRAFANA_PASSWORD>" \
-e PAGERDUTY_API_TOKEN="<PAGERDUTY_API_TOKEN>" \
oncall-migrator python /app/add_users_to_grafana.py
```
Expand All @@ -352,9 +415,9 @@ oncall-migrator python /app/add_users_to_grafana.py
```bash
docker run --rm \
-e MIGRATING_FROM="splunk" \
-e GRAFANA_URL="http://localhost:3000" \
-e GRAFANA_USERNAME="admin" \
-e GRAFANA_PASSWORD="admin" \
-e GRAFANA_URL="<GRAFANA_API_URL>" \
-e GRAFANA_USERNAME="<GRAFANA_USERNAME>" \
-e GRAFANA_PASSWORD="<GRAFANA_PASSWORD>" \
-e SPLUNK_API_ID="<SPLUNK_API_ID>" \
-e SPLUNK_API_KEY="<SPLUNK_API_KEY>" \
oncall-migrator python /app/add_users_to_grafana.py
Expand Down
2 changes: 2 additions & 0 deletions tools/migrators/lib/pagerduty/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,5 @@
UNSUPPORTED_INTEGRATION_TO_WEBHOOKS = (
os.getenv("UNSUPPORTED_INTEGRATION_TO_WEBHOOKS", "false").lower() == "true"
)

MIGRATE_USERS = os.getenv("MIGRATE_USERS", "true").lower() == "true"
27 changes: 18 additions & 9 deletions tools/migrators/lib/pagerduty/migrate.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from lib.oncall.api_client import OnCallAPIClient
from lib.pagerduty.config import (
EXPERIMENTAL_MIGRATE_EVENT_RULES,
MIGRATE_USERS,
MODE,
MODE_PLAN,
PAGERDUTY_API_TOKEN,
Expand Down Expand Up @@ -46,8 +47,12 @@ def migrate() -> None:
session = APISession(PAGERDUTY_API_TOKEN)
session.timeout = 20

print("▶ Fetching users...")
users = session.list_all("users", params={"include[]": "notification_rules"})
if MIGRATE_USERS:
print("▶ Fetching users...")
users = session.list_all("users", params={"include[]": "notification_rules"})
else:
print("▶ Skipping user migration as MIGRATE_USERS is false...")
users = []

oncall_users = OnCallAPIClient.list_users_with_notification_rules()

Expand Down Expand Up @@ -97,8 +102,9 @@ def migrate() -> None:
rules = session.list_all(f"rulesets/{ruleset['id']}/rules")
ruleset["rules"] = rules

for user in users:
match_user(user, oncall_users)
if MIGRATE_USERS:
for user in users:
match_user(user, oncall_users)

user_id_map = {
u["id"]: u["oncall_user"]["id"] if u["oncall_user"] else None for u in users
Expand Down Expand Up @@ -138,11 +144,14 @@ def migrate() -> None:

return

print("▶ Migrating user notification rules...")
for user in users:
if user["oncall_user"]:
migrate_notification_rules(user)
print(TAB + format_user(user))
if MIGRATE_USERS:
print("▶ Migrating user notification rules...")
for user in users:
if user["oncall_user"]:
migrate_notification_rules(user)
print(TAB + format_user(user))
else:
print("▶ Skipping migrating user notification rules as MIGRATE_USERS is false...")

print("▶ Migrating schedules...")
for schedule in schedules:
Expand Down
Loading