Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sprint April 1 - April 12, 2024 #21181

Closed
mariusandra opened this issue Mar 27, 2024 · 9 comments
Closed

Sprint April 1 - April 12, 2024 #21181

mariusandra opened this issue Mar 27, 2024 · 9 comments
Labels
sprint Sprint planning

Comments

@mariusandra
Copy link
Collaborator

Global Sprint Planning

3 things that might take us down

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red


### High priority

-

### Low priority / side quests

-
@mariusandra mariusandra added the sprint Sprint planning label Mar 27, 2024
@mariusandra mariusandra pinned this issue Mar 27, 2024
@mariusandra
Copy link
Collaborator Author

mariusandra commented Mar 27, 2024 •
edited
Loading

Team Productus Analyticus

  • Support hero: Thomas
  • Marius off 2 weeks (entire sprint).

Retro

  • 💙 Support (Michael)
    • Lots of support
  • 🟢 HogQL trends (Marius)
    • Released as feature preview.
    • 1 issue (session duration) and performance tuning remaining.
  • 🟢 HogQL funnels (Thomas)
    • Released as feature preview.
    • Rolled out, had issues with experiments, fixed those, rolling out again today.
  • 🟡 HogQL paths -> funnels (Julian)
    • Also released as a feature preview.
    • PR to connect to funnels in progress, then starting real world testing.

Hang over items from previous sprint

  • HogQL leftovers (paths?)

OKRs Q2 2024

  • HogQL-based querying

    • Convert the remaining legacy queries to HogQL and release to public (Thomas, Julian, Marius)
      • 🟢 Trends? Funnels? Paths? 👈 this sprint
      • 🟠 Cohorts 👈 maybe rewriting for PoEv3
    • Remove legacy querying backend (Thomas, Julian)
      • 🟠 Convert filters to query (insights, notebooks, activity log, experiments) 👈 maybe this sprint
      • 🟢 Clean up or rewrite dashboardLogic 👈 this sprint
    • Missing Product Analytics features (Thomas, Julian)
      • ⚪ Breakdowns (multiple) in literally everything
      • ⚪ Make a list based on GitHub issues from customer requests…
    • Missing HogQL features (Tom, Marius)
      • 🟠 Type system, JSON 👈 maybe this sprint
      • ⚪ Missing things when building funnels

  • Querying and processing performance (Thomas, Julian)

    • Global performance overview dashboards
      • ⚪ Insights
      • ⚪ Exports
      • ⚪ Cohort recalculations
    • Query request tracing
      • ⚪ Possibly query runner Python optimizations
      • ⚪ Exports improvements
    • ⚪ Identify top 5 query optimizations in terms of impact

  • Artificial Hog / Post Intelligence (Michael)

    • ⚪ Ask a question to get a magical insight (aware of your taxonomy)
    • ⚪ Figure out infra for upgrading queries and models
    • ⚪ Product-wide framework for opting into sharing with OpenAI

  • Activation (side quest: Michael)

    • ⚪ Michael to work with Growth to identify optimizations to getting started with Product Analytics

Planning

High priority

We aim to get trends and funnels live for users still this week. Rolling out gradually either starting Friday or latest Monday. Next week the priority is on supporting users, fixing uncovered bugs, continuing the cleanup and other "users asked for it" projects. We're also here to support PoEv3 in any way we can.

  • Thomas: Support & HogQL support
  • Michael: Project environments & support rollover
  • Julian: Legacy logic/filter cleanup, improve dashboard logic & reloads.
  • Tom: HogQL & Data Warehouse. Type system?

Low priority / side quests

  • Experiments to HogQL (Julian? Tom?)
  • Check if there's anything to do for PoEv3 for cohorts.

@benjackwhite
Copy link
Contributor

benjackwhite commented Mar 27, 2024 •
edited
Loading

Team Infra

Support hero: @frank

  • Ben out for half of next sprint
  • Public holidays as well

Hang over items from previous sprint

  1. 🟢 AWS cost usage exercise
  1. 🟡 Spontaneous look into AZ aware routing to try and reduce our biggest cost centre
  • Trying out options for reducing cross az traffic - still work to be done
  1. 🔴 Purchase reserved RDS instances @frankh
  • why? Risk vs cost works out - to be fully confirmed with pipeline team
  1. 🟢 OpenVPN cert updates (only going to worry about CI/CD and infra peeps) @danielxnj
  • why? Current security risk as they have no expiry
  1. 🟡 Get app onto Canary deploys now that we are on contour @danielxnj
  • app.dev is on argo rollouts (not actually canarying but deploying correctly)
  1. 🔴 Test out capture-rs traffic with contour @frankh
  • Had enough ingress issues the last week or so that we took a pause here
  1. 🟡 Get flagged PoC for proper permissions out @benjackwhite
  • So close - just tests to finalise to get merged.

OKR

  1. 💪 Deploy with confidence 🟡
  • 🟢 Manually controlling traffic between two versions of a service
  • 🟢 Automatic canary deploying with rollbacks based on HTTP
  • 🔴 Manual canary deploys for testing scary changes
  1. 💰 Keep costs under control ✅
  2. 👀 More Observability 🔴
  3. 🎯 Soc2 ✅

High priority

  1. Low hanging fruit for cross-AZ traffic optimizations - particularly pg-bouncer in cluster @danielxnj
  • why? By far our biggest cost centre and we can actually get this done
  1. Get pipeline team clued up on how to use contour to gradually migrate to capture-rs @frankh
  • why? They can be fully unblocked and we can finally get capture migrated over
  1. Canary deploys fully rolled out for app @frankh
  • why? Because we are close
  1. Capacity and forecasting dashboard for all databases @frankh
  2. Add OOM and restart alerting for all deployments @danielxnj

Low priority / side quests

  • Improve Kafka alerting
  • Add alerts for ECR vulnerabilities @danielxnj
  • Side quest hackathon for Ben to work with Paul on heatmaps @benjackwhite

@EDsCODE
Copy link
Member

EDsCODE commented Mar 27, 2024 •
edited by timgl
Loading

Team Data Shack

OKR Q2 2024

Objective

Release data warehouse to everyone

  • Key Results:
    • Integration first experience
      • schemas are reliable
      • modeling of each integration is clear
      • Good automatic roll up views and joins
      • Wizard to onboard people
    • Establish a solid pattern to build integrations
    • Complete data warehouse experience in the rest of the app (insights, feature flags, experiments)

James as a Service -> Clickhouse as a Service

  • Key Results:
    • Better Visibility
      • Regularly testing backups
      • Monitoring/alerting
      • Mutations
      • Moves
    • Management
      • Managing/killing mutations
    • Self Serve
      • Schema design feedback (James non blocking
      • Schema management
    • Automation
      • Replace/Upgrade replicas
      • Upgrading to 24.04
      • Disk configs

Retro

Product

  • @EDsCODE Implement data warehouse series logic at hogql layer (vs query by query)
  • @Gilbert09 property typings for data warehouse tables (for autocomplete)
  • Zendesk integration (helping @annikaschmid dogfood data warehouse product) -> should lead to adding row limits

Infra

  • wire up ansible to pull inventory from TF or AWS using tags
  • have US instance up with ansible playbook from galaxy
  • new volumes added to US cluster to increase capacity
  • clean up hobby CI (no more orphaned DO resources)
  • setup CI for both of the above

High priority

Product

  • @EDsCODE High/medium priority fixes suggested by @annikaschmid here Data warehouse feedback #21105
  • @EDsCODE release data warehouse on trends. This is mostly implemented. Need to make sure everything is working after some of the schema issues that we've been fixing
  • Awaiting hogql release for data warehouse on experiments
  • @Gilbert09 property typings for data warehouse tables (for autocomplete)
  • @Gilbert09 zendesk integration v0.2. Support more granular sync selections when first setting up

Infra

  • Add new way to delete data using a join table instead of inline in query
  • Pulumi integration with housewatch -> make sure that when we recreate instances, the volumes are deterministic
  • Mutation visibility in housewatch

@pauldambra
Copy link
Member

pauldambra commented Mar 27, 2024 •
edited
Loading

Team Replay-apotamus

OMG first sprint of the quarter!

Support hero: David

  • Paul - out for one week (although dipping back in for super days)
  • Bank Holiday monday

Retro

  • longer retro tomorrow

items from previous sprint

High priority

✅ iOS session recording
- alpha is ready for release
✅ 5 more Open Source Android apps tested with android replay
- big list of nits and opportuntities here https://docs.google.com/document/d/1gzY3FybfRr84eE7W_QNlCNL21ueAg7W7Evth6AAYNHs/edit
✅ test error clustering with PostHog and ideally with some customers

  • this is out for PostHog
  • really need to validate this - so we ignore productionising it a little longer 🙈

Low priority / side quests

🟡 remove user agent parsing from posthog-js @pauldambra

  • everything is setup for the plugin improvments but not prioritised due to other work
    🔴 stop running replay clients when over quota @pauldambra
  • not prioritised due to other work
    🔴 css-in-js lots of support requests here @daibhin
  • not prioritised due to other work

OKR

  1. OKR, status (red/yellow/green) and action points if yellow/red
  • quality and consolidation
    • mobile replay open beta
    • replay captures every site
    • people can find the valuable recordings
  • Error tracking MVP
  • Hire 2 amazing colleagues
    • 10 interviews this week 🫠

High priority

  • Paul is going to sicily to build heatmaps 3000 with Ben
  • we've got a backlog of nits and improvements to Android replay getting ready for advertised open beta
  • we've got a heavy support backlog
  • so we're starting the quarter with a :fix all the things: sprint

Low priority / side quests

@neilkakkar
Copy link
Collaborator

neilkakkar commented Mar 27, 2024 •
edited
Loading

Team Feature Successsss 🐍

Support hero: @jurajmajerik
Days off:
Juraj: 3 days
Neil: 1 days

Retro

  • Experiments UI draft + results view - basic version with all pre-existing functionality we can release - @jurajmajerik -> Should be good to go by Friday ✅
  • Investigate and fix (or find alternatives) for cohort OOM errors for $bigClients - @neilkakkar -> in progress. Massively overestimated capacity given interviews + planning + support + incident + days off + sick day (as result of former stuff 😮‍💨 )
  • Add property filters to event matching in cohorts - @neilkakkar -> 🚫
  • Find a good flow for restricting users to exposed users in secondary metrics for experiments - @neilkakkar -> 🚫

Hang over items from previous sprint

OKRs

  1. Make sure feature flags can handle 10x current scale
  2. Polish new experiments UI & collect feedback
  3. Add most requested surveys functionality

High priority

  • Collect feedback on the new Experiments UI from our top users - @jurajmajerik

  • Continue improving Experiments UI - @jurajmajerik

    • no results empty state: show checklist for what events are missing. Explore button in the middle of empty state.
    • banners for what to do next
    • data collection participants bug - set default to 20% for happy path, 50% for edges
    • Tooltips (stretch)
    • link to docs (stretch)
    • investigate MAI calculator (stretch)

  • Add property filters to event matching in cohorts - @neilkakkar

  • Find a good flow for restricting users to exposed users in secondary metrics for experiments - @neilkakkar

  • RFC for feature flags rewrite - @neilkakkar

Low priority / side quests

  • Explore experiment metrics easily in insights - @jurajmajerik
  • Temporal queues for feature success - @neilkakkar
  • Setup instrumentation for flip-flopping problem of experiment significance - @neilkakkar

For next sprint

@robbie-c
Copy link
Member

robbie-c commented Mar 27, 2024 •
edited
Loading

Team 🕸️

Off next sprint: none

Last sprint

  • Lost a few days to being ill last sprint
  • Fixed many bugs relating to the backfill, now 95% done on US, 50% on EU
  • HogQL sessions table optimisation
  • Sessions channel type fixes
  • Wrote some docs for web analytics

Hang over items from previous sprint

  • Run sessions backfilll script
  • Add session filters

OKR

  1. Finish the beta to-do list
  2. Find 5 high-ICP reference customers

🔴 switch web analytics over to sessions - aimed to release this week but didn't hit deadline and will be pushed back by person-properties-less events

High priority

  • Pause sessions table work (except nursing the backfill) to work on person-less events
  • Person-less events
    • posthog-js changes
    • UI problem solving whack-a-mole
  • Write more docs for web analytics (e.g. what is a Channel Type?) and add more tooltips to web analytics

Low priority

  • Session filters for all posthog events
  • Add session properties to the property definition endpoint
  • polish and be responsive to support

@raquelmsmith
Copy link
Member

raquelmsmith commented Mar 27, 2024 •
edited by zlwaterfield
Loading

Team Growth

Retro

Retro items

Q1 Goals

✅=finished 🟢=on track to finish this quarter 🟡=might not finish 🔴=won't finish

  • 🟡 Improve activation rates for product analytics and session replay

    • ✅ Polish onboarding flow with Cory
    • 🟢 Run 1 experiment per week
    • 🟢 Do an average of 2 customer interviews per week
    • 🔴 Redefine and improve activation rates for PA & SR
      • Did not really get to this, revisiting for PA in Q2

  • 🟢 Launch pricing for:

    • 🟢 Teams

  • Team offsite!

High priority

Q2 Goals

✅=finished 🟢=on track to finish this quarter 🟡=might not finish 🔴=won't finish

  1. ⚪ Create a flow in product analytics onboarding to fill out a dashboard template using actions (Raquel)
  2. ⚪ Simplify our subscription flows (Zach, supported by Raquel)
  3. ⚪ Launch pricing changes (Bianca, supported by Raquel)
    • Personless events - will help us reach more customers at an affordable price
    • Data warehouse - it's becoming pretty useful, we should charge for it
    • Session replay - we can reduce costs to improve retention and reach more people

This sprint

High priority

  • @raquelmsmith

    • Dashboard creation from onboarding
    • Send emails to prevent surprise bills
      • MessagingRecord table to track emails we send
    • Feature gating for activity panel
      • Migrate customers
    • Feature gating for automatic provisioning
      • Migrate customers
    • Wrap up product intros experiment - it's not successful

  • @xrdt (Out for the first week of this sprint)

    • Support billing for person processing addon
    • Enterprise signup button on website should be "contact us"

  • @zlwaterfield

Low priority

  • CORS issue for Next.js reverse proxy
  • Billing as a product competition/market analysis and breakdown of possible features

Metrics to watch:

@tiina303
Copy link
Contributor

tiina303 commented Mar 27, 2024 •
edited by timgl
Loading

Team Pipeline

Off: Tomas (2 days)
Support: Tomas

Retro

High priority

  • Person property updates don't slow down ingestion - Brett
    • sidetracked by other stuff, e.g. support
  • Launch Person IDs on Events for everyone - Ted & Tomás
    • The squash job runs weekly on Sundays (gated on team IDs because irreversible)
      - blocked on disk space issues -> Jams says not, he will sync with Tomás
    • We have metrics on override table size (total size, highest total per team) to correlate with query performance
    • v3 read path with Analytics: overrides are loaded in a CH dictionary available for query, we monitor its resource usage (memory, cpu) as we backfill values
      - without the dictionary part as discussed that to be perf tested first
    • Overrides for all teams are backfilled to CH and available for query
    • Read path feature flag rolled out to all teams
      - blocked by HogQL insights migration (to avoid incisistencies of data between insights that are not using HogQL yet)
  • Replay overflow implemented, plugin-server detection part - Xavier
    • Likely will ship by EOW

OKR

✅=finished 🟢=on track to finish this quarter 🟡=might not finish 🔴=won't finish

  • ✅ Wrapped up Webhooks Delivery system - Brett
  • ✅ Wrapped up capture-rs posthog-js rollout - Xavier
  • 🔴 Wrapped up New Pipeline UI - Tiina
    • side tracked by CDP, team leading
  • 🟡 Wrapped up PoE (with backfills and squashes) - Ted
    • pending on other teams
  • 🟢 Volume spikes on session replay do not hurt other users - Xavier
  • 🔴 Person property updates don't slow down ingestion - Brett
    • got deprioritised over cloud to cloud migrations & deletions breaking
  • 🔴 Requested data deletion is complete and reliably executed going forward - Tiina
    • turned out to be more complex - tracked in issue

High priority

  • Person processing add-on (Tiina & everyone else)

Low priority / side quests

  • Helping out with HogQL migration (Ted)

@neilkakkar
Copy link
Collaborator

Team Feature Successsss 🐍

Support hero: @neilkakkar
Days off:
Juraj: 1 days
Neil: 1 days

Retro

  • Collect feedback on the new Experiments UI from our top users - @jurajmajerik -> survey is out, some good feedback already, most people love it! Will potentially do some individual reaching out next sprint.
  • Continue improving Experiments UI - @jurajmajerik
    • no results empty state: show checklist for what events are missing. Explore button in the middle of empty state. ✅
    • banners for what to do next - ✅
    • data collection participants bug - set default to 20% for happy path, 50% for edges - no, will rollover.
    • Tooltips (stretch)
    • link to docs (stretch)
    • investigate MAI calculator (stretch)
  • Add property filters [and exact date matching] to event matching in cohorts - @neilkakkar -> ✅
  • Find a good flow for restricting users to exposed users in secondary metrics for experiments - @neilkakkar -> 🔄
  • RFC for feature flags rewrite - @neilkakkar -> ✅

Hang over items from previous sprint

OKRs

  1. Make sure feature flags can handle 10x current scale
  2. Polish new experiments UI & collect feedback
  3. Add most requested surveys functionality

High priority

Low priority / side quests

@mariusandra mariusandra unpinned this issue Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
sprint Sprint planning
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@mariusandra @tiina303 @pauldambra @robbie-c @benjackwhite @neilkakkar @EDsCODE @raquelmsmith