Sprint November 13 - November 24

[timgl]

Global Sprint Planning

3 things that might take us down

from previous sprint issues

solved/delete/or carry forward?

• Clickhouse EU/cloud
• Terraform - James to schedule an incident analysis
• Clickhouse visibility - had an incident, and then we had to sudo command in.
• Gunicorn -> Frank looking at killing both

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red


### High priority

-

### Low priority / side quests

-

[mariusandra]

Project Query Engine

Support hero: Tom, followed by Marius

Retro

Only Tom was working the first week. Second week Julian's onboarding in Cambridge

• Trends insights continued (aggregations) @Gilbert09
  • Feature complete, not released
• Have a great onboarding week
  • In progress
• Async slow lane for queries that take 60 < x < 600 seconds to complete.
  • Didn't get to this.

Hang over items from previous sprint

• Insight refactor, continued.

OKR & Status now

• Project "Finish the darned Query Engine" (Tom, Julian, Marius)
  • 🟡 Finish insight HogQL conversion.
  • 🔴 Support non-event data sources on HogQL insights (BI tools for data warehouse).
  • 🔴 Build tools to measure HogQL query performance, and catch regressions before they hit users.

High priority

• Launch HogQL trends and lifecycle insights @Gilbert09 @mariusandra
  • Finish trends
  • Check for data integrity between the old and new systems.
• Start building the HogQL BI features @Gilbert09
  • Trends graphs from any HogQL query
  • Replicating the go/revenue dashboard.
• Async slow lane for queries that take 60 < x < 600 seconds to complete. @webjunkie

Low priority / side quests

• Performance metric mondays (will be more relevant in the next sprints)
• Convert the Retention, Stickiness, Path and Funnel insights.

[benjackwhite]

Team Re-pay

Support hero: @daibhin

Retro

• PD - incident that took a whole day was frustrating
• PD - felt very hard to diagnose a tricky bug - we're clearly missing an integration test to protect us from core behaviour
• PD - felt (for me) like we had good collaboration

Hang over items from previous sprint

• 🟢 Get the team rename done @benjackwhite
• 🟢 First draft of mobile replay specification @pauldambra
• 🟢 Android recording PoC / spike @marandaneto
  • Got Android SDK sending "canvas" data for demo app, but Canvas isn't supported by our RRWeb Player and is considered unsafe.

OKR

• 📱 Mobile Replay 🟢
  • Android PoC looks promising

High priority

• Investigate whether canvas as a target makes sense for iOS (PoC) @marandaneto
• Figure out Canvas player - either within RRweb or our own player @pauldambra

Low priority / side quests

• Network response payloads (would need our own branch

[robbie-c]

Team Web Analytics

Support hero: @robbie-c

Hang over items from previous sprint

n/a

OKR

🟢 Build the dashboards to match 80% of functionality - work underway and making good progress
⚪ Start working towards the architecture to scale to a general release - not started

High priority

• Web Analytics Opt-In Beta Readiness #18177 - should be this week
• Start opt-in feature preview & be responsive to user feedback
• Searchable pathnames, better pathname widget

Low priority

• Change posthog-js default storage fix: Change default storage type, migrate existing users posthog-js#875

[EDsCODE]

Team Data Warehouse

OKR Q4

Objective

PostHog users can effortlessly bring their business data to PostHog and analyze it 🟢

• Key Results:
  • Be able to import data into data warehouse in one click from most popular SaaS tools (X, Y, Z connectors built)
  • 5 happy users/teams who have imported external data
  • Data Warehouse queries are compatible with insights hogQL

PostHog is moved over to whatever CH infrastructure we will be using for the next 1 year (at least) 🟡

• Key Results:
  • We are either on new infrastructure for EU that we host or on ClickHouse Cloud
  • Queries for large customers are performing faster
  • We can scale more easily (without James bespoke effort)

Retro

Product

• *Get the pricing and stripe connector through the door and deployed for beta users. All of it should be implemented by end of week
• Work through a few more connectors (TBD which ones are important)

Infra

• Complete required performance improvements with ClickHouse.cloud
• Remove EBS volumes on Prod US cluster for cost savings

High priority

Product

• get billing deployed
• have a logging and monitoring dashboard setup
• stream selector interface

Infra

• Complete required performance improvements with ClickHouse.cloud
• Start back on migrations to new CH world

[raquelmsmith]

Team Growth

Support hero: Raquel, then Bianca

Retro

• Fix inability to delete orgs due to foreign key on old billing tables
• Fix revenue forecasting for quarterly billing
  • Still in progress
• Keep iterating on onboarding exp until it's good
• Launch data warehouse pricing https://github.com/PostHog/billing/issues/378
  • Still in progress
• Unsubscribe survey

OKR

• ⚪ Objective 1: Better understand what leads to revenue
  • Reasoning: We're still a bit fuzzy about what metrics are leading indicators for revenue.
  • KR: A company scorecard with actionable insights that help us understand where to invest
  • KR: An understanding (doc or dashboard) of how people use multiple products
• ⚪ Objective 2: Simplify and improve the upgrade process
  • Reasoning: It's currently somewhat confusing and requires too many clicks.
  • KR: Increase the percentage of orgs who upgrade to a paid account
  • KR: Decrease or maintain the number of surprise bills that CS has to refund
  • KR: Automatically subscribe people to the products they use
• ⚪ Bonus Objective 3: Continue experimenting with onboarding and activation metrics
  • KR: Product-specific onboarding performs better than the old onboarding flow
  • KR: 2 experiments for Product Analytics & Session Replay activation metrics each

This sprint

High priority

• Make sure unsub survey response is only sent if there is a response @xrdt fix: Only send unsubscribe survey if response is non-empty #18431
• Roll out new onboarding to everyone @raquelmsmith
  • add the project switcher to the /products screen
  • make mobile friendly
  • make sure people who have existing projects aren't suddenly redirected
    • use snipped_onboarding_completed
• Launch data warehouse pricing https://github.com/PostHog/billing/issues/378
  • Automatically set $500 limit on subscribe to data_warehouse @xrdt
• Make sure people aren't receiving billing limit emails every day @xrdt
• Deal with the incident @raquelmsmith
• Deal with usage reports not landing in the db? -- I was using the wrong date 🤦‍♀

Low priority / side quests

• Change MRR per product reporting to be calculated based on usage, not invoices
• Store an email for every billing customer, not just the subscribed ones
• Website pricing page update Improved pricing UI for both website and product posthog.com#7056 @xrdt https://github.com/PostHog/billing/pull/398

[neilkakkar]

Team Feature Success

Support hero: @neilkakkar

Days off:
Juraj: 1 day
Neil: 1 day
Li: 2 day

Retro

• Feature flag Environments: Feature Flag Environments to make testing easier #13160 - @jurajmajerik - should be ready to roll out to users in this sprint! 🥳

• New feature flag targeting options: Relative dates: Relative date targeting for feature flags / surveys #17392, by distinct ID: Allow targeting feature flags based on distinct ids #13808 - @neilkakkar

  • somewhat sidelined by higher prio bugs in flag matching, but this is working in app and 2 SDKs, others to follow (and then UI) - will rollover.

• Fix toolbar for flags: Also return disabled flags in toolbar feature flags #17616 Toolbar Not Respecting Bootstrap #18079 Toolbar can't override control variant #15481 - @liyiy -> will be done by Fri!

Hang over items from previous sprint

---

OKRs

1. Get 5 reference customers for surveys
2. Get reference customers for feature flags and experiments

High priority

• Finish all 5 server SDKs for targeting - Ruby/Go @liyiy ; node/php @neilkakkar
  • Do geoip updates when touching other server side SDKs: Make GeoIP handling for server side & client side events more seamless #14351
  • (like Disable geoip for capture and decide calls by default posthog-python#98)
• Survey user interviews & quick fixes - @liyiy
  • Surveys: allow customising 'next' and 'close' buttons
• Quota limiting for flags - observability - @neilkakkar
• Scheduled rollouts of flags (and potentially everything else): Automated/scheduled rollouts of feature flags #13842 - @jurajmajerik

Low priority / side quests

• Generate cohort of all current users for whom flag is enabled: easy access to all exposed users (@neilkakkar )
• Stale feature flags: Auto-archive stale feature flags #16497 (@jurajmajerik )

[davemurphysf]

Team Pipeline

Support hero: @bretthoerner / @tkaemming

Retro

• Capture-rs is deployed and capturing traffic for team 2 from PostHog-js @xvello
• Finished migrations of BigQuery (migrated half, other half this Thursday), Postgres, and Redshift app @tomasfarias
• Run PoE experiment in prod and figure out next steps depending on that; have a plan to move forward @bretthoerner

Hang over items from previous sprint

• Make replicator use compose webhook and elements_chain @tiina303 / @davemurphysf

OKR

1. OKR, status (red/yellow/green) and action points if yellow/red

• keep the lights on
  • 🟢, no incidents but Brett was paged every day this week
• Batch exports migrations complete - @tomasfarias
  • 🟢, last half of BigQuery being migrated on Thursday, then it will be cleaning up stragglers
• Webhooks v2 @tiina303/@davemurphysf
  • New delivery system
    • 🟡, new system will be in Rust; @ellie is going to assist @davemurphysf (and maybe @tomasfarias)
  • New interface for webhooks and onevent apps
    • 🟢, draft PR up for UI feat: Pipeline UI transformations #18402

High priority

• Capture-rs ingests 100% of analytic events from posthog-js >= 1.86.0. This means ramping up from 1M/day to 100+M/day @xvello
• Transformations page for pipeline UI @tiina303
• Kill export events buffer @tiina303
  • Batch exports - BQ 2nd half export; stragglers @tomasfarias
• Batch export autoscale & observability @bretthoerner

Low priority / side quests

• Work with Monitoring team to write mini RFC with shape of payload and failures modes of capture @xvello
• PostHog-js to send elements_chain @davemurphysf feat: Create elements chain string as we store it posthog-js#823
• Small refactor on Snowflake export to improve async @tomasfarias
• Make progress on PoE (first on-call/support shift) @tkaemming

[ellie]

Team Infra 🚀

Support hero: @danielxnj

Ellie in Paris next week

Hang over items from previous sprint

• wrap up EU primary key bigint migration @danielxnj
• Nginx Unit rollout - looks to be all working, will run a pod in prod tomorrow and see how it goes, should be able to rollout next week @frankh
• finish rolling out tracing in prod, demo on python usage @ellie

High priority

• PR hobby deploy for testable deployable PRs @danielxnj

• EKS 1.28 (been waiting for the 1.28.2 patch fix to come out on EKS) @frankh

• switch RDS to io optimised @frankh

• polishing up capture-rs with @xvello in Paris, -> 100% of prod @ellie

• working with @davemurphysf on rusty webhooks @ellie

Low priority / side quests

[benjackwhite]

Project Noteforce3000

Hang over items from previous sprint

• 🔵 Dark (and tan mode) mode polish ⭐️ @daibhin
  • Still more to do here but lots of improvements already
• 🔵 New Nav/Top Bar (child issue: PostHog 3000 navigation steps #18090) @Twixes ⭐️
  • should be there by the end of the sprint
• 🔵 Cohorts -> Persons, Annotations -> Data Management (and adding any missing apps) @benjackwhite
• 🔵 Universal Search/Command Palette @thmsobrmlr ⭐️⭐️⭐️
  • Probably rolling over into next sprint...
• 🔵 Right Side Panel Notebooks @benjackwhite
• 🔵 Right Side Panel Feedback @benjackwhite
• 🔵 Right Side Panel Docs @benjackwhite
  • In and feature flagged (waiting on website changes)
• Revert project homepage to "old" mode @daibhin
• Insight fixes for Notebooks @thmsobrmlr
  • It works!
  • .... but its not saving correctly...

High priority

• 🔵 Dark mode final colors ⭐️ @daibhin @corywatilo
• 🔵 Universal Search/Command Palette @thmsobrmlr ⭐️⭐️⭐️
  • ... and get this in Notebooks (stretch goal)
• Tidying @daibhin @Twixes
  • Button styles
  • Condensed table design tweaks
• Insights layout for Notebooks (e.g. resizeable) @benjackwhite
• 3000 support for signup/login and onboarding @benjackwhite @raquelmsmith
• Experiment and instrumentation / dashboards for rollout @Twixes
• Fix feature preview flashing the old UI @benjackwhite
• Fix insight saving of date ranges in notebook @thmsobrmlr

Side quests

• Icons tidying
• 🟣 Query Builder ⭐️⭐️⭐️ (PoC) @corywatilo @Twixes @benjackwhite
• 🟣 Person Feed final work (PoC-ing Canvas concept) @Twixes @daibhin
• 🔵 Right Side Panel Activity @benjackwhite