Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: query path instrumentation #26106

Merged
merged 1 commit into from
Mar 6, 2025

Conversation

praveen-influx
Copy link
Contributor

@praveen-influx praveen-influx commented Mar 6, 2025

  • spans added for buffer, parquet chunks along with number of files that are already in parquet cache along with the sql

NOTE: The parquet cache check for files is done at the point of call to TableProvider::scan, the data may get evicted by the time datafusion actually tries to access it but for a lot of the queries which are running in perf test env, this is a good enough approximation for now

The workflow here is,

  • spin up influxdb3 with extra switches --traces-exporter-jaeger-agent-port 6831 --traces-exporter-jaeger-service-name influxdb3-core --traces-exporter jaeger and
  • jaeger - following docs

Once it's running, issue queries and then in jaeger ui (running on http://localhost:16686), search for

  • service name: influxdb3-core
  • operation: query_database_sql
  • tags: query="select count(*) from cpu_4 where time > now() - interval '15 minutes' limit 1"

image (13)

That should return bunch of traces and drilling into one of those get_db_namespace should show table_chunks span that shows the buffer chunks, parquet chunks and how many parquet chunks are already in the cache

image (12)

image (11)

@praveen-influx praveen-influx force-pushed the prav/query-path-instrumentation branch 2 times, most recently from 6064561 to db77a99 Compare March 6, 2025 16:56
- spans added for buffer, parquet chunks along with number of files that
  are already in parquet cache along with the sql
@praveen-influx praveen-influx force-pushed the prav/query-path-instrumentation branch from db77a99 to 6ff33bc Compare March 6, 2025 16:57
@praveen-influx praveen-influx requested a review from a team March 6, 2025 17:05
Copy link
Contributor

@waynr waynr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@praveen-influx praveen-influx merged commit c724e06 into main Mar 6, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants