Refactor aya-gen into aya-tool and add tplist-like capabilities

[willfindlay]

bcc currently has tplist, a tool that outputs a list of available tracepoints / USDT targets and optionally dumps their arguments in a format that can easily be turned into a C data structure.

I think it would be great if Aya had a self-contained tool that could do the above and possibly more. At the moment, I don't know of a good way to find this information besides 1) depending on bcc for the tplist tool, 2) manually examining the contents of these files, or 3) reading the kernel sources.

To do this, we could write a new aya-info tool that similarly parses information from /sys/kernel/debug/tracing. This new tool could be used to get a list of available tracepoints as well as available targets for tracing programs / kprobes. Other features could also be included, such as the ability to query the availability of specific eBPF features in the kernel. Any functionality should probably also be exported as a library (i.e. so that it could be included in a build.rs file or used to dynamically resolve available targets at runtime).

Does anyone have any thoughts on whether this would be worth pursuing?

[dave-tucker]

Perhaps we could make this a subcommand of aya-gen. For example: - aya-gen tracepoint list - aya-gen tracepoint generate The first would list available tracepoints in the system. The second would generate a Rust struct that could be used in your program. The only annoying thing about this is that access to /sys/kernel/debug usually requires sudo, so it might not be suited to use in a build.rs. And I'm not sure how stable parsing the output is...

…

On Fri, 29 Oct 2021, at 18:57, William Findlay wrote: bcc <https://github.com/iovisor/bcc> currently has `tplist` <https://github.com/iovisor/bcc/blob/master/tools/tplist.py>, a tool that outputs a list of available tracepoints / USDT targets and optionally dumps their arguments in a format that can easily be turned into a C data structure. I think it would be great if Aya had a self-contained tool that could do the above and possibly more. At the moment, I don't know of a good way to find this information besides 1) depending on bcc for the tplist tool, 2) manually examining the contents of these files, or 3) reading the kernel sources. To do this, we could write a new `aya-info` tool that similarly parses information from `/sys/kernel/debug/tracing`. This new tool could be used to get a list of available tracepoints as well as available targets for tracing programs / kprobes. Other features could also be included, such as the ability to query the availability of specific eBPF features in the kernel. Any functionality should probably also be exported as a library for inclusion in build.rs files for example. Does anyone have any thoughts on whether this would be worth pursuing? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#87>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA43JEP4WKUHZ3PCXC5Y4WLUJLOCJANCNFSM5G73NU3Q>.

[willfindlay]

Hmm, good point about build.rs support. Although, to be honest, I envision this being the most useful as a standalone tool during development and a library for use at runtime (e.g. allow the user to perform a check just before attaching a probe). Other aspects (for example, probing for supported BPF features) probably don't require root (but I haven't checked).

I'm not sure I agree about this being a subcommand of aya-gen since most aspects of my proposal are not generating anything per se.

The aya-gen tracepoint generate command, on the other hand, does sound like it would belong in aya-gen. However, I think this is already possible using existing bindings generated from the kernel (i.e., how libbpf already does it).

Here's a slightly crazy idea: What about renaming aya-gen to aya-tool or something similar and then providing the existing aya-gen functionality as a generate subcommand?

[alessandrod]

Here's a slightly crazy idea: What about renaming aya-gen to aya-tool or something similar and then providing the existing aya-gen functionality as a generate subcommand?

I think we should 100% do this

[willfindlay]

Sounds good to me then :)

[willfindlay]

@alessandrod @dave-tucker I just had another thought before I start working on this -- what about making this a cargo subcommand instead? For example cargo aya generate or cargo aya tplist?

[dave-tucker]

@willfindlay Honestly? I'm not sure....

On thinking over the UX, I'm a bit conflicted as to if/how this should be implemented... My preference would be to go standalone, yet I'm also struggling to think of how some of the feature requests in aya-template aren't easily do-able via cargo-generate - i.e aya-rs/aya-template#11, or running cargo-generate again to add another program type.

Speaking of UX, I guess how I'd like this to work is:

1. You use a tool to list all available tracepoints
2. You pass the tracepoint name to a tool to generate a program for you. You get a program skeleton tracepoint program with a bindgen of trace_event_raw_$tracepoint (we would have to handle cases where the name didn't match convention) and code to coerce that from the context. Maybe the check you mentioned for existence of the tracepoint could be included in the generated loader code too. And all of this would be CO-RE ready 🎉

And the relevant things we have right now are:

1. aya-gen which becomes aya-tool - codegen and misc utils
2. cargo-generate and the aya-template repo - project scaffolding

So maybe, given that, we could glue this all together as follows:

1. run sudo aya-tool tracepoint list
2. run cargo generate --git https://github.com/aya-rs/aya-template to generate a new tracepoint project
3. run sudo aya-tool tracepoint generate $tracepoint --out-dir src/tp-ebpf to generate a binding to the right struct from vmlinux for that tracepoint and place it in --out-dir
4. manually add the necessary code to the src/tp-ebpf/main,rs to get the struct from the context

maybe even some of this can be automated further (can cargo-generate be used as a library? or worst case we can fork/exec it)?

[alessandrod]

what about making this a cargo subcommand instead? For example cargo aya generate or cargo aya tplist?

what would we gain from doing that and what would we lose? If cargo <blah> worked like npx <blah> it would be a no brainer, but it doesn't so i'm not sure what the advantage would be.

What comes to mind right now:

Pros: we get the CARGO_* env vars set by cargo
Cons: we need to be generally compatible with the cargo cmdline (not necessarily a bad thing, just an extra requirement)