aya-gen: rework command line

[alessandrod]

The aya-gen command allows generating bindings from BTF info. We should add support for generating bindings from C header files too. Related to #87 but can be done before.

The command today looks like:

# generate bindings using BTF info from /sys/kernel/btf/vmlinux
aya-gen btf-types ethhdr iphdr

I'm proposing we change it to look like:

# generate bindings using BTF info from /sys/kernel/btf/vmlinux
aya-gen generate ethhdr iphdr

# generate bindings using BTF info from a custom location
aya-gen generate --btf /path/to/btf-info ethhdr iphdr

# generate bindings using a header file
aya-gen generate --header /path/to/file.h ethhdr iphdr

So it would still default to BTF, it would optionally allow passing a custom BTF info path, and it would add the possibility to generate bindings from a C header file using --header.

All variations should additionally allow passing arguments to bindgen directly, eg:

# --blocklist-type and --rustified-enum get passed to bindgen directly
aya-gen generate --header /path/to/file.h ethhdr iphdr -- --blocklist-type foo --rustified-enum bar 

[nitishm]

I can pick this up @alessandrod

[alessandrod]

I can pick this up @alessandrod

awesome thank you! 😊

[alessandrod]

@nitishm ping? are you still interested in working on this?

[nitishm]

@nitishm ping? are you still interested in working on this?

Really sorry @alessandrod but I have been really occupied with work lately. I should've updated earlier and that's my mistake. It would be best that I give this up and let someone else take this up. I hope to get back to the project in 2022 when things settle down.

[alessandrod]

I hope to get back to the project in 2022 when things settle down.

All good no need to apologize at all - we all get busy! 😊

[willfindlay]

@alessandrod I'm going to try and take on the more general refactor of aya-gen to aya-tool this week (#87), which will include this change.

[willfindlay]

@alessandrod Slight problem -- In order for this to work as you described, allowing you to pass flags directly to bindgen, we would need to invoke bindgen as a command rather than using the builder API. One potential workaround is to use the builder's command_line_flags() method to spit out some default flags we can pass to bindgen, but that feels super ugly to me. Any thoughts?

[alessandrod]

Yep I've ran into this before. Using the builder, then calling command_line_flags() and spawning is the best we can do I think? (in fact it's what I thought we were doing already)

[willfindlay]

Yep I've ran into this before. Using the builder, then calling command_line_flags() and spawning is the best we can do I think?

Alright.

(in fact it's what I thought we were doing already)

Currently we are just calling the builder's generate() method, but maybe that wasn't always the case.

[albertxos]

consider using BTFGen way for aya-gen:

[PinkJohnOfUs]

aya already supports all the bits to implement it. You only have to point aya-gen to the vmlinux you want to target. It needs to be checked if vmlinux can be passed to bpfloader at runtime. Btfgen generates a stripped down version of BTF input, you can ship this reduced btf along the way to your app in user space.
Aya already resolves relocations, so it'd be matter of making that code more generic so that instead of just applying relocations, it can also spit out a list of type definitions needed:
So it would work in two modes:

1. "parse relocations => return types"
2. "apply relocations"

[albertxos]

@vadorovsky could this be a solution for aya-gen regarding btftool version issue? BTFGen way for aya-gen or do you think a container workaround is enough?

[vadorovsky]

@albertxos I think that could be a solution, yes. Let me have a deeper look on BTFGen (I didn't use it yet) and I will try to find some time to figure out how to integrate it with aya-gen.

[alessandrod]

Is there anything left to do here?

[tamird]

Looks like #297 finished this.