Failed to load code: bad relocation type 1

[reubenhwk]

I compiled llvm + clang 3.9.1, and I compiled my sample like this...

~/local/bin/clang -target bpf -c src/benchmark/factor.c -o factor.o

and I get this error...

Failed to load code: bad relocation type 1

What does this mean, and how do I fix this?

[pchaigno]

Could you show your C code?
You should get a relocation type of 2 for calls to external functions (helpers) and 1 only for maps (which ubpf doesn't support yet):

ubpf/vm/ubpf_loader.c

Lines 185 to 188 in 2e06fd4

	if (ELF64_R_TYPE(r->r_info) != 2) {
	*errmsg = ubpf_error("bad relocation type %u", ELF64_R_TYPE(r->r_info));
	goto error;
	}

[reubenhwk]

It looks to me like the type 1 relocations are labels from my loops? Or maybe it's the internal function? Not sure. If you would, tell me a more about this, point me to some docs, etc. I may attempt to fix it.

typedef struct {
	unsigned f[32];
	unsigned length;
} results_t;

void factor(unsigned n, results_t * output) {
	unsigned * factors = output->f;
	unsigned count = 0;
	unsigned f = 3;

	while (n % 2 == 0) {
		factors[count++] = 2;
		n /= 2;
	}

	while (n > 1) {
		if (n % f == 0) {
			factors[count++] = f;
			n /= f;
		} else {
			f += 2;
		}
	}

	output->length = count;
}

int main() {
	unsigned numbers[] = {100, 1234, 83924, 9849022, 84920, 83492, 2000000001};
	results_t results[7] = {};

	for (int i = 0; i < 7; ++i) {
		factor(numbers[i], &results[i]);
	}

	return 0;
}

[pchaigno]

Could be your factor function. Try declaring it as static inline. As far as I know, ubpf doesn't support calls to other BPF functions as the last Linux BPF does.

[reubenhwk]

There may be issues with the internal functions, but I don't think that's the issue I'm seeing now. It seems to be the read-only section...

int main()
{
	int a[1] = {1};
	return a[0];
}

compiles down to this...

        .text
	.globl	main
	.align	8
main:                                   # @main
# BB#0:                                 # %entry
	mov	r1, 0
	stw	-4(r10), r1
	ld_64	r1, <MCOperand Expr:(.Lmain.a)>
	ldw	r1, 0(r1)
	stw	-8(r10), r1
	ldw	r0, -8(r10)
	ret

	.section	.rodata.cst4,"aM",@progbits,4
	.align	4                       # @main.a
.Lmain.a:
	.long	1                       # 0x1

When I remove the '1' from the initializer list, then the code doesn't have the .rodata section and it works. So that list of .rodata is a type 1 relocation?

[Alan-Jowett]

@pchaigno is there a reference doc for the relocation types in BPF ELF format? Something similar to AMD64 ABI?

I was planning on adding support for maps and other type 1 relocations, but I wasn't able to find a good reference (can't look at the code in Linux kernel to avoid cross-contamination).

[jpsamaroo]

You could check out the libbpf (https://github.com/libbpf/libbpf) project, which comes from the Linux kernel but is dual-licensed with BSD 2-clause and LGPL.

[Alan-Jowett]

Something doesn't match here. lllvm defines only 2 relocation types for EBPF:
R_BPF_64_64 = 1
R_BPF_64_32 = 10

See: llvm/BinaryFormat/ELFRelocs/BPF.def

Where does the type 2 in the code come from?

Likewise a dump of a eBPF program using readelf shows:

Relocation section '.relxdp_redirect' at offset 0x1c60 contains 2 entries:
  Offset          Info           Type           Sym. Value    Sym. Name
000000000058  004700000001 R_BPF_INSN_64     0000000000000000 tx_port
000000000090  004600000001 R_BPF_INSN_64     000000000000001c rxcnt

[Dantali0n]

I would be great if we could somehow disallow clang to emit a BPF relocation type 1 instruction. I'm trying to figure out what generates generates these relocations. So far I have found out that global variables .text section accesses generate the R_BPF_64_64 relocation.

Some things to help debugging I found are:

llvm-objdump --reloc bpf_program.o
llvm-objdumo -d bpf_program.o

I have also changed between several versions of the clang compiler (3.8, 6, 10) and tried changing the options up, unfortunately I haven't been able to prevent the generation of these instructions so far.

Instead of the regular clang -g -O2 -target bpfI have also tried the more elaborate clang -g -O2 -emit-llvm -c file.c -o - | llc-10 -march=bpf -mcpu=probe -filetype=obj -o file.o.

UPDATE

Declaring global variables as static prevents the emitting of this R_BPF_64_64 instruction

[slide]

This occurs with strings as well, e.g. if I have something like the following:

log(0, "This is a test");

The string data goes into the rodata section.

[Alan-Jowett]

If you want to add support for strings, you could build in this commit 4e192c3.

It adds support for emit_string_load and emitting a string table at the end of the generated machine code.

[slide]

I am not using an Intel architecture, so that commit doesn't help. I am also not using JIT (because I am using a non-Intel architecture). I'll take a look and see if I can figure out how to add support for what I need and propose a PR. Thanks for the info though!

[Alan-Jowett]

A workaround is this:

#include <uapi/linux/bpf.h>
#include <bpf/bpf_helpers.h>

#undef bpf_printk
#define bpf_printk(fmt, ...) \
    ({ \
        char ____fmt[] = fmt; \
        bpf_trace_printk(____fmt, sizeof(____fmt), ##__VA_ARGS__); \
    })


SEC("test")
int test()
{
    bpf_printk("Really long string string 1");
    return 0;
}

Produces:

llvm-objdump: warning: 'test.o': failed to parse debug information for test.o
       0:       b7 01 00 00 67 20 31 00 r1 = 3219559
       1:       63 1a f8 ff 00 00 00 00 *(u32 *)(r10 - 8) = r1
       2:       18 01 00 00 6e 67 20 73 00 00 00 00 74 72 69 6e r1 = 7956016061199968110 ll
       4:       7b 1a f0 ff 00 00 00 00 *(u64 *)(r10 - 16) = r1
       5:       18 01 00 00 6f 6e 67 20 00 00 00 00 73 74 72 69 r1 = 7598263559141158511 ll
       7:       7b 1a e8 ff 00 00 00 00 *(u64 *)(r10 - 24) = r1
       8:       18 01 00 00 52 65 61 6c 00 00 00 00 6c 79 20 6c r1 = 7791360861932709202 ll
      10:       7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
      11:       bf a1 00 00 00 00 00 00 r1 = r10
      12:       07 01 00 00 e0 ff ff ff r1 += -32
      13:       b7 02 00 00 1c 00 00 00 r2 = 28
      14:       85 00 00 00 06 00 00 00 call 6
      15:       b7 00 00 00 00 00 00 00 r0 = 0
      16:       95 00 00 00 00 00 00 00 exit

Which has no relocations.

[slide]

That's very helpful, thanks very much!

[mdr78]

@pchaigno & @reubenhwk

Having much the same issue, when building code with CLang-10, with the flags '-O2 -target bpf'.
My sample bpf snippet is below.

#include <netinet/in.h>                                              
#include <vppinfra/types.h>                                          
#include <vnet/ip/ip4_packet.h>                                      
#include <vnet/udp/udp_packet.h>                                     
                                                                     
typedef enum                                                         
  {                                                                  
    BPFTRACE_E_LOGLVL_ERROR = 0,                                     
    BPFTRACE_E_LOGLVL_WARN = 1,                                      
    BPFTRACE_E_LOGLVL_DEBUG = 2,                                     
  } bpftrace_e_loglevel;                                             
                                                                     
/* extern void */                                                    
void bpftrace_log (bpftrace_e_loglevel e, char *, ...);              
                                                                     
u64 entry (void *mem)                                                
{                                                                    
  ip4_header_t *ip = (ip4_header_t *) mem;                           
  udp_header_t *udp = (udp_header_t *) (mem + sizeof (ip4_header_t));
                                                                     
  ip->fragment_id = 2;                                               
  udp->dst_port = htons(8086);                                       
  ip->checksum = ip4_header_checksum(ip);                            
                                                                     
  bpftrace_log (BPFTRACE_E_LOGLVL_ERROR, "Hello World");             
                                                                     
  return 0;                                                          
}   

When I build with CLang and include the call to bpftrace_log, I get the following relocations in my elf object.

Relocation section [ 3] '.rel.text' for section [ 2] '.text' at offset 0x490 contains 2 entries:
  Offset              Type                 Value               Name
  0x0000000000000220  BPF_64_64            000000000000000000  .L.str
  0x0000000000000230  BPF_64_32            000000000000000000  bpftrace_log

Relocation section [ 6] '.rel.eh_frame' for section [ 5] '.eh_frame' at offset 0x4b0 contains 1 entry:
  Offset              Type                 Value               Name
  0x000000000000001c  BPF_64_64            000000000000000000  .text

However when the call to bpftrace_log is omitted, these disappear and all is well - no more errors from ubpf.

Relocation section [ 4] '.rel.eh_frame' for section [ 3] '.eh_frame' at offset 0x430 contains 1 entry:
  Offset              Type                 Value               Name
  0x000000000000001c  BPF_64_64            000000000000000000  .text

The bit I am missing is that CLang seems to emit 2 relocation types for ebpf, neither of which have a relocation type of 2?

// No relocation                
ELF_RELOC(R_BPF_NONE,        0) 
ELF_RELOC(R_BPF_64_64,       1) 
ELF_RELOC(R_BPF_64_32,      10) 

So why is ubpf expecting a relocation type of 2?
What am I missing ...

 if (ELF64_R_TYPE(r->r_info) != 2) {                                          
     *errmsg = ubpf_error("bad relocation type %u", ELF64_R_TYPE(r->r_info)); 
     goto error;                                                              
 }                                                                            

[Dantali0n]

So why is ubpf expecting a relocation type of 2? What am I missing ...

There exists a patch in the wild that addresses this, I haven't tested it:
iomartin@af4a54c

[mdr78]

Yeah - this pretty much what I had written and was testing.

[mdr78]

@Dantali0n so on reflection, what made me nervous about that approach, was that I wasn't certain it would cover every permutation of what Clang might produce. After thinking about it again, I think a much better approach would be to provide a Makefile and a simple example code, and say this is what we support, it then becomes a bounded problem.

@reubenhwk and @pchaigno - what do you think? I can do the work, if people like the idea.

[mdr78]

@Dantali0n

I coded a fix supports both Clang code/function and string relocations, without breaking existing relocations.
The fix is similar to the fix from iomartin, but it also supports string relocations.

All the existing nosetests work as before.
You can find the code in this PR #102.