Rosetta failures in guest Linux environment #6799

kjmph · 2024-10-27T15:46:28Z

kjmph
Oct 27, 2024

When using Rosetta in the guest Linux environment, which is Debian testing in my case, Rosetta fails with VDSO & DT_HASH errors. This seemed to have started in the last week or so. As prior to then, everything was functioning. I'm guessing that it may be because Debian testing updated to a newer 6.11.4-arm64 kernel? Here is an example error:

$ file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
$ /media/rosetta/rosetta hello
assertion failed [hash_table != nullptr]: Failed to find vdso DT_HASH
(Vdso.cpp:78 get_vdso_dynamic_data)
 Trace/breakpoint trap
$ strings /media/rosetta/rosetta | ag "Failed to find vdso DT_HASH"
Failed to find vdso DT_HASH

Note, I can run a Mach-O binary on the host platform with Rosetta, showing it must be related to the ELF header.

$ file hello
hello: Mach-O 64-bit executable x86_64
$ arch -x86_64 ./hello
Hello, World!

And the ELF sections are as such:

$ readelf -S hello
There are 13 section headers, starting at offset 0x3088:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .init             PROGBITS         0000000000401000  00001000
       0000000000000003  0000000000000000  AX       0     0     1
  [ 2] .text             PROGBITS         0000000000401010  00001010
       00000000000006ec  0000000000000000  AX       0     0     16
  [ 3] .fini             PROGBITS         00000000004016fc  000016fc
       0000000000000003  0000000000000000  AX       0     0     1
  [ 4] .rodata           PROGBITS         0000000000402000  00002000
       0000000000000332  0000000000000000   A       0     0     32
  [ 5] .eh_frame         PROGBITS         0000000000402338  00002338
       0000000000000034  0000000000000000   A       0     0     8
  [ 6] .init_array       INIT_ARRAY       0000000000403fe8  00002fe8
       0000000000000008  0000000000000008  WA       0     0     8
  [ 7] .fini_array       FINI_ARRAY       0000000000403ff0  00002ff0
       0000000000000008  0000000000000008  WA       0     0     8
  [ 8] .got              PROGBITS         0000000000403ff8  00002ff8
       0000000000000008  0000000000000000  WA       0     0     8
  [ 9] .got.plt          PROGBITS         0000000000404000  00003000
       0000000000000018  0000000000000008  WA       0     0     8
  [10] .data             PROGBITS         0000000000404018  00003018
       000000000000000c  0000000000000000  WA       0     0     8
  [11] .bss              NOBITS           0000000000404040  00003024
       0000000000000258  0000000000000000  WA       0     0     32
  [12] .shstrtab         STRTAB           0000000000000000  00003024
       0000000000000060  0000000000000000           0     0     1

I don't see DT_HASH, yet I'm not as familiar with this.

Configuration

UTM Version: Version 4.5.4 (100)
macOS Version: 15.0.1 (24A348)
Mac Chip: Apple M1 Max

stv0g · 2024-11-13T22:35:51Z

stv0g
Nov 13, 2024

I am seeing this as well with:

Host:
- Hypervisor: lima v1.0.1 / vz macOS Hypervisor Framwork
- OS: macOS 15.2 Beta (24C5079e)
Guest:
Ubuntu 24.10 (Oracular Oriole)
Kernel: Linux lima-default 6.11.0-9-generic [UTM] Multiple concurrent running VMs #9-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 14 13:46:10 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Given the error message, I am assuming that the vDSO ELF object is missing the DT_HASH table:

tv0g@lima-default:/Users/stv0g$ readelf -a /lib/modules/6.11.0-9-generic/vdso/vdso.so
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           AArch64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          22040 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         3
  Size of section headers:           64 (bytes)
  Number of section headers:         23
  Section header string table index: 22

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .gnu.hash         GNU_HASH         00000000000000e8  000000e8
       0000000000000038  0000000000000000   A       2     0     8
  [ 2] .dynsym           DYNSYM           0000000000000120  00000120
       0000000000000090  0000000000000018   A       3     1     8
  [ 3] .dynstr           STRTAB           00000000000001b0  000001b0
       0000000000000077  0000000000000000   A       0     0     1
  [ 4] .gnu.version      VERSYM           0000000000000228  00000228
       000000000000000c  0000000000000002   A       2     0     2
  [ 5] .gnu.version_d    VERDEF           0000000000000238  00000238
       0000000000000038  0000000000000000   A       3     2     8
  [ 6] .note             NOTE             0000000000000270  00000270
       00000000000000c0  0000000000000000   A       0     0     4
  [ 7] .text             PROGBITS         0000000000000340  00000340
       0000000000000648  0000000000000000  AX       0     0     32
  [ 8] .altinstructions  PROGBITS         0000000000000988  00000988
       0000000000000030  0000000000000000   A       0     0     1
  [ 9] .dynamic          DYNAMIC          00000000000009b8  000009b8
       0000000000000110  0000000000000010  WA       3     0     8
  [10] .rodata           PROGBITS         0000000000000ac8  00000ac8
       0000000000000020  0000000000000008 WAX       0     0     8
  [11] .debug_aranges    PROGBITS         0000000000000000  00000af0
       0000000000000060  0000000000000000           0     0     16
  [12] .debug_info       PROGBITS         0000000000000000  00000b50
       0000000000001aa4  0000000000000000           0     0     1
  [13] .debug_abbrev     PROGBITS         0000000000000000  000025f4
       000000000000040d  0000000000000000           0     0     1
  [14] .debug_line       PROGBITS         0000000000000000  00002a01
       0000000000000b79  0000000000000000           0     0     1
  [15] .debug_str        PROGBITS         0000000000000000  0000357a
       0000000000000922  0000000000000001  MS       0     0     1
  [16] .debug_line_str   PROGBITS         0000000000000000  00003e9c
       00000000000002ac  0000000000000001  MS       0     0     1
  [17] .debug_loclists   PROGBITS         0000000000000000  00004148
       0000000000000d0f  0000000000000000           0     0     1
  [18] .debug_rnglists   PROGBITS         0000000000000000  00004e57
       00000000000001b9  0000000000000000           0     0     1
  [19] .comment          PROGBITS         0000000000000000  00005010
       0000000000000025  0000000000000001  MS       0     0     1
  [20] .symtab           SYMTAB           0000000000000000  00005038
       00000000000003c0  0000000000000018          21    35     8
  [21] .strtab           STRTAB           0000000000000000  000053f8
       0000000000000127  0000000000000000           0     0     1
  [22] .shstrtab         STRTAB           0000000000000000  0000551f
       00000000000000f8  0000000000000000           0     0     1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000ae8 0x0000000000000ae8  R E    0x20
  DYNAMIC        0x00000000000009b8 0x00000000000009b8 0x00000000000009b8
                 0x0000000000000110 0x0000000000000110  R      0x8
  NOTE           0x0000000000000270 0x0000000000000270 0x0000000000000270
                 0x00000000000000c0 0x00000000000000c0  R      0x4

 Section to Segment mapping:
  Segment Sections...
   00     .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d .note .text .altinstructions .dynamic .rodata
   01     .dynamic
   02     .note

Dynamic section at offset 0x9b8 contains 12 entries:
  Tag        Type                         Name/Value
 0x000000000000000e (SONAME)             Library soname: [linux-vdso.so.1]
 0x0000000000000010 (SYMBOLIC)           0x0
 0x000000006ffffef5 (GNU_HASH)           0xe8
 0x0000000000000005 (STRTAB)             0x1b0
 0x0000000000000006 (SYMTAB)             0x120
 0x000000000000000a (STRSZ)              119 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x000000006ffffffc (VERDEF)             0x238
 0x000000006ffffffd (VERDEFNUM)          2
 0x000000000000001e (FLAGS)              SYMBOLIC
 0x000000006ffffff0 (VERSYM)             0x228
 0x0000000000000000 (NULL)               0x0

There are no relocations in this file.

The decoding of unwind sections for machine type AArch64 is not currently supported.

Symbol table '.dynsym' contains 6 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 00000000000008a8   172 FUNC    GLOBAL DEFAULT    7 __[...]@@LINUX_2.6.39
     2: 0000000000000980     8 NOTYPE  GLOBAL DEFAULT    7 __[...]@@LINUX_2.6.39
     3: 0000000000000000     0 OBJECT  GLOBAL DEFAULT  ABS LINUX_2.6.39
     4: 0000000000000880    40 FUNC    GLOBAL DEFAULT    7 __[...]@@LINUX_2.6.39
     5: 0000000000000850    40 FUNC    GLOBAL DEFAULT    7 __[...]@@LINUX_2.6.39

Symbol table '.symtab' contains 40 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 00000000000000e8     0 SECTION LOCAL  DEFAULT    1 .gnu.hash
     2: 0000000000000120     0 SECTION LOCAL  DEFAULT    2 .dynsym
     3: 00000000000001b0     0 SECTION LOCAL  DEFAULT    3 .dynstr
     4: 0000000000000228     0 SECTION LOCAL  DEFAULT    4 .gnu.version
     5: 0000000000000238     0 SECTION LOCAL  DEFAULT    5 .gnu.version_d
     6: 0000000000000270     0 SECTION LOCAL  DEFAULT    6 .note
     7: 0000000000000340     0 SECTION LOCAL  DEFAULT    7 .text
     8: 0000000000000988     0 SECTION LOCAL  DEFAULT    8 .altinstructions
     9: 00000000000009b8     0 SECTION LOCAL  DEFAULT    9 .dynamic
    10: 0000000000000ac8     0 SECTION LOCAL  DEFAULT   10 .rodata
    11: 0000000000000000     0 SECTION LOCAL  DEFAULT   11 .debug_aranges
    12: 0000000000000000     0 SECTION LOCAL  DEFAULT   12 .debug_info
    13: 0000000000000000     0 SECTION LOCAL  DEFAULT   13 .debug_abbrev
    14: 0000000000000000     0 SECTION LOCAL  DEFAULT   14 .debug_line
    15: 0000000000000000     0 SECTION LOCAL  DEFAULT   15 .debug_str
    16: 0000000000000000     0 SECTION LOCAL  DEFAULT   16 .debug_line_str
    17: 0000000000000000     0 SECTION LOCAL  DEFAULT   17 .debug_loclists
    18: 0000000000000000     0 SECTION LOCAL  DEFAULT   18 .debug_rnglists
    19: 0000000000000000     0 SECTION LOCAL  DEFAULT   19 .comment
    20: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS vgettimeofday.c
    21: 0000000000000340     0 NOTYPE  LOCAL  DEFAULT    7 $x
    22: 0000000000000340   524 FUNC    LOCAL  DEFAULT    7 __cvdso_gettimeo[...]
    23: 0000000000000550   764 FUNC    LOCAL  DEFAULT    7 __cvdso_clock_ge[...]
    24: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS note.o
    25: 0000000000000300     0 NOTYPE  LOCAL  DEFAULT    6 $d
    26: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS sigreturn.o
    27: 0000000000000978     0 NOTYPE  LOCAL  DEFAULT    7 $x
    28: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS
    29: fffffffffffff000     0 NOTYPE  LOCAL  DEFAULT  ABS _timens_data
    30: 00000000000009b8     0 OBJECT  LOCAL  DEFAULT  ABS _DYNAMIC
    31: ffffffffffffe000     0 NOTYPE  LOCAL  DEFAULT    1 _vdso_data
    32: 0000000000000ac8     0 OBJECT  LOCAL  DEFAULT  ABS _GLOBAL_OFFSET_TABLE_
    33: 0000000000000ae8     0 NOTYPE  LOCAL  DEFAULT   10 _end
    34: 0000000000000980     0 NOTYPE  LOCAL  DEFAULT    7 VDSO_sigtramp
    35: 0000000000000000     0 OBJECT  GLOBAL DEFAULT  ABS LINUX_2.6.39
    36: 00000000000008a8   172 FUNC    GLOBAL DEFAULT    7 __kernel_clock_getres
    37: 0000000000000980     8 NOTYPE  GLOBAL DEFAULT    7 __kernel_rt_sigreturn
    38: 0000000000000880    40 FUNC    GLOBAL DEFAULT    7 __kernel_gettimeofday
    39: 0000000000000850    40 FUNC    GLOBAL DEFAULT    7 __kernel_clock_g[...]

Histogram for `.gnu.hash' bucket list length (total of 3 buckets):
 Length  Number     % of total  Coverage
      0  0          (  0.0%)
      1  2          ( 66.7%)     40.0%
      2  0          (  0.0%)     40.0%
      3  1          ( 33.3%)    100.0%

Version symbols section '.gnu.version' contains 6 entries:
 Addr: 0x0000000000000228  Offset: 0x00000228  Link: 2 (.dynsym)
  000:   0 (*local*)       2 (LINUX_2.6.39)   2 (LINUX_2.6.39)   2 (LINUX_2.6.39)
  004:   2 (LINUX_2.6.39)   2 (LINUX_2.6.39)

Version definition section '.gnu.version_d' contains 2 entries:
 Addr: 0x0000000000000238  Offset: 0x00000238  Link: 3 (.dynstr)
  000000: Rev: 1  Flags: BASE  Index: 1  Cnt: 1  Name: linux-vdso.so.1
  0x001c: Rev: 1  Flags: none  Index: 2  Cnt: 1  Name: LINUX_2.6.39

Displaying notes found in: .note
  Owner                Data size 	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 3230b130d26e503fef8fbfec8efa5584dc9fa2de
  FDO                  0x0000005c	FDO_PACKAGING_METADATA
    Packaging Metadata: {"type":"deb","os":"ubuntu","name":"linux","version":"6.11.0-9.9","architecture":"arm64"}
  Linux                0x00000004	Unknown note type: (0x00000000)
   description data: 00 0b 06 00
  Linux                0x00000001	OPEN
   description data: 00

0 replies

stv0g · 2024-11-13T22:42:23Z

stv0g
Nov 13, 2024

Okay, I found the breaking change:

https://lore.kernel.org/linux-arm-kernel/[email protected]/T/

torvalds/linux@48f6430

1 reply

kjmph Nov 26, 2024
Author

Thank you for finding that, we at least know why it changed. I don't think the Rosetta use case would register as a large enough use case to reverse that commit, maybe if we pinged Maskray they could weigh in.

oscarbg · 2024-11-19T02:20:26Z

oscarbg
Nov 19, 2024

also found using parallels on Ubuntu 24.10 and Macos 15.2 beta..
has this "kernel bug" already been fixed in 6.12 or in linux-next (6.13?)

4 replies

stv0g Nov 19, 2024

I don't think this will be fixed in Linux. It's probably something Apple needs to fix in Rosetta..

oscarbg Nov 20, 2024

ok thanks for answering..
2 questions then:
a) my first time playing with Rosetta Linux so don't know how fast Apple fixes this kind of Rosetta-Kernel interaction bugs: do you think we can expect this kind of bug to be fixed in a 15.x point release (say 15.3 onwards)? or will have to wait for at least MacOS 16.x at WWDC 25 at earliest to fix..
b)as using under Parallels 20, another question, assuming Apple fixes this bug and using new MacOS with fixed Rosetta, virtualization software needs to catch up and include new updated "rosetta for Linux" or virtualization software in some way "references/symbolic mounts" directly the updated rosetta linux bits so should work with current Parallels software.. example: say I don't want to pay for Parallels 21/having perpetual Parallels 20 license for ex.

kjmph Nov 26, 2024
Author

A) I'm dismayed, I don't think Apple gives much consideration to Rosetta. It works for them, yet there are a number of broken parts of Rosetta (like the AVX, AVX2, and AVX512 instructions). So, other than a campaign by us, I don't know how to raise this to their attention.

B) I can't comment about Parallels, yet UTM, the project this discussion is about, mounts rosetta in the guest VM. So, if Parallels does something similar, one wouldn't need to upgrade any licensing.

oscarbg Nov 27, 2024

nice news!! seems is not enterily due to new kernel.. maybe the kernel flags/options at build time are also in play and ubuntu kernels (and also form mainline ppa) since 6.11 and 6.12 are broken..
so seems this zabbly repo 6.11 version plays well with rosetta
https://ubuntuhandbook.org/index.php/2023/08/install-latest-kernel-new-repository/
maybe once I have time make a diff of build options diff vs ubuntu and search what flag makes it work!!
EDIT: of course would be nice of someone confirms my findings!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rosetta failures in guest Linux environment #6799

{{title}}

Replies: 3 comments 5 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Rosetta failures in guest Linux environment #6799

kjmph Oct 27, 2024

Replies: 3 comments · 5 replies

stv0g Nov 13, 2024

stv0g Nov 13, 2024

kjmph Nov 26, 2024 Author

oscarbg Nov 19, 2024

stv0g Nov 19, 2024

oscarbg Nov 20, 2024

kjmph Nov 26, 2024 Author

oscarbg Nov 27, 2024

kjmph
Oct 27, 2024

Replies: 3 comments 5 replies

stv0g
Nov 13, 2024

stv0g
Nov 13, 2024

kjmph Nov 26, 2024
Author

oscarbg
Nov 19, 2024

kjmph Nov 26, 2024
Author