Islet-cli segfaults, when launched from /shared folder with rmm=islet

[zos]

When islet-cli, or even independently built rsictl, is launched from /shared folder, both clients either segfault (islet-cli) or return error (rsictl) when parsing token. This concerns both attestation retrieval (using 'attest' command) and verification of token from file ('verify -i ').

This only happens when running fvp with --rmm=islet. In case of tf-rmm both commands work properly.

This is the output with errors (using Islet rmm):

# cd /shared
# insmod rsi.ko
[  675.032957] rsi: loading out-of-tree module taints kernel.
[  675.048576] rsi: Initializing
[  675.055726] rsi: Chardev registered with major 246
[  675.067772] rsi: Device created on /dev/rsi
[  675.077194] rsi: Is realm: true
[  675.085222] rsi: RSI version: 1.0
[  675.093383] rsi: Config ret: 0, Bits: 21
# ./islet attest
Getting an attestation report on aarch64.
[  682.158831] rsi: device rsi open
[  682.166550] rsi: ioctl: attestation_token
[  682.324770] rsi: device rsi released
Segmentation fault
# ./rsictl attest
[  693.007918] rsi: device rsi open
[  693.015676] rsi: ioctl: attestation_token
[  693.175656] rsi: device rsi released
Error: VerificationFailed("challenge verification failed")
# ./rsictl verify -i binary.bin
Error: VerificationFailed("challenge verification failed")
# ./islet verify -i binary.bin
Segmentation fault

When client is copied from /shared folder to root '/' folder (it can be done in runtime), then both commands work fine.

This seems to be a problem with 9p/virtio and memory management on the Realm side.

[p-sawicki2]

@bitboom @bokdeuk-jeong @jinbpark Could you confirm that the issue also occurs on your side?

[bitboom]

@bitboom @bokdeuk-jeong @jinbpark Could you confirm that the issue also occurs on your side?

Sorry for the late response. Let me check this issue.

[bitboom]

I was able to reproduce the issue in my environment.
The problem arises from the islet_rmm combined with the 9p filesystem.
And it appears only when verifies the attestation report.

$ /shared/rsictl attest --output=/shared/report # Success
$ /root/rsictl verify --input=/shared/report  # Success
$ /shared/rsictl verify --input=/shared/report  # Fail

I think it would be beneficial to delve deeper into the matter by examining the memory management aspect of the region of the 9p system and MMIO.

[darshanp4]

hello,

I am facing different issue when using islet rmm, while doing attest using ./bin/rsictl attest -o token.bin

facing below error :

rsi: device rsi open
rsi: ioctl: unknown ioctl cmd
rsi: device rsi released
Error: EINVAL

dmesg :
[ 0.000000] RME: Using RSI version 1.0
[ 0.345241] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 1.911383] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[ 12.149515] VFIO - User Level meta-driver version: 0.3
[ 12.516895] registered taskstats version 1
[ 747.676207] rsi: loading out-of-tree module taints kernel.
[ 747.691658] rsi: Initializing
[ 747.699933] rsi: Chardev registered with major 245
[ 747.712592] rsi: Device created on /dev/rsi
[ 747.723240] rsi: Is realm: true
[ 747.731454] rsi: RSI version, ret: RSI_SUCCESS, lower: 1.0, higher: 1.0
[ 747.747861] rsi: RSI config, ret: RSI_SUCCESS, ipa_width_in_bits: 33
[ 884.258661] rsi: device rsi open
[ 884.267063] rsi: ioctl: unknown ioctl cmd
[ 884.281800] rsi: device rsi released

[bokdeuk-jeong]

@p-sawicki2 Could you assign this recurring issue to one of your task members?

[p-sawicki2]

@darshanp4 I'm not sure if your issue is the same as the described here. To make sure that it is not related to the 9p filesystem/rmm could you copy rsictl tool into the /root folder, run it from there and share the results?

Also before running the tool, add printing of cmd code at

islet/realm/linux-rsi/rsi.c

Line 415 in ed25d9d

printk(RSI_ALERT "ioctl: unknown ioctl cmd\n");

[darshanp4]

@p-sawicki2 yeah it doesn't seem to be issue related to 9p filesystem.
it is coming to default case only when i try to attest.

# ./rsictl attest -o token.bin
[  389.320278] rsi: device rsi open
[  389.328870] rsi: rsi default
[  389.337235] rsi: ioctl: unknown ioctl cmd
[  389.356058] rsi: device rsi released
Error: EINVAL
# cd /root/
# ./rsictl attest -o token.bin
[  436.078270] rsi: device rsi open
[  436.086652] rsi: rsi default
[  436.094867] rsi: ioctl: unknown ioctl cmd
[  436.105148] rsi: device rsi released
Error: EINVAL

[p-sawicki2]

@darshanp4 OK, so we know that your issue is not related to that issue.

I suppose that the user space library is out of sync with the rsi.ko driver. Could you, as I wrote earlier, add printing of the numerical value of the cmd argument?

printk(RSI_ALERT` "ioctl: unknown ioctl cmd: %u\n", cmd);

[darshanp4]

rsi: ioctl: unknown ioctl cmd: 3494148290

[p-sawicki2]

@darshanp4 Thanks. Could you create a separate issue, then I'll assign someone from my team to check that?

[darshanp4]

@p-sawicki2 thank you for your quick response, please check #350 and help to assign to someone.