Podman fails to pull large images on macos

[ianks]

Issue Description

When attempting to pull a large image, podman-remote gets stuck and seems to leak memory due to gvproxy crashing.

podman-remote process sample

Steps to reproduce the issue

$ podman machine init --now --cpus 4 --memory 8192 --rootful
$ podman pull --platform linux/amd64 rbsys/x86_64-linux:latest # platform doesnt matter, just needs to be sufficiently large

Describe the results you received

podman pull does not work

Describe the results you expected

podman pull should work

podman info output

host:
  arch: arm64
  buildahVersion: 1.35.3
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc39.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 92.67
    systemPercent: 4.16
    userPercent: 3.17
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "39"
  eventLogger: journald
  freeLocks: 2043
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.7.9-200.fc39.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 7857213440
  memTotal: 8301899776
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.fc39.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-1.fc39.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.fc39.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240326.g4988e2b-1.fc39.aarch64
    version: |
      pasta 0^20240326.g4988e2b-1.fc39.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc39.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 0m 15.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 0
    stopped: 2
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 4872851456
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.0.1
  Built: 1711929600
  BuiltTime: Sun Mar 31 20:00:00 2024
  GitCommit: ""
  GoVersion: go1.21.8
  Os: linux
  OsArch: linux/arm64
  Version: 5.0.1

### Podman in a container

No

### Privileged Or Rootless

Privileged

### Upstream Latest Release

Yes

### Additional environment details

Additional environment details

### Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[kovyrin]

Seeing the same issue on an M3 mac when pulling a pretty large image from a private repository. Small image pulls work as expected.

[kovyrin]

Just ran something like this in a loop: podman machine stop; podman machine start; podman pull <my very large image until it finally succeeded. So the problem seems to be intermittent...

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[edeandrea]

I submitted #23114 yesterday which seems slightly related to this, although I do not experience the symptoms in this issue.

When I try

$ podman machine init --now --cpus 4 --memory 8192 --rootful
$ podman pull --platform linux/amd64 rbsys/x86_64-linux:latest # platform doesnt matter, just needs to be sufficiently large

everything works as expected.

[Luap99]

Does it still happen with the latest podman? I fixed one gvproxy crash in containers/gvisor-tap-vsock#370 which should be part of the latest podman 5.2.5 installer so assume this is fixed