BUG: occ files:scan -> Exception during scan: NOAUTH Authentication required.

[schklom]

Hi,

Summary:

using redis with a password makes some occ commands bug.

Details

I recently reinstalled nextcloud from scratch (all volumes deleted) after deciding to use userns-remap on Docker.
I deactivate the userns for nextcloud and the whole thing works as well as before, with one exception.

BUG:

$ docker exec -it nextcloud bash
I have no name!@3e615422f49e:/var/www/html$ php occ files:scan schklom
Starting scan for user 1 out of 1 (schklom)
Exception during scan: NOAUTH Authentication required.
#0 /var/www/html/lib/private/Memcache/Redis.php(110): Redis->set('b7277d39df73116...', 0, Array)
#1 /var/www/html/lib/private/Lock/MemcacheLockingProvider.php(84): OC\Memcache\Redis->add('files/0182828f2...', 0)
#2 /var/www/html/lib/private/Files/Storage/Common.php(751): OC\Lock\MemcacheLockingProvider->acquireLock('files/0182828f2...', 2, 'home::schklom::sc...')
#3 /var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php(594): OC\Files\Storage\Common->acquireLock('scanner::', 2, Object(OC\Lock\MemcacheLockingProvider))
#4 /var/www/html/lib/private/Files/Cache/Scanner.php(333): OC\Files\Storage\Wrapper\Wrapper->acquireLock('scanner::', 2, Object(OC\Lock\MemcacheLockingProvider))
#5 /var/www/html/lib/private/Files/Utils/Scanner.php(260): OC\Files\Cache\Scanner->scan('', true, 3)
#6 /var/www/html/apps/files/lib/Command/Scan.php(152): OC\Files\Utils\Scanner->scan('/schklom', true, NULL)
#7 /var/www/html/apps/files/lib/Command/Scan.php(208): OCA\Files\Command\Scan->scanFiles('schklom', '/schklom', Object(Symfony\Component\Console\Output\ConsoleOutput), false, true, false)
#8 /var/www/html/3rdparty/symfony/console/Command/Command.php(255): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/html/core/Command/Base.php(169): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /var/www/html/3rdparty/symfony/console/Application.php(1000): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/html/3rdparty/symfony/console/Application.php(271): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/html/3rdparty/symfony/console/Application.php(147): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/html/lib/private/Console/Application.php(215): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/html/console.php(100): OC\Console\Application->run()
#15 /var/www/html/occ(11): require_once('/var/www/html/c...')
#16 {main}
+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 0       | 0     | 00:00:00     |
+---------+-------+--------------+

docker-compose relevant services:

version: "3.8"

services:
  nextcloud:
    image: nextcloud
    user: ${PUID}:${PGID}
    restart: "no"
    security_opt: &sec
      - no-new-privileges:true
    userns_mode: "host"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${DOCKERDATA}/nextcloud:/var/www/html
      - ${DOCKERCONFIG}/nextcloud/memory-limit.ini:/usr/local/etc/php/conf.d/memory-limit.ini:ro # 512M
      - ${DOCKERCONFIG}/nextcloud/upload-max-filesize.ini:/usr/local/etc/php/conf.d/upload-max-filesize.ini:ro # 16G
      - ${DOCKERCONFIG}/nextcloud/redis-session.ini:/usr/local/etc/php/conf.d/redis-session.ini
    depends_on:
      - nextcloud_db
      - nextcloud_redis
    environment: # look at config.php
    networks:
      - nc-db-redis
  
  nextcloud_db:
    image: postgres:13
    container_name: nextcloud_db
    restart: "no"
    security_opt: *sec
    userns_mode: "host"
    user: ${PUID}:${PGID}
    environment:
      - POSTGRES_DB_FILE=/run/secrets/nextcloud_db_database
      - POSTGRES_USER_FILE=/run/secrets/nextcloud_db_user
      - POSTGRES_PASSWORD_FILE=/run/secrets/nextcloud_db_password
      - TZ=${TZ}
    secrets:
      - nextcloud_db_database
      - nextcloud_db_user
      - nextcloud_db_password
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${DOCKERDATA}/nextcloud_db:/var/lib/postgresql/data
    networks:
      - nc-db-redis
    
  nextcloud_redis:
    image: redis:alpine
    container_name: nextcloud_redis
    user: ${PUID}:${PGID}
    restart: "no"
    security_opt: *sec
    userns_mode: "host"
    # Lets redis user a password secret file
    command: bash -c '[ "$$REDIS_PASS_FILE" ] &&
      ( cat "$$REDIS_PASS_FILE" | xargs -0 redis-server --requirepass ) || redis-server'
    environment:
      REDIS_PASS_FILE: /run/secrets/nextcloud_redis_pass
    secrets:
      - nextcloud_redis_pass
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${DOCKERDATA}/nextcloud_redis:/data
    networks:
      - nc-db-redis

config.php

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => 'nextcloud_redis',
    'password' => 'blabla',
    'port' => 6379,
  ),
  'overwritehost' => 'DOMAIN:PORT',
  'overwriteprotocol' => 'https',
  'trusted_proxies' =>
  array (
    0 => 'traefik',
  ),
  'passwordsalt' => 'blablabla',
  'secret' => 'blablablabla',
  'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => 'DOMAIN:PORT',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'pgsql',
  'version' => '20.0.6.1',
  'overwrite.cli.url' => 'https://localhost',
  'dbname' => 'nextcloud',
  'dbhost' => 'nextcloud_db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'oc_schklom',
  'dbpassword' => 'blablabla',
  'installed' => true,
  'instanceid' => 'blablabla',
  'logfile' => '/mylog/nextcloud.log',
  'logtimezone' => 'Europe/Oslo',
);

Temporary fix

Disable the password in redis and adjust config.php.

Notes

The redis container works fine with a password for everything, except some occ commands

It seems like a nextcloud problem to me, but maybe I'm wrong.

Does anyone else have that problem and/or know how to fix it ?

Many thanks to all who try to help :)

[xsolinsx]

I have the same issue, cannot execute commands like occ db:add-missing-indices and occ db:convert-filecache-bigint due to this
thanks for the temporary workaround though

[veerendra2]

Any updates on this issue? I have similar issue while running occ in docker container

www-data@nextcloud:~/html$ php occ -h
An unhandled exception has been thrown:
RedisException: NOAUTH Authentication required. in /var/www/html/lib/private/Memcache/Redis.php:56
Stack trace:
#0 /var/www/html/lib/private/Memcache/Redis.php(56): Redis->get('e0b1cdb2d6d4afb...')
#1 /var/www/html/lib/private/IntegrityCheck/Checker.php(441): OC\Memcache\Redis->get('oc.integrityche...')
...

[schklom]

@veerendra2 For now at least, disable the Redis password, that should fix it.

[xsolinsx]

any news on this?
it makes it impossible to check for real errors inside logs given that...


and this is a low number compared to others I've seen previously

disabling redis password for nextcloud cannot be a long-term solution, this goes on since 2021

[joshtrichards]

Is this merely this? #1608 (comment)

Keep in mind the image uses multiple config files, so merely looking at config.php isn't your true active config. You must use occ config:list system to get the merged configs.

[schklom]

@joshtrichards I did not use the config file to set the redis password, I used

    environment:
      REDIS_HOST_PASSWORD_FILE: /run/secrets/nextcloud_redis_pass
    secrets:
      - nextcloud_redis_pass

My understanding is this should be enough to get NC to use a Redis password. I can try again though.

[joshtrichards]

This does indeed appear to be a bug. The REDIS_HOST_PASSWORD_FILE is used when setting up the PHP session handling:

docker/docker-entrypoint.sh

Lines 108 to 133 in 13f51c4

	if [ -n "${REDIS_HOST+x}" ]; then
	echo "Configuring Redis as session handler"
	{
	file_env REDIS_HOST_PASSWORD
	echo 'session.save_handler = redis'
	# check if redis host is an unix socket path
	if [ "$(echo "$REDIS_HOST" | cut -c1-1)" = "/" ]; then
	if [ -n "${REDIS_HOST_PASSWORD+x}" ]; then
	echo "session.save_path = \"unix://${REDIS_HOST}?auth=${REDIS_HOST_PASSWORD}\""
	else
	echo "session.save_path = \"unix://${REDIS_HOST}\""
	fi
	# check if redis password has been set
	elif [ -n "${REDIS_HOST_PASSWORD+x}" ]; then
	echo "session.save_path = \"tcp://${REDIS_HOST}:${REDIS_HOST_PORT:=6379}?auth=${REDIS_HOST_PASSWORD}\""
	else
	echo "session.save_path = \"tcp://${REDIS_HOST}:${REDIS_HOST_PORT:=6379}\""
	fi
	echo "redis.session.locking_enabled = 1"
	echo "redis.session.lock_retries = -1"
	# redis.session.lock_wait_time is specified in microseconds.
	# Wait 10ms before retrying the lock rather than the default 2ms.
	echo "redis.session.lock_wait_time = 10000"
	} > /usr/local/etc/php/conf.d/redis-session.ini
	fi

But it's not used in the .config/redis.conf.php:

docker/.config/redis.config.php

Line 8 in 13f51c4

'password' => (string) getenv('REDIS_HOST_PASSWORD'),

There is a new PR from @VesperiaArt that will likely fix this. Anyone on this thread may wish to manually test #2232 by replacing their config/redis.conf.php with the contents of the PR.

Note, upon merge, you'll still have to upgrade your config/ folder contents (in existing installations not new ones) since the image doesn't overwrite existing config/ contents for safety (hopefully we'll soon get a warning at container start-up if your config is out-of-date with the latest image contents - via #2120).

[calebjones]

I'm using the docker container with the REDIS_HOST_PASSWORD_FILE environment variable and I ran into this issue. I manually patched redis.config.php per the commit above and now occ doesn't have that issue.

[joshtrichards]

@calebjones Thanks for providing feedback on testing in your environment. It's appreciated. :)