Environemt Variables are not accessible in Maintenance Tasks

[DanielSchwiperich]

We are using environment variables in the parameters.yml. But the manager is not using / accessing them in for example maintenance tasks.

This is what the paramters.yml looks like (it's a docker dev setup, so no rant about the env name pls 😄 )

parameters:
    database_host: db
    database_port: 3306
    database_user: root
    database_password: '%env(DEFAULT_PASSWORD)%'
    database_name: '%env(PROJECT_NAME)%'
    secret: ...

When running the rebuilding contao cache weg get

[Symfony\Component\DependencyInjection\Exception\EnvNotFoundException]  
  Environment variable not found: "DEFAULT_PASSWORD".                     
                                                                          

--------------------------------------------------------
Exception occured: The command "/usr/local/bin/php '-q' '/var/www/share/project/vendor/bin/contao-console' 'cache:clear' '--env=prod' '--no-warmup'" failed.
...

[fritzmg]

Have you confirmed that the environment variables are actually available via the PHP CLI in general?

[aschempp]

I can confirm this issue, the manager currently unsets all environment variables. Long story short, probably something I need to fix :(

[aschempp]

Note to myself: PATH_TRANSLATED could be what CGI reacts on.
https://secure.php.net/manual/de/features.commandline.php#33119

[ngdot]

same problem here. @DanielSchwiperich have you got a workaround?

[Helrunar]

Are there an workaround or any news about that problem? I will need Environment variables in .env in fact of deploying on different stages

[JanMalte]

I just like to add another error backtrace related to this issue.
I'm using the .env files in combination with docker and docker-compose.

root@ab0f81490f05:/var/www/html# ./vendor/bin/contao-console contao:migrate
08:20:51 ERROR     [console] Error thrown while running command "contao:migrate". Message: "An exception occurred in driver: SQLSTATE[HY000] [1045] Access denied for user '%env(MYSQL_USER)%'@'172.26.0.3' (using password: YES)" ["exception" => Doctrine\DBAL\Exception\ConnectionException^ { …},"command" => "contao:migrate","message" => "An exception occurred in driver: SQLSTATE[HY000] [1045] Access denied for user '%env(MYSQL_USER)%'@'172.26.0.3' (using password: YES)"]

In AbstractMySQLDriver.php line 112:
  An exception occurred in driver: SQLSTATE[HY000] [1045] Access denied for user '%env(MYSQL_USER)%'@'172.26.0.3' (using password: YES)  

In Exception.php line 18:
  SQLSTATE[HY000] [1045] Access denied for user '%env(MYSQL_USER)%'@'172.26.0.3' (using password: YES)  

In PDOConnection.php line 38:
  SQLSTATE[HY000] [1045] Access denied for user '%env(MYSQL_USER)%'@'172.26.0.3' (using password: YES)  

contao:migrate [--with-deletes] [--schema-only] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-e|--env ENV] [--] <command>

[aschempp]

@JanMalte this looks like Symfony is not correctly decoding your environment variables, not something about the Contao Manager?

[JanMalte]

The usage of %env(MYSQL_DATABASE)% for database_name (and all the other database parameters) is not supported with contao, as contao escapes those values before creating the internal used DATABASE_URL.

Symfony does not escape the env values and it therefor worked in previous contao versions which didn't escaped the values.

Source:
https://github.com/contao/contao/blob/4.13/manager-bundle/src/ContaoManager/Plugin.php#L587

Documentation:
https://docs.contao.org/dev/reference/config/#database-url
https://docs.contao.org/dev/getting-started/starting-development/#application-configuration
https://symfony.com/doc/4.4/configuration.html#env-file-syntax

Discussions:
contao/contao#4546
https://community.contao.org/de/showthread.php?80370-env-statt-config-parameters-yml

[fritzmg]

DSN paths need to be URL encoded.

This issue is about environment variables allegedly not being resolved.

[fritzmg]

@JanMalte in any case environment variables in the parameters.yaml are currently not supported. It does not make sense to use environment variables there anyway, as the config/parameters.yaml is already environment specifc.

Instead you should use the environment variables directly in your config/config.yaml for example, as already mentioned in contao/contao#4546

Also, this has nothing to do with the Contao Manager.

[JanMalte]

Using .env files or ENV variables to provide credentials ist widly common and easy to adapt in any CI/CD, as the syntax is as easy as possible.

Adding a complete yaml file to your CI/CD or substituting placeholders during CI/CD is much more complicated and produces quite some overhead.

Especially when using docker it is common to pass the variables via .env to avoid packaging the credentials in the container image or having to include multiple files via docker compose.

I'm now using .env files for credentials during the CI/CD and compose the supported DATABASE_ in the docker-compose.yml

services:
  web:
    image: contao-project:apache-bundle
    volumes:
      - ./contao-parameters.yml:/var/www/html/config/parameters.yml
      - contao_uploads:/var/www/html/files/uploads/
    depends_on:
      - db
    environment:
      - MYSQL_DATABASE
      - MYSQL_USER
      - MYSQL_PASSWORD
      - DATABASE_URL=mysql://${MYSQL_USER}:${MYSQL_PASSWORD}@db/${MYSQL_DATABASE}
      - SECRET_KEY
      - MAIL_HOST
      - MAIL_USER
      - MAIL_PASS
      - MAILER_DSN=smtp://${MAIL_USER}:${MAIL_PASS}@${MAIL_HOST}

However, the TRUSTED_PROXIES is not working/supported as $_ENV for the container so I still need a parameters.yml which contains the IP range of the proxy.

framework:
  trusted_proxies: "172.0.1.0/24"

[fritzmg]

You misunderstood. The issue is that you are tyring to use environment variables in your parameters.yml.

[JanMalte]

Yes, just like explained in the symfony docs: https://symfony.com/doc/current/configuration.html#configuration-based-on-environment-variables

Anyway, we don't have to discuss this any further. I just wanted to put the information here in the issue, because I always find it via Google when I stumble across exactly this circumstance in a project.

[fritzmg]

Yes, just like explained in the symfony docs: https://symfony.com/doc/current/configuration.html#configuration-based-on-environment-variables

The Symfony docs use environment variables directly, they don't use the parameters.yml.