Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Auto)fallback to ANSI-less output in CI #147

Open
kjeremy opened this issue Aug 5, 2024 · 6 comments
Open

(Auto)fallback to ANSI-less output in CI #147

kjeremy opened this issue Aug 5, 2024 · 6 comments

Comments

@kjeremy
Copy link

kjeremy commented Aug 5, 2024

I'm trying to integrate this into my CI pipeline on gitlab and am getting output like the following:

$ nix --version
nix (Nix) 2.18.5
$ nix store ping --store $OUR_NIX_STORE
Store URL: unix:///mnt/nix/var/nix/daemon-socket/socket
Version: 2.18.4
Trusted: 1
$ nix profile list
warning: the group 'nixbld' specified in 'build-users-group' does not exist
$ nom build -L $NIX_REMOTE_OPTS "$MRAK_TEST_PKG"
warning: the group 'nixbld' specified in 'build-users-group' does not exist
⏱ 0s⏱ 1s⏱ 2s⏱ 3s⏱ 4s⏱ 4swarning: ignoring untrusted flake configuration setting 'extra-substituters'.
Pass '--accept-flake-config' to trust it
⏱ 4s⏱ 4s⏱ 4s⏱ 4s⏱ 4s⏱ 4s⏱ 4s⏱ 4s⏱ 5scopying path '/nix/store/16ik5rfzbjbl2kxpagqx5rp03hkd32qp-source' from 'https://cache.nixos.org'/
┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 5s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 6s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 7s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 8s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 9s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 10s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 11s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 12s┏━━━ Downloads
┗━ ∑ ↓ 1 │ ↓ 0 │ ⏸ 0 │ ⏱ 13s┏━━━ Downloads │ Host
┃ │ │ │ localhost
┃ │ ↓ 1 │ │ https://cache.nixos.org/
┗━ ∑ ↓ 0 │ ↓ 1 │ ⏸ 0 │ ⏱ 13s ┏━━━ Downloads │ Host
┃ │ │ │ localhost
┃ │ ↓ 1 │ │ https://cache.nixos.org/
┗━ ∑ ↓ 0 │ ↓ 1 │ ⏸ 0 │ ⏱ 13s ┏━━━ Downloads │ Host

I tried setting TERM as in https://docs.gitlab.com/ee/ci/yaml/script.html#job-log-output-is-not-formatted-as-expected-or-contains-unexpected-characters but it doesn't seem to make a difference.

@maralorn
Copy link
Owner

maralorn commented Aug 5, 2024

Yeah, that is to be expected.

The purpose of nom is to print a live status report below the log. I am not aware that this is possible in something like gitlab ci. (because there are afaict no control codes available to remove previous lines.)

Am I wrong in thinking this is not possible or do you have some other ideas how nom should behave?

@maralorn
Copy link
Owner

maralorn commented Nov 1, 2024

If anyone has a good suggestion what to do here, please share. Until then I am closing this as not planned.

@maralorn maralorn closed this as not planned Won't fix, can't repro, duplicate, stale Nov 1, 2024
@layus
Copy link

layus commented Jan 25, 2025

Dumping notes here just in case:

Using ncurses it seems there is a way to write to the terminal (using newterm) without scrambling stdout. Maybe that would work well enough in cases where ther is no terminal per se.

See https://stackoverflow.com/questions/8371877/ncurses-and-linux-pipeline and https://stackoverflow.com/questions/53696818/how-to-i-make-python-curses-application-pipeline-friendly.

A bit of work to move to a proper ncurses since nom is ANSII escapes based.

@maralorn
Copy link
Owner

I still don't understand why we would do that. I see no point in using nom in a CI pipeline.

@layus
Copy link

layus commented Jan 26, 2025 via email

@maralorn
Copy link
Owner

Aaah, that makes sense.

@maralorn maralorn reopened this Jan 26, 2025
@maralorn maralorn changed the title Output is bad in gitlab-ci (Auto)fallback to ANSI-less output in CI Jan 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants