in the previous sections, we learned how to run default programs in a shell, create our own, and manipulate and redirect their input and output.
In this section, we will discuss how to best manage our processes to make the most of today's powerful machines, safely.
Often programs can take long time to complete, e.g.: copying large amount of files with cp
, or making complex figures or calculations. In these cases, we want to free our terminal window so that we can work on something else in the meanwhile.
This can be done by sending the processes to background.
If we run any program in the terminal window, it will stop us from typing more lines until it finishes---because it's running in the foreground.
E.g. if we write a program to wait 5 s and then print a message (technically 2 sequential programs), we won't be able to use the terminal until we see the print message:
my_program.sh:
#!/usr/bin/env bash
sleep 5
echo "the wait is over"
Execution:
$my_teminal$: my_program.sh
the wait is over
This is the "standard" way to run a program.
In order to be able to type something else while a program runs, we need to send program to the background by adding a final "&":
$my_teminal$: my_program.sh &
[1] 12345
$my_teminal$: echo "banana"
banana
$my_teminal$:
$my_teminal$:
$my_teminal$: the wait is over
[1]+ Done my_program.sh
In the above example, we ran the program in the background, entered another command (echo "banana"
) while the first one was running, and finally got its output ("the wait is over").
Importantly, in the example above, the shell gave us two extra lines:
[1] 12345
-> info to identify our program in the background.[1]+ Done my_program.sh
-> when the program finishes, it notifies us.
Both fore- and background programs are tied to our terminal window or shell, which means that they will stop if we accidentally close it.
For long, non-interactive programs, we may want to make sure that they cannot be accidentally closed.
There are many ways to do it, but we will focus on detaching a process from the shell using nohup
.
The program nohup
("NoHangUP") detaches the program from the current terminal, making independent from it. A program ran with nohup
will not stop even if we close all terminal windows. If we use nohup
in conjunction with the background character &
in the example from above:
$my_teminal$: nohup my_program.sh &
[1] 12345
appending output to nohup.out
$my_teminal$:
$my_teminal$:
[1]+ Done nohup my_program.sh
As you can see the results are similar to putting my_program.sh
in the background, but we don't see the output! This is because nohup automatically redirects the standard output to a file called nohup.out
:
$my_teminal$: cat nohup.out
the wait is over
This is necessary because the program is not attached to our terminal anymore, so it cannot output to it.
When we use nohup
, we can close all terminal windows and our program will run until the end or until the computer shuts down.
For both background and detached processes, the user looses direct control over them, so that they cannot be stoppped with Ctrl+C
for instance.
Processes in UNIX are uniquely identified by a unique number consisting of multiple digits: the Process ID or PID.
In the examples above, for each process we sent to the background the terminal gave us the PID of our program, "12345".
In order to list all background processes (a.k.a. "jobs"), run jobs -l
, which will give us the job number, PID, status and name of the program we ran:
$my_teminal$: jobs -l
[1]+ 12345 Running my_program
If we do it while our example is running, we will get:
$my_teminal$: my_program.sh &
[1] 12345
$my_teminal$: jobs -l
[1]+ 12345 Running my_program.sh
$my_teminal$:
$my_teminal$:
$my_teminal$:
$my_teminal$: the wait is over
[1]+ Done my_program.sh
Jobs however only reports the programs that were started in the current terminal. If we started a nohup
program in another terminal, it will not be listed.
The command ps
or "Process Status", will return exactly that. With the aux
option to inlcude processes for all users and the ones with no controlling terminal, we can repeat our example:
$my_teminal$: nohup my_program.sh &
[1] 12345
appending output to nohup.out
$my_teminal$: ps aux | grep my_prog
user 12345 0.0 0.0 34150324 1408 s002 S 3:52pm 0:00.02 bash my_program.sh
user 12346 0.0 0.0 34122828 824 s002 R+ 3:52pm 0:00.00 grep my_prog
$my_teminal$:
$my_teminal$:
$my_teminal$:
[1]+ Done nohup my_program.sh
ps aux
will list the PID as the second item of the string.
The program htop
allows a comprehensive view of all processes currently ongoing on a machine,
Once we identified the PID of the program we are interested in, we are able to terminate it.
To terminate a program with a certain PID PIDofprogram
even if it's hung up, run:
$my_teminal$: kill -9 PIDofprogram
The "-9" option will kill also a stuck program.
For processes with a known name, we can also use killall
, which will kill all subprocesses as well. Useful to tame problematic apps, e.g.:
$my_teminal$: killall Firefox