General Computing
If you have any plans to work with robotics, or high-performance systems, you are most likely looking at using Linux in some form. Linux is not much different from other operating systems (such as Windows or OSX), but there are a few key differences in its design. There are two main design ideologies: "The Unix Philosophy", and "Everything is a File". Aside from these points, the front end of Linux system can be made to feel very similar to other operating systems.
- Write programs that do one thing and do it well
- Write programs to work together
- Write programs to handle text streams, because that is a universal interface
The preliminary idea of "Everything is a File" is that all items in the system, documents, hard-drives, keyboards, printers, network sockets, etc., are simply streams of data. This approach allows many types of tools to integrate easily with all sorts of devices without any need for specific interfaces. It also allows for many general tools for searching and filtering data that are compatible with nearly anything you could use them for.
Within the Linux community, there is a fairly large split between those who prefer the Graphical User Interface (GUI), and those for the Command Line Interface (CLI). Either way, the environment is still Linux. The preference for a normal user will come down to both the device, and the usage of the environment.
For example, it is recommended for on-board computers, which will be running most of the time with out a screen plugged in, to be run without any sort of GUI environment set to startup by default, as it will save computing resources for needed computations. For a full sized computer, such as one that will be used to develop software on, it is probably a good idea to run a GUI to make life easier for the developer.
By this point, you have probably heard of the Ubuntu distribution. Like all other Linux distributions (to be brief), Ubuntu is simply a collection of supported software that is packaged together in a certain manner. Going from here, there are multiple desktop environments that can be used, each with different levels of ability. In Ubuntu, there are multiple versions of prepackaged desktop environments.
| Name | Ubuntu Distro | Description |
|---|---|---|
| GNOME | Ubuntu | A feature packed desktop for maximum user friendliness |
| KDE | Kubuntu | A sleek desktop with a mix of usability and speed |
| XFCE | Xubuntu | A light-weight full-featured desktop |
| LXDE | Lubuntu | The most light-weight full-featured desktop |
For the first entrance into Linux, the desktop environment XFCE is recommended for a few reasons: it is very familiar for most users, it runs well on most devices, and it is highly configurable. For SoC type devices, if a GUI is desired, LXDE is recommended as it is the least processor-intensive out of all the options.
Do not be afraid to pick one without too much background knowledge, as it is possible to install and test multiple at once, to get familiar with the options without any pressure.
Regardless of the choice to use a GUI or CLI, you will no doubt use a a terminal at some point throughout your experience with Linux, and as such, you should ensure you're comfortable with the command line. If you are using a GUI, you should try to get the hang of navigating around and doing basic tasks, such as doing some basic file editing or installing software, in a Virtual Terminal so that you are able to at least familiar enough to set up a GUI from the CLI or connect to a remote device over SSH.
Some basic CLI tools to remember (you can use the "-h" and "--help" options to find out more information):
- ls - List directory
- cd - Change directory
- cp - Copy file
- mv - Move (or rename) file
- cat - Concatenates (prints) a files contents
- nano - A basic text editor (use "CTRL+X" to exit)
Some file system notes:
- You can reference the current directory with "." (e.g. "nano ./notes.txt")
- You can reference the parent directory with ".." (e.g. "nano ../parent_notes.txt")
- You can reference the home directory with "~" (e.g. "nano ~/my_notes.txt")
- You can stack directory references (e.g. "cd ../../" to go up two directory levels)
- Hidden files and directories start with a "." (e.g. "nano ~/.bashrc" to access a hidden file in the home directory)
Some more advanced tools:
- tmux - Allows for detached sessions (keeps processes alive if disconnected), terminals in 1 window, multiple windows, etc.
Before we can dive into using Linux, we will need to install it in some manner.
The easiest way to try Linux in a semi-permanent fashion is to use virtualization. The typical method would be to use a hypervisor, such as "VirtualBox", to effectively emulate the entire system. This method will only create a Virtual Machine which exist as files on your hard drive, and are therefore very easy to uninstall when you are done with
As a suggestion, go check out VirtualBox for further information.
There are main considerations that you should check before using a Virtual Machine:
- A VM will make it difficult (but not impossible) to communicate from a remote computer to the guest computer.
- VM support for 3D software (specifically software using OpenGL or DirectX) will most likely not run inside the VM (although there is some exceptions).
- They provide a super-easy backup solution (after you install ROS, etc., you can copy-paste the VM to make a backup. You can also make Snapshots of the VM to roll back to at a later date).
- They provide a risk-free environment (nothing done on the Guest should be able to affect Host, also provides immediate protection from accidentally deleting files on the Host)
- Guest machines should not be used for anything that is time-critical (real time computations can be delayed significantly if the Host is under high load, and the Guest may think it is still running in real-time)
One critical note to allow proper networking to the Virtual Machine, you will have to change the networking mode on to bridge networking (the exact setting in VirtualBox is labeled as "Bridged Adapter").
The "proper" way to install Linux is by actually installing it to the hard drive. This will get around all the disadvantages that the Virtual Machines suffer, but does require a bit more commitment and setup on the software side. If you are all ready running an operating system, such as Windows, your best option is to look into "dual booting", which will allow you to switch back and forth between the two systems.
For further information on a good place to start, go check out the Ubuntu Dual Boot guide for further information.
As a start point, or as a reliable backup, it may be desirable to configure a login prompt over a hardware serial line. In modern systems, first you must identify the serial port (in this example, we want to start the serial on "/dev/ttyS2"), then it should be as simple as running the following commands:
systemctl enable [email protected]
systemctl start [email protected]Or (as an example, to use the header Tx/Rx on a Raspberry Pi):
systemctl enable [email protected]
systemctl start [email protected]Networking in Linux is a fairly open experience, as the system will allow nearly any combination of configurations if it is set up correctly. Unfortunately, such a setup takes time, and may not be the most appropriate for a system that can afford to be much simpler.
The best way to list the available networking devices is with the following command:
ifconfig
For most systems, there are two primary options for getting a basic connection up and running, either configuring a connection manually, or using a network manager to handle all of the hard work in the background. If you are using a system that has a pre-packaged OS (like Ubuntu) you probably want to skip to the Automated/Managed section.
For most devices, the best place to start is by getting the ethernet port configured (if it isn't already).
Note: Going forward, if you are on a fresh system, you may find that there one device as already in the following files, the "loopback" interface. THIS MUST NOT BE DISABLED UNDER ANY CIRCUMSTANCE, as your terminal and many other system functions. It will look something like this for reference:
auto lo
iface lo inet loopback
For most backwards compatibility, edit the file "/etc/interfaces" and ensure the following lines are there.
For DHCP (best when using some kind of router):
auto eth0
iface eth0 inet dhcp
For a static address (best if you need to do a direct connection):
iface eth0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.254
The system will need to be rebooted for the settings to be applied.
For newer systems, the folder "/etc/interfaces.d/" can be used to configure multiple setups and devices. To use this option, simply create a file "/etc/interfaces.d/eth0.cfg" and add the same lines as above to it. This won't be covered here for brevity.
Before continuing, note that a user may need to be in the "netdev" group to control these network managers:
sudo usermod -aG netdev $USERRemember to log out and back in to make sure the group permissions are updated.
If you are using a user-friendly OS (like Ubuntu), there's a good change that NetworkManager is already set up and configured, so you should probably start there.
Note: With managed network solutions, there is a very important point to make: only one solution can be used at a time. If multiple solutions are used, there will be issues with your network connectivity. This means that once you install a network manager, you must disable the lines for devices configured in the "Quick Setup" process.
As far as network managers go, there are 3 main contenders (as of time of writing):
-
NetworkManager - A very configurable, expandable, and scalable network manager
- Start with
sudo systemctl start NetworkManager.service - Can be controlled through the command line with
nmcli - Can be controlled through a GUI with
nm-applet - Can be configured to run multiple connections at once
- Start with
-
wicd - A simple, reliable, and scripting-friendly network manager
- Start with
sudo systemctl start wicd.service - Can be controlled through the command line with
wicd-cli - Can be controlled through the command line as an interface with
wicd-curses - Can be controlled through a GUI with
wicd-gtk - Can only have one connection active at any one time
- Start with
-
connman - A light-weight networking solution aimed at ARM-based devices
- Start with
sudo systemctl start connman.service - Can be controlled through the command line with
connmanctl - Can be controlled through a GUI with
cmst - Can only have one connection active at any one time
- Start with
Wireless networking can be configured in multiple different ways, depending on the application of your system.
The easiest way to configure and use one or more wireless devices is to have some form of wireless-capable router. This will allow you to to many different configurations, and will usually handle DHCP, so you do not need to worry about static IPs
Allows the creation a dynamic networks using only the wireless chips on the devices that are connecting to the network. This is very useful when trying to create a network where multiple devices need to talk to each other in the field, or where using a router will be inconvenient.
A newer configuration where a single device will act as a host/router, allowing other devices to connect to it and will assign IP addresses to each device. This is most useful for devices that need to be connected to for initial configuration, but do not need a permanent connection, or as an initial interface before a permanent configuration is set up.
Due to the way the 802.11 (WiFi) standard is defined, there is a random and variable delay for a device to wait while the channel is busy. Combined with general noise and interference, this can cause WiFi to be very unreliable, in respect to trying to maintain a low-latency transmission. Out of previous experience, WiFi cannot realistically be relied on for anything that requires a latency <50ms.
Other issues to consider is how to handle disconnections, or devices not connecting in the first place.
Power Save Polling
One very common issue with wireless networking is power saving settings on WiFi devices. Most commonly, this is in the form of a setting called Power Save Polling (PSP or PS-Poll). It is highly recommended that you disable this feature for any type of latency-critical connections).
The most common symptom of PSP causing latency issues is one-way latency to wireless devices. This can be tested for on a network with a wireless device, and some access to a wired device that supports some form of ping command. If the pings from the wireless device to the wired device look something like the following:
Pinging 192.168.0.1 with 32 bytes of data:
Reply from 192.168.0.1: time=3ms
Reply from 192.168.0.1: time=3ms
Reply from 192.168.0.1: time=4ms
Reply from 192.168.0.1: time=6ms
Reply from 192.168.0.1: time=4ms
And the pings from the wired device to the wireless device look something like the following:
Pinging 192.168.0.2 with 32 bytes of data:
Reply from 192.168.0.2: time=244ms
Reply from 192.168.0.2: time=59ms
Reply from 192.168.0.2: time=79ms
Reply from 192.168.0.2: time=4ms
Reply from 192.168.0.2: time=100ms
Then your devices are most likely are suffering from PSP.
For Windows devices:
Control Panel > Power Options > Change Plan Settings (for current/all plans)
Wireless Adapter Settings > Power Saving Mode > Setting
Set this to "Maximum Performance"
For Mac devices:
No fix currently known
For Linux devices:
sudo iw dev wlan0 set power_save offOr:
sudo iwconfig wlan0 power offIf you are running Linux using virtualization, you may encounter that there are difficulties when trying to establish a 2-way connection to/from the guest machine (a primary example on this is when setting up Distributed ROS in such an environment). By default, VirtualBox is configured to operate using a NAT network adapter, which will cause your IP addresses so look something like the following:
Host IP: 192.168.1.8
Guest IP: 10.0.0.5
To allow the guest machine to access the network adapter without the constraints introduced by the NAT network adapter, you must switch it to Bridged Mode:
- Shutdown the Guest Machine.
- Edit the settings Guest Image Settings, and go to "Network".
- Switch the "Adapter 1" attachment to "Bridged".
- Save the changes and start up the Guest Machine.
- Check that the assigned IP address's between the host and guests are the same.
Host IP: 192.168.1.8
Guest IP: 192.168.1.8
For most systems that are running without some form of easy access (e.g. an on-board computer), it is often desirable to have some form of method to access them remotely. The most versatile method is by using Secure Shell (SSH). SSH is a tool that allows us to log in as any user through a remote IP address.
With most distributed OS images, SSH will be enabled by default, all that is needed to log in is a username/password and the remote system's IP address. Most Linux systems will come with the SSH client pre-installed, and if the local system is running something like Windows, a program like "Putty" will need to be used. On Linux, simply run the command:
ssh user@remote-ipFor more powerful systems, it is also possible to achieve a Virtual Remote Connection (VNC), which will allow access to a desktop GUI on the remote system. As a good starting point, look into "TigerVNC" for more information.
Working on a distributed network can bring some unforeseen consequences to how you have to plan out data processing and handling. The main 2 issues that will affect a network are latency (the time it takes for data to be sent) and time syncing (having 2 independent computers tracking "current time" separately).
To get around this, most complex systems will implement 2 other systems:
- A dedicated local network for the computers to converse over without worrying about other computers clogging up available bandwidth (usually through the use of an router)
- A method for synchronizing system time across multiple devices
To combat the issues of time synchronization, there are many methods that can be used, however, for the sake of simplicity, we will use a client-server model. Fortunately for us, most large computer-based organizations (should) run a time server. Although this is not on the local subnet (or router), we (hopefully) have a gigabit connection to the timeserver through the network, so for the purposes of the occasional sync (as we don't really have to worry about long-term bandwidth congestion), this will do quite nicely for our purposes.
This method uses the systemd-timesyncd method for time syncronisation with a server, and is appropriate for systems where only approximate synchronization is required (>1ms accuracy). First, ensure you have access to the timedatectl, you should get a response that looks like:
Local time: Fri 2019-10-25 18:47:19 AEST
Universal time: Fri 2019-10-25 08:47:19 UTC
RTC time: Fri 2019-10-25 08:47:19
Time zone: Australia/Brisbane (AEST, +1000)
System clock synchronized: no
NTP service: inactive
RTC in local TZ: no
Next, open the file /etc/systemd/timesyncd.conf and perform the following configuration to set your time server (QUT's timeserver for example):
[Time]
NTP=time.qut.edu.au
FallbackNTP=0.oceania.pool.ntp.org 1.oceania.pool.ntp.org 2.oceania.pool.ntp.org 3.oceania.pool.ntp.org
Where you could use any time server for the fallback NTP connection.
Next, enable and (re)start the time syncronisation daemon:
sudo systemctl enable systemd-timesyncd.service
sudo systemctl restart systemd-timesyncd.service
Finally, verify your configuration, use timedatectl show-timesync --all:
LinkNTPServers=
SystemNTPServers=time.qut.edu.au
FallbackNTPServers=0.oceania.pool.ntp.org 1.oceania.pool.ntp.org 2.oceania.pool.ntp.org 3.oceania.pool.ntp.org
ServerName=time.qut.edu.au
ServerAddress=(null)
RootDistanceMaxUSec=5s
PollIntervalMinUSec=32s
PollIntervalMaxUSec=34min 8s
PollIntervalUSec=0
Frequency=0
And check if NTP is active with timedatectl (it may take a few seconds to activate):
Local time: Fri 2019-10-25 18:59:56 AEST
Universal time: Fri 2019-10-25 08:59:56 UTC
RTC time: Fri 2019-10-25 08:59:56
Time zone: Australia/Brisbane (AEST, +1000)
System clock synchronized: no
NTP service: active
RTC in local TZ: no
If is lists inactive, the syncronisation simply hasn't completed yet.
First off, we need to install and start the package called "chrony" on all computers that will be using network. Next, we just have to tell chrony where to look to find the best timeserver. Open the file "/etc/chrony/chrony.conf" (on Ubuntu-type systems) and scroll down to the first line that is uncommented, it should look something like:
server 0.pool.ubuntu...
OR:
pool 0.pool.ubuntu...
In either case, comment this line out (add a '#' to the front of the line), and just above it we will enter the location of our time server (QUT's timeserver for example):
server time.qut.edu.au iburst
This will configure chrony to only look for this server to sync with. If your network blocks external time servers (as QUT does), this means that your PC will only be able to sync with this time server when you're on the network. If you would like to time sync with a public server when you are not on the network (which is probably a good idea anyway), you could uncomment the original time server, but have it second in the list.
Another fairly critical setting to configure is the "makestep" setting. This will tell chrony that it is allowed to perform a large initial jump in time during the first few time syncs. The following setting will large jumps to occur if the time delta is larger than 1.0 seconds, but only on the first 3 time syncs. Find the and change the setting to the following (or just enter it near the top of the config file):
makestep 1.0 3
Finally, run the following command to restart chrony:
sudo systemctl restart chrony.serviceYou can now run the following to see what server chrony has synced with (although it may take a few seconds):
chronyc tracking -aIf the server you entered is listed near the top, then success!
When you run a program, it will generally be set to a priority common to all other software that has been run at a user level (i.e. not system-critical software). This means that all user-run processes will be given an equal chance to perform any necessary calculations. It may however be desirable for a program that is will use up a lot of resources, if given the chance, to run at a lower priority than other programs, to allow for a fairer sharing of resources. Likewise, it may be desirable for some mission-critical programs to run at a higher priority, to ensure that they are operating with the best possible conditions.
In Linux, the priority of a process is controlled through two different scales. For most use cases, the scale of niceness is what we can use to tell Linux to pay more attention to a process, or to be more likely to give it system resources when requested. The niceness is a scale of -20 to +19, where; -20 will tell Linux to treat it with the highest priority, 0 is the default, and 19 will tell Linux that the process can but put on hold if a more important process needs to run.
To run a program with a lower priority (a python script for example):
nice -10 python script.pyOr with a higher priority:
nice --10 python script.pyTo reassign a program a different priority:
renice -n 10 -p 3534Where -n 10 will set the niceness value to 10, and -p 3534 is the target process ID.
It may be possible to use pgrep to get the process name (take care using this in practice):
renice -n -10 -p $(pgrep some_program_name)To set a negative niceness, you will have to execute the command as a root user (with sudo).
The weight is roughly equivalent to 1024 / (1.25 ^ nice_value). As the nice value decreases the weight increases exponentially. The time-slice allocated for the process is proportional to the weight of the process divided by the total weight of all run-able processes (Reference).
Note: you should take care of assigning the niceness value. Setting a niceness of -20 may cause your programs to interfere with system-level programs (not good!). It is recommended that you set mission-critical programs to something around -10 to -15, and set non-critcal programs to something around +10 to +15. This way, leaving other programs that run at the default user level (0) will still remain in good shape, and you won't interfere with system-critical programs.
In Linux, there is some devices that may not be accessible without the correct permissions.
Unlike a normal desktop computer set up to be used in a GUI, Linux won't automatically mount a USB flash drive so it can be accessed by a user.
First we must create a mount point (a path that will allow us to access the device). Then we have to give the correct permissions so a user can read and write to the device (in this case, we give read/write for all users).
sudo mkdir /media/usb
sudo chmod g+rw,o+rw,u+rw /media/usbNext we can mount the device to the path we created (assuming you know the device and partition you want to mount).
sudo mount /dev/sdX /media/usb Lastly, remember to un-mount the device before you remove it from the system (we also use sync to check all data has been transferred)
sync
sudo umount /media/usb Before we can use a serial device such as a USB-Serial Converter, we must add our user to the "dialout" group. This will allow us to use any "/dev/ttyX" device.
sudo usermod -aG dialout $USERRemember to log out and back in to make sure the group permissions are updated.
If we are using a pre-installed system, it is a good idea to remove the "modem manager" package, as it tends to really mess with serial communications. As an example for Ubuntu:
sudo apt remove modemmanagerBefore we can use a video device such as a webcam, we must add our user to the "video" group. This will allow us to use any "/dev/videoX" device.
sudo usermod -aG video $USERRemember to log out and back in to make sure the group permissions are updated.