-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathos-concepts
138 lines (137 loc) · 8.07 KB
/
os-concepts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
#OS Concepts
##Process Management
- Process: a program in execution or running instance of a program.
- Foreground or Interactive Process: user initiated process, requires interaction.
- Background or non-interactive process: start the process, ctrl z it and the "bg". To bring back use "fg" or by doing "Process_name &"
- Daemons: Background processes that start at system startup and keep running as service. can be controlled using init process.
- Process Creation
- using the system() function
- using fork() and exec() function
- Basically new process is created when an exisiting process makes an exact copy of itself, this child process has same enviorment as its parent but new process ID (this is called Forking). The address space of the child process is overwritten with the new process data using an exec call to the system.
- Processes are identified by their PID or PPID (Parent's PID)
- Init Process
- first program executed when the Linux System boots up.
- PID == 1
- pidof systemd == 1
************************
[1]+ Stopped top
root@ip-172-26-3-150:~# jobs
[1]+ Stopped top
root@ip-172-26-3-150:~# bg
[1]+ top &
root@ip-172-26-3-150:~# fg
************************
- State of Process
- running
- waitiing
- stopped
- zombie : parent process is dead or it has been halted but still exists in process table.
- Process view Commands
- ps
- top
- glances
- Process Kill or Stop command
- kill -l ;; Will list different signals can be sent to a process.
- nice/renice
- used to adjust the priority of a process. default priority == 0 , range -20 to 19 with lower being better.
##Sockets
- It is one end of an interprocess communication channel established between client/server.
- Clinet side socket establishment steps
- Create a socket with the socket() system call
- Connect the socket to the address of the server using the connect() system call
- Send and receive data. There are a number of ways to do this, but the simplest is to use the read() and write() system calls.
- Server side socket establishment steps
- Create a socket with the socket() system call
- Bind the socket to an address using the bind() system call. For a server socket on network, an address consists of a port number on the host machine.
- Listen for connections with the listen() system call
- Accept a connection with the accept() system call. This call typically blocks until a client connects with the server.
- Send and receive data
- Types of Sockets
- stream sockets - TCP, continuous steam of characters.
- datagram sockets - UDP, read entire messages at once.
##Processes and threads
- Process: Running instance of a computer program.
- It consists of allocated memory which holds the program code, its data and a heap for dynamic memory allocations.
- It is not the unit for multi-tasking in destop operating systems.
- Thread: It is the default unit - the task - of CPU usage. Code executed in a single thread is what we usually refer to as sequential or synchronous execution.
- supported on multithreaded OS and can be created with system calls.
- They have their own call stacks, virtual CPU and (often) local storage but share the application's heap, data, codebase and resources (such as file handles) with the other threads in the same process.
- Context switching: switching between threads
- It is done at frequent intervals by the kernel, creating the illusion that our programs are running in parallel, whereas in reality, they are running concurrently but sequentially in short slices.
- CPU Bound
- scientific computation
- (in-mem) data analysis
- simulations
- I/O Bound
- reading from/writing to Disk
- access peripherials like mic or cam etc
- reading/writing to Network Sockets
- reading from stdin
- Blocking vs non-blocking
- blocking: waiting for the device to be ready
- non-blocking: eg polling periodically until ready
- synchronous vs asynchronous
- synchronous: executing the operation (e.g. read or write) initiated by the program
- asynchronous: executing the operation as response to an event from the kernel (asynchronous / event driven)
- synchronous, blocking I/O: Eg Cooking food by focusing on one recipe at a time, not multi-tasking at all.
- Synchronous, non-blocking I/O: Eg Cooking rice dish and other dishs at the same time, make rice changes when needed otherwise do other dishes.
- Asynchronous, non-blocking I/O: Eg Get a sou chef, assign work to them as needed and during non-interuptions work on your dishes.
##Filesystems of yesterday and today
- Ext2
- Ext2 stands for second extended file system.
- Ext2 does not have journaling feature, thereby considered better on usb, flash etc
- Maximum individual file size can be from 16 GB to 2 TB
- Overall ext2 file system size can be from 2 TB to 32 TB
- Creating ext2 filesystem # mke2fs /dev/sda1
- Ext3
- Ext3 stands for third extended file system.
- allows journaling
- Journaling has a dedicated area in the file system, where all the changes are tracked. When the system crashes, the possibility of file system corruption is less because of journaling.
- Types of Journalling
- Journal – Metadata and content are saved in the journal.
- Ordered – Only metadata is saved in the journal. Metadata are journaled only after writing the content to disk. This is the default.
- Writeback – Only metadata is saved in the journal. Metadata might be journaled either before or after the content is written to the disk. - Performant
- You can convert a ext2 file system to ext3 file system directly (without backup/restore).
- Ext4
- Ext4 stands for fourth extended file system.
- Maximum individual file size can be from 16 GB to 16 TB
- Overall maximum ext4 file system size is 1 EB (exabyte). 1 EB = 1024 PB (petabyte).
- Directory can contain a maximum of 64,000 subdirectories (as opposed to 32,000 in ext3). max files 4 billion.
- Several other new features are introduced in ext4: multiblock allocation, delayed allocation, journal checksum. fast fsck, etc. All you need to know is that these new features have improved the performance and reliability of the filesystem when compared to ext3.
- In ext4, you also have the option of turning the journaling feature “off”.
- XFS
- high-performance 64 bit journaling file system
- scalable/parallel I/O based I/O threads and file system bandwidth
- journaling for metadata operations
- online defragmentation
- suspend/resume I/O
- maximum file system size of 8 exabytes for 64 bit file system, max file size 8 ebytes.
- max files 2 to power 64
- cannot be shrunk or size changed
- Uses B+ tree implementation for searching files and allocating space. This makes searching and fetching the data much faster.
- OpenZFS
- OpenZFS is designed as a copy-on-write file system which means even when data is being modified, it is done by writing a new data block first and then getting rid of the old data block.
- Supports checksums which means silent data corruptions when your disks start to fail is reported, if not automatically fixed.
- OpenZFS is a 128-bit file system with some super large file zise/directories support
##Directory Structure
- root
- bin: Essential user command binaries
- bash, cat, cp etc
- etc: (everything to configure) Config files for the system
- crontab, hosts, http etc
- sbin: Essential system binaries
- fdisk, getty, ifconfig etc
- usr: read-only user app support data and binaries
- usr/bin: most user commands
- usr/include: standard include files for "C" code
- usr/lib: obj, bin,lib for coding and packages
- usr/local: local software with sub tree structure
- usr/share: static data sharable across all architectures
- var: variable data files
- lib, log, cache, opt etc sub directories
- dev: device files incl. /dev/null
- home: user home directories
- lib: libraries and kernel modules
- mnt: mount files for temp filesystems
- opt: optional software applications
- proc: process and ketnel information files