Fundamental Concepts
Prev: history-and-standards Next: system-programming-concepts
The Core Operating System: The Kernel
The term kernel is used to refer to the central software that manages and allocates computer resources, like CPU, RAM, and devices.
The kernel exists to make writing programs easier, as it provides an abstraction that manages the limited resources of a computer for other programs.
Tasks performed by the Kernel
The kernel performs some tasks such as:
- Process scheduling: running one or more processes, then swapping them in and out in turns. Linux is a preemptive multitasking operating system, which means that multiple processes are governed by the kernel process scheduler, which moves processes in and out, instead of having processes choosing how long to run.
- Memory management: memory such as RAM has to be shared between
processes in an efficient fashion. Linux has virtual memory management,
which does the following:
- Processes are isolated, so one process can’t read or modify the memory of another process.
- Only part of the process needs to be kept in memory, allowing for better CPU utilization.
- File System Management: The kernel provides a file system, allowing files to be created, read, updated, and deleted.
- Process management: the kernel can create new processes and manage them through their lifecycle.
- Device Access: The kernel governs access to devices such as USBs, hard drives, keyboards, mice, and monitors.
- Networking: The kernel transmits and receives network messages on behalf of user processes.
- System Calls: The kernel allows user space applications to get access to kernel owned resources with a stable system call API.
- User management: The kernel manages users, so their software can run isolated from other users on the same computer.
Kernel mode and User mode
Modern processors run in at least two modes, one is called kernel mode, which has access to the entire computer, such as devices, memory, and I/O, and user mode, where access is limited to memory that has been labeled for user mode. The system call interface allows for switching into kernel mode and running code that user mode generally cannot.
The Shell
The shell is a program designed to read commands and execute appropriate programs in response to those commands. Many shells are in use, but the most popular one is bash. Others include the bourne shell (sh), C Shell (csh), korn (ksh), fish, and oil.
Users and Groups
Each user is uniquely identified, and a user may belong to one or more groups.
Users
Each user on the system has a unique login name, and a corresponding
user id (UID). This information is stored in the system password file
/etc/password, which also includes the Group ID (GID), the
home directory, and the login shell. Normally the passwords are stored
in a separate shadow password file, only readable to superusers.
Groups
Each user may be part of one or more groups. File access and other
administrative tasks can be granted to groups, so they allow for better
administration. The group file is stored in /etc/group,
which has group names, group ID (GID), and User list, which are the
members of the group.
Superuser
One user, the superuser, has special privileges. They have the ID 0,
and the login name root by default. They can access any
file in the system and send signals to any process in the system.
Single Directory Hierarchy, Directories, Links, Files
The Linux Directory Hierarchy looks like this:
In the file system, each file is marked with a type, demarcating it from files such as devices, pipes, sockets, directories, and links.
A directory is a special file which has a table of filenames along
with references to the corresponding files. A filename + reference is
called a link, and files may have multiple links, and thus, many names.
A directory may contain links to other directories. Every directory has
two references, . which refers to itself, and
.., which is the parent directory. The parent directory for
the root directory, /, points to itself.
A symbolic link provides an alternative name for a file. When traversing a symbolic link, the kernel dereferences the link until it finds a normal file. Circular links stop being followed after some time to prevent infinite recursion.
Filenames can have up to 255 characters, and contain characters
except slashes and the null character \0. The 65 characters
in [-._a-zA-Z0-9] is referred to as the portable filename
character set, and are a safe set of characters to use for
filenames.
Pathnames are a string consisting of / and a series of
filenames separated by slashes. A pathname is read from left to
right.
Each file has an associated user ID, and group ID. When accessing a file, the system divides users into three categories, those who own the file, those who are in the group indicated on the file, and the rest of the world. Three permission bits are also set for each of these, denoting who can read, write, and execute the files. On a directory, these three bits allow a user to list the files in the directory, write, to write inside the directory, and execute allows files inside the directory to be accessed.
File I/O Model
In Unix, I/O is universal, so the same system calls
(open, read, write,
close, etc) work on all types of files, including devices.
The kernel translates I/O requests onto the appropriate file system or
device. The kernel provides one file type, a sequence of bytes that can
be accessed with lseek commands.
Programs
Programs normally exist as source code, which must be converted to binary, which the computer understands. Some languages, called scripting languages, use as interpreter to run source code directly, requiring no compilation.
Processes
A process is an instance of an executing program. When a program is executed, the kernel loads the code of the program into virtual memory, allocates space for its stack and variables and sets up kernel data structures. The kernel provides the process with the illusion that it has all the memory of the system, but in the background the kernel is managing those resources.
A process has four parts:
Text: the instructions of the program Data: the static variables in the program Heap: an area where programs can dynamically allocate extra memory Stack: an area that grows and shrinks as functions are called and dynamical variables are allocated.
Process creation
Processes can be created using the fork system call.
Fork copies the process, and makes two copies. One is called the parent
process, which continues executing as normal, and the other process is
called the child process. The child process may then use
execve to change what program it’s running.
Process termination
A process can terminate by requesting its own termination with
_exit, or by being killed by a signal. The process yields a
termination status, which can be read by the parent process using
wait.
Memory Mappings
The mmap system call can be used to create a new memory
mapping in the calling process’s virtual address space. A file mapping
maps a region of a file into the calling process’ virtual memory,
allowing the process to access bytes in the files in the corresponding
regions. The pages are automatically loaded from the file as required.
An anonymous mapping doesn’t have a corresponding file, where instead
the pages of the mapping are initialized to 0.
Interprocess Communication and Synchronization
Linux implements a way for processes to communicate with each other, IPC.
- signals inform a process that an event has occurred
- pipes/FIFOs are used to transfer data between processes
- sockets transfer data from one process to another, on either the same host or another computer
- file locking allows a process to lock parts of a file in order to prevent other processes from reading/updating the file
- message queues to exchange messages between processes
- semaphores to synchronize processes
- shared memory to synchronize memory that processes share
Signals
Signals are software interrupts, and are represented by an event
starting with SIG, like SIGKILL for the kill
signal to a process. The kernel can send signals to processes and
processes can block or handle all signals except for
SIGKILL, wherein it automatically terminates.
Threads
Threads are the runners of a program in a process. Threads can communicate with each other using shared memory or semaphores. They have different stacks.
Process Groups and Shell Job Control
Each program executed by the shell is started in its own process.
$ ls -l | sort -k5n | less will spawn three processes,
which are grouped into a process group.
Sessions, Controlling Terminals, and Controlling Processes
A session is a collection of jobs. Each process has the same session identifier, and the process that created the session’s process id becomes the session id.
Sessions have a controlling terminal, which can have at most one
session. Sessions may have one foreground process, and many background
process groups (started with the & character).
Pseudoterminals
A pseudoterminal is a pair of connected virtual devices, one master and one slave. The pair provides an IPC channel allowing data to be transferred in both directions between the two devices.
They are mainly used for creating an interface that behaves like a terminal, the slave would be the server that receives terminal input, and the master would be the program that drives input to the terminal.
Date and Time
There are two kinds of time:
- Real time: measured from either calendar time or some artificial time, like the start of a process.
- Process time: also called CPU time, the amount of CPU time that a process has used since starting. This is divided into system CPU time, and user time.
Client-Server Architecture
A client-server architecture is an application split into two parts:
- a client, which asks the server to carry out an action by sending a message
- a server, which reads messages from the client, performs an action, and sends back a response to the client
Clients and servers may be on the same computer or different computers.
Realtime
Realtime applications need to respond in a bounded and timely fashion to input. Some examples might be aircraft navigation systems, or assembly lines.
Linux doesn’t qualify as a real time operating system, because realtime goals of responsiveness conflict with multiuser time-sharing operating systems, which linux is. Some linux facilities allow for asynchronous operations.
The /proc file system
Linux provides a special file system called /proc, which
is virtual and has information about the processes running on the
computer, as well as metadata relating to it.
Prev: history-and-standards Next: system-programming-concepts