Chapter 33: Threads: Further Details

This chapter provides further details on various aspects of POSIX threads. We discuss the interaction of threads with aspects of the traditional Unix API; in particular, signals and the process control primitives (fork(), exec(), and _exit()). We also provide an overview of the two POSIX threads implementations available on Linux--LinuxThreads and NPTL--and discuss where each of these implementations deviates from the SUSv3 specification of Pthreads.

33 Threads: Further Details
33.1 Thread Stacks
33.2 Threads and Signals
        33.2.1 How the Unix Signal Model Maps to Threads
        33.2.2 Manipulating the Thread Signal Mask
        33.2.3 Sending a Signal to a Thread
        33.2.4 Dealing with Asynchronous Signals Sanely
33.3 Threads and Process Control
33.4 Thread Implementation Models
33.5 Linux Implementations of POSIX Threads
        33.5.1 LinuxThreads
        33.5.2 NPTL
        33.5.3 Which Threading Implementation?
33.6 Advanced Features of the Pthreads API
33.7 Summary
33.8 Exercises

Chapter 32: Threads: Thread Cancellation

Typically, multiple threads execute in parallel, with each thread performing its task until it decides to terminate by calling pthread_exit() or returning from the thread’s start function.

Sometimes, it can be useful to cancel a thread; that is, to send it a request asking it to terminate now. This could be useful, for example, if a group of threads is performing a calculation, and one thread detects an error condition that requires the other threads to terminate. Alternatively, a GUI-driven application may provide a cancel button to allow the user to terminate a task that is being performed by a thread in the background; in this case, the main thread (controlling the GUI) needs to tell the background thread to terminate.

In this chapter, we describe the POSIX threads cancellation mechanism.

32 Threads: Thread Cancellation
32.1 Canceling a Thread
32.2 Cancellation State and Type
32.3 Cancellation Points
32.4 Testing for Thread Cancellation
32.5 Cleanup Handlers
32.6 Asynchronous Cancelability
32.7 Summary
32.8 Exercises

Chapter 31: Threads: Thread Safety and Per-thread Storage

This chapter continues the discussion of the POSIX threads API. It covers thread-safe functions and one-time initialization. We also discuss how to use thread-specific data or thread-local storage to make an existing function thread-safe without changing the function's interface.

31 Threads: Thread Safety and Per-thread Storage
31.1 Thread Safety (and Reentrancy Revisited)
31.2 One-time Initialization
31.3 Thread-specific Data
        31.3.1 Thread-specific Data from the Library Function’s Perspective
        31.3.2 Overview of the Thread-specific Data API
        31.3.3 Details of the Thread-specific Data API
        31.3.4 Employing the Thread-specific Data API
        31.3.5 Thread-specific Data Implementation Limits
31.4 Thread-local Storage
31.5 Summary
31.6 Exercises

Chapter 30: Threads: Thread Synchronization

In this chapter, we describe two tools that threads can use to synchronize their actions: mutexes and condition variables. Mutexes allow threads to synchronize their use of a shared resource, so that, for example, one thread doesn't try to access a shared variable at the same time as another thread is modifying it. Condition variables perform a complementary task: They allow threads to inform each other that a shared variable (or other shared resource) has changed state.

30 Threads: Thread Synchronization
30.1 Protecting Accesses to Shared Variables: Mutexes
        30.1.1 Statically Allocated Mutexes
        30.1.2 Locking and Unlocking a Mutex
        30.1.3 Performance of Mutexes
        30.1.4 Mutex Deadlocks
        30.1.5 Dynamically Initializing a Mutex
        30.1.6 Mutex Attributes
        30.1.7 Mutex Types
30.2 Signaling Changes of State: Condition Variables
        30.2.1 Statically Allocated Condition Variables
        30.2.2 Signaling and Waiting on Condition Variables
        30.2.3 Testing a Condition Variable’s Predicate
        30.2.4 Example Program: Joining Any Terminated Thread
        30.2.5 Dynamically Allocated Condition Variables
30.3 Summary
30.4 Exercises

Chapters 38, 39, and 40 are in copyedit

Chapter 37 came back from coyedit. Chapters 38, 39, and 40 are now in copyedit.

Chapter 29: Threads: Introduction

In this and the next few chapters, we describe POSIX threads, often known as Pthreads. We won't attempt to cover the entire Pthreads API, since it is rather large. Various sources of further information about threads are listed at the end of this chapter.

These chapters mainly describe the standard behavior specified for the Pthreads API. In Section 33.5, we discuss those points where the two main Linux threading implementations--LinuxThreads and Native POSIX Threads Library (NPTL)--deviate from the standard.

In this chapter, we provide an overview of the operation of threads, and then look at how threads are created and how they terminate. We conclude with a discussion of some factors that may influence the choice of a multithreaded approach versus a multiprocess approach when designing an application.

29 Threads: Introduction
29.1 Overview
29.2 Background Details of the Pthreads API
29.3 Thread Creation
29.4 Thread Termination
29.5 Thread IDs
29.6 Joining with a Terminated Thread: pthread_join()
29.7 Detaching a Thread: pthread_detach()
29.8 Thread Attributes
29.9 Threads Versus Processes
29.10 Summary
29.11 Exercises

Chapter 28: Process Creation and Program Execution in More Detail

This chapter extends the material presented in Chapters 24 to 27 by covering a variety of topics related to process creation and program execution. We describe process accounting, a kernel feature that writes an accounting record about each process on the system as it terminates. We then look at the Linux-specific clone() system call, which is the low-level API that is used to create threads on Linux. We follow this with some comparisons of the performance of fork(), vfork(), and clone(). We conclude with a summary of the effects of fork() and exec() on the attributes of a process.

28 Process Creation and Program Execution in More Detail
28.1 Process Accounting
28.2 The clone() System Call
        28.2.1 The clone() flags Argument
        28.2.2 Extensions to waitpid() for Cloned Children
28.3 Speed of Process Creation
28.4 Effect of exec() and fork() on Process Attributes
28.5 Summary
28.6 Exercises

Chapter 37 is in copyedit

Chapter 34 already came back from copyedit, and Chapters 35 and 36 also went to copyedit and came back already. Now Chapter 37 is in copyedit.

Chapter 27: Program Execution

This chapter follows from our discussion of process creation and termination in the previous chapters. We now look at how a process can use the execve() system call to replace the program that it is running by a completely new program. We then show how to implement the system() function, which allows its caller to execute an arbitrary shell command.

27 Program Execution
27.1 Executing a New Program: execve()
27.2 The exec() Library Functions
        27.2.1 The PATH Environment Variable
        27.2.2 Specifying Program Arguments As a List
        27.2.3 Passing the Caller's Environment to the New Program
        27.2.4 Executing a File Referred to by a Descriptor: fexecve()
27.3 Interpreter Scripts
27.4 File Descriptors and exec()
27.5 Signals and exec()
27.6 Executing a Shell Command: system()
27.7 Implementing system()
27.8 Summary
27.9 Exercises

Chapter 26: Monitoring Child Processes

In many application designs, a parent process needs to know when one of its child processes changes state--when the child terminates or is stopped by a signal. This chapter describes two techniques used in monitoring child processes: the wait() system call (and its variants) and the use of the SIGCHLD signal.

26 Monitoring Child Processes
26.1 Waiting on a Child Process
        26.1.1 The wait() System Call
        26.1.2 The waitpid() System Call
        26.1.3 The Wait Status Value
        26.1.4 Process Termination from a Signal Handler
        26.1.5 The waitid() System Call
        26.1.6 The wait3() and wait4() System Calls
26.2 Orphans and Zombies
26.3 The SIGCHLD Signal
        26.3.1 Establishing a Handler for SIGCHLD
        26.3.2 Delivery of SIGCHLD for Stopped Children
        26.3.3 Ignoring Dead Child Processes
26.4 Summary
26.5 Exercises

Chapter 25: Process Termination

This chapter describes what happens when a process terminates. We begin by describing the use of exit() and _exit() to terminate a process. We discuss the use of exit handlers to automatically perform cleanups when a process calls exit(). We also consider some interactions between fork(), stdio buffers, and exit().

25 Process Termination
25.1 Terminating a Process: _exit() and exit()
25.2 Details of Process Termination
25.3 Exit Handlers
25.4 Interactions Between fork(), stdio Buffers, and _exit()
25.5 Summary
25.6 Exercises

Chapter 24: Process Creation

In this and the next three chapters, we look at how a process is created and terminates, and how a process can execute a new program. This chapter covers process creation. However, before diving into that subject, we present a short overview of the main system calls covered in these four chapters.

24 Process Creation
24.1 Overview of fork(), exit(), wait(), and execve()
24.2 Creating a New Process: fork()
        24.2.1 File Sharing Between Parent and Child
        24.2.2 Memory Semantics of fork()
24.3 The vfork() System Call
24.4 Race Conditions after fork()
24.5 Avoiding Race Conditions by Synchronizing with Signals
24.6 Summary
24.7 Exercises

Chapter 23: Timers and Sleeping

A timer allows a process to schedule a notification for itself to occur at some time in the future. Sleeping allows a process (or thread) to suspend execution for a period of time. This chapter describes the interfaces used for setting timers and for sleeping. It covers the following topics:

• the classical Unix APIs for setting interval timers (setitimer() and alarm()) to notify a process when a certain amount of time has passed;
• the APIs that allow a process to sleep for a specified interval;
• the POSIX.1b clocks and timers APIs; and
• the Linux-specific timerfd facility, which allows the creation of timers whose expirations can be read from a file descriptor.

23 Timers and Sleeping
23.1 Interval Timers
23.2 Scheduling and Accuracy of Timers
23.3 Setting Timeouts on Blocking Operations
23.4 Suspending Execution for a Fixed Interval (Sleeping)
        23.4.1 Low-resolution Sleeping: sleep()
        23.4.2 High-resolution Sleeping: nanosleep()
23.5 POSIX Clocks
        23.5.1 Retrieving the Value of a Clock: timer_gettime()
        23.5.2 Updating the Value of a Clock: timer_gettime()
        23.5.3 Obtaining the Clock ID of a Specific Process or Thread
        23.5.4 Improved High-resolution Sleeping: clock_nanosleep()
23.6 POSIX Interval Timers
        23.6.1 Creating a Timer: timer_create()
        23.6.2 Arming and Disarming a Timer: timer_settime()
        23.6.3 Retrieving the Current Value of a Timer: timer_gettime()
        23.6.4 Deleting a Timer: timer_delete()
        23.6.5 Notification via a Signal (SIGEV_SIGNAL)
        23.6.6 Timer Overruns
        23.6.7 Notification via a Thread (SIGEV_THREAD)
23.7 Timers That Notify via File Descriptors: the timerfd API
23.8 Summary
23.9 Exercises

Chapter 34 is in copyedit

Chapters 29 to 33 are back from copyedit. Chapter 34 has gone to copyedit.

Chapter 22: Signals: Advanced Features

In this chapter, we continue our discussion of signals, covering a number of more advanced topics, including:

• core dump files;
• special cases regarding signal delivery, disposition, and handling;
• synchronous and asynchronous generation of signals;
• when and in what order signals are delivered;
• interruption of system calls by signal handlers, and how to automatically restart interrupted system calls;
• realtime signals;
• using sigsuspend() to set the process signal mask wait for a signal to arrive;
• using sigwaitinfo() (and sigtimedwait()) to synchronously wait for a signal to arrive;
• using signalfd() to receive a signal via file descriptor; and
• the older BSD and System V signal APIs.

22 Signals: Advanced Features
22.1 Core Dump Files
22.2 Special Cases for Signal Delivery, Disposition, and Handling
22.3 Interruptible and Uninterruptible Process Sleep States
22.4 Hardware-generated Signals
22.5 Synchronous and Asynchronous Signal Generation
22.6 Timing and Order of Signal Delivery
22.7 Implementation and Portability of signal()
22.8 Realtime Signals
        22.8.1 Sending Realtime Signals
        22.8.2 Handling Realtime Signals
22.9 Waiting for a Signal Using a Mask: sigsuspend()
22.10 Synchronously Waiting for a Signal
22.11 Fetching Signals via a File Descriptor
22.12 Interprocess Communication with Signals
22.13 Earlier Signal APIs (System V and BSD)
22.14 Summary
22.15 Exercises

Chapter 21: Signals: Signal Handlers

This chapter continues the description of signals begun in the previous chapter. It focuses on signal handlers, and extends the discussion started in Section 20.4. Among the topics we consider are the following:

• how to design a signal handler, which necessitates a discussion of reentrancy and async-signal-safe functions;
• alternatives to performing a normal return from a signal handler, in particular, the use of a nonlocal goto for this purpose;
• handling of signals on an alternate stack;
• the use of the sigaction() SA_SIGINFO flag to allow a signal handler to obtain more detailed information about the signal that caused its invocation; and
• how a blocking system call may be interrupted by a signal handler, and how the call can be restarted if desired.

21 Signals: Signal Handlers
21.1 Designing Signal Handlers
        21.1.1 Signals Are Not Queued (Revisited)
        21.1.2 Reentrant and Async-signal-safe Functions
        21.1.3 Global Variables and the sig_atomic_t Data Type
21.2 Other Methods of Terminating a Signal Handler
        21.2.1 Performing a Nonlocal Goto from a Signal Handler
        21.2.2 Terminating a Process Abnormally: abort()
21.3 Handling a Signal on an Alternate Stack: sigaltstack()
21.4 The SA_SIGINFO Flag
21.5 Interruption and Restarting of System Calls
21.6 Summary
21.7 Exercises

Chapter 20: Signals: Fundamental Concepts

This chapter and next two chapters discuss signals. Although the fundamental concepts are quite simple, our discussion is quite lengthy, since there are many details to cover.

This chapter covers the following topics:

• the various different signals and their purposes;
• the circumstances in which the kernel may generate a signal for a process, and the system calls that one process may use to send a signal to another process;
• how a process responds to a signal by default, and the means by which a process can change its response to a signal, in particular, through the use of a signal handler, a programmer-defined function that is automatically invoked on receipt of a signal;
• the use of a process signal mask to block signals, and the associated notion of pending signals; and
• how a process can suspend execution and wait for the delivery of a signal.

20 Signals: Fundamental Concepts
20.1 Concepts and Overview
20.2 Signal Types and Default Actions
20.3 Changing Signal Dispositions: signal()
20.4 Introduction to Signal Handlers
20.5 Sending Signals: kill()
20.6 Checking for the Existence of a Process
20.7 Other Ways of Sending Signals: raise() and killpg()
20.8 Displaying Signal Descriptions
20.9 Signal Sets
20.10 The Signal Mask (Blocking Signal Delivery)
20.11 Pending Signals
20.12 Signals Are Not Queued
20.13 Changing Signal Dispositions: sigaction()
20.14 Waiting for a Signal: pause()
20.15 Summary
20.16 Exercises

Chapter 19: Monitoring File Events

Some applications need to be able to monitor files or directories in order to determine whether events have occurred for the monitored objects. For example, a graphical file manager needs to be able to determine when files are added or removed from the directory that is currently being displayed, or a daemon may want to monitor its configuration file in order to know if the file has been changed.

Starting with kernel 2.6.13, Linux provides the inotify mechanism, which allows an application to monitor file events. This chapter describes the use of inotify.

19 Monitoring File Events
19.1 Overview
19.2 The inotify API
19.3 inotify Events
19.4 Reading inotify Events
19.5 Queue Limits and /proc Files
19.6 An Older System for Monitoring File Events: dnotify
19.7 Summary
19.8 Exercises

Chapter 18: Directories and Links

In this chapter, we conclude our discussion of file-related topics by looking at directories and links. After an overview of their implementation, we describe the system calls used to create and remove directories and links. We then look at library functions that allow a program to scan the contents of a single directory and to walk through (i.e., examine each file in) a directory tree.

Each process has two directory-related attributes: a root directory, which determines the point from which absolute pathnames are interpreted, and a current working directory, which determines the point from which relative pathnames are interpreted. We look at the system calls that allow a process to change both of these attributes.

18 Directories and Links
18.1 Directories and (Hard) Links
18.2 Symbolic (Soft) Links
18.3 Creating and Removing (Hard) Links: link() and unlink()
18.4 Changing the Name of a File: rename()
18.5 Working with Symbolic Links: symlink() and readlink()
18.6 Creating and Removing Directories: mkdir() and rmdir()
18.7 Removing a File or Directory: remove()
18.8 Reading Directories: opendir() and readdir()
18.9 File Tree Walking: nftw()
18.10 The Current Working Directory of a Process
18.11 Operating Relative to a Directory File Descriptor
18.12 Changing the Root Directory of a Process: chroot()
18.13 Resolving a Pathname: realpath()
18.14 Parsing Pathname Strings: dirname() and basename()
18.15 Summary
18.16 Exercises

Chapter 17: Access Control Lists

Section 15.4 described the traditional Unix (and Linux) file permissions scheme. For many applications, this scheme is sufficient. However, some applications need finer control over the permissions granted to specific users and groups. To meet this requirement, many Unix systems implement an extension to the traditional Unix file permissions model known as access control lists (ACLs). ACLs allow file permissions to be specified per user or per group, for an arbitrary number of users and groups. Linux provides ACLs from kernel 2.6 onward.

This chapter provides a description of ACLs and a brief tutorial on their use. It also describes some of the library functions used for manipulating and retrieving ACLs.

17 Access Control Lists
17.1 Overview
17.2 ACL Permission-Checking Algorithm
17.3 Long and Short Text Forms for ACLs
17.4 The ACL_MASK Entry and the ACL Group Class
17.5 The getfacl and setfacl Commands
17.6 Default ACLs and File Creation
17.7 ACL Implementation Limits
17.8 The ACL API
17.9 Summary
17.10 Exercises