Chapter 42: Advanced Features of Shared Libraries

The previous chapter covered the fundamentals of shared libraries. This chapter describes a number of advanced features of shared libraries, including the following:

• dynamically loading shared libraries;
• controlling the visibility of symbols defined by a shared library;
• using linker scripts to create versioned symbols;
• using initialization and finalization functions to automatically execute code when a library is loaded and unloaded;
• shared library preloading; and
• using LD_DEBUG to monitor the operation of the dynamic linker.

42 Advanced Features of Shared Libraries
42.1 Dynamically Loaded Libraries
        42.1.1 Opening a Shared Library
        42.1.2 Diagnosing Errors from the dlopen API
        42.1.3 Obtaining the Address of a Symbol: dlsym()
        42.1.4 Closing a Shared Library: dlclose()
        42.1.5 Obtaining Information about Loaded Symbols: dladdr()
        42.1.6 Accessing Symbols in the Main Program
42.2 Controlling Symbol Visibility
42.3 Linker Version Scripts
        42.3.1 Controlling Symbol Visibility with Version Scripts
        42.3.2 Symbol Versioning
42.4 Initialization and Finalization Functions
42.5 Preloading Shared Libraries
42.6 Monitoring the Dynamic Linker: LD_DEBUG
42.7 Summary
42.8 Exercises

See you at LCA 2010!

One of my proposals got accepted for LCA 2010, in Wellington, New Zealand! As usual, there look to be lots of interesting papers at LCA. I'm presenting a tutorial on writing secure privileged programs, which will draw heavily on Chapter 38 of my book.

Capter 41: Fundamentals of Shared Libraries

Shared libraries are a technique for placing library functions into a single unit that can be shared by multiple processes at run time. This technique can save both disk space and RAM. This chapter covers the fundamentals of shared libraries. The next chapter covers a number of advanced features of shared libraries.

41 Fundamentals of Shared Libraries
41.1 Object Libraries
41.2 Static Libraries
41.3 Overview of Shared Libraries
41.4 Creating and Using Shared Libraries--A First Pass
        41.4.1 Creating a Shared Library
        41.4.2 Position-independent Code
        41.4.3 Using a Shared Library
        41.4.4 The Shared Library Soname
41.5 Useful Tools for Working with Shared Libraries
41.6 Shared Library Versions and Naming Conventions
41.7 Installing Shared Libraries
41.8 Compatible Versus Incompatible Libraries
41.9 Upgrading Shared Libraries
41.10 Specifying Library Search Directories in an Object File
41.11 Finding Shared Libraries at Run Time
41.12 Run-time Symbol Resolution
41.13 Using a Static Library Instead of a Shared Library
41.14 Summary
41.15 Exercises

Chapter 40: Login Accounting

Login accounting is concerned with recording which users are currently logged in to the system, and recording past logins and logouts. This chapter looks at the login accounting files and the library functions used to retrieve and update the information they contain. We also describe the steps that an application providing a login service should perform in order to update these files when a user logs in and out.

40 Login Accounting
40.1 Overview of the utmp and wtmp Files
40.2 The utmpx API
40.3 The utmpx Structure
40.4 Retrieving Information from the utmp and wtmp Files
40.5 Retrieving the Login Name: getlogin()
40.6 Updating the utmp and wtmp Files for a Login Session
40.7 The lastlog File
40.8 Summary
40.9 Exercises

Chapter 39: Capabilities

This chapter describes the Linux capabilities scheme, which divides the traditional all-or-nothing Unix privilege scheme into individual capabilities that can be independently enabled or disabled. Using capabilities allows a program to perform some privileged operations, while preventing it from performing others.

39 Capabilities
39.1 Rationale for Capabilities
39.2 The Linux Capabilities
39.3 Process and File Capabilities
39.3.1 Process Capabilities
39.3.2 File Capabilities
39.3.3 Purpose of the Process Permitted and Effective Capability Sets
39.3.4 Purpose of the File Permitted and Effective Capability Sets
39.3.5 Purpose of the Process and File Inheritable Sets
39.3.6 Assigning and Viewing File Capabilities from the Shell
39.4 The Modern Capabilities Implementation
39.5 Transformation of Process Capabilities during exec()
39.5.1 Capability Bounding Set
39.5.2 Preserving root Semantics
39.6 Effect on Process Capabilities of Changing User IDs
39.7 Changing Process Capabilities Programmatically
39.8 Creating Capabilities-only Environments
39.9 Discovering the Capabilities Required by a Program
39.10 Older Kernels and Systems without File Capabilities
39.11 Summary
39.12 Exercises

Chapters 43 to 44 are in copyedit

Chapters 41 and 42 already went to copyedit a while back, and by now chapters 37 to 42 came back from copyedit. Chapters 43 and 44 are now in copyedit.

Chapter 38: Writing Secure Privileged Programs

Privileged programs have access to features and resources (files, devices, and so on) that are not available to ordinary users. A program can run with privileges by two general means:

• The program was started under a privileged user ID. Many daemons and network servers, which are typically run as root, fall into this category.

• The program has its set-user-ID or set-group-ID permission bit set. When a set-user-ID (set-group-ID) program is execed, it changes the effective user (group) ID of the process to be the same as the owner (group) of the program file.
  

If a privileged program contains bugs, or can be subverted by a malicious user, then the security of the system or an application can be compromised. From a security viewpoint, we should write programs so as to minimize both the chance of a compromise and the damage that can be done if a compromise does occur. These topics form the subject of this chapter.

38 Writing Secure Privileged Programs
38.1 A Checklist for Secure Programming
38.2 Summary
38.3 Exercises

Chapter 37: Daemons

This chapter examines the characteristics of daemon processes and looks at the steps required to turn a process into a daemon. We also look at how to log messages from a daemon using the syslog facility.

37 Daemons
37.1 Overview
37.2 Creating a Daemon
37.3 Guidelines for Writing Daemons
37.4 Using SIGHUP to Reinitialize a Daemon
37.5 Logging Messages and Errors Using syslog
37.5.1 Overview
37.5.2 The syslog API
37.5.3 The /etc/syslog.conf File
37.6 Summary
37.7 Exercises

Chapter 36: Process Resources

Each process consumes system resources such as memory and CPU time. This chapter looks at resource-related system calls. We begin with the getrusage() system call, which allows a process to monitor the resources that it has used or that its children have used. We then look at the setrlimit() and getrlimit() system calls, which can be used to change and retrieve limits on the calling process's consumption of various resources.

36 Process Resources
36.1 Process Resource Usage: getrusage()
36.2 Process Resource Limits: getrlimit() and setrlimit()
36.3 Details of Specific Resource Limits
36.4 Summary
36.5 Exercises

Choosing a title for the book

No Starch Press and I have been kicking around some ideas for the title of the book, but so far we've still not settled on one. And I'm curious what other people might think. So this blog post asks for readers' opinions on the candidates.

Before I mention the titles we've thought about, it's worth mentioning that a couple of possibly useful titles are already taken (since otherwise people might suggest them to me): Advanced Linux Programming (Mitchell et al., 2001, Sams) and Linux System Programming (Love, 2007, O’Reilly).

When thinking of the titles below, I try to keep in mind two important aspects to the title of a book (which are not always in perfect harmony):

• Is the title an accurate reflection of the subject of the book?

• Does the title help sell the book? For example, does it help the book get found in online searches? Is it memorable? Does it get good shelf placement in bookshops? And so on.

1. Linux System and Network Programming

This has been my working title for the book since its inception, and captures two points that are important for marketing:

• The focus on Linux.

• The inclusion of a significant section on network programming. This is important because there are whole books devoted to network programming (most obviously, the books of W. Richard Stevens) so there is an obvious market. Although the material that is purely about network programming is only about 10% of the book (i.e., 150 pages), the fact that a lot of related topics are covered (e.g., process creation, program execution, secure programming, I/O multiplexing, threaded programming) means that the book's coverage of network programming is in effect quite substantial.

Against this title choice:

• It doesn't capture an important aspect of the book: the book focuses heavily on portability, and clearly delineates Linux extensions from "standard" Unix. This means the book really can be used by system programmers on other Unix systems. (The current version of the preface makes this point strongly.)

• It could be confusing if I one day write a book whose subject is more purely "Linux Network Programming" (which is something I might one day do).

• It could be considered a little prosaic.
  

2. Linux/UNIX System and Network Programming

This title attempts to keep "Network" while trying also to address the portable focus of the book. But is it a little clunky as a title?

3. Linux and UNIX System Programming

This title captures the book's focus on both Linux specifically and Unix generally: on the one hand, the book constantly drills down into the specifics of Linux, while on the other hand, it keeps a close eye on Unix standards and portability (and occasionally drills down into details about other Unix systems). (Again, see the preface.)

This title is more concise than Linux/UNIX System and Network Programming, but, conversely, it omits mention of the network programming component of the book. The question is, if we're going for a shorter title, is it more important from a marketing point of view to mention Unix or to mention networking?

Like Linux System and Network Programming, this title might be considered prosaic. On the other hand, it's solid, and could be a title that has a good shelf life.

4. The Craft of Linux Programming

I like this title quite a lot. It has a few resonances. One is the comparison of "craft" with "art", which carries over quite happily into computing. For example, Knuth’s The Art of Computer Programming is exactly that: a (magnificent) work of art about the fundamentals of computing. By contrast, my book is about a lot of the everyday tools that are required for the craft of system programming. In addition, the English word craft derives from the German Kraft (I'm not German, but I live in Germany). Kraft can be translated as power, which seems a reasonable summary of a goal of the book: to provide its readers with the power to write effective system programs.

The title is quirky, and maybe less searchable than others. On the other hand, it has the virtue of being distinctive. Stevens' classic had a somewhat quirky and distinctive title as well, one that distinguished it from the pack: Advanced Programming in the Unix Environment. So, in the end, I'm undecided about whether quirkiness is a marketing plus or minus.

5. Advanced Programming in the Linux Environment

Because of its resonance with another title, inevitably, I have to preface discussion of this title with a couple of remarks. First, it was not my idea (it was someone else's suggestion), and I was initially rather against it. Second, to my surprise, informal canvassing of some of my technical reviewers showed that several of them were quite warm to the title (and none of them disliked it), even though most (probably all) of them are well aware of the other book.

Of course, the title is a play on Stevens' Advanced Programming in the Unix Environment (which was in turn a play on an earlier quirkily titled classic, Kernighan & Pike, The Unix Programming Environment).

Against this title:

• Maybe it looks presumptuous and/or cheeky to choose such an obviously close title.

• It doesn't mention Unix or Network (see my comments on the other titles above).

For this title:

• In terms of Linux titles, it's reasonably distinctive.

• The flip side of cheeky and presumptuous is that the title is also a homage and conveys something of my intent in writing the book. Any book that tries to seriously cover the area of system programming on Unix-type systems owes a debt to Stevens' work, both in terms of the rich technical resources that his books provide, and in terms of the wonderful clarity of his writing. As a system programming text, my book certainly aims at something like the technical depth of Stevens (while at the same time: broadening the range of topics covered, addressing the latest standards, and providing in-depth specifics of the most widely-used Unix-type system). And I continue to be amazed by the clarity of his writing, and attempt to get somewhere close to that clarity (inevitably, it's hard for me to judge how well I've done on that score, since I'm too close to my own writing and ideas to be a good judge).

Summary

In summary, the titles that we're so far considering are the following:

1. Linux System and Network Programming
2. Linux/UNIX System and Network Programming
3. Linux and UNIX System Programming
4. The Craft of Linux Programming
5. Advanced Programming in the Linux Environment
6. Something else?

I'd be interested to hear your thoughts. You can send me votes and thoughts by email at mtk AT man7.org. You may also want to add comments to this blog post; but if you want to vote, I'd really prefer email. (Don't worry: I won't add you to any email list!) You can vote for as many titles as you like, but your vote will only make a meaningful difference if you vote for more than zero of the titles, and you don't vote for them all. Also, you might have ideas for other titles (option 6 above), or other thoughts about the titles I've listed above, and I’d welcome those.

Update: if you do email a vote to me, it would also be interesting to hear the reasons why you prefer or dislike certain titles. Thanks!

Chapter 35: Process Priorities and Scheduling

This chapter discusses various system calls and process attributes that determine when and which processes obtain access to the CPU(s). We begin by describing the nice value, a process characteristic that influences the amount of CPU time that a process is allocated by the kernel scheduler. We follow this with a description of the POSIX realtime scheduling API. This API allows us to define the policy and priority used for scheduling processes, giving us much tighter control over how processes are allocated to the CPU. We conclude with a discussion of the system calls for setting a process's CPU affinity mask, which determines the set of CPUs on which a process running on a multiprocessor system will run.

35 Process Priorities and Scheduling
35.1 Process Priorities (Nice Values)
35.2 Overview of Realtime Process Scheduling
35.2.1 The SCHED_RR Policy
35.2.2 The SCHED_FIFO Policy
35.2.3 The SCHED_BATCH and SCHED_IDLE Policies
35.3 Realtime Process Scheduling API
35.3.1 Realtime Priority Ranges
35.3.2 Modifying and Retrieving Policies and Priorities
35.3.3 Relinquishing the CPU
35.3.4 The SCHED_RR Time Slice
35.4 CPU Affinity
35.5 Summary
35.6 Exercises

Chapter 34: Process Groups, Sessions, and Job Control

Process groups and sessions form a two-level hierarchical relationship between processes: a process group is a collection of related processes, and a session is a collection of related process groups. The meaning of the term related in each case is explained in the course of this chapter.

Process groups and sessions are abstractions defined to support shell job control, which allows interactive users to run commands in the foreground or in the background. The term job is often used synonymously with the term process group.

This chapter describes process groups, sessions, and job control.

34 Process Groups, Sessions, and Job Control
34.1 Overview
34.2 Process Groups
34.3 Sessions
34.4 Controlling Terminals and Controlling Processes
34.5 Foreground and Background Process Groups
34.6 The SIGHUP Signal
        34.6.1 Handling of SIGHUP by the Shell
        34.6.2 SIGHUP and Termination of the Controlling Process
34.7 Job Control
        34.7.1 Using Job Control within the Shell
        34.7.2 Implementing Job Control
        34.7.3 Handling Job-control Signals
        34.7.4 Orphaned Process Groups (and SIGHUP Revisited)
34.8 Summary
34.9 Exercises