Many system programs need to be able to allocate extra memory for dynamic data structures (e.g., linked lists and binary trees), whose size depends on information that is available only at run time. In this chapter, we describe the functions that are used to allocate memory on the heap or the stack.
7 Memory Allocation
7.1 Allocating Memory on the Heap
7.1.1 Adjusting the Program Break: brk() and sbrk()
7.1.2 Allocating Memory on the Heap: malloc() and free()
7.1.3 Implementation of malloc() and free()
7.1.4 Other Methods of Allocating Memory on the Heap
7.2 Allocating Memory on the Stack: alloca()
7.3 Summary
7.4 Exercises
2009-08-02
2009-07-31
Chapter 6: Processes
In this chapter, we look at the structure of a process, paying particular attention to the layout and contents of a process’s virtual memory. We also examine some of the attributes of a process. In later chapters, we examine further process attributes (for example, process credentials in Chapter 9, and process priorities and scheduling in Chapter 35). In Chapters 24, 25, 26, and 27, we look at how processes are created, how they terminate, and how they can be made to execute new programs.
6 Processes
6.1 Processes and Programs
6.2 Process ID and Parent Process ID
6.3 Memory Layout of a Process
6.4 Virtual Memory Management
6.5 The Stack and Stack Frames
6.6 Command-line Arguments (argc, argv)
6.7 Environment List
6.8 Performing a Nonlocal Goto: setjmp() and longjmp()
6.9 Summary
6.10 Exercises
6 Processes
6.1 Processes and Programs
6.2 Process ID and Parent Process ID
6.3 Memory Layout of a Process
6.4 Virtual Memory Management
6.5 The Stack and Stack Frames
6.6 Command-line Arguments (argc, argv)
6.7 Environment List
6.8 Performing a Nonlocal Goto: setjmp() and longjmp()
6.9 Summary
6.10 Exercises
Chapters 24 to 28 are in copyedit
Chapters 20 and 21 came back copyedit. Chapters 24 to 28 have gone to copyedit.
2009-07-30
Thanks, LWN
The kind folks at LWN.net picked up on this blog. Given the relatively small amount of publicity I've made so far for the blog, their article easily generated the best day of traffic so far. Thanks, LWN!
Chapter 5: File I/O: Further Details
In this chapter, we extend the discussion of file I/O that we started in the previous chapter.
In continuing the discussion of the open() system call, we explain the concept of atomicity--the notion that the actions performed by a system call are executed as a single uninterruptible step. This is a necessary requirement for the correct operation of many system calls.
We introduce another file-related system call, the multipurpose fcntl(), and show one of its uses: fetching and setting open file status flags.
Next, we look at the kernel data structures used to represent file descriptors and open files. Understanding the relationship between these structures clarifies some of the subtleties of file I/O discussed in subsequent chapters. Building on this model, we then explain how to duplicate file descriptors.
We then consider some system calls that provide extended read and write functionality. These system calls allow us to perform I/O at a specific location in a file without changing the current file offset, and to transfer data to and from multiple buffers in a program.
We briefly introduce the concept of nonblocking I/O, and describe some extensions provided to support I/O on very large files.
Since temporary files are used by many system programs, we’ll also look at some library functions that allow us to create and use temporary files with randomly generated unique names.
5 File I/O: Further Details
5.1 Atomicity and Race Conditions
5.2 File Control Operations: fcntl()
5.3 Open File Status Flags
5.4 Relationship Between File Descriptors and Open Files
5.5 Duplicating File Descriptors
5.6 File I/O at a Specified Offset: pread() and pwrite()
5.7 Scatter-gather I/O: readv() and writev()
5.8 Truncating a File: truncate() and ftruncate()
5.9 Nonblocking I/O
5.10 I/O on Large Files
5.11 The /dev/fd Directory
5.12 Creating Temporary Files
5.13 Summary
5.14 Exercises
In continuing the discussion of the open() system call, we explain the concept of atomicity--the notion that the actions performed by a system call are executed as a single uninterruptible step. This is a necessary requirement for the correct operation of many system calls.
We introduce another file-related system call, the multipurpose fcntl(), and show one of its uses: fetching and setting open file status flags.
Next, we look at the kernel data structures used to represent file descriptors and open files. Understanding the relationship between these structures clarifies some of the subtleties of file I/O discussed in subsequent chapters. Building on this model, we then explain how to duplicate file descriptors.
We then consider some system calls that provide extended read and write functionality. These system calls allow us to perform I/O at a specific location in a file without changing the current file offset, and to transfer data to and from multiple buffers in a program.
We briefly introduce the concept of nonblocking I/O, and describe some extensions provided to support I/O on very large files.
Since temporary files are used by many system programs, we’ll also look at some library functions that allow us to create and use temporary files with randomly generated unique names.
5 File I/O: Further Details
5.1 Atomicity and Race Conditions
5.2 File Control Operations: fcntl()
5.3 Open File Status Flags
5.4 Relationship Between File Descriptors and Open Files
5.5 Duplicating File Descriptors
5.6 File I/O at a Specified Offset: pread() and pwrite()
5.7 Scatter-gather I/O: readv() and writev()
5.8 Truncating a File: truncate() and ftruncate()
5.9 Nonblocking I/O
5.10 I/O on Large Files
5.11 The /dev/fd Directory
5.12 Creating Temporary Files
5.13 Summary
5.14 Exercises
2009-07-28
Chapter 4: File I/O: The Universal I/O Model
With this chapter, we start to look in earnest at the system call API. We start with files, since they are central to the Unix philosophy. The focus of this chapter is the system calls used for performing file input and output.
We begin by introducing the concept of a file descriptor, and then look at the system calls that constitute the so-called universal I/O model. These are the system calls that open and close a file, and read and write data. In addition, we look at the system call that is used to seek to a random location in a file.
We focus on I/O on disk files. However, much of the material covered here is relevant for later chapters, since the same system calls are used for performing I/O on all types of files, such as pipes and terminals.
We continue the discussion of file I/O with further details in Chapter 5. One other aspect of file I/O, buffering, is complex enough to deserve its own chapter. In Chapter 13, we cover I/O buffering in the kernel and in the stdio library.
4 File I/O: The Universal I/O Model
4.1 Overview
4.2 Universality of I/O
4.3 Opening a File: open()
4.4 Reading from a File: read()
4.5 Writing to a File: write()
4.6 Closing a File: close()
4.7 Changing the Current File Offset: lseek()
4.8 Operations Outside the Universal I/O Model: ioctl()
4.9 Summary
4.10 Exercises
We begin by introducing the concept of a file descriptor, and then look at the system calls that constitute the so-called universal I/O model. These are the system calls that open and close a file, and read and write data. In addition, we look at the system call that is used to seek to a random location in a file.
We focus on I/O on disk files. However, much of the material covered here is relevant for later chapters, since the same system calls are used for performing I/O on all types of files, such as pipes and terminals.
We continue the discussion of file I/O with further details in Chapter 5. One other aspect of file I/O, buffering, is complex enough to deserve its own chapter. In Chapter 13, we cover I/O buffering in the kernel and in the stdio library.
4 File I/O: The Universal I/O Model
4.1 Overview
4.2 Universality of I/O
4.3 Opening a File: open()
4.4 Reading from a File: read()
4.5 Writing to a File: write()
4.6 Closing a File: close()
4.7 Changing the Current File Offset: lseek()
4.8 Operations Outside the Universal I/O Model: ioctl()
4.9 Summary
4.10 Exercises
2009-07-27
Chapter 3: System Programming Concepts
This chapter covers various topics that are prerequisites for system programming. We begin by introducing system calls and detailing the steps that occur during their execution. We then consider library functions and how they differ from system calls, and couple this with a description of the (GNU) C library.
Whenever we make a system call or call a library function, we should always check the return status of the call in order to determine if it was successful. We describe how to perform such checks, and present a set of functions that are used in most of the example programs in this book to diagnose errors from system calls and library functions.
We conclude by looking at various issues related to portable programming, specifically the use of feature test macros and the standard system data types defined by SUSv3.
3 System Programming Concepts
3.1 System Calls
3.2 Library Functions
3.3 The Standard C Library; The GNU C Library (glibc)
3.4 Handling Errors from System Calls and Library Functions
3.5 Notes on the Example Programs in This Book
3.5.1 Command-line Options and Arguments
3.5.2 Common Functions and Header Files
3.6 Portability Issues
3.6.1 Feature Test Macros
3.6.2 System Data Types
3.6.3 Miscellaneous Portability Issues
3.7 Summary
3.8 Exercises
Whenever we make a system call or call a library function, we should always check the return status of the call in order to determine if it was successful. We describe how to perform such checks, and present a set of functions that are used in most of the example programs in this book to diagnose errors from system calls and library functions.
We conclude by looking at various issues related to portable programming, specifically the use of feature test macros and the standard system data types defined by SUSv3.
3 System Programming Concepts
3.1 System Calls
3.2 Library Functions
3.3 The Standard C Library; The GNU C Library (glibc)
3.4 Handling Errors from System Calls and Library Functions
3.5 Notes on the Example Programs in This Book
3.5.1 Command-line Options and Arguments
3.5.2 Common Functions and Header Files
3.6 Portability Issues
3.6.1 Feature Test Macros
3.6.2 System Data Types
3.6.3 Miscellaneous Portability Issues
3.7 Summary
3.8 Exercises
2009-07-25
Chapter 2: Fundamental Concepts
This chapter introduces a range of concepts related to Linux system programming. It is intended for readers who have worked with other operating systems, or who have only limited experience with Linux or another Unix implementation.
2 Fundamental Concepts
2.1 The Core Operating System: The Kernel
2.2 The Shell
2.3 Users and Groups
2.4 Single Directory Hierarchy, Directories, Links, and Files
2.5 File I/O Model
2.6 Programs
2.7 Processes
2.8 Memory Mappings
2.9 Static and Shared Libraries
2.10 Interprocess Communication and Synchronization
2.11 Signals
2.12 Threads
2.13 Process Groups and Shell Job Control
2.14 Sessions, Controlling Terminals, and Controlling Processes
2.15 Pseudoterminals
2.16 Date and Time
2.17 Client-server Architecture
2.18 Realtime
2.19 The /proc File System
2.20 Summary
2 Fundamental Concepts
2.1 The Core Operating System: The Kernel
2.2 The Shell
2.3 Users and Groups
2.4 Single Directory Hierarchy, Directories, Links, and Files
2.5 File I/O Model
2.6 Programs
2.7 Processes
2.8 Memory Mappings
2.9 Static and Shared Libraries
2.10 Interprocess Communication and Synchronization
2.11 Signals
2.12 Threads
2.13 Process Groups and Shell Job Control
2.14 Sessions, Controlling Terminals, and Controlling Processes
2.15 Pseudoterminals
2.16 Date and Time
2.17 Client-server Architecture
2.18 Realtime
2.19 The /proc File System
2.20 Summary
2009-07-24
LCA 2010 Proposals
I've submitted a talk and some tutorial proposals for LCA 2010, which takes place 18-23 January in Wellington, New Zealand. The tutorials are all related to subjects in my book. I'll get to know whether any proposals got accepted in September. Either way, I'm hoping to be at LCA 2010.
2009-07-23
Chapter 23 is in copyedit
One more chapter sent of to the publisher, to make sure my copyeditor does not run out of things to do.
Chapters 20, 21, and 22 are in copyedit
Chapters 17, 18, and 19 came back copyedit. The copyeditor has chapters 20, 21, and 22 now.
2009-07-22
Chapter 1: History and Standards
Chapter 1 is in two main parts. The first part presents short histories of Unix, the C programming language, the GNU project, and the Linux kernel. The second part looks at the various standards that have evolved as C and Unix grew more popular.
Here's the chapter ToC:
1 History and Standards
1.1 A Brief History of Unix and C
1.2 A Brief History of Linux
1.2.1 The GNU Project
1.2.2 The Linux Kernel
1.3 Standardization
1.3.1 The C Programming Language
1.3.2 The First POSIX Standards
1.3.3 X/Open Company and The Open Group
1.3.4 SUSv3 and POSIX.1-2001
1.3.5 SUSv4 and POSIX.1-2008
1.3.6 Unix Standards Timeline
1.3.7 Implementation Standards
1.3.8 Linux, Standards, and the Linux Standard Base
1.4 Summary
Here's the chapter ToC:
1 History and Standards
1.1 A Brief History of Unix and C
1.2 A Brief History of Linux
1.2.1 The GNU Project
1.2.2 The Linux Kernel
1.3 Standardization
1.3.1 The C Programming Language
1.3.2 The First POSIX Standards
1.3.3 X/Open Company and The Open Group
1.3.4 SUSv3 and POSIX.1-2001
1.3.5 SUSv4 and POSIX.1-2008
1.3.6 Unix Standards Timeline
1.3.7 Implementation Standards
1.3.8 Linux, Standards, and the Linux Standard Base
1.4 Summary
Subscribe to:
Posts (Atom)
