2014-09-30

TLPI fifth print run

The files for the fifth printing of The Linux Programming Interface went to the printer a few weeks ago, and I expect it will soon be available via retailers. Thanks to all those who submitted errata that were incorporated in this print run.

2014-07-23

Linux/UNIX System Programming course scheduled for October

I've scheduled a further 5-day Linux/UNIX System Programming course to take place in Munich, Germany, for the week of 6-10 October 2014.

The course is intended for programmers developing system-level, embedded, or network applications for Linux and UNIX systems, or programmers porting such applications from other operating systems (e.g., Windows) to Linux or UNIX. The course is based on my book, The Linux Programming Interface (TLPI), and covers topics such as low-level file I/O; signals and timers; creating processes and executing programs; POSIX threads programming; interprocess communication (pipes, FIFOs, message queues, semaphores, shared memory),  network programming (sockets), and server design.
     
The course has a lecture+lab format, and devotes substantial time to working on some carefully chosen programming exercises that put the "theory" into practice. Students receive a copy of TLPI, along with a 600-page course book containing the more than 1000 slides that are used in the course. A reading knowledge of C is assumed; no previous system programming experience is needed.

Some useful links for anyone interested in the course:
Questions about the course? Email me via training@man7.org.

2014-06-05

Linux/UNIX System Programming course in July

I've scheduled a further 5-day Linux/UNIX System Programming course to take place in Munich, Germany, for the week of 21-25 July 2014.

The course is intended for programmers developing system-level, embedded, or network applications for Linux and UNIX systems, or programmers porting such applications from other operating systems (e.g., Windows) to Linux or UNIX. The course is based on my book, The Linux Programming Interface (TLPI), and covers topics such as low-level file I/O; signals and timers; creating processes and executing programs; POSIX threads programming; interprocess communication (pipes, FIFOs, message queues, semaphores, shared memory),  network programming (sockets), and server design.
     
The course has a lecture+lab format, and devotes substantial time to working on some carefully chosen programming exercises that put the "theory" into practice. Students receive a copy of TLPI, along with a 600-page course book containing the more than 1000 slides that are used in the course. A reading knowledge of C is assumed; no previous system programming experience is needed.

Some useful links for anyone interested in the course:
Questions about the course? Email me via training@man7.org.

2014-02-19

Further Linux/UNIX System Programming courses scheduled

I've scheduled two further public iterations of my Linux/UNIX System Programming course in Munich, Germany, in April and June, and I hope to announce a San Francisco date soon as well. (I'm also available to give on-demand tailored versions of the course onsite at customer premises, in Europe, the US, and further afield.)
          
The 5-day course is intended for programmers developing system-level, embedded, or network applications for Linux and UNIX systems, or programmers porting such applications from other operating systems (e.g., Windows) to Linux or UNIX. The course is based on my book, The Linux Programming Interface (TLPI), and covers topics such as low-level file I/O; signals and timers; creating processes and executing programs; POSIX threads programming; interprocess communication (pipes, FIFOs, message queues, semaphores, shared memory),  network programming (sockets), and server design.
     
The course has a lecture+lab format, and devotes substantial time to working on some carefully chosen programming exercises that put the "theory" into practice. Students receive a copy of TLPI, along with a 600-page course book containing the more than 1000 slides that are used in the course. A reading knowledge of C is assumed; no previous system programming experience is needed.

Some useful links for anyone interested in the course:
Questions about the course? Email me via training@man7.org.

2014-02-14

System call naming and numbering

Various commentators have remarked that the name of the proposed new renameat2() system call seems inconsistent; see, for example, these comments in an LWN.net article that discusses renameat2(). The idea is that when system calls are numbered in this fashion, the number should reflect the number of arguments, rather than the being a "version" number. Thus, with five arguments, the new system call should be called renameat5(), in the fashion of other system calls such as accept4() and signalfd4().

The truth is that there has been little consistency in the numbering conventions used for new system calls. In a few cases, such numbering has represented successive "versions" of a system call (for example, mmap2() and sync_file_range2()).  In several other cases, the numbering has reflected the argument count.  And in at least one fortuitous case, the numbering has reflected the argument count and the "version" (dup(), dup2(), and dup3()).

The use of "version numbers" in the names of new system calls doesn't have have any strong arguments in its favor. However, the convention of
naming system calls according to the number of arguments seems even worse, for a number of reasons.  Among these are the following:
  • The number of arguments exposed by a system call may differ from the number of arguments exposed by the corresponding wrapper function. For example, the pselect6() system call has six arguments, while the C library pselect() wrapper function has just 5 arguments.
  • A new "version" of a system call might have the same number of arguments as the numbered system call it replaces. That precise situation hasn't occurred so far, but there has been at least one case where a system call named according to its number of arguments has had the same number of arguments as the call it supersedes: epoll_create() and epoll_create1() both have the same number of arguments.
  • Finally, if a system call supports a flags argument, then it is possible that a subsequent enhancement to the system call may allow additional arguments to be specified, depending on the use of special bit values in the flags argument (clone(), fcntl(), mremap(), and open() are existing examples of such variadic system calls). In that case, the naming system no longer bears any relation to the number of arguments in the system call.
The ideal solution, of course, is to avoid the naming problem altogether,
by avoiding the need to create new versions of systems calls, which was the main point of my recent LWN.net article, Flags as a system call API design pattern.

2014-01-22

Simplified Chinese translation of TLPI published

The Simplified Chinese translation of TLPI has just been published by Posts and Telecommunications Press. I'm still waiting to receive my copies, but in the meantime, here's a picture from the publisher's website:


Information about all of the translations of TLPI can be found on my translations page.

2013-09-23

Linux/UNIX System Programming course, Munich, October 2013

I will be running another iteration of my Linux/UNIX System Programming course in Munich, Germany, during the week of 21-25 October 2013. The course is intended for programmers developing system-level, embedded, or network applications for Linux and UNIX systems, or programmers porting such applications from other operating systems (e.g., Windows) to Linux or UNIX. Among the topics covered in the course are low-level file I/O, signals, timers, creating processes and executing programs, programming with POSIX threads, interprocess communication (pipes, FIFOs, message queues, semaphores, shared memory), and network programming using sockets. You can expect to work fairly hard (and also learn a lot) during the week

For detailed information about the content of the course, prerequisites, and other details, see http://man7.org/training/.

In addition to course materials, all participants will receive a copy of my book, The Linux Programming Interface.

The course will be priced (lower than usual) at 2000 euro plus VAT, and the class size will be kept quite small (I've yet to determine if the maximum will be 6 or 8). If you are interested in the course, please email me at training@man7.org and I'll send you information for course registration. Questions regarding the course can be sent to the same address.

2013-07-17

Fourth print run (and request for bug reports)

The publisher will shortly be preparing a fourth print run of The Linux Programming Interface. That print run will incorporate all of the outstanding errata.

If you've been reading TLPI and noticed any errors or typos, now would be a good time to report them, so that fixes can be included in the fourth printing.

2013-05-14

Adding further man pages to the HTML renderings on man7.org

As noted in my last post, I've expanded the set of HTML man page renderings at man7.org/linux/man-pages/ to include some projects other than man-pages. Currently, man pages from 37 projects are now rendered, with about 1750 pages in all. The projects that I have so far included have a bias that matches my interests: man-pages, projects related to low-level C and system programming (e.g., the ACL and extended attribute libraries), toolchain projects (e.g., gcc, gdb, Git, coreutils, binutils, util-linux), and other relevant tools (kmod, strace, ltrace, procps, expect) and tools relevant to manual pages (e.g., groff, man-db).

Although there are some other sites around that have renderings of a much larger set of pages, I am (so far) resisting the temptation to take a kitchen-sink approach on man7.org. Nevertheless, I'm open to adding further projects to the set, if they seem relevant. If you think there is a project that should be added to the rendered set, drop a note to man-pages@man7.org with the following information:
  • Name of the project.
  • Project description.
  • URL for the web site the project.
  • (If you know it:) URL of a web page that provides information on how to report bugs in the man pages (or email list address).
  • Source URL for the man pages of the project. The project should provide pages by one of the following means one of the following:
    • Ideally: the URL of a Git repository for the project.
    • The URL of a Bazaar or Mercurial repository.
    • An HTTP or FTP address for the location that is updated with the latest release tarball on each release of the project.
    • If nothing else: the URL of a CVS or Subversion repository. Note: if there is a Git read-only mirror of the CVS or Subversion repository, that is preferred.
  • Instructions on how to build the man pages for the project. These instructions should be minimal, in the sense that they require the minimum CPU effort to build just the man pages. In other words, if possible, I'd like to avoid building the entire project just to obtain the manual pages.
  • Approximate number of manual pages in the project (actual pages, excluding links).

More man pages now rendered on man7.org

There are several web sites that provide renderings of a comprehensive set of Linux man pages. However, those sites typically have a number of faults.

For example, the renderings are either for out-of-date versions of the man pages (on one site, the rendered pages are close to ten years old) or the pages provide no timestamp information indicating the age of the pages. In many other cases, sites provide no information about the origin of the rendered pages, give no information about the extraction date or the project version from which the manual pages were taken, and provide no information on how to report manual page bugs to the corresponding upstream projects. Providing that information was the main goal when I started adding the COLOPHON sections to the man-pages pages in December 2007 (man-pages-2.69).

Furthermore, the renderings on many sites are either unattractive (obviously, a subjective judgement) or simply broken (for example, it looks like none of the groff tables in the pages at http://linux.die.net/man/ are rendered, so that, for example, the table of systems calls in the syscalls(2) man page does not appear, making the page essentially useless). Finally, most of the sites provide no obvious information on how to report bugs in the man page renderings.

One of the few sites that does a reasonable job on the above criteria is the http://manpages.courier-mta.org/ site maintained by Sam Varshavchik. It is probably no coincidence that Sam has also provided numerous bug reports on formatting issues in the man-pages page set over the last several years. However, Sam provides a rather less comprehensive set of pages than on the other sites, taking pages from just seven projects.

So, it seems there's room out there for a web site that does a better job on many of the above criteria by providing a comprehensive set of page renderings that: (a) are up to date; (b) carefully document the origin of the rendered pages; (c) describe where to report bugs in the man pages; and (d) explain where to report bugs in the renderings.

With those goals in mind, I've extended the set of pages that are rendered at http://man7.org/linux/man-pages/ to include pages in addition to those provided by the man-pages project. Each page includes a COLOPHON section that shows the name and URL of the project from which the page comes, the URL of the tarball or source code repository from which the page was extracted, the date when the page was extracted, and (where I know it) information on where to report bugs in the manual page. So far, I've added about 35 projects to the set, with the pages for each project being taken either from the latest release tarball or directly from the project's source code repository. This raises the number of rendered pages at man7.org from the around 950 pages in man-pages to around 1750 with the addition of the other projects. (More projects may be added in the future; I'll say more on that later.) A full list of the projects and pages that are rendered can be found in the project page directory.

Sometimes different projects provide the same manual page. On all sites that I know of, where such conflicts occur, just one version of the page is rendered. I've dealt with such conflicts in a different way. One of the versions is treated as canonical (here, I've currently followed the lead of Fedora by choosing the page that it treats as canonical, though I may adjust that approach in the future), but I provide renderings of the other versions at different URLs, with cross page links between the various versions. Thus, for example, three of the projects that I handle provide versions of the kill(1) man page, and the three version are rendered at the following URLs:
The "@" syntax in the URLs is used to distinguish the various versions of the page. For completeness, the canonical version of each page also has an equivalent "@" link, so that the util-linux version of the page is also available at http://man7.org/linux/man-pages/man1/kill.1@util-linux.html. The intention is that the "@" URL scheme should be stable so that manual pages from specific projects can be referenced. If anyone sees any problems with the URL scheme http://.../page-name.section@project.html, please let me know (soon): I'll adjust the scheme if necessary.

2013-03-12

Revisiting kernel capability usage statistics

Revisiting my earlier statistics on capability use in the Linux 3.2 kernel source, things are not getting better for  CAP_SYS_ADMIN. The statistics below are for Linux 3.9-rc2. By comparison with Linux 3.2, total uses of CAP_* constants in the kernel sources have risen by 10.6% (1242 versus 1167) and total uses of CAP_SYS_ADMIN have risen by slightly more: 11.1% (502 versus 451). This article remains relevant, and digging a bit deeper, overly broad range seems to be a problem that afflicts not just CAP_SYS_ADMIN and CAP_NET_ADMIN about also (at least) CAP_SYS_RAWIO, as the discussion in this thread on the proposed CAP_COMPROMISE_KERNEL capability shows.

Capability#uses#files
CAP_AUDIT_CONTROL22
CAP_AUDIT_WRITE11
CAP_BLOCK_SUSPEND32
CAP_CHOWN42
CAP_DAC_OVERRIDE21
CAP_DAC_READ_SEARCH53
CAP_FOWNER118
CAP_FSETID97
CAP_IPC_LOCK149
CAP_IPC_OWNER11
CAP_KILL22
CAP_LEASE11
CAP_LINUX_IMMUTABLE1414
CAP_MAC_ADMIN285
CAP_MAC_OVERRIDE52
CAP_MKNOD33
CAP_NET_ADMIN399188
CAP_NET_BIND_SERVICE1512
CAP_NET_BROADCAST00
CAP_NET_RAW2012
CAP_SETFCAP32
CAP_SETGID116
CAP_SETPCAP22
CAP_SETUID94
CAP_SYS_ADMIN502257
CAP_SYS_BOOT22
CAP_SYS_CHROOT22
CAP_SYSLOG22
CAP_SYS_MODULE53
CAP_SYS_NICE148
CAP_SYS_PACCT11
CAP_SYS_PTRACE115
CAP_SYS_RAWIO6943
CAP_SYS_RESOURCE3825
CAP_SYS_TIME1911
CAP_SYS_TTY_CONFIG115
CAP_WAKE_ALARM21
Total1242654

2012-11-18

Japanese translation of TLPI will be published in December

The Japanese translation of TLPI (published by O'Reilly Japan) will come out next month. Japanese readers can read (the Japanese translation of) a short preface that I wrote for the translation.

Although I will never be able to read it, I have a couple of reasons to believe that the translation will be excellent. First, the translation was done by a kernel developer, Junjiro Okajima (developer of the aufs file system). Second, along the way Junjiro has been extremely attentive in reading TLPI and checking details. As a consequence, he contributed a large number of errata reports (making up more than 20% of all current errata reports for TLPI), ranging from typos to corrections and improvements to rather deep technical details presented in the English text.