The book "Linux API. Comprehensive Guide »

Hi, Habrozhiteli! Recently, we published the fundamental work of Michael Kerriska on the software interfaces of the Linux operating system. The book provides almost a complete description of the system programming API running Linux.

We will now look at the “Execution Flows: Introduction” section. We will first take a look at the overview of how threads work, and then focus on how they are created and completed. In the end, several factors will be considered that should be considered when choosing between two different approaches to designing applications - multi-threaded and multi-process.

29.1. Short review

By analogy with processes, execution threads are a mechanism for simultaneous execution of several parallel tasks within a single application. As shown in fig. 29.1, a single process may contain multiple threads. All of them are executed within a single program independently of each other, sharing common global memory — including initialized / uninitialized data and heap segments (the traditional UNIX process is just a special case of a multi-threaded process; it consists of a single thread).
')
In fig. 29.1 some simplifications are allowed. In particular, the location of stacks for each of the threads may intersect with shared libraries and shared memory locations; this depends on the order in which the threads were created, the libraries were loaded, and the common chunks were added. In addition, the location of the stacks for threads may vary depending on the Linux distribution.

Threads in the process can run simultaneously. In multiprocessor systems, parallel execution of threads is possible. If one thread is blocked due to I / O, another can continue to work (sometimes it makes sense to create a separate thread that deals solely with I / O, although alternative I / O models are often more appropriate; for more details see chapter 59) .

In some situations, threads have an advantage over processes. Consider the traditional UNIX approach to ensuring competitive execution by creating multiple processes. Take, for example, the network server model in which the parent process accepts incoming connections and creates separate child processes using the fork () call to communicate with each client (see section 56.3). This allows you to simultaneously serve multiple connections. Such an approach usually shows itself well, but in some situations it leads to the following limitations.

• The exchange of information between processes has its own difficulties. Since the parent and the child do not share the memory (in addition to the text segment, intended solely for reading), we are forced to use some form of interprocess communication to exchange data.
• Creating a process using fork () consumes relatively many resources. Even if we use the copy-on-write method described in section 24.2.2, we still have to duplicate various process attributes, such as tables with memory pages and file descriptors; this means that the fork () call still takes a significant amount of time. Threads help get rid of both of these problems.
• The exchange of information between threads is simple and fast. To do this, just need to copy the data to common variables (global or heap). But in order to avoid problems that may arise in a situation where several threads try to update the same information at once, one has to apply the synchronization methods described in Chapter 30.
• Creating a stream takes less time than creating a process - usually we have at least a tenfold performance gain (in Linux, the threads are implemented using the clone () system call; the speed differences between it and the fork () call are shown in Table 28.3). The process of creating a stream is faster because many attributes are simply separated instead of directly copied, as in the case of fork (). In particular, there is no need to duplicate pages of memory (by copying while writing) and tables with pages.

In addition to global memory, threads also share a number of other attributes (this is when attributes are global for the whole process, and not for individual threads). Among them are the attributes listed below.

- Identifiers of the process and its parent.
- Process group and session identifiers.
- Managing terminal.
- Process credentials (user and group identifiers).
- Open file descriptors.
- Record locks created by calling fcntl ().
- Signal actions.
- Information relating to the file system: umask, current and root directory.
- Interval timers (setitimer ()) and POSIX timers (timer_create ()).
- The semaphore cancellation values ​​(semadj) in System V.
- Restrictions on resources.
- CPU time consumed (derived from times ()).
- consumed resources (obtained from getrusage ()).
- The nice value (set with setpriority () and nice ()).

The following are attributes that are unique to each individual stream:

- Stream ID (see section 29.5).
- Mask signal.
- Data related to a specific flow (see section 31.3).
- Alternative signal stack (sigaltstack ()).
- Variable errno.
- Floating point settings (see env (3)).
- Policy and priority planning in real time (see sections 35.2 and 35.3).
- Binding to the CPU (applies only to Linux, described in section 35.4).
- Features (applies only to Linux, described in chapter 39).
- Stack (local variables and function call layout information).

As can be seen in fig. 29.1, all stacks related to individual threads are located within the same virtual address space. This means that threads, having suitable pointers, can exchange data through each other's stacks. This is convenient, but requires caution when writing code to resolve the dependency arising from the fact that a local variable remains valid only for the lifetime of the stack in which it is located (if the function returns a value, the memory used by its stack can be re-enabled during the subsequent function call; if the thread terminates, the section of memory in which its stack was located is formally available to another thread). Incorrect work with this dependency can lead to errors that will be difficult to track.

29.2. Pthreads API Overview

In the late 1980s and early 1990s, there were several different software interfaces for working with threads. In 1995, the POSIX threads API was described in the POSIX.1 standard, which later became part of SUSv3. The Pthreads software interface is based on several concepts. We will get acquainted with them, considering in detail its implementation.

Pthreads data types
The Pthreads program interface defines a number of data types, some of which are listed in Table. 29.1. Most of them will be described on the following pages.


The SUSv3 standard does not contain details on how exactly these types of data should be presented, so portable applications should consider them opaque. This means that the program should not depend on the structure or contents of variables of any of these types. In particular, we cannot compare such variables using the == operator.

Threads and errno
In the traditional UNIX programming interface, the errno variable is global and integer. However, this is not enough for multi-threaded programs. If a thread calls a function that writes an error to the global variable errno, this can be confusing to other threads, which also call functions and check the value of errno. In other words, the result will be a race condition. Thus, in multi-threaded programs, each thread has its own separate errno instance. In Linux (and in most UNIX implementations), approximately one approach is used: errno is declared as a macro that expands into a function call that returns a variable value of the lvalue type that is unique to each individual thread (since the lvalue value is editable, we we can still write assignment operations like errno = value in multi-threaded programs.

Summarizing the above: the errno mechanism was integrated into threads so that the error notification procedure is fully consistent with the traditional approach used in UNIX program interfaces.

The value returned by functions in pthreads
Traditionally, system calls and some functions return 0 if successful and –1 if an error occurred. To indicate the error itself, the variable errno is used. Functions in the Pthreads API behave differently. If successful, the return value is 0, but a positive value is used if an error occurs. This is one of those values ​​that can be assigned to the errno variable in traditional UNIX system calls.

Since each reference to errno in a multi-threaded program incurs the overhead associated with calling a function, our program does not assign the value returned by a function from the Pthreads structure directly to this variable. Instead, we use the intermediate variable and our diagnostic function errExitEN () (see subsection 3.5.2), as shown below:

pthread_t *thread; int s; s = pthread_create(&thread, NULL, func, &arg); if (s != 0) errExitEN(s, "pthread_create"); 

Compiling Pthreads based programs
On Linux, programs that use the Pthreads API should be compiled with the cc -pthread parameter. Among the actions of this parameter are the following.

• The preprocessor macro _REENTRANT is declared. This allows access to declarations of several reentrant (reintroduced) functions.
• The program is linked with the libpthread library (equivalent to the -lpthread parameter).

The specific compilation options for multi-threaded programs vary depending on the implementation (and compiler). Some systems (such as Tru64) also use cc -pthread, although Solaris and HP-UX use the cc -mt option.

29.3. Creating threads

Immediately after starting the program, the final process consists of one stream, which is called the source or main stream. In this section, you will learn how to create additional threads.

To create a new thread, use the pthread_create () function.


The new thread starts execution by calling the function specified as the start value and taking the arg argument (that is, start (arg)). The thread that called pthread_create () continues to work by executing the instruction following this call (this corresponds to the behavior of the wrapper function around the clone () system call from the glibc library described in section 28.2).

The arg argument is declared as void *. This means that we can pass to the start function a pointer to an object of any type. It usually points to a variable in global space or in a heap, but we can also use the value NULL. If we need to pass several arguments to the start function, we can provide a pointer to the structure containing these arguments as separate fields as arg. We can even specify arg as an integer (int) using the appropriate type conversion.

Strictly speaking, C standards do not describe the results of casting int to void * and vice versa. However, most compilers allow this operation and generate a predictable result — that is, int j == (int) ((void *) j).

The value returned by the start function is also of type void * and can be interpreted as the arg argument. Below, when considering the pthread_join () function, you will learn how this value is used.

Care should be taken when casting the value returned by the initial stream function to an integer. The point is that the PTHREAD_CANCELED value returned when the stream is canceled (see Chapter 32) is usually implemented as an integer number converted to the void * type. If the initial function returns this value, another thread executing pthread_join () will mistakenly take it as a cancellation notification. In applications that allow cancellation of streams and use integers as values ​​returned from the initial functions, it is necessary to ensure that in the streams ending normally, these values ​​do not match the constant PTHREAD_CANCELED (whatever it is in the current implementation Pthreads). Portable applications should do the same, but with all the implementations they can work with.

The thread argument points to a buffer of type pthread_t, in which, before returning the function pthread_create (), writes the unique identifier of the thread created. With this identifier you can refer to this thread in future calls to Pthreads.

The SUSv3 standard states separately that the buffer pointed to by thread does not need to be initialized before starting a new thread. That is, the new thread can start working before the pthread_create () function returns. If a new thread needs to get its own identifier, it must use the pthread_self () function (described in section 29.5) for this.

The attr argument is a pointer to the pthread_attr_t object, which contains the various attributes of the new thread (to which we will return in section 29.8). If attr is set to NULL, the stream will be created with default attributes — this is what we will do in most of the examples in this book.

The program does not know to which thread the scheduler will allocate processor time after calling pthread_create () (in multiprocessor systems, both threads can run simultaneously on different CPUs). Programs that explicitly rely on a specific planning procedure are subject to the same kinds of race conditions as described in section 24.4. If we need to guarantee a particular order of execution, we must use one of the synchronization methods discussed in Chapter 30.

29.4. Thread termination

Flow execution stops for one of the following reasons.

• The initial function executes the return statement, specifying the return value for the stream.
• The thread calls the pthread_exit () function (described below).
• The thread is canceled using the pthread_cancel () function (described in section 32.1).
• Any of the threads calls exit () or the main thread executes a return statement (inside the main () function), which causes all threads in the process to terminate immediately.

The pthread_exit () function terminates the calling thread and specifies the return value that can be obtained from another thread using the pthread_join () function.

 include <pthread.h> void pthread_exit(void *retval); 

Calling pthread_exit () is equivalent to executing a return statement within the initial function of a thread, except that pthread_exit () can be called from any code that is started by the initial function.

The retval argument stores the value returned by the stream. The value pointed to by retval should not be on the stack of the thread itself, since after the end of the pthread_exit () call, its contents become undefined (this portion of the process’s virtual memory can be immediately allocated to the stack for the new thread). The same applies to the value passed with the return statement in the initial function.

If the main thread calls pthread_exit () instead of exit () or a return statement, the other threads will continue.

29.5. Thread identifiers

Each thread within the process has its own unique identifier. It is returned to the calling thread by the pthread_create () function. In addition, using the pthread_self () function, a thread can get its own identifier.

 include <pthread.h> pthread_t pthread_self(void); 

Returns the ID of the calling thread.

The thread identifiers inside the application can be used as follows.

• Involved by various Pthreads functions to determine which thread they are running. Examples include the functions pthread_join (), pthread_detach (), pthread_cancel () and pthread_kill (); they are all described in this and subsequent chapters.
• In some applications, it may make sense to tag dynamic data structures with the identifier of a particular stream. So we can determine the creator and "owner" of the structure; it also allows you to define a stream that must perform some subsequent actions with the data structure.

The pthread_equal () function allows to check for two identities of threads for identity.


For example, to check whether the caller's identifier matches the value stored in the tid variable, you can write the following code:

 if (pthread_equal(tid, pthread_self()) printf("tid matches self\n"); 

The need for the pthread_equal () function arises from the fact that the pthread_t data type must be perceived as opaque. In Linux, it is of type unsigned long, but on other systems it can be a pointer or a structure.

In the NPTL library, pthread_t really is a pointer that is cast to an unsigned long type.

The SUSv3 standard does not require the pthread_t type to be scalar. It may be a structure. Thus, the code for outputting the stream identifier presented above is not portable (although it works on many systems, including Linux, and can be useful during debugging):

 pthread_t thr; printf("Thread ID = %ld\n", (long) thr); /* ! */ 

In Linux, thread IDs are unique to all processes. However, on other systems this may not be the case. The SUSv3 standard states separately that portable applications cannot rely on these identifiers to define threads in other processes. It also indicates that stream libraries can reuse these identifiers after attaching a completed stream with the pthread_join () function or after the disconnected thread has completed (the pthread_join () function will be discussed in the next section, and the disconnected streams in section 29.7).

The POSIX and regular thread identifiers returned by the gettid () system call (available only on Linux) are not the same thing. The POSIX flow identifier is assigned and maintained by the thread library implementation. The regular thread identifier is returned by a call to gettid () and is a number (similar to the process identifier) ​​that is assigned by the kernel. And although the NPTL library implementation uses unique thread identifiers issued by the kernel, applications often do not need to be aware of them (besides, working with them makes it impossible for applications to be portable between different systems).

29.6. Joining a completed stream

The pthread_join () function waits for the end of the thread indicated by the thread argument (if the thread has already terminated, it returns immediately). This operation is called accession.


If the retval argument is a non-zero pointer, the function receives a copy of the return value of the completed stream, that is, the value specified when the thread executed a return statement or a call to pthread_exit ().

A call to the pthread_join () function for the identifier of an already attached thread can lead to unpredictable consequences; for example, we can join a thread that was created later and reuses the same identifier.

If the thread is not disconnected (see section 29.7), we must join it with the pthread_join () function. If we fail to do this, the completed stream will become an analogue of the “zombie” process (see section 26.2). In addition to the waste of resources, this can lead to the fact that we can no longer create new threads (in the event that a sufficient number of zombie flows accumulate).

The procedure that the pthread_join () function performs on threads is similar to the action of calling waitpid () in the context of processes. But there are noticeable differences between them.

If the retval argument is a non-zero pointer, the function receives a copy of the return value of the completed stream, that is, the value specified when the thread executed a return statement or pthread_exit ().

A call to the pthread_join () function for the identifier of an already attached thread can lead to unpredictable consequences; for example, we can join a thread that was created later and reuses the same identifier.

If the thread is not disconnected (see section 29.7), we must join it with the pthread_join () function. If we fail to do this, the completed stream will become an analogue of the “zombie” process (see section 26.2). In addition to the waste of resources, this can lead to the fact that we can no longer create new threads (in the event that a sufficient number of zombie flows accumulate).

The procedure that the pthread_join () function performs on threads is similar to the action of calling waitpid () in the context of processes. But there are noticeable differences between them.

• Threads do not have a hierarchy. Any thread in the process can use the pthread_join () function to join another thread in the same process. Imagine, for example, that flow A created flow B, which later created flow C; in this case, flow A may join flow B and vice versa. This is different from the hierarchical relationships between processes. If the parent process created a descendant using fork (), he and he alone can wait for that descendant using the wait () call. There is no such relationship between the thread that calls the pthread_create () function and the thread that is being created.
• It is impossible to join “any thread” (in the case of processes, we can do this by calling waitpid (–1, & status, options)); there is also no non-blocking join operation (similar to the waitpid () call with the WNOHANG flag). Similar results can be achieved using conditional variables; An example of this will be shown in subsection 30.2.4.

The restriction related to the fact that the pthread_join () function can attach threads only if there is a specific identifier was created intentionally. The idea is that the program should only join those threads that it "knows" about. The problem of “joining an arbitrary stream” arises from the fact that streams do not have a hierarchy, so in this way we could really join any stream, including the private one created by the library function (using conditional variables described in section 30.2 .4, allows joining only to known streams). As a result, the library would no longer be able to join this stream to gain its status, and attempts to join the already attached stream would lead to errors. In other words, the operation of “joining an arbitrary stream” is incompatible with the modular architecture of the application.

Sample program
The program shown in Listing 29.1 creates a new thread and joins it.

Listing 29.1. A simple program using the Pthreads library

 ______________________________________________________________threads/simple_thread c #include <pthread.h> #include "tlpi_hdr.h" static void * threadFunc(void *arg) { char *s = (char *) arg; printf("%s", s); return (void *) strlen(s); } int main(int argc, char *argv[]) { pthread_t t1; void *res; int s; s = pthread_create(&t1, NULL, threadFunc, "Hello world\n"); if (s != 0) errExitEN(s, "pthread_create"); printf("Message from main()\n"); s = pthread_join(t1, &res); if (s != 0) errExitEN(s, "pthread_join"); printf("Thread returned %ld\n", (long) res); exit(EXIT_SUCCESS); } ______________________________________________________________threads/simple_thread.c 

Running this program, we will see the following:

 $ ./simple_thread Message from main() Hello world Thread returned 12 

The order of output of the first two lines depends on how the scheduler uses two threads.

29.7. Disconnect thread

By default, threads are joinable; this means that after completion, their status can be obtained from another thread using the pthread_join () function. Sometimes the status returned by the stream does not matter; we just need the system to automatically free up resources and delete the stream when it is finished. In this case, we can mark the thread as disconnected by using the pthread_detach () function and specifying the thread identifier in the thread argument.


As an example of using the pthread_detach () function, we can use the following call, in which the thread disconnects itself:

 pthread_detach(pthread_self()); 

If the thread has already been disconnected, we can no longer get its return status using the pthread_join () function. We also cannot make it rejoinable again.

A disconnected thread does not become resistant to an exit () call made in another thread or to a return statement executed in the main program. In any of these situations, all threads within the process are immediately terminated, regardless of whether they are attached or not. In other words, the pthread_detach () function is simply responsible for the behavior of the thread after its completion, but not for the circumstances in which it terminates.

29.8. Stream Attributes

As mentioned earlier, the attr argument of the pthread_create () function, which is of type pthread_attr_t, can be used to set attributes that are used when creating a new thread. We will not go into the consideration of these attributes (for details, look for them in the links listed at the end of the chapter) or study the prototypes of various Pthreads functions that allow you to work with the pthread_attr_t object. We simply note that these attributes contain information such as the location and size of the flow stack, its scheduling policy and priority (this is similar to real-time scheduling policies and process priorities described in sections 35.2 and 35.3), as well as information about , whether the stream is attached or disconnected.

An example of using these attributes is shown in Listing 29.2, where a thread is created that is disconnected at the time of its appearance (and not as a result of a subsequent call to pthread_detach ()). At the very beginning, this code initializes the structure with attributes using default values, then sets the attributes necessary to create the detached stream, and then creates a new stream using this structure. At the end of the creation procedure, an object with attributes is deleted as unnecessary.

Listing 29.2. Creating a stream with a “disconnecting” attribute

 __________________________________________________   threads/detached_attrib.c #include <pthread.h> #include "tlpi_hdr.h" static void * threadFunc(void *x) { return x; } int main(int argc, char *argv[]) { pthread_t thr; pthread_attr_t attr; int s; s = pthread_attr_init(&attr); /*     */ if (s != 0) errExitEN(s, "pthread_attr_init"); s = pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED); if (s != 0) errExitEN(s, "pthread_attr_setdetachstate"); s = pthread_create(&thr, &attr, threadFunc, (void *) 1); if (s != 0) errExitEN(s, "pthread_create"); s = pthread_attr_destroy(&attr); /*    */ if (s != 0) errExitEN(s, "pthread_attr_destroy"); s = pthread_join(thr, NULL); if (s != 0) errExitEN(s, "pthread_join failed as expected"); exit(EXIT_SUCCESS); } __________________________________________________   threads/detached_attrib.c 

29.9. Comparing threads and processes

In this section, we briefly consider several factors that should be considered when choosing between threads and processes as the basis for your application. Let's start by discussing the benefits of a multi-threaded approach.

• Sharing data between threads is easy. The same data exchange, but between processes requires more costs (for example, creating a common memory segment or using a pipeline).
• Creating a thread takes less time than creating a process; threads can also benefit in terms of context switching speeds. However, threads have certain drawbacks compared to processes.
• , , ( 31.1). .
• (, ) , . , .
• Each thread competes for the ability to use the final address space of its process. In particular, the stack and local storage of each thread consumes part of the process's virtual memory, which makes it inaccessible to other threads. Although the address space is quite large (for example, 3 GB for the x86-32 architecture), this factor can be a significant limitation, which hinders the process of creating a large number of threads or threads that require a lot of memory. Separate processes, on the other hand, can involve the entire range of free virtual memory, limited only by the size of the RAM and the paging space.

The following are additional points that may affect the choice between threads and processes.

• ( , ). 33.2.
• (, ). .
• , (, , , , () ). , , .

29.10. Summary

In multithreaded processes in the same program simultaneously run different threads. All of them have common global variables and a bunch, but each of them has its own separate stack for local variables. Different threads of the same process also share a number of attributes, including the process ID, open file descriptor, signal actions, current directory, and resource limits.

A key feature of threads is the simpler exchange of information compared to processes; for this reason, some software architectures are better placed on a multi-threaded approach than on a multiprocess one. In addition, in some situations, threads may exhibit better performance (for example, a thread is created faster than a process), but this factor is usually secondary when choosing between threads and processes.

Threads are created using the pthread_create () function. Any thread can complete independently of the others, using the pthread_exit () function (if you call exit () on any of the threads, all of them will be completed immediately). If a thread has not been marked as disconnected (for example, by calling pthread_detach ()), it must be connected by another thread through the pthread_join () function, which returns the exit code of the attached thread.

»More information about the book can be found on the publisher's website.
» Table of Contents
» Excerpt

For Habrozhiteley 20% discount coupon - Linux