Understanding and Using Threads in Python

If you do not know what a thread is then we’ll have to take it from the beginning. From an end user's perspective, think about applications that are running as carrying out various tasks. An application is usually nothing but a single process, though sometimes they are made of multiple cooperating processes. A process contains at least one thread and the primary thread is called the main thread. The primary threads can create other threads. All the other threads created later from the main thread live in the same process as the main one.

So, your code is run by a thread, not a process. A process is a container for threads. Application is only conceptual—we call the process or cooperating processes as the application. The flow of your program is governed by threads, not by the process. Process is a container and separation of memory and resources. The operating system sends various signals to the process and one or more threads inside the process can handle that. The OS identifies various open resources by process, not by the threads. A process exits when the main thread of the process returns.

Every threads has its own execution path. So, if you want to create another execution path like your main program you have to create and run another thread, as result threads need their own runtime stacks. You can manually exit from a thread or the thread automatically exits when the callable used to run the thread exits.

Getting Started

To get started with threading in Python you need to have Python installed on your system. You can use any plain text editor, code editor or IDE for writing code. I am going to use Python 3.6 and the IDLE provided with Python installation. Create a python script file named py_thread.py.

To work with threads in Python you need to use the threading module from the Python standard library. To get the current thread object you can call the currentThread() function from the module. To get the current thread name from any thread you can call getName() on the thread object. So, our initial code looks like the following:

import threading

current_th = threading.currentThread()
thread_name = current_th.getName()

print("The name of the current thread is : " + thread_name)

Output:

The name of the current thread is : MainThread

Creating a New Thread Object

There are several ways of creating a new thread. Here I’m discussing the basic and widely used way of creating a thread. A new Thread object can be created by instantiating the Thread class from the threading module. In the constructor you have to pass a callable which can be a pure function or another callable object with the target keyword argument to constructor. If the callable accepts positional parameters then you can pass them as a tuple with the args keyword argument. Keyword arguments can be passed as the kwargs keyword argument as a dictionary. You can specify the name of the thread with the name keyword argument. You can also pass group and daemon keyword arguments.

import threading

def target_fn():
   current_th = threading.currentThread()
   thread_name = current_th.getName()
   print("The name of the current thread is : " + thread_name + "\n")


th1 = threading.Thread(target=target_fn, name="Thread One")
th2 = threading.Thread(target=target_fn, name="Thread Two")
th3 = threading.Thread(target=target_fn, name="Thread Three")

Hit run if everything is alright.

Executing a Thread

Creating a thread only creates a Thread object in the memory. To execute the thread you have to call the start() method on the thread object. This method will not block, it will return immediately. The start() method is used to start the thread activity. This method calls the run() method internally in a different thread of control to call the specified target function to perform your desired tasks.

import threading

def target_fn():
   current_th = threading.currentThread()
   thread_name = current_th.getName()
   print("The name of the current thread is : " + thread_name + "\n")

th1 = threading.Thread(target=target_fn, name="Thread One")
th2 = threading.Thread(target=target_fn, name="Thread Two")
th3 = threading.Thread(target=target_fn, name="Thread Three")

th1.start()
th2.start()
th3.start()

Output:

The name of the current thread is : Thread One

The name of the current thread is : Thread Two

The name of the current thread is : Thread Three

Depending on various situations you may not see this result. Again the result seems to be sequential to the starting of the thread but that is not true. Threads can run in any sequence. If you want to guarantee the sequence of thread executing, you have take some special steps.

Sleep, Join and Parallel Execution of Threads

A thread can wait for some other tasks to be finished. To do so it can go to sleep (other mechanism are available too). The sleep() function from the time module can be used for this purpose. The sleep() function takes seconds as a parameter. You can provide milliseconds by providing fraction of a second as a floating point number. Remember that the sleep() function blocks the running thread.

A thread is run in parallel with other threads including main thread. When the main thread reaches the end, it exits. As a result other threads get killed. Let's try to prove it. We are going to put our newly created three threads to sleep. To do so we can can call sleep() function inside the target function.

import threading
import time

def target_fn():
   current_th = threading.currentThread()
   thread_name = current_th.getName()
   time.sleep(5)
   print("The name of the current thread is : " + thread_name + "\n")


th1 = threading.Thread(target=target_fn, name="Thread One")
th2 = threading.Thread(target=target_fn, name="Thread Two")
th3 = threading.Thread(target=target_fn, name="Thread Three")

th1.start()
th2.start()
th3.start()

Run the script and you will see no result out of your Python program. When the other threads are waiting for few seconds the main thread has reached the end.

Let's put the main thread to sleep for more seconds than others. As a result the main thread will block for that specified time and other threads will get some time to breath and run. Also set different sleeping time for seeing a different result.

import threading
import time

def target_fn(sleeping_time=5):
   current_th = threading.currentThread()
   thread_name = current_th.getName()
   time.sleep(sleeping_time)
   print("The name of the current thread is : " + thread_name + "\n")


th1 = threading.Thread(target=target_fn, name="Thread One", kwargs={'sleeping_time' : 6})
th2 = threading.Thread(target=target_fn, name="Thread Two", kwargs={'sleeping_time' : 1})
th3 = threading.Thread(target=target_fn, name="Thread Three")

th1.start()
th2.start()
th3.start()

time.sleep(10)

Output:

The name of the current thread is : Thread Two

The name of the current thread is : Thread Three

The name of the current thread is : Thread One

The order of execution is different now.

We don’t know how much time other threads will take to perform their task so we cannot blindly put our main thread to sleep for a random number of seconds. We have a better way of serving our purpose. We can call the join() method on the thread objects. The join method will block until the thread completes its tasks.

import threading
import time

def target_fn(sleeping_time=5):
   current_th = threading.currentThread()
   thread_name = current_th.getName()
   time.sleep(sleeping_time)
   print("The name of the current thread is : " + thread_name + "\n")


th1 = threading.Thread(target=target_fn, name="Thread One", kwargs={'sleeping_time' : 6})
th2 = threading.Thread(target=target_fn, name="Thread Two", kwargs={'sleeping_time' : 1})
th3 = threading.Thread(target=target_fn, name="Thread Three")

th1.start()
th2.start()
th3.start()

th1.join()
th2.join()
th3.join()

Hit run to see the result. We did not put the main thread to sleep this time. Still the program is working properly.

Terminating a Thread

A thread cannot be terminated from outside of the thread. A thread can exit itself from inside of the thread or from inside of the target callable or any other code called from within the target callable. To exit from a thread, call the exit() function from the sys module.

import threading
import time
import sys

def target_fn(sleeping_time=5):
   current_th = threading.currentThread()
   thread_name = current_th.getName()
   if thread_name != "Thread Two":
       print("Exiting -> " + thread_name)
       sys.exit()

   time.sleep(sleeping_time)
   print("The name of the current thread is : " + thread_name + "\n")


th1 = threading.Thread(target=target_fn, name="Thread One", kwargs={'sleeping_time' : 6})
th2 = threading.Thread(target=target_fn, name="Thread Two", kwargs={'sleeping_time' : 1})
th3 = threading.Thread(target=target_fn, name="Thread Three")

th1.start()
th2.start()
th3.start()

th1.join()
th2.join()
th3.join()

Output:

Exiting -> Thread OneExiting -> Thread Three

The name of the current thread is : Thread Two

Global Interpreter Lock (GIL)

Global Interpreter Lock or GIL is both a feature and a problem in Python. GIL is a lock that is used to prevent more than one thread to execute Python bytecode at the same time. That means if one thread is running your Python program or code another thread is halted. GIL is both pessimistic and sadistic. It's always a burning topic to remove the lock from the Python interpreter. But at the end of every discussion the result is always negative. The creator of Python Guido van Rossum once said that GIL is always going to stay in the Python interpreter.

After some interval the GIL is released for other threads to use it. If there are more than one thread present in the program one of them acquires the lock according to priority and after the same interval it releases the lock for other threads to use it. GIL is a feature because it helps mitigate the risk of accidentally executing unexpected parts of the program's bytecode at the same time. That means parts of the Python interpreter are not thread safe.

GIL is actually not present in the Python programming language. It is present in the different implementations of the language. The most widely used Python implementation is CPython which is the Python interpreter you download from python.org. It is implemented in C and thus the name is CPython. CPython always had GIL and it will always be there. There are other implementations of Python programming language. PyPy, Jython, IronPython are some of other popular implementations. Jython is Java implementation and it runs on JVM. IronPython is C# implementation of Python and it runs on CLR. These implementations do not have GIL and you can take the full potential of multithreading in Python on these two platforms. These two implementation are not updated often like CPython. If that is a problem for you and you need threads badly then you are out of luck.

GIL was not a problem when we did not have multi-core machines, but in the modern world we all have multi-core machines. Even hexacore mobile devices are widely available now. In the modern world GIL not only a problem, it's a big problem. We cannot take full potential of multicore devices due to GIL. Some people say that we can use process instead of threads. Multiprocess can never be an alternative to multithread. Threads are lightweight and don’t not need separate virtual memory; communication between different threads is much less costly. For a concurrent application threads are a must.

Conclusion

In many real world program, multithreading is a must. Sometimes in our program we have to wait for some other results to come. But in that period our program hangs on one point instead of doing other tasks. To avoid such blocking we can use threads to execute long running or blocking task in parallel with other tasks. When you are comfortable with multithreading described in this article, you should read the official python reference on threading.

Need to brush up on your Python? Check out the top 8 places to learn Python online—from basics to advance.

And stop by the homepage to search and compare thousands of SDK, APIs, and other dev tools.