Chapter 13: Concurrency & Parallelism

Objectives

• Use threading.Thread and Lock for simple threads.
• Leverage multiprocessing.Process and pools for CPU-bound work.
• Employ concurrent.futures for a unified interface.
• Write asyncio coroutines with async/await.
• Understand the Python GIL and when to use threads vs processes.

1. threading

Spawn lightweight threads:

import threading

def worker(n):
    print(f"Thread {n} starting")
    # simulate work
    import time; time.sleep(0.5)
    print(f"Thread {n} done")

threads = []
for i in range(3):
    t = threading.Thread(target=worker, args=(i,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

Protect shared data with a Lock:

lock = threading.Lock()
counter = 0

def safe_increment():
    global counter
    with lock:
        temp = counter
        temp += 1
        counter = temp

2. multiprocessing

Run code in separate processes:

from multiprocessing import Process, Pool

def f(x):
    return x*x

if __name__ == "__main__":
    # individual processes
    procs = [Process(target=worker, args=(i,)) for i in range(2)]
    for p in procs:
        p.start()
    for p in procs:
        p.join()

    # pool of workers
    with Pool(4) as pool:
        print(pool.map(f, [1,2,3,4]))

Multiprocessing bypasses the GIL for CPU-bound tasks.

3. concurrent.futures

Unified high-level API:

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def square(x): return x*x

with ThreadPoolExecutor(max_workers=3) as executor:
    results = list(executor.map(square, range(5)))
    print("Threads:", results)

with ProcessPoolExecutor(max_workers=3) as executor:
    results = list(executor.map(square, range(5)))
    print("Processes:", results)

4. asyncio

Write asynchronous coroutines:

import asyncio

async def say_after(delay, msg):
    await asyncio.sleep(delay)
    print(msg)

async def main():
    task1 = asyncio.create_task(say_after(1, "hello"))
    task2 = asyncio.create_task(say_after(2, "world"))
    print("Started tasks")
    await task1
    await task2

asyncio.run(main())

Use asyncio.gather() to run coroutines concurrently.

Exercises

1. Write a threaded counter that increments a shared variable 10 000 times safely.
2. Compute factorials of numbers 1–10 in parallel using a process pool.
3. Use ThreadPoolExecutor to fetch URLs concurrently (use requests).
4. Write an asyncio TCP echo server and client.