
Mastering Concurrency in Python
Description
Alles über E-Books | Antworten auf Fragen rund um E-Books, Kopierschutz und Dateiformate finden Sie in unserem Info- & Hilfebereich.
Python is one of the most popular programming languages, with numerous libraries and frameworks that facilitate high-performance computing. Concurrency and parallelism in Python are essential when it comes to multiprocessing and multithreading; they behave differently, but their common aim is to reduce the execution time. This book serves as a comprehensive introduction to various advanced concepts in concurrent engineering and programming.
Mastering Concurrency in Python starts by introducing the concepts and principles in concurrency, right from Amdahl's Law to multithreading programming, followed by elucidating multiprocessing programming, web scraping, and asynchronous I/O, together with common problems that engineers and programmers face in concurrent programming. Next, the book covers a number of advanced concepts in Python concurrency and how they interact with the Python ecosystem, including the Global Interpreter Lock (GIL). Finally, you'll learn how to solve real-world concurrency problems through examples.
By the end of the book, you will have gained extensive theoretical knowledge of concurrency and the ways in which concurrency is supported by the Python language
More details
Other editions
Additional editions

Content
- Cover
- Title Page
- Copyright and Credits
- Dedication
- About Packt
- Contributors
- Table of Contents
- Preface
- Chapter 1: Advanced Introduction to Concurrent and Parallel Programming
- Technical requirements
- What is concurrency?
- Concurrent versus sequential
- Example 1 - checking whether a non-negative number is prime
- Concurrent versus parallel
- A quick metaphor
- Not everything should be made concurrent
- Embarrassingly parallel
- Inherently sequential
- Example 2 - inherently sequential tasks
- I/O bound
- The history, present, and future of concurrency
- The history of concurrency
- The present
- The future
- A brief overview of mastering concurrency in Python
- Why Python?
- Setting up your Python environment
- General setup
- Downloading example code
- Summary
- Questions
- Further reading
- Chapter 2: Amdahl's Law
- Technical requirements
- Amdahl's Law
- Terminology
- Formula and interpretation
- The formula for Amdahl's Law
- A quick example
- Implications
- Amdahl's Law's relationship to the law of diminishing returns
- How to simulate in Python
- Practical applications of Amdahl's Law
- Summary
- Questions
- Further reading
- Chapter 3: Working with Threads in Python
- Technical requirements
- The concept of a thread
- Threads versus processes
- Multithreading
- An example in Python
- An overview of the threading module
- The thread module in Python 2
- The threading module in Python 3
- Creating a new thread in Python
- Starting a thread with the thread module
- Starting a thread with the threading module
- Synchronizing threads
- The concept of thread synchronization
- The threading.Lock class
- An example in Python
- Multithreaded priority queue
- A connection between real-life and programmatic queues
- The queue module
- Queuing in concurrent programming
- Multithreaded priority queue
- Summary
- Questions
- Further reading
- Chapter 4: Using the with Statement in Threads
- Technical requirements
- Context management
- Starting from managing files
- The with statement as a context manager
- The syntax of the with statement
- The with statement in concurrent programming
- Example of deadlock handling
- Summary
- Questions
- Further reading
- Chapter 5: Concurrent Web Requests
- Technical requirements
- The basics of web requests
- HTML
- HTTP requests
- HTTP status code
- The requests module
- Making a request in Python
- Running a ping test
- Concurrent web requests
- Spawning multiple threads
- Refactoring request logic
- The problem of timeout
- Support from httpstat.us and simulation in Python
- Timeout specifications
- Good practices in making web requests
- Consider the terms of service and data-collecting policies
- Error handling
- Update your program regularly
- Avoid making a large number of requests
- Summary
- Questions
- Further reading
- Chapter 6: Working with Processes in Python
- Technical requirements
- The concept of a process
- Processes versus threads
- Multiprocessing
- Introductory example in Python
- An overview of the multiprocessing module
- The process class
- The Pool class
- Determining the current process, waiting, and terminating processes
- Determining the current process
- Waiting for processes
- Terminating processes
- Interprocess communication
- Message passing for a single worker
- Message passing between several workers
- Summary
- Questions
- Further reading
- Chapter 7: Reduction Operators in Processes
- Technical requirements
- The concept of reduction operators
- Properties of a reduction operator
- Examples and non-examples
- Example implementation in Python
- Real-life applications of concurrent reduction operators
- Summary
- Questions
- Further reading
- Chapter 8: Concurrent Image Processing
- Technical requirements
- Image processing fundamentals
- Python as an image processing tool
- Installing OpenCV and NumPy
- Computer image basics
- RGB values
- Pixels and image files
- Coordinates inside an image
- OpenCV API
- Image processing techniques
- Grayscaling
- Thresholding
- Applying concurrency to image processing
- Good concurrent image processing practices
- Choosing the correct way (out of many)
- Spawning an appropriate number of processes
- Processing input/output concurrently
- Summary
- Questions
- Further reading
- Chapter 9: Introduction to Asynchronous Programming
- Technical requirements
- A quick analogy
- Asynchronous versus other programming models
- Asynchronous versus synchronous programming
- Asynchronous versus threading and multiprocessing
- An example in Python
- Summary
- Questions
- Further reading
- Chapter 10: Implementing Asynchronous Programming in Python
- Technical requirements
- The asyncio module
- Coroutines, event loops, and futures
- Asyncio API
- The asyncio framework in action
- Asynchronously counting down
- A note about blocking functions
- Asynchronous prime-checking
- Improvements from Python 3.7
- Inherently blocking tasks
- concurrent.futures as a solution for blocking tasks
- Changes in the framework
- Examples in Python
- Summary
- Questions
- Further reading
- Chapter 11: Building Communication Channels with asyncio
- Technical requirements
- The ecosystem of communication channels
- Communication protocol layers
- Asynchronous programming for communication channels
- Transports and protocols in asyncio
- The big picture of asyncio's server client
- Python example
- Starting a server
- Installing Telnet
- Simulating a connection channel
- Sending messages back to clients
- Closing the transports
- Client-side communication with aiohttp
- Installing aiohttp and aiofiles
- Fetching a website's HTML code
- Writing files asynchronously
- Summary
- Questions
- Further reading
- Chapter 12: Deadlocks
- Technical requirements
- The concept of deadlock
- The Dining Philosophers problem
- Deadlock in a concurrent system
- Python simulation
- Approaches to deadlock situations
- Implementing ranking among resources
- Ignoring locks and sharing resources
- An additional note about locks
- Concluding note on deadlock solutions
- The concept of livelock
- Summary
- Questions
- Further reading
- Chapter 13: Starvation
- Technical requirements
- The concept of starvation
- What is starvation?
- Scheduling
- Causes of starvation
- Starvation's relationship to deadlock
- The readers-writers problem
- Problem statement
- The first readers-writers problem
- The second readers-writers problem
- The third readers-writers problem
- Solutions to starvation
- Summary
- Questions
- Further reading
- Chapter 14: Race Conditions
- Technical requirements
- The concept of race conditions
- Critical sections
- How race conditions occur
- Simulating race conditions in Python
- Locks as a solution to race conditions
- The effectiveness of locks
- Implementation in Python
- The downside of locks
- Turning a concurrent program sequential
- Locks do not lock anything
- Race conditions in real life
- Security
- Operating systems
- Networking
- Summary
- Questions
- Further reading
- Chapter 15: The Global Interpreter Lock
- Technical requirements
- An introduction to the Global Interpreter Lock
- An analysis of memory management in Python
- The problem that the GIL addresses
- Problems raised by the GIL
- The potential removal of the GIL from Python
- How to work with the GIL
- Implementing multiprocessing, rather than multithreading
- Getting around the GIL with native extensions
- Utilizing a different Python interpreter
- Summary
- Questions
- Further reading
- Chapter 16: Designing Lock-Based and Mutex-Free Concurrent Data Structures
- Technical requirements
- Lock-based concurrent data structures in Python
- LocklessCounter and race conditions
- Embedding locks in the data structure of the counter
- The concept of scalability
- Analysis of the scalability of the counter data structure
- Approximate counters as a solution for scalability
- The idea behind approximate counters
- Implementing approximate counters in Python
- A few considerations for approximate counter designs
- Mutex-free concurrent data structures in Python
- The impossibility of being lock-free in Python
- Introduction to the network data structure
- Implementing a simple network data structure in Python and race conditions
- RCU as a solution
- Building on simple data structures
- Summary
- Questions
- Further reading
- Chapter 17: Memory Models and Operations on Atomic Types
- Technical requirements
- Python memory model
- The components of Python memory manager
- Memory model as a labeled directed graph
- In the context of concurrency
- Atomic operations in Python
- What does it mean to be atomic?
- The GIL reconsidered
- Innate atomicity in Python
- Atomic versus nonatomic
- Simulation in Python
- Summary
- Questions
- Further reading
- Chapter 18: Building a Server from Scratch
- Technical requirements
- Low-level network programming via the socket module
- The theory of server-side communication
- The API of the socket module
- Building a simple echo server
- Building a calculator server with the socket module
- The underlying calculation logic
- Implementing the calculator server
- Building a non-blocking server
- Analyzing the concurrency of the server
- Generators in Python
- Asynchronous generators and the send method
- Making the server non-blocking
- Summary
- Questions
- Further reading
- Chapter 19: Testing, Debugging, and Scheduling Concurrent Applications
- Technical requirements
- Scheduling with APScheduler
- Installing APScheduler
- Not a scheduling service
- APScheduler functionalities
- APScheduler API
- Scheduler classes
- Executor classes
- Trigger keywords
- Common scheduler methods
- Examples in Python
- Blocking scheduler
- Background scheduler
- Executor pool
- Running on the cloud
- Testing and concurrency in Python
- Testing concurrent programs
- Unit testing
- Static code analysis
- Testing programs concurrently
- Debugging concurrent programs
- Debugging tools and techniques
- Debugging and concurrency
- Summary
- Questions
- Further reading
- Assessments
- Other Books You May Enjoy
- Index
System requirements
File format: ePUB
Copy protection: Adobe-DRM (Digital Rights Management)
System requirements:
- Computer (Windows; MacOS X; Linux): Install the free reader Adobe Digital Editions prior to download (see eBook Help).
- Tablet/smartphone (Android; iOS): Install the free app Adobe Digital Editions or the app PocketBook before downloading (see eBook Help).
- E-reader: Bookeen, Kobo, Pocketbook, Sony, Tolino and many more (not Kindle).
The file format ePub works well for novels and non-fiction books – i.e., „flowing” text without complex layout. On an e-reader or smartphone, line and page breaks automatically adjust to fit the small displays.
This eBook uses Adobe-DRM, a „hard” copy protection. If the necessary requirements are not met, unfortunately you will not be able to open the eBook. You will therefore need to prepare your reading hardware before downloading.
Please note: We strongly recommend that you authorise using your personal Adobe ID after installation of any reading software.
For more information, see our ebook Help page.