Effective Python
125 Specific Ways to Write Better Python
3rd Edition
Published on 4. December 2024
672 pages
978-0-13-817224-4 (ISBN)
Description
Master the art of Python programming with 125 actionable best practices to write more efficient, readable, and maintainable code.
Python is a versatile and powerful language, but leveraging its full potential requires more than just knowing the syntax. Effective Python: 125 Specific Ways to Write Better Python, 3rd Edition is your comprehensive guide to mastering Python's unique strengths and avoiding its hidden pitfalls. This updated edition builds on the acclaimed second edition, expanding from 90 to 125 best practices that are essential for writing high-quality Python code.
Drawing on years of experience at Google, Brett Slatkin offers clear, concise, and practical advice for both new and experienced Python developers. Each item in the book provides insight into the "Pythonic" way of programming, helping you understand how to write code that is not only effective but also elegant and maintainable. Whether you're building web applications, analyzing data, writing automation scripts, or training AI models, this book will equip you with the skills to make a significant impact using Python.
Key Features of the 3rd Edition:
Expanded Content: Now with 125 actionable guidelines, including 35 entirely new items.
Updated Best Practices: Reflects the latest features in Python releases up to version 3.13.
New Chapters: Additional chapters on how to build robust programs that achieve high performance.
Advanced Topics: In-depth coverage of creating C-extension modules and interfacing with native shared libraries.
Practical Examples: Realistic code examples that illustrate each best practice.
Python is a versatile and powerful language, but leveraging its full potential requires more than just knowing the syntax. Effective Python: 125 Specific Ways to Write Better Python, 3rd Edition is your comprehensive guide to mastering Python's unique strengths and avoiding its hidden pitfalls. This updated edition builds on the acclaimed second edition, expanding from 90 to 125 best practices that are essential for writing high-quality Python code.
Drawing on years of experience at Google, Brett Slatkin offers clear, concise, and practical advice for both new and experienced Python developers. Each item in the book provides insight into the "Pythonic" way of programming, helping you understand how to write code that is not only effective but also elegant and maintainable. Whether you're building web applications, analyzing data, writing automation scripts, or training AI models, this book will equip you with the skills to make a significant impact using Python.
Key Features of the 3rd Edition:
Expanded Content: Now with 125 actionable guidelines, including 35 entirely new items.
Updated Best Practices: Reflects the latest features in Python releases up to version 3.13.
New Chapters: Additional chapters on how to build robust programs that achieve high performance.
Advanced Topics: In-depth coverage of creating C-extension modules and interfacing with native shared libraries.
Practical Examples: Realistic code examples that illustrate each best practice.
More details
Series
Edition
3rd edition
Language
English
Place of publication
Boston
United States
Publishing group
Pearson Education (US)
Target group
Professional and scholarly
File size
10,23 MB
ISBN-13
978-0-13-817224-4 (9780138172244)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Book
02/2025
3rd Edition
Addison Wesley
€56.00
Shipment within 15-20 days
Person
Brett Slatkin is a Principal Software Engineer at Google in the Office of the CTO, focusing on emerging technologies. He co-founded Google Surveys, launched Google Cloud's first product (App Engine), and co-created the PubSubHubbub protocol-all using Python. Brett has been writing Python code professionally for the past 19 years and has made numerous contributions to open-source projects.
Content
Preface xvii
Acknowledgments xxiii
About the Author xxv
Chapter 1: Pythonic Thinking 1
Item 1: Know Which Version of Python You're Using 1
Item 2: Follow the PEP 8 Style Guide 3
Item 3: Never Expect Python to Detect Errors at Compile Time 6
Item 4: Write Helper Functions Instead of Complex Expressions 8
Item 5: Prefer Multiple-Assignment Unpacking Over Indexing 11
Item 6: Always Surround Single-Element Tuples with Parentheses 16
Item 7: Consider Conditional Expressions for Simple Inline Logic 19
Item 8: Prevent Repetition with Assignment Expressions 24
Item 9: Consider match for Destructuring in Flow Control; Avoid When if Statements Are Sufficient 30
Chapter 2: Strings and Slicing 41
Item 10: Know the Differences Between bytes and str 41
Item 11: Prefer Interpolated F-Strings over C-Style Format Strings and str.format 47
Item 12: Understand the Difference Between repr and str when Printing Objects 58
Item 13: Prefer Explicit String Concatenation over Implicit, Especially in Lists 62
Item 14: Know How to Slice Sequences 67
Item 15: Avoid Striding and Slicing in a Single Expression 70
Item 16: Prefer Catch-All Unpacking Over Slicing 72
Chapter 3: Loops and Iterators 77
Item 17: Prefer enumerate over range 77
Item 18: Use zip to Process Iterators in Parallel 79
Item 19: Avoid else Blocks After for and while Loops 82
Item 20: Never Use for Loop Variables After the Loop Ends 85
Item 21: Be Defensive when Iterating over Arguments 87
Item 22: Never Modify Containers While Iterating over Them; Use Copies or Caches Instead 92
Item 23: Pass Iterators to any and all for Efficient Short-Circuiting Logic 98
Item 24: Consider itertools for Working with Iterators and Generators 102
Chapter 4: Dictionaries 109
Item 25: Be Cautious when Relying on Dictionary Insertion Ordering 109
Item 26: Prefer get over in and KeyError to Handle Missing Dictionary Keys 117
Item 27: Prefer defaultdict over setdefault to Handle Missing Items in Internal State 122
Item 28: Know How to Construct Key-Dependent Default Values with __missing__ 124
Item 29: Compose Classes Instead of Deeply Nesting Dictionaries, Lists, and Tuples 127
Chapter 5: Functions 135
Item 30: Know That Function Arguments Can Be Mutated 135
Item 31: Return Dedicated Result Objects Instead of Requiring Function Callers to Unpack More Than Three Variables 138
Item 32: Prefer Raising Exceptions to Returning None 142
Item 33: Know How Closures Interact with Variable Scope and nonlocal 145
Item 34: Reduce Visual Noise with Variable Positional Arguments 150
Item 35: Provide Optional Behavior with Keyword Arguments 153
Item 36: Use None and Docstrings to Specify Dynamic Default Arguments 157
Item 37: Enforce Clarity with Keyword-Only and Positional-Only Arguments 161
Item 38: Define Function Decorators with functools.wraps 166
Item 39: Prefer functools.partial over lambda Expressions for Glue Functions 169
Chapter 6: Comprehensions and Generators 173
Item 40: Use Comprehensions Instead of map and filter 173
Item 41: Avoid More Than Two Control Subexpressions in Comprehensions 176
Item 42: Reduce Repetition in Comprehensions with Assignment Expressions 178
Item 43: Consider Generators Instead of Returning Lists 182
Item 44: Consider Generator Expressions for Large List Comprehensions 184
Item 45: Compose Multiple Generators with yield from 186
Item 46: Pass Iterators into Generators as Arguments Instead of Calling the send Method 188
Item 47: Manage Iterative State Transitions with a Class Instead of the Generator throw Method 195
Chapter 7: Classes and Interfaces 201
Item 48: Accept Functions Instead of Classes for Simple Interfaces 201
Item 49: Prefer Object-Oriented Polymorphism over Functions with isinstance Checks 205
Item 50: Consider functools.singledispatch for Functional-Style Programming Instead of Object-Oriented Polymorphism 210
Item 51: Prefer dataclasses for Defining Lightweight Classes 217
Item 52: Use @classmethod Polymorphism to Construct Objects Generically 230
Item 53: Initialize Parent Classes with super 235
Item 54: Consider Composing Functionality with Mix-in Classes 240
Item 55: Prefer Public Attributes over Private Ones 245
Item 56: Prefer dataclasses for Creating Immutable Objects 250
Item 57: Inherit from collections.abc Classes for Custom Container Types 260
Chapter 8: Metaclasses and Attributes 265
Item 58: Use Plain Attributes Instead of Setter and Getter Methods 265
Item 59: Consider @property Instead of Refactoring Attributes 270
Item 60: Use Descriptors for Reusable @property Methods 274
Item 61: Use __getattr__, __getattribute__, and __setattr__ for Lazy Attributes 279
Item 62: Validate Subclasses with __init_subclass__ 285
Item 63: Register Class Existence with __init_subclass__ 293
Item 64: Annotate Class Attributes with __set_name__ 299
Item 65: Consider Class Body Definition Order to Establish Relationships Between Attributes 303
Item 66: Prefer Class Decorators over Metaclasses for Composable Class Extensions 310
Chapter 9: Concurrency and Parallelism 319
Item 67: Use subprocess to Manage Child Processes 320
Item 68: Use Threads for Blocking I/O; Avoid for Parallelism 324
Item 69: Use Lock to Prevent Data Races in Threads 330
Item 70: Use Queue to Coordinate Work Between Threads 333
Item 71: Know How to Recognize When Concurrency Is Necessary 344
Item 72: Avoid Creating New Thread Instances for On-Demand Fan-out 349
Item 73: Understand How Using Queue for Concurrency Requires Refactoring 353
Item 74: Consider ThreadPoolExecutor When Threads Are Necessary for Concurrency 361
Item 75: Achieve Highly Concurrent I/O with Coroutines 364
Item 76: Know How to Port Threaded I/O to asyncio 368
Item 77: Mix Threads and Coroutines to Ease the Transition to asyncio 381
Item 78: Maximize Responsiveness of asyncio Event Loops with async-friendly Worker Threads 389
Item 79: Consider concurrent.futures for True Parallelism 393
Chapter 10: Robustness 399
Item 80: Take Advantage of Each Block in try/except/else/finally 399
Item 81: assert Internal Assumptions and raise Missed Expectations 404
Item 82: Consider contextlib and with Statements for Reusable try/finally Behavior 408
Item 83: Always Make try Blocks as Short as Possible 412
Item 84: Beware of Exception Variables Disappearing 414
Item 85: Beware of Catching the Exception Class 416
Item 86: Understand the Difference Between Exception and BaseException 419
Item 87: Use traceback for Enhanced Exception Reporting 424
Item 88: Consider Explicitly Chaining Exceptions to Clarify Tracebacks 428
Item 89: Always Pass Resources into Generators and Have Callers Clean Them Up Outside 436
Item 90: Never Set __debug__ to False 442
Item 91: Avoid exec and eval Unless You're Building a Developer Tool 445
Chapter 11: Performance 447
Item 92: Profile Before Optimizing 448
Item 93: Optimize Performance-Critical Code Using timeit Microbenchmarks 453
Item 94: Know When and How to Replace Python with Another Programming Language 458
Item 95: Consider ctypes to Rapidly Integrate with Native Libraries 462
Item 96: Consider Extension Modules to Maximize Performance and Ergonomics 467
Item 97: Rely on Precompiled Bytecode and File System Caching to Improve Startup Time 475
Item 98: Lazy-Load Modules with Dynamic Imports to Reduce Startup Time 478
Item 99: Consider memoryview and bytearray for Zero-Copy Interactions with bytes 485
Chapter 12: Data Structures & Algorithms 493
Item 100: Sort by Complex Criteria Using the key Parameter 493
Item 101: Know the Difference Between sort and sorted 499
Item 102: Consider Searching Sorted Sequences with bisect 501
Item 103: Prefer deque for Producer-Consumer Queues 504
Item 104: Know How to Use heapq for Priority Queues 509
Item 105: Use datetime Instead of time for Local Clocks 519
Item 106: Use decimal When Precision Is Paramount 523
Item 107: Make pickle Serialization Maintainable with copyreg 526
Chapter 13: Testing and Debugging 533
Item 108: Verify Related Behaviors in TestCase Subclasses 533
Item 109: Prefer Integration Tests over Unit Tests 541
Item 110: Isolate Tests From Each Other with setUp, tearDown, setUpModule, and tearDownModule 547
Item 111: Use Mocks to Test Code with Complex Dependencies 550
Item 112: Encapsulate Dependencies to Facilitate Mocking and Testing 559
Item 113: Use assertAlmostEqual to Control Precision in Floating Point Tests 563
Item 114: Consider Interactive Debugging with pdb 565
Item 115: Use tracemalloc to Understand Memory Usage and Leaks 570
Chapter 14: Collaboration 575
Item 116: Know Where to Find Community-Built Modules 575
Item 117: Use Virtual Environments for Isolated and Reproducible Dependencies 576
Item 118: Write Docstrings for Every Function, Class, and Module 582
Item 119: Use Packages to Organize Modules and Provide Stable APIs 588
Item 120: Consider Module-Scoped Code to Configure Deployment Environments 593
Item 121: Define a Root Exception to Insulate Callers from APIs 595
Item 122: Know How to Break Circular Dependencies 600
Item 123: Consider warnings to Refactor and Migrate Usage 605
Item 124: Consider Static Analysis via typing to Obviate Bugs 613
Item 125: Prefer Open Source Projects for Bundling Python Programs over zipimport and zipapp 621
Index 627
Acknowledgments xxiii
About the Author xxv
Chapter 1: Pythonic Thinking 1
Item 1: Know Which Version of Python You're Using 1
Item 2: Follow the PEP 8 Style Guide 3
Item 3: Never Expect Python to Detect Errors at Compile Time 6
Item 4: Write Helper Functions Instead of Complex Expressions 8
Item 5: Prefer Multiple-Assignment Unpacking Over Indexing 11
Item 6: Always Surround Single-Element Tuples with Parentheses 16
Item 7: Consider Conditional Expressions for Simple Inline Logic 19
Item 8: Prevent Repetition with Assignment Expressions 24
Item 9: Consider match for Destructuring in Flow Control; Avoid When if Statements Are Sufficient 30
Chapter 2: Strings and Slicing 41
Item 10: Know the Differences Between bytes and str 41
Item 11: Prefer Interpolated F-Strings over C-Style Format Strings and str.format 47
Item 12: Understand the Difference Between repr and str when Printing Objects 58
Item 13: Prefer Explicit String Concatenation over Implicit, Especially in Lists 62
Item 14: Know How to Slice Sequences 67
Item 15: Avoid Striding and Slicing in a Single Expression 70
Item 16: Prefer Catch-All Unpacking Over Slicing 72
Chapter 3: Loops and Iterators 77
Item 17: Prefer enumerate over range 77
Item 18: Use zip to Process Iterators in Parallel 79
Item 19: Avoid else Blocks After for and while Loops 82
Item 20: Never Use for Loop Variables After the Loop Ends 85
Item 21: Be Defensive when Iterating over Arguments 87
Item 22: Never Modify Containers While Iterating over Them; Use Copies or Caches Instead 92
Item 23: Pass Iterators to any and all for Efficient Short-Circuiting Logic 98
Item 24: Consider itertools for Working with Iterators and Generators 102
Chapter 4: Dictionaries 109
Item 25: Be Cautious when Relying on Dictionary Insertion Ordering 109
Item 26: Prefer get over in and KeyError to Handle Missing Dictionary Keys 117
Item 27: Prefer defaultdict over setdefault to Handle Missing Items in Internal State 122
Item 28: Know How to Construct Key-Dependent Default Values with __missing__ 124
Item 29: Compose Classes Instead of Deeply Nesting Dictionaries, Lists, and Tuples 127
Chapter 5: Functions 135
Item 30: Know That Function Arguments Can Be Mutated 135
Item 31: Return Dedicated Result Objects Instead of Requiring Function Callers to Unpack More Than Three Variables 138
Item 32: Prefer Raising Exceptions to Returning None 142
Item 33: Know How Closures Interact with Variable Scope and nonlocal 145
Item 34: Reduce Visual Noise with Variable Positional Arguments 150
Item 35: Provide Optional Behavior with Keyword Arguments 153
Item 36: Use None and Docstrings to Specify Dynamic Default Arguments 157
Item 37: Enforce Clarity with Keyword-Only and Positional-Only Arguments 161
Item 38: Define Function Decorators with functools.wraps 166
Item 39: Prefer functools.partial over lambda Expressions for Glue Functions 169
Chapter 6: Comprehensions and Generators 173
Item 40: Use Comprehensions Instead of map and filter 173
Item 41: Avoid More Than Two Control Subexpressions in Comprehensions 176
Item 42: Reduce Repetition in Comprehensions with Assignment Expressions 178
Item 43: Consider Generators Instead of Returning Lists 182
Item 44: Consider Generator Expressions for Large List Comprehensions 184
Item 45: Compose Multiple Generators with yield from 186
Item 46: Pass Iterators into Generators as Arguments Instead of Calling the send Method 188
Item 47: Manage Iterative State Transitions with a Class Instead of the Generator throw Method 195
Chapter 7: Classes and Interfaces 201
Item 48: Accept Functions Instead of Classes for Simple Interfaces 201
Item 49: Prefer Object-Oriented Polymorphism over Functions with isinstance Checks 205
Item 50: Consider functools.singledispatch for Functional-Style Programming Instead of Object-Oriented Polymorphism 210
Item 51: Prefer dataclasses for Defining Lightweight Classes 217
Item 52: Use @classmethod Polymorphism to Construct Objects Generically 230
Item 53: Initialize Parent Classes with super 235
Item 54: Consider Composing Functionality with Mix-in Classes 240
Item 55: Prefer Public Attributes over Private Ones 245
Item 56: Prefer dataclasses for Creating Immutable Objects 250
Item 57: Inherit from collections.abc Classes for Custom Container Types 260
Chapter 8: Metaclasses and Attributes 265
Item 58: Use Plain Attributes Instead of Setter and Getter Methods 265
Item 59: Consider @property Instead of Refactoring Attributes 270
Item 60: Use Descriptors for Reusable @property Methods 274
Item 61: Use __getattr__, __getattribute__, and __setattr__ for Lazy Attributes 279
Item 62: Validate Subclasses with __init_subclass__ 285
Item 63: Register Class Existence with __init_subclass__ 293
Item 64: Annotate Class Attributes with __set_name__ 299
Item 65: Consider Class Body Definition Order to Establish Relationships Between Attributes 303
Item 66: Prefer Class Decorators over Metaclasses for Composable Class Extensions 310
Chapter 9: Concurrency and Parallelism 319
Item 67: Use subprocess to Manage Child Processes 320
Item 68: Use Threads for Blocking I/O; Avoid for Parallelism 324
Item 69: Use Lock to Prevent Data Races in Threads 330
Item 70: Use Queue to Coordinate Work Between Threads 333
Item 71: Know How to Recognize When Concurrency Is Necessary 344
Item 72: Avoid Creating New Thread Instances for On-Demand Fan-out 349
Item 73: Understand How Using Queue for Concurrency Requires Refactoring 353
Item 74: Consider ThreadPoolExecutor When Threads Are Necessary for Concurrency 361
Item 75: Achieve Highly Concurrent I/O with Coroutines 364
Item 76: Know How to Port Threaded I/O to asyncio 368
Item 77: Mix Threads and Coroutines to Ease the Transition to asyncio 381
Item 78: Maximize Responsiveness of asyncio Event Loops with async-friendly Worker Threads 389
Item 79: Consider concurrent.futures for True Parallelism 393
Chapter 10: Robustness 399
Item 80: Take Advantage of Each Block in try/except/else/finally 399
Item 81: assert Internal Assumptions and raise Missed Expectations 404
Item 82: Consider contextlib and with Statements for Reusable try/finally Behavior 408
Item 83: Always Make try Blocks as Short as Possible 412
Item 84: Beware of Exception Variables Disappearing 414
Item 85: Beware of Catching the Exception Class 416
Item 86: Understand the Difference Between Exception and BaseException 419
Item 87: Use traceback for Enhanced Exception Reporting 424
Item 88: Consider Explicitly Chaining Exceptions to Clarify Tracebacks 428
Item 89: Always Pass Resources into Generators and Have Callers Clean Them Up Outside 436
Item 90: Never Set __debug__ to False 442
Item 91: Avoid exec and eval Unless You're Building a Developer Tool 445
Chapter 11: Performance 447
Item 92: Profile Before Optimizing 448
Item 93: Optimize Performance-Critical Code Using timeit Microbenchmarks 453
Item 94: Know When and How to Replace Python with Another Programming Language 458
Item 95: Consider ctypes to Rapidly Integrate with Native Libraries 462
Item 96: Consider Extension Modules to Maximize Performance and Ergonomics 467
Item 97: Rely on Precompiled Bytecode and File System Caching to Improve Startup Time 475
Item 98: Lazy-Load Modules with Dynamic Imports to Reduce Startup Time 478
Item 99: Consider memoryview and bytearray for Zero-Copy Interactions with bytes 485
Chapter 12: Data Structures & Algorithms 493
Item 100: Sort by Complex Criteria Using the key Parameter 493
Item 101: Know the Difference Between sort and sorted 499
Item 102: Consider Searching Sorted Sequences with bisect 501
Item 103: Prefer deque for Producer-Consumer Queues 504
Item 104: Know How to Use heapq for Priority Queues 509
Item 105: Use datetime Instead of time for Local Clocks 519
Item 106: Use decimal When Precision Is Paramount 523
Item 107: Make pickle Serialization Maintainable with copyreg 526
Chapter 13: Testing and Debugging 533
Item 108: Verify Related Behaviors in TestCase Subclasses 533
Item 109: Prefer Integration Tests over Unit Tests 541
Item 110: Isolate Tests From Each Other with setUp, tearDown, setUpModule, and tearDownModule 547
Item 111: Use Mocks to Test Code with Complex Dependencies 550
Item 112: Encapsulate Dependencies to Facilitate Mocking and Testing 559
Item 113: Use assertAlmostEqual to Control Precision in Floating Point Tests 563
Item 114: Consider Interactive Debugging with pdb 565
Item 115: Use tracemalloc to Understand Memory Usage and Leaks 570
Chapter 14: Collaboration 575
Item 116: Know Where to Find Community-Built Modules 575
Item 117: Use Virtual Environments for Isolated and Reproducible Dependencies 576
Item 118: Write Docstrings for Every Function, Class, and Module 582
Item 119: Use Packages to Organize Modules and Provide Stable APIs 588
Item 120: Consider Module-Scoped Code to Configure Deployment Environments 593
Item 121: Define a Root Exception to Insulate Callers from APIs 595
Item 122: Know How to Break Circular Dependencies 600
Item 123: Consider warnings to Refactor and Migrate Usage 605
Item 124: Consider Static Analysis via typing to Obviate Bugs 613
Item 125: Prefer Open Source Projects for Bundling Python Programs over zipimport and zipapp 621
Index 627
