Publishing Python Packages
Test, share, and automate your projects
1st Edition
Published on 28. February 2023
248 pages
978-1-63835-168-9 (ISBN)
Description
Create masterful, maintainable Python packages! This book includes pro tips for design, automation, testing, deployment, and even release as an open source project!
In Publishing Python Packages you will learn how to:
Build extensions and console script commands
Use tox to automate packaging, installing, and testing
Build a continuous integration pipeline using GitHub Actions
Improve code quality and reduce manual review using black, mypy, and flake8
Create published documentation for your packages
Keep packages up to date with pyupgrade and Dependabot
Foster an open source community using GitHub features
Publishing Python Packages teaches you how to easily share your Python code with your team and the outside world. Learn a repeatable and highly automated process for package maintenance that's based on the best practices, tools, and standards of Python packaging. This book walks you through creating a complete package, including a C extension, and guides you all the way to publishing on the Python Package Index. Whether you're entirely new to Python packaging or looking for optimal ways to maintain and scale your packages, this fast-paced and engaging guide is for you.
Foreword by David Beazley.
About the technology
Successful Python packages install easily, run flawlessly, and stay reliably up to date. Publishing perfect Python packages requires a rigorous process that supports systematic testing and review, along with excellent documentation. Fortunately, the Python ecosystem includes tools and techniques to automate package creation and publishing.
About the book
Publishing Python Packages presents a practical process for sharing Python code in an automated and scalable way. Get hands-on experience with the latest packaging tools, and learn the ins and outs of package testing and continuous integration. You'll even get pro tips for setting up a maintainable open source project, including licensing, documentation, and nurturing a community of contributors.
What's inside
Build extensions and console script commands
Improve code quality with automated review and testing
Create excellent documentation
Keep packages up to date with pyupgrade and Dependabot
About the reader
For intermediate Python programmers.
About the author
Dane Hillard has spent the majority of his development career using Python to build web applications.
Table of Contents
PART 1 FOUNDATIONS
1 The what and why of Python packages
2 Preparing for package development
3 The anatomy of a minimal Python package
PART 2 CREATING A VIABLE PACKAGE
4 Handling package dependencies, entry points, and extensions
5 Building and maintaining a test suite
6 Automating code quality tooling
PART 3 GOING PUBLIC
7 Automating work through continuous integration
8 Authoring and maintaining documentation
9 Making a package evergreen
10 Scaling and solidifying your practices
11 Building a community
In Publishing Python Packages you will learn how to:
Build extensions and console script commands
Use tox to automate packaging, installing, and testing
Build a continuous integration pipeline using GitHub Actions
Improve code quality and reduce manual review using black, mypy, and flake8
Create published documentation for your packages
Keep packages up to date with pyupgrade and Dependabot
Foster an open source community using GitHub features
Publishing Python Packages teaches you how to easily share your Python code with your team and the outside world. Learn a repeatable and highly automated process for package maintenance that's based on the best practices, tools, and standards of Python packaging. This book walks you through creating a complete package, including a C extension, and guides you all the way to publishing on the Python Package Index. Whether you're entirely new to Python packaging or looking for optimal ways to maintain and scale your packages, this fast-paced and engaging guide is for you.
Foreword by David Beazley.
About the technology
Successful Python packages install easily, run flawlessly, and stay reliably up to date. Publishing perfect Python packages requires a rigorous process that supports systematic testing and review, along with excellent documentation. Fortunately, the Python ecosystem includes tools and techniques to automate package creation and publishing.
About the book
Publishing Python Packages presents a practical process for sharing Python code in an automated and scalable way. Get hands-on experience with the latest packaging tools, and learn the ins and outs of package testing and continuous integration. You'll even get pro tips for setting up a maintainable open source project, including licensing, documentation, and nurturing a community of contributors.
What's inside
Build extensions and console script commands
Improve code quality with automated review and testing
Create excellent documentation
Keep packages up to date with pyupgrade and Dependabot
About the reader
For intermediate Python programmers.
About the author
Dane Hillard has spent the majority of his development career using Python to build web applications.
Table of Contents
PART 1 FOUNDATIONS
1 The what and why of Python packages
2 Preparing for package development
3 The anatomy of a minimal Python package
PART 2 CREATING A VIABLE PACKAGE
4 Handling package dependencies, entry points, and extensions
5 Building and maintaining a test suite
6 Automating code quality tooling
PART 3 GOING PUBLIC
7 Automating work through continuous integration
8 Authoring and maintaining documentation
9 Making a package evergreen
10 Scaling and solidifying your practices
11 Building a community
More details
Person
Dane Hillard has spent the majority of his development career using Python to build web applications.
Content
- Intro
- inside front cover
- Publishing Python Packages
- Copyright
- brief contents
- contents
- front matter
- foreword
- preface
- acknowledgments
- about this book
- Who should read this book?
- How this book is organized: a roadmap
- About the code
- liveBook discussion forum
- about the author
- about the cover illustration
- Part 1. Foundations
- 1 The what and why of Python packages
- 1.1 What is a package, anyway?
- 1.1.1 Standardizing packaging for automation
- 1.1.2 The contents of a distribution package
- 1.1.3 The challenges of sharing software
- 1.2 How packaging helps you
- 1.2.1 Enforcing cohesion and encapsulation through packaging
- 1.2.2 Promoting clear ownership of code
- 1.2.3 Decoupling implementation from usage
- 1.2.4 Filling roles by composing small packages
- Summary
- 2 Preparing for package development
- 2.1 Managing Python virtual environments
- 2.1.1 Creating virtual environments with venv
- Summary
- 3 The anatomy of a minimal Python package
- 3.1 The Python build workflow
- 3.1.1 Parts of the Python build system
- 3.2 Authoring package metadata
- 3.2.1 Required core metadata
- 3.2.2 Optional core metadata
- 3.2.3 Specifying a license
- 3.3 Controlling source code and file discovery
- 3.4 Including non-Python files in a package
- Summary
- Part 2. Creating a viable package
- 4 Handling package dependencies, entry points, and extensions
- 4.1 A package for calculating vehicle drift
- 4.2 Creating a C extension for Python
- 4.2.1 Creating C extension source code
- 4.2.2 Integrating Cython into a Python package build
- 4.2.3 Installing and profiling your C extension
- 4.2.4 Build targets for binary wheel distributions
- 4.2.5 Specifying required Python versions
- 4.3 Offering command-line tools from a Python package
- 4.3.1 Creating commands with Setuptools entry points
- 4.4 Specifying dependencies for Python packages
- Answer to exercises
- Summary
- 5 Building and maintaining a test suite
- 5.1 Integrating a testing setup
- 5.1.1 The pytest testing framework
- 5.1.2 Adding test coverage measurement
- 5.1.3 Increasing test coverage
- 5.2 Addressing testing tedium
- 5.2.1 Addressing repetitive, data-driven tests
- 5.2.2 Addressing frequent package installation
- 5.2.3 Configuring test environments
- 5.2.4 Tips for quicker and safer testing
- Answers to exercises
- Summary
- 6 Automating code quality tooling
- 6.1 The true power of tox environments
- 6.1.1 Creating nondefault tox environments
- 6.1.2 Managing dependencies across tox environments
- 6.2 Analyzing type safety
- 6.2.1 Creating a tox environment for type checking
- 6.2.2 Configuring mypy
- 6.3 Creating a tox environment for code formatting
- 6.3.1 Configuring black
- 6.4 Creating a tox environment for linting
- 6.4.1 Configuring flake8
- Answers to exercises
- Summary
- Part 3. Going public
- 7 Automating work through continuous integration
- 7.1 The continuous integration workflow
- 7.2 Continuous integration with GitHub Actions
- 7.2.1 A high-level GitHub Actions workflow
- 7.2.2 Understanding GitHub Actions terminology
- 7.2.3 Starting a GitHub Actions workflow configuration
- 7.3 Converting manual tasks to GitHub Actions
- 7.3.1 Running a job multiple times with a build matrix
- 7.3.2 Building Python package distributions for a variety of platforms
- 7.4 Publishing a package
- Summary
- 8 Authoring and maintaining documentation
- 8.1 Some quick philosophy on documentation
- 8.2 Starting your documentation with Sphinx
- 8.2.1 Automating documentation refresh during development
- 8.2.2 Automating extraction of code documentation
- 8.3 Publishing documentation to Read the Docs
- 8.3.1 Configuring Read the Docs
- 8.4 Documentation best practices
- 8.4.1 What to document
- 8.4.2 Prefer linking over repetition
- 8.4.3 Use consistent, empathetic language
- 8.4.4 Avoid assumptions and create context
- 8.4.5 Create visual interest and coherent structure
- 8.4.6 Powering up your documentation
- Summary
- 9 Making a package evergreen
- 9.1 Choosing a package-versioning strategy
- 9.1.1 Direct and indirect dependencies
- 9.1.2 Python dependency specifiers and dependency hell
- 9.1.3 Semantic versioning and calendar versioning
- 9.2 Getting the most out of GitHub
- 9.2.1 The GitHub dependency graph
- 9.2.2 Mitigating security vulnerabilities with Dependabot
- 9.3 Thresholding test coverage
- 9.4 Updating Python syntax using pyupgrade
- 9.5 Reducing rework using pre-commit hooks
- Answers to exercises
- Summary
- Part 4. The long haul
- 10 Scaling and solidifying your practices
- 10.1 Creating a project template for future packages
- 10.1.1 Creating a cookiecutter configuration
- 10.1.2 Extracting a cookiecutter template from an existing project
- 10.2 Using namespace packages
- 10.2.1 Converting an existing package to a namespace package
- 10.3 Scaling packaging in your organization
- 10.3.1 Private package repository servers
- Summary
- 11 Building a community
- 11.1 Your README needs to make a value proposition
- 11.2 Provide supporting documentation for different user types
- 11.3 Establish, provide, and enforce a code of conduct
- 11.4 Conveying the project's road map, status, and changes
- 11.4.1 Using GitHub projects for kanban management
- 11.4.2 Use GitHub labels to track status for individual tasks
- 11.4.3 Track high-level changes in a log
- 11.5 Gather consistent information with issue templates
- 11.6 Go forth
- Summary
- Appendix A. Installing asdf and python-launcher
- A.1 Installing asdf
- A.2 Installing python-launcher
- Answer to exercise A.1
- Appendix B. Installing pipx, build, tox, pre-commit, and cookiecutter
- B.1 Installing pipx
- B.2 Installing build
- B.3 Installing tox
- B.4 Installing pre-commit
- B.5 Installing cookiecutter
- index
- inside back cover
