1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
|
There is a significant amount of Python project tooling. This document collects my personal recommendations on how to set up a Python project.
It is not meant to reflect the best or most common practices, just my personal taste.
# Use pipx
Pipx is a tool that installs Python packages to your user environment. It creates an isolated environment for every tool, so if you install multiple packages they won't have version conflicts. It also takes care of adding a module's entrypoints to your user path.
Pipx is useful for two purposes:
* To install tools such as poetry
* To let other users install your software easily
# Use Poetry
> [!NOTE]
> I have been using [rye](https://rye.astral.sh/) recently.
> Not enough to recommend it unconditionally over Poetry, but I am liking it so far.
> Consider testing it.
> It uses [Python redistributable builds](https://github.com/indygreg/python-build-standalone) to manage Python versions, so you can choose to use more modern Python versions.
When using third-party dependencies in your Python code, it is highly interesting to avoid installing any project-specific dependency outside the project.
To achieve that, traditionally virtualenvs are used; those are miniature Python installations where you can install any library you want. Virtualenvs need to be explicitly activated to be used, so it is easy to have a virtualenv for each Python project you are working on.
Poetry is a tool that leverages virtualenvs to manage a project's dependencies, managing virtualenvs automatically.
There are many similar tools such as pipenv and there are many multiple ways to specify a project's dependencies (`setup.py`, `requirements.txt`, etc.); Poetry provides a convenient way to do everything.
You can install poetry using pipx.
Consider reading [some brief notes about Python dependency management](dependency_handling.md).
# Test your code
Write the necessary amount of tests so you can make changes to your code with confidence.
If you find yourself iterating over a piece of code slowly, try to isolate the code you are writing so it can be tested in isolation for faster iteration.
## Use pytest for testing
Python provides *two* testing frameworks in its standard library, but they have some limitations:
* `unittest` is an xUnit-style testing framework which follows non-PEP-8 naming conventions (probably because it copied the Java's jUnit), so extra work needs to be done to make your test cases PEP-8 compliant
* `doctest` is a tool which allows you to run tests embedded in comments. For some code, it is great and helps you provide good, up-to-date documentation. However, a significant amount of code is awkward to test using `doctest`.
Use `doctest` whenever you can, but outside that, use `pytest` to write PEP-8-compliant tests.
Ensure that your test suite runs correctly by running `pytest` without any arguments.
Use plain Python's `assert` statements to check assertions in your tests; `pytest` does some magic to provide nice error messages on failed assertions.
## Gate your changes with testing
Set up your version control so changes cannot be made to your main codeline without passing continuous integration tests (and possibly, code review).
# Perform automated code formatting and static checking
> [!NOTE]
> I have been using [ruff](https://github.com/astral-sh/ruff) recently.
> Not enough to recommend it unconditionally over flake8/black, but I am liking it so far.
> Consider testing it.
> It requires slightly less configuration and it comes with more lints.
## Use Black
Use Black to format your code.
## Use flake8
Use `flake8` to gate changes. Use `flake8-black` to prevent committed code which does not follow Black style.
# Version control
## Use a minimal gitignore file
Keep editor-specific ignores in a personal `excludesfile`. Do not include patterns in gitignore which do not match anything generated by documented and supported development procedures.
## Keep your code together
All the code you modify as part of the project should be kept in a single repository so you can make atomic changes. If you find yourself making changes across multiple repositories and having to coordinate them, consider merging those repositories.
Use git submodules or similar mechanisms to refer to code you modify that must be kept external.
Use git subrepo to publish parts of the repository outside the main repository if needed.
# Support multiple modern versions of Python
Unless you have a specific requirement to support Python 2, don't.
It is reasonable to support multiple versions of Python 3 from 3.4 onwards. Supporting the oldest versions might limit the features you can use (although features from more modern versions have been backported), so evaluate which operating systems and versions you need to support and try to support Python versions readily available for them (in Linux, by using mainline distro repos, for instance).
Even if you are not running your code using the latest versions of Python, try to support all the newest available versions.
Use continuous integration to run your tests in all supported versions of Python.
This implies that development should be possible to do without using a specific version of Python, so pyenv or similar is not strictly needed.
# Use ipython and ipdb
Add ipython and ipdb as development dependencies.
# Versioning
Unless you have a specific requirement to support multiple versions of your code or to distribute to a platform that *requires* versioning (such as pypi), do not explicitly version your code but allow implicit versioning (e.g. it should be possible to identify which Git commit deployed code comes from).
# Documentation
Provide a `README` containing:
* The purpose of the code
* How to use the code
* How to develop the code
If the `README` becomes unwieldly, separate usage instructions to `USAGE` and/or development instructions to `HACKING`.
Provide docstrings detailing the external interface of Python modules. Provide internal comments in modules detailing implementation.
If you are developing a library/framework, consider using Sphinx. Sphinx can create a documentation website for a Python project, taking advantage of docstrings.
# Distribution
If your code can be executed from a command line, consider documenting installation via `pipx`.
If your code has dependencies that are not trivial to install (such as Pandas), consider publishing a Docker image or using dependencies that are simpler to install. Design your Docker images so rebuilding the image on most changes is fast.
|
