1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
|
# Python Modules Primer
## Prerequisites
These instructions assume a Linux environment. A macOS environment is similar, but not identical. A Windows environment is more different.
## Previous knowledge
### A refresher on the PATH variable
If you execute the following command in your terminal:
```
$ echo hello
```
, the shell searches for the echo command in the directories listed in your PATH environment variable. You can display your PATH variable by running:
```
$ echo $PATH
/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
```
The contents of the PATH variable depend on your particular environment.
If you run the following command:
```
$ which echo
/usr/bin/echo
```
The which command prints where the shell locates the echo command.
### A refresher on shell scripts
If you create a file named foo.sh with the following contents:
```
#!/bin/sh
echo hello
```
, you define a "shell script". The first line indicates that this shell script is executed by using the /bin/sh command. The rest of the file are commands to be executed by the shell command. These commands behave as if you typed them into your terminal, so if you execute this script, the command "echo hello" will be executed, printing hello.
If you try to run foo.sh like you run the echo command, by typing its name, it does not work:
```
$ foo.sh
bash: foo.sh: command not found...
```
, because the shell looks for the foo.sh in the directories listed in the PATH variable. Unless you created the foo.sh file in a directory like /usr/bin, the shell will not find the foo.sh command.
A solution to this problem is to specify the path to the foo.sh file, instead of relying on the PATH variable. However, if you do this, you face a second problem.
```
$ ./foo.sh
bash: ./foo.sh: Permission denied
```
This happens because only files with the executable permission can be executed in this way. To solve this, add the executable permission; then it works:
```
$ chmod +x foo.sh
$ ./foo.sh
hello
```
## The import statement in Python
### Importing from the Python standard library
Run the following commands by using the Python REPL:
```
$ python3
>>> import datetime
>>> datetime.datetime.now()
datetime.datetime(2023, 9, 11, 21, 53, 16, 331236)
```
import works in a similar way to running a command in the shell.
Python searches a number of directories looking for the datetime module.
To see which directories are searched, run:
```
$ python3
>>> import sys
>>> sys.path
['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '/home/alex/.local/lib/python3.9/site-packages', '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages']
```
sys.path is a list of the directories that the import command searches.
The contents of `sys.path` depend on your operating system and Python installation method.
In my system, the /usr/lib64/python3.9 directory contains the datetime.py module.
```
$ head /usr/lib64/python3.9/datetime.py
"""Concrete date/time and related types.
See http://www.iana.org/time-zones/repository/tz-link.html for
time zone and DST data sources.
"""
__all__ = ("date", "datetime", "time", "timedelta", "timezone", "tzinfo",
"MINYEAR", "MAXYEAR")
...
```
/usr/lib64/python3.9 contains the modules in the Python standard library.
### Importing your Python files
If you create a file with the a.py name:
```
def f():
return 2
```
, and another with the b.py name:
```
import a
print(a.f())
```
, then:
```
$ python b.py
2
```
This works, because sys.path contains '', which means "the current directory".
(sys.path is very similar to the PATH variable. However, sys.path contains the current directory by default, whereas PATH does not.)
When "import a" is executed, then Python searches the directories in sys.path for an a.py file; it is found when checking the '' path. When "import datetime" is executed, Python searches in the current directory (because '' comes first in the path), doesn't find it, but then finds it in the following /usr/lib64/python3.9 directory. Python iterates over the sys.path directories, and loads the first matching file.
## Installing libraries
When writing Python software, sometimes it is enough with the modules included in the standard library. However, frequently you want to use other libraries. To use Python libraries, you must install them using the pip program.
The pip program is not part of the python3 package in some Linux distributions, and comes from the python3-pip package.
The pip program can download libraries from pypi.org, the Python package index, and install them. pip installs libraries to a "Python environment".
Old versions of pip defaulted to installing libraries to the "system" Python environment. In a Linux system, the system Python environment is located in a directory such as /usr/lib64/python3.9. By default, normal Linux users cannot write to /usr, so installing a package would fail.
Modern versions of pip detect that they cannot write to the "system" Python environment, and then redirect the install to the "user" Python environment. The "user" Python environment is in a directory such as ~/.local/lib/python3.9.
You could use a command such as "sudo pip install" to grant pip the privileges required to write to /usr. However, this can make a Linux system unusable. Most Linux systems use software that uses the "system" Python environment. Altering the "system" Python environment can break such software. Do not run "sudo pip install" with root privileges unless you know why you need this.
If you use a modern pip (or use the --user option), you can install libraries to the "user" Python environment. However, this is problematic because a Python environment can only contain a single version of a Python library. If you have two different Python programs that different versions of the same library, then these two programs cannot coexist in the "user" Python environment.
In general, Python virtual environments are used to address this problem.
## Creating Python virtual environments
If you run:
```
$ python3 -m venv <some path>
```
This will create a directory with the path you specify, with the following contents:
```
<some path>
├── bin
│ ├── activate
│ ├── pip
│ ├── python
├── include
├── lib
│ └── python3.9
```
The python and pip commands are copies of the same commands from the "system" Python environment.
But these commands work differently from the "system" Python environment commands:
```
$ <some path>/bin/python
>>> import sys
>>> sys.path
['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '<some path>/lib64/python3.9/site-packages', '<some path>/lib/python3.9/site-packages']
```
sys.path uses the lib directories in the virtual environment.
When you use the `pip` program from the virtual environment, it installs the libraries to the virtual environment.
You can create as many virtual environments as you need, and you can install different versions of libraries to each virtual environment.
## Activating Python environments
You can run the python and pip commands by specifying the full path, like we did when executing the foo.sh command earlier.
By default, if you run python, the shell will invoke the python command from the "system" Python environment because it is in a directory included in the PATH variable. If you specify the full path, you override this.
To save typing, the bin directory of a virtual environment contains an activate file. The activate file is a "special" shell script that must be invoked like this:
```
$ . <some path>/bin/activate
```
. is a special shell command that the activate script requires to work correctly.
activate alters your path, so that the bin directory in your virtual environment comes first in your path.
```
$ echo $PATH
/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
$ . <some path>/bin/activate
(some path) $ echo $PATH
<some path>/bin:/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin
```
, and thus if you run python, <some path>/bin/python will be executed instead of /usr/bin/python.
Besides changing your prompt to indicate the virtual environment is activated, activate only alters your PATH. activate is not mandatory to use a virtual environment. For example, when running the Python command, if you specify the path of the Python executable in a virtual environment, the command will execute as if the virtual environment had been activated. Virtual environment management tools also have commands that can run commands inside a virtual environment without activating it. Activation can save time, but it is also more error-prone than more explicit means of using virtual environments.
|
