conda list vs. pip list

In the blog¹, I used Python pip to update the Matplotlib package. However, I found that if I use command conda list and pip list to inspect the Matplotlib package version respectively, I will get different results.

conda list:

1
2
3
4
5
6
7
8
9
10
11
In: conda list
# packages in environment at D:\Softwares\anaconda3:
#
# Name                    Version                   Build  Channel
...  
matplotlib                3.5.2            py39haa95532_0  
matplotlib-base           3.5.2            py39hd77b12b_0  
matplotlib-inline         0.1.6            py39haa95532_0  
... 

Note: you may need to restart the kernel to use updated packages.

pip list:

1
2
3
4
5
6
7
8
[In]: pip list
Package                       Version
----------------------------- --------------------
...
matplotlib                    3.9.0
matplotlib-inline             0.1.6
...
Note: you may need to restart the kernel to use updated packages.

NOTE: The full list of Conda packages and Pip packages are shown in Appendix.

This remind me of that Conda and Pip actually are different package management tools²³, although their functions overlap in a way, and many users use them, myself included, to manage packages simultaneously and don’t mind their nuances.

As said in blog¹, if I inspect the Matplotlib version after updating by pip update matplotlib --user command, I find the version is 3.9.0 and the script can work correctly. Which means that, although there are two-version Matplotlib packages, Python will first invoke that managed by Pip. So the next question is coming, what is the precedence order of those packages?

Python module precedence order

According to Python official documentation⁴ (and other reference⁵), when a module is imported, Python interpreter will first search for the module in (1) the built-in module list given by sys.builtin_module_names, and then (2) the directories given by sys.path (in a descending order).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
In: sys.builtin_module_names
Out: 
('_abc',
 '_ast',
 '_bisect',
 '_blake2',
 '_codecs',
 '_codecs_cn',
 '_codecs_hk',
 '_codecs_iso2022',
 '_codecs_jp',
 '_codecs_kr',
 '_codecs_tw',
 '_collections',
 '_contextvars',
 '_csv',
 '_datetime',
 '_functools',
 '_heapq',
 '_imp',
 '_io',
 '_json',
 '_locale',
 '_lsprof',
 '_md5',
 '_multibytecodec',
 '_opcode',
 '_operator',
 '_peg_parser',
 '_pickle',
 '_random',
 '_sha1',
 '_sha256',
 '_sha3',
 '_sha512',
 '_signal',
 '_sre',
 '_stat',
 '_statistics',
 '_string',
 '_struct',
 '_symtable',
 '_thread',
 '_tracemalloc',
 '_warnings',
 '_weakref',
 '_winapi',
 '_xxsubinterpreters',
 'array',
 'atexit',
 'audioop',
 'binascii',
 'builtins',
 'cmath',
 'errno',
 'faulthandler',
 'gc',
 'itertools',
 'marshal',
 'math',
 'mmap',
 'msvcrt',
 'nt',
 'parser',
 'sys',
 'time',
 'winreg',
 'xxsubtype',
 'zlib')

1
2
3
4
5
6
7
8
9
10
11
12
13
14
In: sys.path
Out: 
['D:\\Softwares\\anaconda3\\python39.zip',
 'D:\\Softwares\\anaconda3\\DLLs',
 'D:\\Softwares\\anaconda3\\lib',
 'D:\\Softwares\\anaconda3',
 '',
 'C:\\Users\\<USER_NAME>\\AppData\\Roaming\\Python\\Python39\\site-packages',
 'D:\\Softwares\\anaconda3\\lib\\site-packages',
 'D:\\Softwares\\anaconda3\\lib\\site-packages\\win32',
 'D:\\Softwares\\anaconda3\\lib\\site-packages\\win32\\lib',
 'D:\\Softwares\\anaconda3\\lib\\site-packages\\Pythonwin',
 'D:\\Softwares\\anaconda3\\lib\\site-packages\\IPython\\extensions',
 'C:\\Users\\<USER_NAME>\\.ipython']

NOTE: '' denotes the current folder.

Apparently, those installed Python packages are stored in those sys.path directories. If query the Pip-managed Matplotlib package location⁶:

1
2
3
4
In: import matplotlib
In: print(matplotlib.__file__)

C:\Users\<USER_NAME>\AppData\Roaming\Python\Python39\site-packages\matplotlib\__init__.py

the result shows that the package is located at the 6th directory in sys.path. As for the Matplotlib managed by conda, I don’t find an easy way to get the package installment location. So, I select an exclusive package backports (see Appendix), and print its location:

1
2
3
4
In: import backports
In: print(backports.__file__)

D:\Softwares\anaconda3\lib\site-packages\backports\__init__.py

As can be seen, those Conda packages are in D:\Softwares\anaconda3\lib\site-packages, which is the 7th search directory in sys.path.

After, in the D:\Softwares\anaconda3\lib\site-packages directory, I find a matplotlib folder, and the _version.py file in it shows that this is version 3.5.2:

1
2
3
4
5
6
7
# D:\Softwares\anaconda3\Lib\site-packages\matplotlib\_version.py

# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
__version__ = version = '3.5.2'
__version_tuple__ = version_tuple = (3, 5, 2)

which is indeed consistent with the conda list result.

To sum up, Matplotlib 3.9.0 managed by Pip is stored in the 6th directory in sys.path, while Matplotlib 3.5.2 managed by Conda is in the 7th directory in sys.path. Which is why the former is importing when executing import matplotlib. And it also means Pip packages have a HIGHER calling order than Conda packages if both packages have the same name.

Appendix

About “Note” column in the following table:

(1) SAME (262 packages): the package and whose version are the same in Conda and Pip managers;

(2) CONDA (88 packages): exclusive package in Conda;

(3) PIP (8 packages): exclusive package in Pip;

(4) CASE DIFFERENCE (33 packages): Conda package and Pip package have the same version, but there is a case difference between the both, such as: bottleneck (1.3.5, Conda package) vs. Bottleneck (1.3.5, Pip package), and heapdict (1.0.1, Conda package) vs. HeapDict (1.0.1, Pip package), and markupsafe (2.0.1, Conda package) vs. MarkupSafe (2.0.1, Pip package).

(5) UNDERSCORE OR DASH (17 packages): Conda package and Pip package have the same version, but it is kind of different in aspect of connection symbol (_ of Conda package and - of Pip package) in package name, such as: jupyter_console (6.4.3, Conda package) vs. jupyter-console (6.4.3, Pip package), ipython_genutils (0.2.0, Conda package) vs. ipython-genutils (0.2.0, Pip package), and conda-package-streaming (0.7.0, Conda package) vs. conda_package_streaming (0.7.0, Pip package).

(6) DIFFERENT VERSION (2 packages): Conda package and Pip package have the same name, but different versions, such as matplotlib (3.5.2, Conda package) vs. matplotlib (3.9.0, Pip package), scikit-learn-intelex (2021.6.0, Conda package) vs. scikit-learn-intelex (2021.20221004.171935, Pip package).

(7) DIFFERENT PACKAGE NAME (1 packages): Conda package and Pip package have the same version, and seem represent the same package, like ruamel_yaml (0.15.100, Conda package) vs. ruamel-yaml-conda(0.15.0, Pip package).

References