Decoding dependencies of your python package

Python dependencies are one of the most difficult problems that one encounters while working in any Python-based system. If the underlying Python package dependencies are not correctly resolved, then you could run into many run-time issues when executing some Python code.

Sometimes knowing about what dependencies are required by a given Python package may help in understanding why unexpected run time errors happen when you are trying to consume some module from your Python package. In this article, we will learn about some command line utilities that help you learn about the dependencies of your Python package and how you can visualize these dependencies effectively.

Prerequisites

To learn about the dependencies of a Python package you would need the following setup and command line tools:-

1. A Python environment

2. Any Python package installed in the environment (I will use data-understand in this article)

3. pip (You can install pip using the command python -m pip install --upgrade pip)

4. deptree (You can install deptree using pip install deptree)

Use pip list to inspect the environment

You can use pip list to get a bird's eye view of which Python packages are installed in your Python environment.

<>➜ / $ pip list 
Package                  Version
------------------------ ----------
cryptography             41.0.1
cycler                   0.11.0
data-understand          0.0.6
.
.
.

Use pip show to view dependencies

You can use pip show <python-package-name> to see the required dependencies of python-package-name. Below is the output of pip show data-understand.

<>➜ / $ pip show data-understand
Name: data-understand
Version: 0.0.6
Summary: Utility package for generating insights for datasets
Home-page: https://github.com/ggupta2005/data.understand
Author: Gaurav Gupta
Author-email: ggupta2005@gmail.com
License: 
Location: /usr/local/python/3.10.4/lib/python3.10/site-packages
Requires: fpdf2, matplotlib, nbformat, numpy, pandas, raiutils, seaborn
Required-by:

As per the above output, the required dependencies of data-understand are fpdf2, matplotlib, nbformat, numpy, pandas, raiutils and seaborn.

Use deptree to visualize dependencies

The problem with pip show is that it doesn't show what is the version of a given required dependency. deptree is a command line utility that helps with visualizing the pinned versions of the dependencies of a Python package.

To visualize the dependency graph of dependencies for a given python package you can execute deptree <python-package-name>. Let's look at the dependency graph for data-understand.

<> ➜ / $ deptree data-understand
data-understand==0.0.6  # data-understand
  pandas==1.5.3  # pandas<2.0.0
    python-dateutil==2.8.2  # python-dateutil>=2.8.1
      six==1.16.0  # six>=1.5
    pytz==2023.3  # pytz>=2020.1
    numpy==1.24.2  # numpy>=1.21.0; python_version >= "3.10"
  nbformat==5.9.0  # nbformat
    fastjsonschema==2.16.3  # fastjsonschema
    jsonschema==4.17.3  # jsonschema>=2.6
      attrs==22.2.0  # attrs>=17.4.0
      pyrsistent==0.19.3  # pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0
    jupyter-core==5.3.0  # jupyter-core
      platformdirs==3.2.0  # platformdirs>=2.5
      traitlets==5.9.0  # traitlets>=5.3
    traitlets==5.9.0  # traitlets>=5.1
  matplotlib==3.7.1  # matplotlib
    contourpy==1.0.7  # contourpy>=1.0.1
      numpy==1.24.2  # numpy>=1.16
    cycler==0.11.0  # cycler>=0.10
    fonttools==4.39.3  # fonttools>=4.22.0
    kiwisolver==1.4.4  # kiwisolver>=1.0.1
    numpy==1.24.2  # numpy>=1.20
    packaging==23.0  # packaging>=20.0
    Pillow==9.5.0  # pillow>=6.2.0
    pyparsing==3.0.9  # pyparsing>=2.3.1
    python-dateutil==2.8.2  # python-dateutil>=2.7
      six==1.16.0  # six>=1.5
  numpy==1.24.2  # numpy
  fpdf2==2.7.3  # fpdf2
    defusedxml==0.7.1  # defusedxml
    Pillow==9.5.0  # Pillow!=9.2.*,>=6.2.2
    fonttools==4.39.3  # fonttools>=4.34.0
  seaborn==0.12.2  # seaborn
    numpy==1.24.2  # numpy!=1.24.0,>=1.17
    pandas==1.5.3  # pandas>=0.25
      python-dateutil==2.8.2  # python-dateutil>=2.8.1
        six==1.16.0  # six>=1.5
      pytz==2023.3  # pytz>=2020.1
      numpy==1.24.2  # numpy>=1.21.0; python_version >= "3.10"
    matplotlib==3.7.1  # matplotlib!=3.6.1,>=3.1
      contourpy==1.0.7  # contourpy>=1.0.1
        numpy==1.24.2  # numpy>=1.16
      cycler==0.11.0  # cycler>=0.10
      fonttools==4.39.3  # fonttools>=4.22.0
      kiwisolver==1.4.4  # kiwisolver>=1.0.1
      numpy==1.24.2  # numpy>=1.20
      packaging==23.0  # packaging>=20.0
      Pillow==9.5.0  # pillow>=6.2.0
      pyparsing==3.0.9  # pyparsing>=2.3.1
      python-dateutil==2.8.2  # python-dateutil>=2.7
        six==1.16.0  # six>=1.5
  raiutils==0.4.0  # raiutils
    numpy==1.24.2  # numpy
    pandas==1.5.3  # pandas
      python-dateutil==2.8.2  # python-dateutil>=2.8.1
        six==1.16.0  # six>=1.5
      pytz==2023.3  # pytz>=2020.1
      numpy==1.24.2  # numpy>=1.21.0; python_version >= "3.10"
    requests==2.28.2  # requests
      charset-normalizer==3.1.0  # charset-normalizer<4,>=2
      idna==3.4  # idna<4,>=2.5
      urllib3==1.26.15  # urllib3<1.27,>=1.21.1
      certifi==2022.12.7  # certifi>=2017.4.17
    scikit-learn==1.2.2  # scikit-learn
      numpy==1.24.2  # numpy>=1.17.3
      scipy==1.10.1  # scipy>=1.3.2
        numpy==1.24.2  # numpy<1.27.0,>=1.19.5
      joblib==1.2.0  # joblib>=1.1.1
      threadpoolctl==3.1.0  # threadpoolctl>=2.0.0
    scipy==1.10.1  # scipy
      numpy==1.24.2  # numpy<1.27.0,>=1.19.5

From the above output, we can see that one of the direct dependencies of data-understand, pandas is pinned at 1.5.3. The good thing about deptree is that it recursively gives the dependencies of dependencies. So its output is helpful in case one of the indirect dependencies of your Python package is causing any run-time problems.