Pipenv Tutorial - the official package control for python

Why use Pipenv:

• Pipenv is essentially a combination of pip and virtualenv (i.e. makes virtual environment), but with better version control
• It solves the version conflict for packages and sub-dependencies
• It could automatically install the latest package/dependency that meet all the requirements
• It is the new STANDARD for python packages and is OFFICIALLY used

How to use it

• Installation: pip install pipenv

  • After it is installed, use pipenv instead of pip in the future for any package install

• Create/Enter virtual environment: pipenv shell

  • This will create a virtual environment if one doesn’t already exist in a default location.
  • Here is a list of environment variable to setup in order to change the default behavior
  • You can force the creation of a Python 2 or 3 environment with the arguments --two and --three respectively.
    • More specific version can be specified as -python 3.6
  • Quit virtual environment: ctrl + D

• Install package: same as using pip, but use pipenv instead

  • After the installation, two files are generated: Pipfile and Pipfile.lock, which are used for package version control
  • If the package cannot be installed due to package conflict, use the --skip-lock argument to skip the package version lock.
    • For example, pipenv install django --skip-lock

• Install from version control system (i.e. github): pipenv install -e git+https://github.com/requests/requests.git#egg=requests

• Install package for different environment (i.e. dev environment): pipenv install pytest --dev

  • Providing the --dev argument will put the dependency in a special [dev-packages] location in the Pipfile. These development packages only get installed if you specify the --dev argument with pipenv install.
  • The different sections separate dependencies needed only for development from ones needed for the base code to actually work.

• Lock the environment for production: pipenv lock

  • This will create/update your Pipfile.lock, which you’ll never need to (and are never meant to) edit manually. You should always use the generated file.

• Install the environment in production: pipenv install --ignore-pipfile

  • This tells Pipenv to ignore the Pipfile for installation and use what’s in the Pipfile.lock.
  • The lock file enables deterministic builds by taking a snapshot of all the versions of packages in an environment (similar to the result of a pip freeze).

• Setup the development environment: pipenv install --dev

  • This installs all the dependencies needed for development, which includes both the regular dependencies and those you specified with the --dev argument during install.
  • When an exact version isn’t specified in the Pipfile, the install command gives the opportunity for dependencies (and sub-dependencies) to update their versions.

Pipenv’s Dependency Resolution Approach

• Pipenv will attempt to install sub-dependencies that satisfy all the requirements from your core dependencies.
• If there are conflicting dependencies, pipenv will not be able to create a lock file and wil output an error
• Show dependency graph for top-level dependencies and their sub-dependencies: pipenv graph
• Show dependency graph for the sub-dependencies with the parent: pipenv graph --reverse

The Pipfile

• Pipfile intends to replace requirements.txt.

• Pipenv is the official package management tool recommended by Python itself.

• You shouldn’t have any sub-dependencies in your Pipfile.

  • You should only include the packages you actually import and use. Pipenv will install the dependencies automatically.

The Pipfile.lock

• This file enables deterministic builds by specifying the exact requirements for reproducing an environment.
• It contains exact versions for packages and hashes to support more secure verification.
• Its in JSON format
• You should never change this file by hand.

Pipenv Extra Features

• Open a third-party package in your default editor: pipenv open flask

  • This will open the flask package in the default editor
  • You can specify a program with an EDITOR environmental variable.
    • i.e. EDITOR=subl for sublime text

• Run a command in the virtual environment without launching a shell: pipenv run <insert command here>

• Check for security vulnerabilities: pipenv check

• Uninstall a package: pipenv uninstall numpy

• Wipe all the installed packages from virtual environment: pipenv uninstall --all

  • You can replace --all with --all-dev to just remove dev packages.

• Pipenv supports the automatic loading of environmental variables when a .env file exists in the top-level directory.

  • When you pipenv shell to open the virtual environment, it loads your environmental variables from the file.

• To find out where your virtual environment is: pipenv --venv

• To find out where your project home is: pipenv --where

• Generate requirements.txt file from Pipfile:

  • pipenv lock -r > requirements.txt
  • pipenv lock -r -d > dev-requirements.txt

What if you already have a requirements.txt file

• pipenv install should automatically detect the requirements.txt and convert it to a Pipfile

  • You should change Pipfile to only specify exact versions if requirements.txt file uses exact versions

• If things break because of a dependency:

  • Update code to function with the new version of the dependency.
    • If backward compatibility with previous versions of the dependency isn’t possible, you have to bump the required version in Pipfile like numpy = ">=1.15"
  • Restrict the version of the dependency in the Pipfile to be < the version that just broke your code: numpy = ">=1.14.1,<1.15"

• You could just install from the requirement file as usual: pipenv install -r requirements.txt

  • If you have a dev version or something similar, you can add the --dev argument as follows: pipenv install -r dev-requirements.txt --dev

Work flow for any project

1. Get pipenv: pip install pipenv
2. Create a virtual environment: pipenv shell
3. Setup local development: pipenv install --dev
4. Work on the code, use version control such as git
5. After done development, lock the environment: pipenv lock
6. When finished working, quit environment: ctrl + D
7. Setup production: pipenv install --ignore-pipfile
8. Repeate steps 3-6 for development

Reference

• Pipenv: A Guide to the New Python Packaging Tool
• Basic Usage of Pipenv
• Pipenv Cheatsheet | feici02’s tidbits