Tutorial 4 - Virtual Environments

A virtual environment is a tool that helps to keep dependencies required by different projects separate by creating isolated Python virtual environments for them. Virtual environments create containers for each project, so that the Python interpreter, libraries, and scripts installed in one virtual environment are isolated from those installed in other virtual environments, as well as they are isolated from the libraries and packages that are installed in the main Python installation.

Therefore, where we create and activate a virtual environment for a specific project, the project becomes an independent application, which is isolated from the system-installed Python and from all other Python libraries. This project-specific virtual environment provides its own Python interpreter, and its own pip to install libraries, separated from other Python libraries.

By creating and managing separate environments for different projects, there is no chance of breaking currently installed packages in other environments. It also helps with reproducibility among developers and researchers working on large projects. For instance, if you have several different projects with their own virtual environments, they can have different versions of a library: for example, one project can use TensorFlow 1.12, and another can use TensorFlow 2.5. This way, we won’t worry whether an update to the TensorFlow library in the main system-installed Python would impact the code in all our projects.

There are several tools for managing virtual environments. Python 3.3 provides a standard module venv, which is most commonly used for managing virtual environments. It allows to manage separate package versions for different projects. When creating a new project, we can simply create a new virtual environment.

• The full official documentation for venv can be found here

• The full official user guide for venv can be found here

• The PEP proposal for venv can be found here

If you are looking for practical examples, it is recommended to consult the user guide. However, if you are looking for more information about specific details of venv, consulting the full documentation is recommended.

Installing venv

The module venv should be included in the standard Python library.

If for some reason it is not available on Linux Systems, you will need to install the python3-venv package using the following command :

sudo apt install python3-venv (Ubuntu/Debian-based system)

Creating a Virtual Environment

To create a virtual environment called test_env, run the following command:

python3 -m venv path_to_new_virtual_environment/test_env (in Unix/MacOS)

python -m venv path_to_new_virtual_environment\test_env (in Windows)

If we want to create a virtual environment in the current working directory, we can omit the path_to_new_virtual_environment in the above code.

Activating a Virtual Environment

Before we can start installing packages in the virtual environment, we must activate it. Doing so will put the virtual environment-specific Python and pip executables in your shell’s PATH.

To activate a virtual environment test_env, run the following command:

source path_to_virtual_environment/test_env/bin/activate (in Unix/MacOS)

path_to_virtual_environment\test_env\Scripts\activate (in Windows)

If you are in the same directory where the virtual environment is located, you don’t need to specify the path to the virtual environment, and you can just use test_env/bin/activate or test_env\Scripts\activate.

To confirm that the virtual environment has been activated you can check the location of your Python interpreter:

which python or whereis python (in Unix/MacOS)

where python (in Windows)

As long as the environment is active, you’ll be able to import packages installed in the environment.

To leave the environment run:

deactivate

Installing Packages

To install packages, first make sure that the environment is active. Installing packages is done simply with with pip, as you would normally install packages. For example, to install requests (a popular library for making HTTP requests):

python3 -m pip install requests (in Unix/MacOS)

pip install requests (in Windows)

To check the list of all packages installed in the newly created virtual environment, use:

pip list

Similarly, we can generate a text file listing all installed libraries in a virtual environment with:

pip freeze > requirements.txt

This can be convenient, because if other users would like to replicate your virtual environment, instead of installing all libraries one by one, they can just run:

pip install -r requirements.txt

Conda Environment

Conda Env is a self-contained and isolated workspace within the Conda package management system, similar to venv. It allows to create and manage specific environments for different projects or applications, each with its own set of packages and dependencies.

The full official documentation for Conda Env can be found here.

Conda Env is pre-installed with Anaconda.

Creating a Conda Env with Commands

1. Open the terminal or an Anaconda Prompt.

2. Type: conda create --n myenv (or you can specify the python version: conda create -n myenv python=3.9).

3. Type y to proceed.

To activate a Conda Env, type:

conda activate myenv

To list all installed conda environments, type:

conda env list

To activate the base conda environment from the command prompt in Windows, just type:

conda activate

To deactivate the Conda Env you are in, type:

conda deactivate

Install Packages in a Conda Env

To install scikit-learn, for example, using Conda commands, type:

conda install -c anaconda scikit-learn

Or, you can use pip install as in:

pip install -U scikit-learn

The main difference between using conda install and pip install is in the dependency resolution approach.

• conda install: conda has a more powerful dependency resolution mechanism. It ensures that all dependencies (both Python and non-Python) are compatible with each other. It often prevents conflicting versions by resolving the entire environment when installing or updating packages.

• pip install: pip only handles Python dependencies and doesn’t handle system or non-Python libraries. Installing packages with pip can sometimes result in dependency conflicts, especially if different packages require incompatible versions of the same dependency.

BACK TO TOP