Installing WSL on Windows 11 and setting up environment for Data Science

In this blog we will look at setting up local environment with WSL with a focus on Data Science.

Let's quickly understand WSL. WSL stands for Windows Subsystem for Linux. It's a feature introduced in Microsoft Windows 10 (usable in Windows 11) that enables users to run native Linux command-line tools and utilities directly on Windows, without the need for a virtual machine or dual boot setup.

WSL provides a Linux-compatible environment for running popular open-source tools such as the Bash shell, and common Linux command-line utilities like sed, awk, and grep. This makes it easier for developers and users who are already familiar with Linux to use Windows and take advantage of the features and performance of the Windows operating system while still having access to the familiar tools and utilities they rely on in Linux.

Install WSL and Distro

To get started open the Command Line as an Administrator and run the below command. I've selected ubuntu as my main Distro but you can select whichever Distro you prefer. Refer to official documentation.

wsl --install -d DISTRO-NAME
# e.g wsl --install -d ubuntu

Install Miniconda

Miniconda is a lightweight version of the Anaconda Python distribution. It contains an open-source distribution of Python for scientific computing that includes many commonly used scientific packages and tools.

Unlike the full Anaconda distribution, which includes a large number of packages, Miniconda only includes a minimal set of packages required to get started with Python development, such as the Python interpreter, the conda package manager, and a few essential libraries.

Miniconda allows users to create a custom environment with only the packages they need for a specific project, making it a more lightweight and flexible option for Python development. It also allows users to easily install, update, and remove packages using the conda package manager, which simplifies the management of dependencies and ensures that all required packages are compatible with each other.

Steps to Install

First step is to open the WSL terminal is to download the shell script to install Miniconda. You can also refer to official documentation to install Miniconda (https://repo.anaconda.com/miniconda/).

sudo apt-get update
sudo apt-get install wget
wget https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-Linux-x86_64.sh

You can now check it downloaded is correctly

sha256sum Miniconda3-py39_4.12.0-Linux-x86_64.sh

After this you run the bash scrip to install Miniconda Run shell script

bash ./Miniconda3-py39_4.12.0-Linux-x86_64.sh

Install Docker

Docker is a platform that enables developers to build, package, and deploy applications in lightweight, portable containers. Containers are a type of virtualization technology that allow multiple applications to run on a single host operating system without interfering with each other.

Docker provides a set of tools and APIs that make it easy to create and manage containers, and also includes a registry for storing and sharing container images. With Docker, developers can build an application and package it along with all of its dependencies into a single container, which can then be easily distributed and run on any system that supports Docker.

Steps to Install

Download docker from https://docs.docker.com/desktop/windows/wsl/#download and install on your windows device normally.

Open Docker and tick the checkbox for Use the WSL 2 based engine.

Next, check the integration with your default WSL distro or manually select the distro

Tips and tricks

Clean up folders in WSL

Quick tip deleting *:Zone.Identifier files which occur because of copying files from local to WSL2 environment

find . -name "*:Zone.Identifier" -type f -delete

Test running docker GPU

You may want to check that WSL can detect and use your gpu. Below command is a quick way to run a benchmark using docker.

 sudo docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

How to increase size of virtual disk size of WSL2

You can refer to this web page to increase the virtual size of the WSL installationhttps://learn.microsoft.com/en-us/windows/wsl/disk-space#how-to-locate-the-vhdx-file-and-disk-path-for-your-linux-distribution