Coding¶

Environments¶

To avoid impacting the laptop (Mac) python core installation, use virtual environment:

python3 -m venv .venv
source .venv/bin/activate

Then in each main folder there is a requirements.txt to get the necessary modules.

There are a lot of other solutions we can use, like the Amazon scikit-learn image. The SageMaker team uses this repository to build its official Scikit-learn image. we can build an image via:

docker build -t sklearn-base:1.2-1 -f https://raw.githubusercontent.com/aws/sagemaker-scikit-learn-container/master/docker/1.2-1/base/Dockerfile.cpu .

VSCode¶

Jupyter Notebook¶

• To select an environment, use the Python: Select Interpreter command from the Command Palette (⇧⌘P)
• Use Create: New Jupyter Notebook from command Palette
• Select a kernel using the kernel picker in the top right.
• Within a Python Notebook, it's possible to view, inspect, sort, and filter the variables within the current Jupyter session, using Variables in toolbar.
• We can offload intensive computation in a Jupyter Notebook to other computers by connecting to a remote Jupyter server. Use server URL with security token.

Run Kaggle image¶

As an alternate Kaggle has a more complete docker image to start with.

# CPU based
docker run --rm -v $(pwd):/home -it gcr.io/kaggle-images/python /bin/bash
# GPU based
docker run -v $(pwd):/home --runtime nvidia --rm -it gcr.io/kaggle-gpu-images/python /bin/bash

Conda¶

Conda provides package, dependency, and environment management for many languages.

On Mac M1 we need ARM64 architecture.

1. Install miniconda: projects/miniconda
2. To create a conda environment named "torch", in miniconda3 folder do: conda env create -f torch-conda-nightly.yml -n torch

3. Activate conda environment:

   conda activate torch
   

4. Register environment: python -m ipykernel install --user --name pytorch --display-name "Python 3.9 (pytorch)"

5. Install the following: conda install pytorch pandas scikit-learn
6. Start Jupiter: jupiter notebook
7. Execute the notebook in to test test-env.ipynb

Install Conda on Windows.

Run Jupyter notebooks¶

• We can use jupyter lab (see installation options) or conda and miniconda.

  conda install -y jupyter
  

JupyterLab¶

The following works as of April 2023:

pip3 install jupyterlab
# build the assets
jupyter-lab build
# The path is something like
# /opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/share/jupyter/lab
# Start the server
jupyter-lab

Once started, in VScode select a remote Python kernel and Jupiter extension to run the notebook inside it.

Important Python Libraries¶

numpy¶

• Array computing in Python. Getting started
• NumPy dimensions are called axes.

import numpy as np
a = np.array([2, 3, 4])
b = np.ones((2, 3, 4), dtype=np.int16)
c = np.zeros((3, 4))

```python
A = np.array([[1, 1], [0, 1]])
B = np.array([[2, 0], [3, 4]])
A @ B
A.dot(B)
```

scipy¶

SciPy is a collection of mathematical algorithms and convenience functions built on NumPy .

• Get a normal distribution function: use the probability density function (pdf)

from scipy.stats import norm
x = np.arange(-3, 3, 0.01)
y=norm.pdf(x)

MatPlotLib¶

Persent figure among multiple axes, from our data.

• Classic import

import matplotlib.pyplot as plt
import numpy as np

import matplotlib as mpl

• See Notebook

Seaborn¶

Seaborn provides a high-level interface for drawing attractive and informative statistical graphics. Based on top of MatPlotLib and integrated with Pandas.

See the introduction for different examples

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_theme()
sns.relplot(
    data=masses_data,
    x="age", y="shape", 
    hue="density", size="density"
)
plt.show()

PyTorch¶

Via conda or pip, install pytorch torchvision torchaudio.

Example of getting started code in deep-neural-net folder.

Summary of the library and deeper studies

Code samples¶