Python Versions, Modules, Homebrew, pyenv

Meet your default python

You are on Mac OS. When you run python in your terminal you see this. You have Python 2.7.10.

This is your default python. You can do things like add 8+8 or print things with print 12345

Meet your modules

From your interpreter you can see your available modules by asking for “help”

> help("modules)"

Note that you have a bunch of modules – including “antigravity”!!! Try it!

> import antigravity

But where are these modules?

For this you need to import the “sys” module – which gives you access to the “system” – and print your “path” – which will print *most* of the folders that python will search for your modules.

import sys

print(sys.path)

* Note: if you ls into any of these folders, you will find a bunch of modules

Wait, every tutorial and most companies now uses Python version 3x

Most tutorials will tell you to brew install python to get the latest version of python.

Brew install walkthrough

the main job of brew is to make it easier to install packages to your mac

Step 1: download Homebrew

brew hosts its package information here as “formulae“. (get it? a brew formula?)

*Note that the python package is version 3.70 as of today.

This entire list is simply a list of packages with additional information including where to download it.

So what happens when I run brew install?

When you run the command, brew will look for the package in the formulae list and run the installation script for the package.

Brew installs *all* packages into its own directory at  /usr/local/Cellar .

Take a look with $ ls /usr/local/Cellar.

After installation, brew creates a symlink which is basically a shortcut that will allow your system to run the new command

Step 2: $ brew install python --verbose

* Note this line that symlinks Cellar/python/3.7.0/bin/python3 with python3

 

Wait, not all libraries use the latest version of python.

As of today, tensorflow DOES NOT support version 3.7. Can I downgrade my python version with brew?

Well, not easily… if you already have the older version downloaded, you can run brew switch python #.#.

But if you don’t have it downloaded, you’ll need to revert brew and do unrecommended stuff.

So how do I juggle versions of python?

pyenv walkthrough

The main job of pyenv is to manage which version of python you prefer to run.

How?

pyenv prepends a directory to your $PATH variable and hijacks every python or pip command you run. Then it tries to find your “preferred” python version to run.

How does it know my preferred version?

pyenv will first check the PYENV_VERSION environment variable (if set), then will move onto finding a .python-version file in the current directory, then look in the parent directory, then that parents directory, all the way up to the root. If it doesn’t find that file, it will search for a $(pyenv root)/version file and if it doesn’t find that, it will simply use your “system” python.

Step 1: brew install pyenv

Step 2: add the pyenv init command to your .bash_profile with this command

$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile

Step 3: run your bash profile to make sure the changes propagate

source ~/.bash_profile

Now, if you run python you will see that you are still on version 2.7.10 (or whatever your system python was from the start). This is because you have not yet set your preferred python anywhere else, and the search landed on your system python.

Use pyenv install --list to list all available python versions

Step 4: install a version of python that works with your favorite library

$ pyenv install 3.5.5

* I’m choosing 3.5.5 which I believe is the latest compatible with tensorflow

Step 5: create a .python-version file in your project

$ echo 3.5.5 > .python-version

Step 6: run python

$ python

 

Bonus Points:

You will eventually want to manage your package versions

Step 1: create a virtual environment in a directory we’ll call env (you can choose any name)

$ virtualenv env --system-site-packages -p `pyenv which python`

Step 2: activate your environment 

$ source env/bin/activate

Now, anytime you run pip install anything all of your packages will be downloaded into your new env folder without being installed in your global python.

Step 3: pip install to your hearts desire

pip install --upgrade tensorflow

Levels of installing python

Level 0: “I just want to play with python”

Method 1: Use and online REPL like this: https://repl.it/languages/python

Level 1: “I want python on my computer”

Method 1: just download the latest version of python and go crazy.

Method 2: download brew, use brew to download the latest version of python and go crazy.

Level 2:”I want to properly manage python packages”

Method 1: use virtualenv to create virtually separated environments for each of your projects. Make sure to “freeze” your requirements to a file so you record every version number of every downloaded package.

Extra credit: use virtualenv-wrapper to manage all the virtualenv‘s you will eventually create

Level 9000: “I need to manage package versions AND python versions across computers”

Method 1: use pyenv to manage python versions per project and check this into version control, virtualenv to manage package versions

Extra credit: use pyenv-virtualenv which is a pyenv plugin to “automate” the creation of virtualenv‘s

 

* other methods are appreciated