Python Versions, Modules, Homebrew, pyenv

Step 1: Meet your default python

You are on Mac OS. When you run python in your terminal you see this. You have Python 2.7.10.

This is your default python. You can do things like add 8+8 or print things with print 12345

Step 2: Meet your modules

From your interpreter you can see your available modules by asking for “help”

> help("modules)"

Note that you have a bunch of modules – including “antigravity”!!! Try it!

> import antigravity

But where are these modules?

For this you need to import the “sys” module – which gives you access to the “system” – and print your “path” – which will print *most* of the folders that python will search for your modules.

import sys

print(sys.path)

* Note: if you ls into any of these folders, you will find a bunch of modules

Wait, every tutorial and most companies now uses Python version 3x

Most tutorials will tell you to brew install python to get the latest version of python.

Brew install walkthrough

the main job of brew is to make it easier to install packages to your mac

Step 1: download Homebrew

brew hosts its package information here as “formulae“. (get it? a brew formula?)

*Note that the python package is version 3.70 as of today.

This entire list is simply a list of packages with additional information including where to download it.

So what happens when I run brew install?

When you run the command, brew will look for the package in the formulae list and run the installation script for the package.

Brew installs *all* packages into its own directory at  /usr/local/Cellar .

Take a look with $ ls /usr/local/Cellar.

After installation, brew creates a symlink which is basically a shortcut that will allow your system to run the new command

Step 2: $ brew install python --verbose

* Note this line that symlinks Cellar/python/3.7.0/bin/python3 with python3

 

Wait, not all libraries use the latest version of python.

As of today, tensorflow DOES NOT support version 3.7. Can I downgrade my python version with brew?

Well, not easily… if you already have the older version downloaded, you can run brew switch python #.#.

But if you don’t have it downloaded, you’ll need to revert brew and do unrecommended stuff.

So how do I juggle versions of python?

pyenv walkthrough

The main job of pyenv is to manage which version of python you prefer to run.

How?

pyenv prepends a directory to your $PATH variable and hijacks every python or pip command you run. Then it tries to find your “preferred” python version to run.

How does it know my preferred version?

pyenv will first check the PYENV_VERSION environment variable (if set), then will move onto finding a .python-version file in the current directory, then look in the parent directory, then that parents directory, all the way up to the root. If it doesn’t find that file, it will search for a $(pyenv root)/version file and if it doesn’t find that, it will simply use your “system” python.

Step 1: brew install pyenv

Step 2: add the pyenv init command to your .bash_profile with this command

$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile

Step 3: run your bash profile to make sure the changes propagate

source ~/.bash_profile

Now, if you run python you will see that you are still on version 2.7.10 (or whatever your system python was from the start). This is because you have not yet set your preferred python anywhere else, and the search landed on your system python.

Use pyenv install --list to list all available python versions

Step 4: install a version of python that works with your favorite library

$ pyenv install 3.5.5

* I’m choosing 3.5.5 which I believe is the latest compatible with tensorflow

Step 5: create a .python-version file in your project

$ echo 3.5.5 > .python-version

Step 6: run python

$ python

 

Bonus Points:

You will eventually want to manage your package versions

Step 1: create a virtual environment in a directory we’ll call env

Note: env is arbitrary but a good convention – a folder called env will be created where you run this command

$ virtualenv env --system-site-packages -p `pyenv which python`

Step 2: activate your environment 

$ source env/bin/activate

Now, anytime you run pip install <anything> all of your packages will be downloaded into your new env folder without being installed in your global python PATH.

Step 3: pip install to your hearts desire

pip install --upgrade tensorflow

Levels of installing python

This is an opinionated route towards python management. Every computer may have its own set up.

Level 0: “I just want to play with python”

Method 1: Use and online REPL like this: https://repl.it/languages/python

Level 1: “I want python on my computer”

Download brew, use brew to download the latest version of python and go crazy.

$ brew install python

Level 2: “I want to manage python packages for a project”

Use virtualenv to create virtually separated environments for each of your projects. Make sure to “freeze” your requirements to a file so you record every version number of every downloaded package.

(Optional): use virtualenv-wrapper to manage all the virtualenv‘s you will eventually create

Level 3: “I need to manage python versions now”

Use pyenv to manage python versions per project and check this into version control. pyenv uses a .python-version file in your project to determine which python to use for the project.

virtualenv can still be used to manage package versions

Optional: use pyenv-virtualenv which is a pyenv plugin to “automate” the creation of virtualenv‘s