Postgres tutorial – installing with brew + basics

Step 1: Install postgres

brew install postgres

Step 2: Start postgres server

brew services start postgresql

Step 3: Create a database

Lets call our database “book”

$ createdb book

Step 4: open the psql shell

List all databases in terminal with: $ psql --list

Adding the name of the db after the command psql will open the shell to that db

$ psql book

Your psql shell has different command. For the rest of the article, we will be using the psql shell.

List your databases in psql shell with: \l

* When checking the docs make sure you check know your postgres version with: select version();

Step 5: Create a new table

list your current database tables with: \dt

Create format:

CREATE TABLE <table_name> (<field_name>, <field_type>);

Lets create fields for “body”, “date” and “code”:

CREATE TABLE dids (body text, date timestamp, code text);

Inspect your table schema with: \d <table_name>, e.g. \d dids

Step 6: Add some table rows

Remember: SINGLE QUOTES ONLY and remember the ending semi colon

INSERT INTO dids (body, date)

  VALUES ('paired with Corey', NOW());

INSERT INTO dids (body, date)

  VALUES ('remembered that SQL is SINGLE QUOTES ONLY', NOW());

 

OOPS… – we realize here that our table does not have a primary key

Step 7: Alter your table columns (Schema migration)

Your tables will likely evolve throughout time – you may want to add fields to you table, or like above, we forget to add a primary key.

Add a column for “primary key” that auto increments

ALTER TABLE dids ADD COLUMN id serial primary key;

look at the table schema with: \d <table_name>

Step 8: Query your new database!

select * from dids

 

Installed with:

Homebrew 1.7.1

postgres (PostgreSQL) 10.4

Heroku: Node, Express, PostgreSQL Setup

Step 1: Install Heroku

Install the heroku cli with homebrew

$ brew install heroku/brew/heroku

Check your heroku installation: 

$ heroku --version

Note: heroku will check for the latest version every* time you run the command

Check what apps you have already created:

$ heroku apps

Step 2: Create a new named heroku app

$ heroku create <app_name>

If you run this command with the <app_name> heroku will choose a new random name for you.

In the picture above, https://theptrk.herokuapp.com is the new url for your blank app.

* this is the default site for your new app that sends a 502 HTTP code

Check that your app has been created

$ heroku apps

Step 3: Initialize your app to your heroku git url

Create a folder, cd into it and initialize your git repo.

$ mkdir theptrk; cd theptrk; git init

You now need to set up your git remote url

“””With Github you might be used to using the word origin to mean your Github url, but the convention for heroku is to name this remote “heroku”. Note that this remote name is completely arbitrary and has nothing to do with the platform or url or anything.”””

Use git remote --verbose to check if you have any remote urls set up. There should be none

Get your apps “info” with info

$ heroku info <app_name>

With your git url add your remote:

$ git remote add heroku <git_url>

$ git push --set-upstream heroku master

* Notice that once you set this and push, you no longer need to specify the --app=<app_name>

** Note that heroku commands return different things depending on where you are.

 

Step 4: Set up node.js, install express.js

Create a package.json 

This file will be used by node/npm to indicate meta data, dependencies and other project level info

$ npm init

Install express.js

$ npm install express --save

Set the node version package.json to the one on your local machine.

You can check your local version with $ node --version

"engines": {
   // use your local version to stay consistent
   // if you want to push a newer node version, update your local node version
   // use nvm for the best way to handle local node versions
  "node": "10.6.0"
},

Create a “start” script

This is the script which is run whenever the command npm start is run.

Heroku will use this “start’ script as the default script if no Procfile is found.

"scripts": {
  "start": "node index.js"
}

curl express hello world (link)

curl -L https://bit.ly/express-hello-world > index.js

Run this server locally with 

$ heroku local web or $ npm start

Check out your website at http://localhost:5000/

At this point, you can git push heroku master

Step 5: Installing PostgreSQL instance for your app

Postgres instances are created and assigned to individual apps as “add-ons”.

Use this command to see all your existing add-ons.

$ heroku addons

** Note that heroku addons performs differently depending on where you are. If heroku recognizes that you are in a heroku app, it will only show you the add-ons for only that app. cd to a directory that is not a heroku app (say your home directory) and heroku will show you all add-ons for your account.

Create your PostgreSQL instance:

$ heroku addons:create heroku-postgresql:hobby-dev --app=<app_name>

Get more info with: $ heroku pg:info*

* if you navigated to a different directory, you may need to specify the app like this heroku pg:info --app theptrk

Congrats! You now have a heroku app instance and PostgreSQL instance.

Connecting to your PostgreSQL instance:

If you want to connect your app to your db, note that heroku automatically created a DATABASE_URL config variable for you. Type heroku config to see your current config variables. You can access this variable with process.env.DATABASE_URL in your application code.

Python Versions, Modules, Homebrew, pyenv

Meet your default python

You are on Mac OS. When you run python in your terminal you see this. You have Python 2.7.10.

This is your default python. You can do things like add 8+8 or print things with print 12345

Meet your modules

From your interpreter you can see your available modules by asking for “help”

> help("modules)"

Note that you have a bunch of modules – including “antigravity”!!! Try it!

> import antigravity

But where are these modules?

For this you need to import the “sys” module – which gives you access to the “system” – and print your “path” – which will print *most* of the folders that python will search for your modules.

import sys

print(sys.path)

* Note: if you ls into any of these folders, you will find a bunch of modules

Wait, every tutorial and most companies now uses Python version 3x

Most tutorials will tell you to brew install python to get the latest version of python.

Brew install walkthrough

the main job of brew is to make it easier to install packages to your mac

Step 1: download Homebrew

brew hosts its package information here as “formulae“. (get it? a brew formula?)

*Note that the python package is version 3.70 as of today.

This entire list is simply a list of packages with additional information including where to download it.

So what happens when I run brew install?

When you run the command, brew will look for the package in the formulae list and run the installation script for the package.

Brew installs *all* packages into its own directory at  /usr/local/Cellar .

Take a look with $ ls /usr/local/Cellar.

After installation, brew creates a symlink which is basically a shortcut that will allow your system to run the new command

Step 2: $ brew install python --verbose

* Note this line that symlinks Cellar/python/3.7.0/bin/python3 with python3

 

Wait, not all libraries use the latest version of python.

As of today, tensorflow DOES NOT support version 3.7. Can I downgrade my python version with brew?

Well, not easily… if you already have the older version downloaded, you can run brew switch python #.#.

But if you don’t have it downloaded, you’ll need to revert brew and do unrecommended stuff.

So how do I juggle versions of python?

pyenv walkthrough

The main job of pyenv is to manage which version of python you prefer to run.

How?

pyenv prepends a directory to your $PATH variable and hijacks every python or pip command you run. Then it tries to find your “preferred” python version to run.

How does it know my preferred version?

pyenv will first check the PYENV_VERSION environment variable (if set), then will move onto finding a .python-version file in the current directory, then look in the parent directory, then that parents directory, all the way up to the root. If it doesn’t find that file, it will search for a $(pyenv root)/version file and if it doesn’t find that, it will simply use your “system” python.

Step 1: brew install pyenv

Step 2: add the pyenv init command to your .bash_profile with this command

$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile

Step 3: run your bash profile to make sure the changes propagate

source ~/.bash_profile

Now, if you run python you will see that you are still on version 2.7.10 (or whatever your system python was from the start). This is because you have not yet set your preferred python anywhere else, and the search landed on your system python.

Use pyenv install --list to list all available python versions

Step 4: install a version of python that works with your favorite library

$ pyenv install 3.5.5

* I’m choosing 3.5.5 which I believe is the latest compatible with tensorflow

Step 5: create a .python-version file in your project

$ echo 3.5.5 > .python-version

Step 6: run python

$ python

 

Bonus Points:

You will eventually want to manage your package versions

Step 1: create a virtual environment in a directory we’ll call env (you can choose any name)

$ virtualenv env --system-site-packages -p `pyenv which python`

Step 2: activate your environment 

$ source env/bin/activate

Now, anytime you run pip install anything all of your packages will be downloaded into your new env folder without being installed in your global python.

Step 3: pip install to your hearts desire

pip install --upgrade tensorflow

Levels of installing python

Level 0: “I just want to play with python”

Method 1: Use and online REPL like this: https://repl.it/languages/python

Level 1: “I want python on my computer”

Method 1: just download the latest version of python and go crazy.

Method 2: download brew, use brew to download the latest version of python and go crazy.

Level 2:”I want to properly manage python packages”

Method 1: use virtualenv to create virtually separated environments for each of your projects. Make sure to “freeze” your requirements to a file so you record every version number of every downloaded package.

Extra credit: use virtualenv-wrapper to manage all the virtualenv‘s you will eventually create

Level 9000: “I need to manage package versions AND python versions across computers”

Method 1: use pyenv to manage python versions per project and check this into version control, virtualenv to manage package versions

Extra credit: use pyenv-virtualenv which is a pyenv plugin to “automate” the creation of virtualenv‘s

 

* other methods are appreciated

what is curl?

curl is tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, POP3 etc)

What can I do with curl?

You can get the weather:

$ curl https://wttr.in

You can download hacker news

$ curl https://news.ycombinator.com/

You can read a file from Github

$ curl https://raw.githubusercontent.com/facebook/react/master/README.md

You can even download files with the > command

curl -L https://bit.ly/express-hello-world > index.js

 

curl express hello world

The official express.js hello world example may crash on certain deployments that rely on a certain PORT.

Make sure you use the PORT set on process.env if it is set.

const PORT = process.env.PORT || 3000

Complete example (code):

const express = require('express')
const app = express()
const PORT = process.env.PORT || 3000
app.get('/', (req, res) => res.send('Hello World!'))
app.listen(PORT, () => console.log(`Example app listening on port ${PORT}!`))

Expert level: How to curl this into an index.js file

I’ve created a bit.ly link that links to the raw version of a file called “hello-world” in my repo “code

How to: curl it and create an index.js file

curl -L https://bit.ly/express-hello-world > index.js

 


More about curl

What if we build a knowledge map?

What is a core unit of knowledge?

Can you map out a graph of competence based on individual nodes of ability?

What does it take to implement a merge sort?

Unit: Have a computer

Unit: install a programming language

Unit: download a text editor

Unit: know how to type

… etc, etc,

This can possibly be infinite…

 

But what if you added a goal?

Goal: Implement binary search in javascript with arrays

I need to learn

  • how to iterate over a smaller and smaller range of indices of an array and terminate when done
  • how to return from a function
  • how to change the value of variables
  • how to perform a conditional check given an element
  • how to access elements by index
  • how to get the index of the last element in an array
  • how to access an element based on the midpoint of x and y
  • how to create multiple variables,
    • one pointing to x (type number)
    • one pointing to y (type number)
    • one to indicate success or failure (type boolean)
  • how to create functions in js
  • how to create variables in js
  • how to find the midpoint of the range (x, y)
  • how to create an array

This is likely what you will need to be “competent” in before you can achieve your goal, at least for a certain implementation.

You might know how to do none, one or many of these bullet points.

But knowing where you are is the important part.

And maybe if you knew where you stood, and you shared it, the people trying to help you know where to start.

 

Question: are these units reusable? 

Yes, you’ll likely reuse a lot of these

Question: who decides what goes in this list?

Anyone with a clear goal. The units of competence can be the “first principles” of ability that can be directed towards a goal. And everyone has different goals. In fact everyone may have different implementations for a goal. So maybe the goals are personal, but the units are universal.

Question: What is competence?

The ability to do something.

Ask yourself. Do you know how to do something, do you know how to do anything?

How do you know?

 


Deeper Questions:

So could you then take on the task of mapping ALL of your knowledge?

And would knowledge equal competence?

did.txt file

Goal: create an insanely simple “did” file accessible by terminal

Time flies by when you’re learning how to code. Its super important to take a second every once in a while to simple write down what you did during the past mental sprint. Writing down what you learned solidifies the knowledge.

Cue the did.txt file

This file is simply an ongoing timestamped list of what you’ve done. There are many other options, “Notes” on Mac, OneNote but since we do not want to lose the flow of coding, we need this in the terminal

Version 1. A simple txt file

Type in command into your terminal and simply type out what you did.

$ vim ~/did.txt

example did.txt file

Version 2. A simple txt file with time stamps

vim on your command line allows options and this includes running “ex commands”. Here we run the r read command and read the date command into the file as a timestamp at the top of the file.

$ vim +'r!date' ~/did.txt

Version 3. A simple txt file with natural time stamps

Its likely more natural for you to type at the bottom of the file so with normal Go we move the cursor to the bottom before reading from the date command.

$ vim +'normal Go' +'r!date' ~/did2.txt

example did.txt file

Final Version. An alias to open did.txt

Final Step: Create your alias and add this to your .bash_profile.

alias did="vim +'normal Go' +'r!date' ~/did.txt"

Congrats!

Now running $ did in your terminal will bring you to your very own did file!

example did.txt file

 

For more about vim

Set up your programming environment

*wip

Must have for your Mac

Get homebrew – no longer drag an icon to install

Homebrew – “the missing package manager for macOS” will make it so much easier to download, upgrade and maintain your packages. It will install packages super transparently in /usr/local/Cellar/* and symlink commands to /bin.

Must have for node.js

nvm

a version manager for node. You will likely use many different node versions if you are pulling down different production repos, nvm will help you install and switch the the correct versions.

node.js

use nvm to install the latest version of node.js

$ nvm install node

 

TODO rust, iOS, python, scala, screen sharing

Weather reports from your terminal with curl

What is curl?

curl is a program that is included in your Mac OSX distribution used for transferring data from or to a server. It uses many supported protocols (HTTP, HTTPS, FTP, SMTP and more).

Do you have curl?

Find the location of the curl program (executable) with $ which curl. This will return /usr/bin/curl if you’re on Mac.

How do I use curl?

A simple http GET request can be made like this:

$ curl google.com

How to deal with 301 redirect 

Notice that you get a 301 message that is address has been “moved”. You can add the flag -L, --location (choose either) to “follow” redirects

$ curl -L google.com

$ curl --location google.com

How do I use curl to get the weather?

There is a service called wttr that is set up to return weather reports when you curl their service. This time we will use http for added security.

For a full local weather report:

$ curl https://wttr.in

Should I use http?

For todays weather report:

$ curl https://wttr.in?0

You can also specify a city or ski resort.

For todays weather report for New York:

curl https://wttr.in/~NewYork?0

Check if its snowy in your favorite ski resort – Charlotte Pass, Snowy Monaro, NSW, Australia

* Notice that wttr is returning special terminal characters to change the color. You can save these to examine this in your text editor.

Congrats! You got the weather!

Note that you can curl any web service, even ones you build yourself.

Want to build your own weather service? We’ll do that next in the Part II of this series.

Bonus Section:

Can I curl my own websites?

Yes! If you followed our github pages tutorial you’ll find you can curl your own website AND YOUR STATIC FILES

$ curl -L theptrk.github.io  <- website

$ curl -L theptrk.github.io/api/message.txt <- static txt file

 

*Update – this post got onto the hackernews front page and since then the API for wttr.in has been down. **Update, the API is back up!