Author: theptrk

Categories
Machine Learning

Understanding Dimensions – Linear, Logistic Regression

In the forums of the Coursera course “Machine Learning” by Andrew Ng, there are many questions regarding the dimensions of the input matrix and theta.

Here is an intuitive guide to understanding the dimensions. Click the link below!

Understanding-Dimensions-Linear-Logistic-Regression.pdf

Screen Shot 2020-02-23 at 10.36.33 PM

In Summary: 

 

The dimensions of your input matrix (X) is typically clear and driven from your data. 

[ # of examples, # of input variables ]

The dimensions of theta are driven from your input variables. In linear regression and binary classification, the number of outputs per example is typically 1:

[ # of input variables, # of outputs per example ]

Categories
Machine Learning

Notes for Coursera ML Course Week 1-5

I am currently taking the Machine Learning Coursera course by Andrew Ng and I’m loving it! I’ve started compiling my notes in handwritten and illustrated form and wanted to share it here. If you are taking the course you can follow along 🙂

AI Cartoons Week 1 – 5 (PDF download link)

Sign up for a notification on the finished PDF here

* Note these are for Weeks 1-5

If you have feedback, I would love to chat on Discord here

Here’s a sample:

Categories
vim

vim jj to esc mapping

For a single vim session (safe version)

:inoremap jj <Esc>

For every vim session (still safe)

Save this to your .vimrc file in your home directory. 

" this is a comment so you remember this later
" insert mode; no recursive; map; <from>; <to>
inoremap jj <Esc>

* change jj to anything you want, fd , jk even ;;

What is happening?

i means we are creating a mapping for the insert mode

nore means we designate “no recursive” mode (see below for more)

map means we are mapping 🗺

jj is the commands we are mapping from

<Esc> is the command we are mapping to

No change needed: Ctrl + [

If you already have Caps Lock mapped to Ctrl, then a solid alternative is: Ctrl + [.

Why use the safe version?

When you create mappings in vim, there is a recursive lookup to all other mappings.

We use inoremap instead of imap above because the nore (not recursive) portion, will avoid recursive mapping. An example of recursive mapping is this :nmap dd jddk where the dd in the right hand definition will actually map to the left hand into an infinite loop. Read more here.

Sources:

  1. Stack Overflow
  2. Learn Vim The Hard Way
  3. vim.fandom
  4. In vim > :help inoremap

Categories
Unix

How to change your command line prompt

Your command line welcomes you with a prompt.

Default OSX terminal may look like this.

PS1 is the variable used by your shell to determine your command line prompt

RPS1 determines your right hand prompt.

Note, you can always `echo` this variable to see what these are currently set to.

By default, mine is set to %n@%m %1~ %#

These % (percent) symbol combinations are prompt expansions (docs)

  • %n– is the special variable $USERNAME (echo this to check)
  • @ – literally the “@“ sign,
  • %m – hostname up to the first “.”
  • %1~ – current working directory for “1” ancestor
  • Additionally you may like..
  • %~ – current working directory with home aliased to “~”
  • %D – date in yy-mm-dd (International) format
  • %W – date in mm-dd-yy (US) format
  • %t – time in am/pm format
  • %* – time in 24 hour format

Note, your prompt can even accept emoji’s 🚀

command+control+space brings up the emoji keyboard on Macs

How to apply this to every new terminal window

If you are using bash, add your PS1=... command to your ~/.bash_profile file.

If you are using zsh, add your PS1=... command to your ~/.zshrc file

* emoji’s are not available on all machines, but great on those are.

Categories
Coding

UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xf1 in position 4: invalid continuation byte

TLDR: Convert your problem file with Sublime Text by opening the file and using “Save with encoding” as utf-8. Alternatively, use iconv -t UTF-8//TRANSLIT -c Zip_Zhvi_SingleFamilyResidence.csv > new_file.csv

When does this error happen?

I wanted to parse the housing data from Zillow at their research page. Zip code is a great measure of single family home real estate values.

zillow research page time series by zipcode.png

However, when I download this data set as “Zip_Zhvi_SingleFamilyResidence.csv”, I could not simply load this data into pandas.

pandas_read_csv_UnicodeDecodeError.png

This last line seemed like the clue:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 4: invalid continuation byte

Well, what format is that file?

Using a Mac, we can use file -I <file_name>

file_type.png

Oh, great! its “us-ascii”, we just pass that encoding into pandas right?

pandas_error_with_encoding.png

Oh maybe, I need to specify the encoding I want. WHY PANDAS, WHY!?

pandas_error_with_encoding_again.png

Why does this error happen?

Some encoding error has occurred, maybe because you accidentally opened Excel before opening ipython or Zillow saves in a crazy format.

Awesome, lets just convert it

Let’s use the *nix program iconv to convert the file. According to the man page (man iconv), “The iconv program converts text form one encoding to another encoding. Great!man_iconv.png

Let’s use this.

iconv -f us-ascii -t utf-8 < Zip_Zhvi_SingleFamilyResidence.csv > new_zip_code_file.csv

iconv_failure.png

“cannot convert”

But iconv, that’s your only job… you know, unix philosophy, one program, one job done well etc etc.

Turns out if you use “//TRANSLIT” appended to the encoding, characters are transliterated when needed and
possible (man page)

Solution 1 – iconv with //TRANSLIT

> iconv -t UTF-8//TRANSLIT -c Zip_Zhvi_SingleFamilyResidence.csv > new_file.csv

> mv new_file.csv Zip_Zhvi_SingleFamilyResidence.csv

Solution 2 (easier to remember) – Sublime Text

Is there a better free editor than Sublime? Be a good citizen and buy your license.

Step 1: Open your file in Sublime Text

Step 2: Save with Encoding > UTF-8

DONE!

grizz_celebrating.gif

read_csv to your hearts desire 🙂

ipython> data = pd.read_csv("new_file.csv")

Categories
Javascript

Save time: npm install nodemon

Nodemon will allow you to start your node.js project and make changes without manually shutting down and restarting your server.

Many people are scared of installing nodemon globally, so here’s a guide to install nodemon locally in your node.js project.

Step 1: Install nodemon 

$ npm install nodemon --save

Step 2: Update your npm scripts object in package.json

"scripts": {

    ...

    "start": "nodemon index.js

}

Note: If you used the express generator you will change "nodemon index.js" to your entry point into your server, which is likely “nodemon ./bin/www"

Step 3: Run your new npm script

$ npm start

Congrats!

Now our JavaScript changes will restart your server automatically

* Note, many frameworks compile and cache templates so changing your html/jade files may not restart your server.

Categories
Coding

Todidlist.com started as a vim shortcut

Hi all!

Todidlist.com started as a vim shortcut that I wrote about way back last year here.

I wanted to have the most accessible “did list” and vim served me so so well. 

However, once I got a new computer, I realized that my did list was missing a key feature: ubiquity. 

There was one simple trait that was missing from my $ did command and that was the ability to see my “Dids” on every computer and my mobile devices.

So, www.todidlist.com was born! Check it out and let me know how it treats you.

Thanks for all the support!

Categories
Javascript

Setting up Redis on Heroku for ExpressJS

By default, your express application will store session data in memory. That means if your server restarts all users will need to log back in. Additionally, this does not scale to more than one instance, leaks memory and does other mean things. While this works while developing your local computer (notice how you always need to log in after making a code change), you will want something better for production.

Heroku even warns you if you do this. Try running $ heroku logs --tail to see this.

Welcome Redis

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.

Step 1: Create the heroku add on

$ heroku addons:create heroku-redis:hobby-dev

This will create a REDIS_URL in the heroku config that you will use to connect. Use $ heroku config:get REDIS_URL to see this value.

Step 2: Download the connect-redis package for your project

$ npm install -s connect-redis

Step 3: Pass in express-session into connect-redis

Step 4: Create an instance of RedisStore as the store property of your session configuration

Note: your local computer probably does not have REDIS_URL set.

Step 5: Push to heroku and try your new session store

$ git push heroku master

How do you run your project locally now?

Option 1: default to memory session store locally

Check your NODE_ENV setting and if its not production we will default the store property to null.

Option 2: default to using a redis instance locally

Run these commands and connect to your instance directly.

Screen Shot 2019-04-07 at 2.51.07 PM.png

Gotcha Error: req.flash() requires sessions

If you try running your project locally, you will notice that your session store fails to initialize and things break – this happens since you likely do not have the REDIS_URL set.

Production ready: 

Note that redis on hobby dev will not allow SSL and to secure a production redis, you will need to pay for a production plan and add “stunnel” to your buildpack. (link)

Categories
Javascript

Create an interactive terminal cli with node

Let’s create a tool that will track our some text entries.

Step 1: Create a new npm project and install the necessary libraries

$ mkdir my_cli && cd my_cli && npm init -y && npm i --save inquirer chalk

chalk – helps color the terminal prompts

inquirer – makes creating interactive prompts a breeze

Step 2: create interactive terminal prompts

Step 3: save or append to a file (did.txt)

Full code here

Categories
Coding

Sublime Text shortcut on command line (subl)

TLDR: Make sure ~/bin is in your $PATH and then run:

ln -s /Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl ~/bin/subl

The Sublime Text documentation “OS X Command Line” references the command line tool, subl to work with files on the command line.

This allows you to open the entire current directory in sublime like this:

$ subl .

Screen Shot 2020-04-05 at 9.38.38 AM

This allows you to open a specific file or create a new file like this:

$ subl <file_name>

But subl is not available by default

The command $ subl is not immediately accessible when you download Sublime. Without symlink-ing the command, you will have to type the full path ​to the original subl

This is how you would use the full path to open a file.

$ /Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl <file_name>

Where does your computer look for commands? 

Your computer will look for commands in your “load paths”. Find your current load paths by running this command:

$ echo $PATH

You will see all load paths including these default OS X paths:

/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

You can list the commands in any directory: $ ls /usr/bin

Step 1: Create a symlink in your “~/bin” directory

We will place this symlink in the ~/bin directory. This is simpler because it does not require admin or root access.

Run this command in your terminal:

ln -s /Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl ~/bin/subl

What happened?

ln means link, -s means we want a symbolic link, which will act as a shortcut to the original command within the Sublime Text application.

We will create a symlink at ~/bin/subl that will point to the original Sublime command. Deleting this link will have no effect on the original file.

Step 2: Troubleshooting

Not found: If ~/bin is not in your $PATH variable your will get an error.

Note: ~/bin is different than /bin

Append ~/bin to your $PATH anywhere in your bash profile or zshrc

PATH=$PATH:~/bin

Screen Shot 2020-04-05 at 9.36.52 AM

Still not found: You have to create a new instance of your terminal to see the changes. Alternatively your can run source ~/.bash_profile or source ~/.zshrc to re-run your shells startup configuration