Patrick

local inference with llama-cpp-python

December 31, 2023December 31, 2023theptrkUncategorized

Step: install the package Step: download a model “TheBloke” on huggingface (link) has a ton of models in “GGUF” format (format introduced by llama.cpp) Click on any of these different quantized (reduced precision) models and find the “download” link Put them in your project somewhere like a “models” directory Step: create your python (.py) or…

rsync with python without extra files

December 29, 2023February 3, 2024theptrkCommand line

Your goal is to send your python project to your server from your local machine. Your python project includes a ton of files you wouldn’t want to send to a server so a simple rsync is not enough. Option 1: rsync with a bunch of exclude flags Option 2: Shell Function Option 3: Shell script

tiktoken: get number of tokens from string + openai pricing

November 22, 2023November 22, 2023theptrkOpenAI

Get the number of tokens The encoding cl100k_base is used by gpt-4, gpt-3.5-turbo, text-embedding-ada-002.This is how you encode a string into the tokens. You can abstract this into a separate function. Source: openai-cookbook (github) OpenAI Pricing As of November 22, 2023 Splitting chunks for to manage Context Window Limits gpt-3.5-turbo-1106 has a limit of 16,385…

Creating a form with HTMX, Django

November 14, 2023November 15, 2023theptrkhtmx

Step 1: Activate htmx (link) for django: Render a csrf token somewhere Step 2: Create the form with hx-post Case: Targeting a container div Case: Loading states extension (docs) Setup: import script, set style to avoid flash, add header Disable submit, add a loading indicator

Django and pgvector for semantic search

November 6, 2023November 9, 2023theptrkdjango, LLM, python

Goal: Semantic Search Setup pgvector You have a model with a field you want to make semantically searchable. Installation comes from: https://github.com/pgvector/pgvector-python Step: install pgvector on your machine Step: create a migration to activate Step: add a VectorField to your model Step: make migrations and migrate Setup sentence transformers for embeddings We will use the…

Can ChatGPT do math? NO! But why?

October 18, 2023November 6, 2023theptrkLLM

ChatGPT can “kind of” do math, but with limited precision. 1+1, 2+2, it’s great at these additions. Here’s GPT-3.5 Here’s GPT-4 But why? Interestingly, GPT knows the way it was trained and the inherent limitations of using a language model to produce accurate arithmetic.

How to create a chrome extension (react vite boilerplate)

July 28, 2023August 20, 2024theptrkchrome extension, Javascript

How to use the boilerplate Step: Use a great boilerplate template Step: Copy the repo, change the details and you are set up! The 6 Areas of a chrome extension Step: Learn the different parts files in the “pages” directory content: stuff you want to add to the dom devtools: devtools tab AND panel: the…