LLM Knowledge Dump
Prompting
This blog post is a great place to start learning about prompt engineering. The OpenAI Prompt Engineering Guide is another good resource.
The Prompt Report: A Systematic Survey of Prompting Techniques surveys over 1,500 prompting papers and condenses the findings into a list of 58 distinct prompting techniques. Also see this podcast with the papers author. The instructor library has also written and documented examples of each prompt technique in the paper.
Declarative Self-improving Python (DSPy) is a radically different approach to prompting. Think of it like a training run to optimize your prompts.
Sampling
XTC Sampler for Creative Writing
Libraries
instructor is so good OpenAI copied it's API for their own structured outputs API. Structured outputs are the real killer app of LLM's. See this post where I use instructor to extract and validate transaction data from my bank statements.
RAGatouille is SOTA for RAG. Basically obsoletes standard approaches that just use cosine similarity on dense embeddings.
Also rans:
Tools
llm is a command line tool in the UNIX tradition. Since it reads input from `stdin`` it's great for scripting.
cat myscript.py | llm "explain this code"
files-to-prompt pairs well with llm.
files-to-prompt django_app/models.py django_app/views.py django_app/forms.py | \
llm "write unit tests for my view functions" > tests/test_views.py
aider is an open source AI pair programmer.
Whenever aider edits a file, it commits those changes with a descriptive commit message. This makes it easy to undo or review aider’s changes.
Aider uses a concise map of your whole git repository that includes the most important classes and functions along with their types and call signatures. This helps aider understand the code it’s editing and how it relates to the other parts of the codebase. The repo map also helps aider write new code that respects and utilizes existing libraries, modules and abstractions found elsewhere in the codebase.
aichat is my go to for interactive chat. I don't like any of the web based chat ui's.
Vision Models
- Segment and count objects in images with just a text prompt. Ex: 'segment the cats'
- Small enough to run fast on local hardware.
- Easily fine-tuned to new tasks.
- SOTA for RAG on for PDF documents.
Misc Resources
- hu-po is a small channel but his videos are fantastic. Here's one on RAG and another on OpenAI's O1 Reasoning Model.
- Simon Willison's Blog
- /r/LocalLLaMA
- How I Use AI
- Applied LLMs