AI Developer

ML Engineer is a hot new job. It’s the boys and girls who train and deploy models. I heard the word AI developer the other day, and I’ll refer to it as AI application engineers. People who use AI to solve use cases. NOTE: This is not what most developers do today. What most developers do today is ask how can AI do the things I would have done (e.g. write the function).

• AI Developer, vs a Software Engineer using AI
• Programs, Human Code, AI.
• Design Patterns
  • Maximize the workflow and computation in your programs.
  • Provide context to your model for Ground your model
  • Use multiple AIs to maximize the value.
  • Human in the loop
• Use Cases and Applications
• Tech Deep Dive - Models
  • Model and Service Dashboards
  • Quality vs Speed vs Prices
  • Llama 3
  • Tools + Libraries
  • Commercial vs Open Source Models
• Data Access
• Related Posts
• Several posts on this topic:

AI Developer, vs a Software Engineer using AI

Today, most software engineers are thinking how do I use AI to be a better programmer.

An AI developer would instead do:

• Be the PM, nail the use case.
• Figure out how much of it AI can do.
• Close the gap.

Programs, Human Code, AI.

At the hand-wavy, programs are workflow and computation, where computation is defined recursively.

In the before times, all workflow and computation was defined in explicit code and the code was written by humans.

The first stage/category of AI evolves from having the AI help write the code, to having the AI write highly specified code itself, to writing code from very vague specifications.

The next stage/category of AI is having AI do leaf computation. This evolved from very specialized (digit recognition), spell checking, to more general - generate an image.

The final stage/category of AI when AI does the higher level of workflow as well. Ask the user what they want, and do it.

Just like there are autonomous driving levels:

1. Human writes code
2. AI helps write code
3. AI writes code
4. AI writes code from vague specifications
5. AI does leaf computation
6. AI does higher level workflow
7. AI does everything

The AI developer thrives to get the AI to the highest level possible

Design Patterns

Maximize the workflow and computation in your programs.

1. Understand use case
2. Make crappy prompt
3. Use meta prompt builder to help write better prompt
4. Put prompt in PromptFoo to evaluate and track over time
5. Figure out cheapest/fastest model to meet your needs
6. Figure out gaps AI is doing with current generation of AI
7. Gap fill

Provide context to your model for Ground your model

Currently more complex, figuring it out.

Use multiple AIs to maximize the value.

• Run same prompt through multiple AIs
• Show user multiple outputs
• Use AI to judge and merge
• Very helpful when you want the best thinking (e.g. commit messages).

Human in the loop

• AI can still be wrong. Put a human in the loop

Use Cases and Applications

• Writing git commit messages
• Spellchecking a file at a time
• Summarize changes in a repo over a period of time. E.g. What I wrote some random week

Tech Deep Dive - Models

Model and Service Dashboards

• Open AI Usage
• Claude Usage
• Gemini Usage - Sheesh I can’t even figure it out
• VAPI - for voice access
• Modal - For FaaS capability

Quality vs Speed vs Prices

• See performance dashboards
• Groq is really exciting because it’s crazy fast. Like 200 Tokens/S. But it can only run open source models, so much less useful. Looks like it can run

Llama 3

• Super exciting based on quality, and by being hosted on Groq making it super fast.

Tools + Libraries

• Langchain
  • Gives you a unified API to use multiple models

Commercial vs Open Source Models

Data Access

• RAG
• Raptor
• DSpy

Related Posts

Several posts on this topic:

---

---

---

---

---