Creating tested, reliable AI applications (Practical AI #295)

Changelog Master Feed

It can be frustrating to get an AI application working amazingly well 80% of the time and failing miserably the other 20%. How can you close the gap and create something that you rely on? Chris and Daniel talk through this process, behavior testing, and the flow from prototype to production in this episode. They also talk a bit about the apparent slow down in the release of frontier models.

Join the discussion

Changelog++ members save 10 minutes on this episode because they made the ads disappear. Join today!

Sponsors:

• Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes.
• Timescale – Purpose-built performance for AI Build RAG, search, and AI agents on the cloud and with PostgreSQL and purpose-built extensions for AI: pgvector, pgvectorscale, and pgai.
• Eight Sleep – Up to $600 off Pod 4 Ultra Go to eightsleep.com/changelog and use the code CHANGELOG. You can try it for free for 30 days - but we’re confident you will not want to return it (we love ours). Once you experience AI-optimized sleep, you’ll wonder how you ever slept without it. Currently shipping to: United States, Canada, United Kingdom, Europe, and Australia.

Featuring:

• Chris Benson – Twitter, GitHub, LinkedIn, Website
• Daniel Whitenack – Twitter, GitHub, Website

Show Notes:

• MLOps Community “Agents in Production” event

Something missing or broken? PRs welcome!

Capitole

1. Welcome to Practical AI (00:00:00)

2. Sponsor: Fly (00:00:57)

https://fly.io/

3. Thanksgiving preparations (00:03:33)

4. Agents in production (00:04:57)

5. AI ceiling & current hype (00:06:27)

6. Level of transformation (00:08:39)

7. Current models are mostly good enough (00:10:49)

8. Sponsor: Timescale (00:16:55)

https://www.timescale.com

9. Robust AI workflows (00:19:34)

10. Finding the right workflow (00:24:39)

11. Transition from notebook to code (00:30:47)

12. Sponsor: Eight Sleep (00:34:06)

https://www.eightsleep.com/changelog

13. Testing and integrating (00:36:44)

14. Sketching out a good framework (00:40:07)

15. Roles have shifted (00:47:23)

16. Outro (00:49:20)