The Secret Sauce for Deploying LLM Applications into Production

Speaker: Josh Reini, Developer Advocate, OSS Lead, TruEra
AI adoption in enterprise is no longer just a trend; it's a necessity for staying competitive. Generative AI applications have started to make their way into production in a big way for use cases including customer support, content generation, summarization, and more. Klarna is a great example of this adoption, where they have moved AI assistants into production that handle more than two-thirds of their support volume - doing the work of 700 full-time agents.
Gaining the required confidence to deploy these apps at scale can be challenging, and structured evaluation has gained recognition as the secret sauce on the path from science experiment to customer value. Evaluation frameworks, such as TruLens OSS, can play a critical role in this journey by allowing developers to run experiments faster and gain systematic validation for production readiness.
This talk will walk through how to log and evaluate LLM applications, the critical decisions that must be made along the way, and how systematic evaluation accelerates the path to production.