AI Engineer End-to-End Workflow: Productivity Challenges and Their Solutions

Nithya Narasimhan is an AI advocate at Microsoft.

The focus of the talk is on productivity challenges and solutions for AI engineers, particularly in the journey from catalog to code to cloud.

An AI engineer is someone who can pick a model, customize it through various techniques, evaluate it for quality, and deploy it for real-world applications.

'Exposure to AI' refers to a task being exposed to AI if AI-powered software can reduce the time needed to complete the task by at least 50%.

Contoso Chat is a generic retail application used to demonstrate the integration of a chatbot into a retail website to improve customer interaction and sales.

The desired features are conversational interaction, grounding responses in catalog data, context awareness, and safety from malicious activities.

The three stages are ideation, augmentation, and operationalization.

Infrastructure as code allows defining infrastructure needs through code, ensuring consistency across team members and enabling use of tools like Azure Developer CLI for provisioning.

The Azure AI Inference API provides a unified abstraction to interact with various models, allowing developers to swap models with minimal effort and enhance productivity during the ideation phase.

AI-assisted evaluation involves using another AI to grade the responses of the first AI, often referred to as 'LLM as a judge,' to scale evaluation against large datasets.