AI Coding Summit London

Minko Getschev, AI lead at Google, shares insights on building effective agent skills, including architecture, skill creation best practices, and evaluation methods. Agent decision-making, planning, and hybrid architectures are explored, with a focus on the React loop for tool utilization. Context expansion and management, utilizing tools like MCP and CLIs, enhance agent power. The discussion includes comparisons between MCP and CLIs in agent systems, emphasizing context handling and debugging. Agent skills are structured as procedural instructions, with an open standard focused on tasks and workflow logic. Best practices for skill workflow involve avoiding redundancy, optimizing front matter, and focusing on predictable execution. Utilization and management of agent skills are crucial for performance optimization. Continuous skill evaluation, testing, and improvement are highlighted, with SkillGrade for evaluating agent skills. Skill workflow execution analysis involves identifying failures, log analysis, and rerunning workflows for successful execution.

Daniel Avila discusses Cloud Code templates and components for AI coding, emphasizing the importance of proper configuration and incorporating skills. Control permissions, settings, commands, sub-agents, and skills are essential for efficient workflow in Cloud Code. Cloud Code development benefits from skills injecting context for AI agents and providing clarity in configurations. Managing skill invocation, model selection, and advanced skill configuration are crucial for optimizing Cloud Code capabilities. Efficiently utilizing file references, scripts, and skill context enhance the functionality of well-crafted skills. Community interaction on GitHub and effective skill sharing practices contribute to successful team collaboration in Cloud Code development.

Introduction to LLMs from a Developer's Perspective, YouTuber with AI Coding Daily channel, evaluates new models and versions on YouTube, gaining traction and positive feedback. Exploring the Best Models: My Benchmark on 18 LLMs, evaluating cost, points, and competition among models for day-to-day use. Measuring Models: Prompt Methodology, Tests on PHP, Laravel, React, TypeScript, and CSV, with emphasis on correct data usage and benchmark awareness. Model Evaluation: Chinese models' progression, benchmark awareness, and Opus and GPT superiority. Model Pricing and Quality Comparison: Opus, GPT, and Composer 2.5 cost-effective options. Composer 2.5 and GPT 5.4 mini offer quality at low prices. Choosing models based on project needs and budget constraints. Discussion on local LLM investment challenges, SONnet 5 benchmark performance, and exploration of new models like Proton Luma 2.0. Importance of skill definitions for model performance and limitations in local model implementation. Discussion on the importance of model harness, setup considerations, and evaluation methodologies for LLMs.

Vladimir Novik discusses the evolution of AI systems in coding, highlighting the shift in job requirements and challenges faced in scaling systems. Challenges in swarm orchestration include reducing reliability due to compounding errors and structural challenges in transitioning to a swarm system. Orchestrator Coding focuses on system properties over tools for code evolution. Enforcing architectural decisions and mechanical enforcement are crucial for system reliability. Orchestration disciplines prioritize dispatch, isolation, and durable handoffs, emphasizing autonomous systems and event-triggered actions. Mastery of orchestrated workflows involves resilience, autonomy, and disciplined manual and automated workflows. Advanced workflow techniques stress exact commands, linear workflow skills, caution in commits, and iterative improvement strategies. Automation enhancement strategies aim to overcome cost and parallelism challenges. Coding transformation journey involves validating and evolving code with orchestration. Model orchestration layers and architecture comparisons highlight the importance of model evaluation and verification gates. Tool comparisons and orchestrator communication strategies improve system orchestration. Architecture enforcement mechanisms utilize multiple layers and tools for code analysis and infrastructure testing.

Daniel, a consultant from Germany, discusses AI-generated code and challenges in rapid code shipping. The Pharoes AI case study highlights code reviews and challenges in agile development. Incidence per pull request and code churn issues are discussed in software development. AI-assisted coding and quality gates are introduced. Developing with Angular and TypeScript emphasizes the importance of basics, AI sub-agents, and pull request review services. Utilizing hooks for deterministic code handling and enhancing code handling with various hooks are crucial. Transitioning to behavior-driven development, utilizing tools like PlayWrite and Jira integration, and evaluating code review strategies are key points. Optimizing code review practices, smart integration of AI and hooks, advanced code review models, ownership in code reviews, software metrics, and quality gates challenges are also covered.

Today's discussion covers challenges posed by AI models becoming more adept at deception, hidden AI thought processes, addressing coding agent security risks, the future of agent deployment and security, real-time monitoring and security policies, developer experience and automated systems, model behavior analysis and agency projection, product offering and developer risk reduction, futuristic concepts and developer risk management, data storage and risk severity management, deception in model behavior and user security, policy transparency and model adaptation, model failure monitoring and safety considerations.

Exploring AI engineering and agent-assisted development through Claude's Code on GitHub. Analysis of Cloud Code commits, repository scale, and language distribution. The impact of agents on coding in various languages like Python, TypeScript, Rust, Dart, and Zig, facilitating easier coding in traditionally more complex languages. The significance of stars on repositories in Cloud Code projects and the observation of growth patterns post-Christmas season. The impact of Christmas on exponential growth in commits per day, the current doubling time trend, and a focused analysis on the top 100 popular repositories in Cloud Code projects. Highlighting key differences in commits between AI and non-AI repositories, focusing on feature contributions, bug fixes, and UI changes. Exploring diverse use cases of Cloud Code, highlighting how Clickhouse utilizes extensive Cloud Code for optimization, CI testing, and infrastructure, fostering engineers' focus on more impactful tasks like feature development and bug fixes, along with the emerging trend of Cloud Skill repositories. The growing trend of active projects utilizing AI contributions, specifically Cloud Code's significant impact on real, critical software projects beyond AI-focused repositories. Exploring models and coding agents that self-identify like Cloud Code, particularly the aggressive nature of Cloud in generating code, as observed with GitHub Copilot and other models like Mistral. Projects adopting AI development observe an increase in GitHub issues due to code production, leading to functional issues. The popularity growth of Clickhouse may be linked to active use of Claude to accelerate development, not directly boosting popularity. Metrics tracking revealed exponential growth in the use of short codes, focusing on valuable contributions to critical code with continued usage.

Talk on software engineering challenges with AI agents and codebase integrity, emphasizing human input in development processes and the need for automated quality assurance tools. Importance of deterministic tooling in catching problems during agent operations, highlighting the rapid generation of maintainability issues by agents and the need for a faster feedback loop in code quality assurance. Importance of rules files and deterministic checks for codebase integrity, emphasizing the necessity of enforcing rules through tools and hooks. Fellow as a fast, deterministic codebase analyzer ensuring code quality by verifying changes and complexities within the loop. TypeScript and JavaScript's flexibility, Fellow combining specialized tools for code-based consistency checks, and the importance of the loop in the development process. The agent's interaction with the bouncer, resolving issues, and the process of identifying and addressing code inconsistencies through audits and actions. Utilizing tools like KNIP and JS CPD enhances code quality by detecting dead code and addressing duplication efficiently, leading to faster feedback loops and improved code writing processes.

Rudhankariyam discusses AI-driven coding to develop self-maintaining AI tools. Introduced an open-source project, App Store Connect CLI, as a case study for AI-driven coding. Implemented automation for App Store publishing using agents and APIs. Managed fewer PRs in bulk, used codex and cursor cloud automations efficiently. Added aliases for hallucinated cloud code commands, improved descriptions and examples. Implemented separate auditor agent for rigorous code review. Emphasized human decision-making for product direction and initial rules in automation strategies. Evaluate model quality based on error rates and user expectations. Highlighted the importance of human judgment in software development amidst AI advancements.

The speaker delves into loop engineering's role in automating decisions and enhancing productivity in AI development workflows. Loop engineering is highlighted as a tool to build and automate tasks, even for creating programming languages like Kerst, with a focus on rapid project development. The importance of repetition in loops, automation criteria, and the suitability of loop engineering for full-blown apps are emphasized. Discussions include read-only tests, verifier and backlog agents, and the necessity of a clear definition of done for effective loop implementation.

Presentation on building a customer support bot with Langraff, Amazon Bedrock as base. LangChain Python library for LLMs. RAG - Retrieval Augmented Generation for accessing current information. LangChain using Bedrock and Amazon Knowledge Base. LangRaph classifies queries intelligently. Using LandGraph on top of LangChain for multiple scenarios with human intervention. Connected to Amazon Bedrock for knowledge base. No guessing with large language models; rely on a connected knowledge base for accurate responses. Chat GPT creation involves memory for context; tools like Land Graph aid in this process. Land Graph usage for flow creation; Demo with Kiro agent ID and new file uploads. Setting up AWS MCP server with agent skills; Using AWS agent toolkit for best practices in service setup; Flow for intent routing and order refund eligibility. Handling Order Refund Requests Using Agent Tools and Lookup; Case Sensitivity in Order Number Search; Validating Order Eligibility for Refund. Uploading Knowledge Base to AWS MCP Server; Verification of Email Address for Refund; Human in the Loop for Refund Approval. Human in the Loop Feature of LaneGraph; Uploading Documents to Bedrock; Utilizing Vectors for Knowledgebase Queries. Utilizing AWS Agent Toolkit for Workflow Best Practices; Connecting to KnowledgeBases for Querying; Building App with Mastering LaneGraph. Generating Design and Requirements for App Development; Utilizing Agent Skills in Spec-Driven Workflow; Ensuring Accuracy with Mastering Langraff Skill. Detailing Setup for Coding Assistant with AWS Agent Toolkit; Requesting Feedback and Interaction; Importance of Langraff in Graph-based Solutions. Enhancing Chatbot Prompt Suggestions Dynamically based on Conversation Context; Utilizing In-Memory Thread for Follow-up Questions in Langraff; Swapping Question Prompts with Tags for User-friendly Experience. Utilizing Different Models for Chatbot Tasks; Implementing Security Measures; Continuous Knowledge Base Updates for Accuracy.

The Talk delves into the migration journey of over 500 micro frontends, emphasizing the use of AI agents for efficient migration. It discusses the challenges of handling manual PRs and code mods at scale, highlighting the importance of reliable tooling. The balance between automation and human judgment is crucial, with deterministic lanes handling basic tasks and human intervention reserved for complex issues. Scalable agent orchestration, workflow automation, safe PR launches, and stable testing are key themes, emphasizing the collaboration between machines and human judgment for successful migrations.

Sam, a forward deployed engineer turned founder, discusses building production agents in agriculture and the trend of companies creating their own agents from scratch. The challenges of maintaining and debugging agents are highlighted, along with the complexities of agent debugging and the importance of agent observability. Advanced analysis techniques for monitoring agent behavior, setting up real-time behavior signals, defining mission-critical signals, and optimizing agent orchestration are explored. Privacy considerations in observability and efficient context management for agents are also discussed, emphasizing the use of open-source models and the challenges in multi-agent workflows.

The Talk covers various aspects of deception and security in AI and data systems. It discusses the historical use of misinformation during World War II to influence decision-making. AI model poisoning through malicious data input and the challenges of ensuring data integrity in real-time pipelines are highlighted. The importance of peripheral information attacks, data contracts, stream governance, and anomaly detection in maintaining AI system security is emphasized. Strategies for protecting data integrity, detecting anomalies, and choosing appropriate AI models are also discussed.

Bernie emphasizes using AI to enhance rendering performance, with a focus on debugging and the evolution of computing speed standards. Rendering optimization challenges persist despite advancements in multi-threaded development. Debugging CSS and rendering performance complexities require tools like profilers for JavaScript efficiency analysis. AI automation can improve performance by understanding rendering engine intricacies. AI aids in identifying and resolving performance bugs, investigating rendering issues, and optimizing icon rendering. Tailwind's impact on performance and leveraging AI for CI checks showcase AI's role in troubleshooting and long-term learning in software development.