#artificial intelligence

AI is transforming software engineering by using agents to help with coding. Agents can autonomously complete tasks and make decisions based on data. Collaborative AI and automation are opening new possibilities in code generation. Bolt is a powerful tool for troubleshooting, bug fixing, and authentication. Code generation tools like Copilot and Cursor provide support for selecting models and codebase awareness. Cline is a useful extension for website inspection and testing. Guidelines for coding with agents include defining requirements, choosing the right model, and frequent testing. Clear and concise instructions are crucial in AI-generated code. Experienced engineers are still necessary in understanding architecture and problem-solving. Energy consumption insights and sustainability are discussed in the Talk.

Today we will discuss React server components with AI and how to build a better search experience using them. We will learn how to make a Next.js app AI-enabled using the Vercel AI SDK. The Vercel AI SDK's streamUI function with the GPT 4.0 model will be used to make suggestions interactive. We will explore the use of history and conversation in AI and how to continue the conversation and read the result. The concept of generative UI with the vector database will be introduced, along with querying the database for movies. We will process user queries and return movies based on them. The power of React server components in enhancing UI will be demonstrated. In summary, the Talk covers vector embeddings, natural language search, and generative UI.

AI design challenges include bias, safety, and security. Trust and transparency are important in AI. Design principles for AI include user control, fighting bias, and promoting good decision-making. AI can enable the design process and investors expect to see it included in products. AI empowers individuals to create and share ideas, but managing expectations is crucial.

In this Talk, the speaker demonstrates how to create an AI chat bot that can answer questions based on information it was never trained on. They build a basic RAG pipeline in just five minutes using live coding. The speaker also shows how to create embeddings and a vector database, set up a vector search index and endpoint, and modify the chat route to enhance the chat bot's capabilities. The program is run and tested, and the Talk concludes with an invitation to join a workshop for more information.

AI in game development has evolved rapidly, with generative AI being a focus. However, game developers like Romero Games have concerns about ethics and prefer using AI to automate processes and make creative work easier. AI has been used in games for decades, from path-finding AI to decision trees. Procedural world building and advanced AI technology are pushing the boundaries of FPS games. Different teams within a company have different approaches to the use of AI, depending on their specific needs and requirements.

Web AI in JavaScript allows for running machine learning models client-side in a web browser, offering advantages such as privacy, offline capabilities, low latency, and cost savings. Various AI models can be used for tasks like background blur, text toxicity detection, 3D data extraction, face mesh recognition, hand tracking, pose detection, and body segmentation. JavaScript libraries like MediaPipe LLM inference API and Visual Blocks facilitate the use of AI models. Web AI is in its early stages but has the potential to revolutionize web experiences and improve accessibility.

Codium is a generative AI assistant for software development that offers code explanation, test generation, and collaboration features. It can generate tests for a GraphQL API in VS Code, improve code coverage, and even document tests. Codium allows analyzing specific code lines, generating tests based on existing ones, and answering code-related questions. It can also provide suggestions for code improvement, help with code refactoring, and assist with writing commit messages.

Join Christoph from Nakazawa Tech in building the AI for Athena Crisis, a game where the AI performs actions just like a player. Learn about the importance of abstractions, primitives, and search algorithms in building an AI for a video game. Explore the architecture of Athena Crisis, which uses immutable persistent data structures and optimistic updates. Discover how to implement AI behaviors and create a class for the AI. Find out how to analyze units, assign weights, and prioritize actions based on the game state. Consider the next steps in building the AI and explore the possibility of building an AI for a real-time strategy game.

This Talk discusses building a voice-activated AI assistant using web APIs and JavaScript. It covers using the Web Speech API for speech recognition and the speech synthesis API for text to speech. The speaker demonstrates how to communicate with the Open AI API and handle the response. The Talk also explores enabling speech recognition and addressing the user. The speaker concludes by mentioning the possibility of creating a product out of the project and using Tauri for native desktop-like experiences.

This talk explores the use of AI in web development, including tools like GitHub Copilot and Fig for CLI commands. AI can generate boilerplate code, provide context-aware solutions, and generate dummy data. It can also assist with CSS selectors and regexes, and be integrated into applications. AI is used to enhance the podcast experience by transcribing episodes and providing JSON data. The talk also discusses formatting AI output, crafting requests, and analyzing embeddings for similarity.

Discussing building better UIs faster with AI, emphasizing the use of LLMs as predictive text models and the need to maximize their smart aspects in coding and UI development. Exploring file context management in Cursor IDE and demonstrating quick UI updates using AI for coding efficiency and real-time results. Discussing advanced workflows for efficient development using feedback-driven approaches and incorporating test-driven development for iterative and automated testing in Cursor IDE. Exploring advanced prompt tricks in Cursor IDE for efficient development by writing tests first, updating code iteratively, and enabling autorun mode for automated testing and bash commands. Cursor's Annoying Prompt Behavior, Automated Testing Benefits, and Test-Driven Development for Reliable Code. AI Feedback Importance and AI Testing Complexity with Clear Goals. AI Challenges, Visual Front-End Focus, and Fusion Preview. Integration of Figma Designs for Visual Tweaks, Exploring New Features and Design Systems, Updating Customers Tab and Making Visual Edits, Making Carousel and Visual Updates, Granular Design Updates and AI Integration, Dynamic Updates and Component Refactoring, Tagging Builder Bot for Automated Code Updates.

Manoj Sureddy discusses building a toolkit for prompt engineering with LLM-based solutions, emphasizing the need for a structured approach like React. The toolkit provides a structured approach for prompt development, ensuring organized and reusable templates for various LLM-based solutions. Integration with version control and CI-CD pipeline for automated evaluations, advanced quality evaluation mechanisms using Gemma, and integration of human in the loop evaluations. Focus on maintaining prompt quality, subjective metrics in evaluations, and insights on prompt drift, versioning, real user feedback, and evaluation automation.

Today's Talk discusses the transformation of performance monitoring with AI in Node.js. AI-powered tools can analyze massive amounts of data and provide actionable insights, making performance debugging faster and more efficient. Traditional profiling techniques are often time-consuming and require deep expertise. AI-powered diagnostics analyze data and provide recommendations for optimizing performance. AI-driven performance analysis correlates performance issues with system metrics and generates code optimization suggestions. Advancements in performance debugging include detecting CPU bottlenecks, memory leaks, and combining multiple data sources for context-aware insights. The AI-powered demo showcases the detection and optimization of performance issues in a Node.js application. Integrating AI-powered performance monitoring is straightforward and requires a mindset of continuous learning and curiosity in performance optimization.

Today's Talk explored combining Node.js and artificial intelligence to build smart applications. Two types of models, closed and open source, were discussed, with closed models accessed via an API and open source models hosted locally or on the cloud. The use of RAC (Retrieve of Method Generation) to enhance models was explained, along with the process of obtaining data and improving retrieval performance. Chunking, embedding, and vector databases were introduced as techniques for organizing and transforming data. The Talk also covered data retrieval using semantic search and the generation of human-readable answers using models. The concept of enhancing models with agents was discussed, focusing on how agents break down complex goals and utilize external functions. The use of tools and the React pattern in implementing agents was highlighted. Lastly, the Talk touched on implementing agent patterns and enhancing LLMs, with demos and code available for further exploration.

Today's Talk explores productivity challenges and solutions for AI engineers, focusing on an application called Contoso Chat. The end-to-end workflow is examined from a productivity perspective, introducing the concept of Exposure to AI. Building an AI application involves ideation, augmentation, and operationalization stages. Provisioning and setup are crucial steps, with infrastructure as code being a productivity tool. GitHub Codespaces and development containers provide consistent development environments. Azure AI Inference API allows easy iteration and prototyping. AI-assisted evaluation involves training AI models and using evaluators for grading responses. Custom evaluators can be created. Overall, the Talk emphasizes the importance of productivity throughout the AI engineer's journey.

Welcome to my talk on AI powered development. We'll explore the implications of AI tools and where the value lies in software development. We're transitioning to a new age where AI plays a significant role in coding. The new workflow involves asking AI for code, verifying it, and continuing to build. AI tools are available in the market, offering different options for developers. It's important to choose the right approach that aligns with your goals. Keeping IDE and AI separate allows for better control. The value of AI-powered development lies in architecture, scalability, and creativity. We're in the early stages of AI-powered development, presenting a significant advantage for those entering the field now.

Welcome to my talk at React Day Berlin. Today, we'll discuss vector search and embeddings for React developers. Traditional search is broken, resulting in poor user experiences. Converting data into embeddings unlocks more accurate search, content recommendations, and document comparisons. Vector search measures the closeness or similarity of vectors using distances. In vector search, the closest relevant documents are found based on similarities between vectors. There are multiple ways to calculate the distance between data in vector search. Generative AI applications often deal with both textual and image data, making vector search useful. By converting queries and data into vectors, semantic meaning is considered, allowing for more accurate results. React and Next.js developers have access to a range of toolings, including BlankChain and AI SDK from Vercel. Retrieval augmented generation enhances search results by converting data into vectors and generating more coherent text. The user query is converted into a vector and matched using vector search, and the response is generated using the AISDK from Vercel.

I'm Steve Ruiz, the founder and CEO of Tealdraw. TLDR is a canvas app in React that allows developers to build with their SDK. TLDR pays attention to details that enhance the canvas experience and can be used to interact with different elements. TLDR's demos showcase its versatility, but may not perform well on social media. TLDR's AI collaboration features allow users to quickly turn their drawings into AI-generated images. The Make Real project helps users create user interfaces by drawing and providing prompts, generating working prototypes. The system behind Make Real can handle complex interactions and ambitious projects. OpenAI integration and the Steel Draw Computer enable building workflows and multi-stage generations. The goal is to bring the Canvas from being specialized to a primitive for the web. Teal Draw is free for open source and non-commercial use, with options to purchase licenses for additional features.

I'm doing a quick lightning talk today, talking about saying no to boilerplate and teaching you how to build an AI app in just minutes. We're going to talk about RAG, v0, fine-tuning for sales AI to SDK, and then talk a little bit about Entity Resolution and your AI toolkit. RAG is a hot topic in chatbot development and allows for creating chatbots with a deeper understanding of specific use cases. Rag offers a versatile AISDK that allows for easy model switching, augmentation, and fine-tuning. Entity resolution is important for resolving entities across multiple points of data, with use cases in personalized marketing, healthcare, and fraud detection.

Today's Talk explores the concept of Retrieval Augmented Generation (RAG) and its application in building chatbots. RAG involves giving language models more context by retrieving relevant data. This is achieved by creating vectors for each talk and using cosine similarity to compare them. The talk emphasizes the limitations of word-based vectors and the benefits of using embedding models and vector databases. By replacing word-based models with vector search, content can be sorted and retrieved more efficiently. RAG, along with large language models and AI, has the potential to enhance scalability and unleash new possibilities.

Let's start diving into the presentation on AI and accessibility. AI is a simulation of human intelligence by machines, and this presentation will focus on narrow AI. Generative AI has achieved great things in accessibility, and AI advancements have improved various applications for people with disabilities. OpenAI's newest model allows blind people to access AI through Be My Eyes. Challenges with AI include bias and accuracy. AI can be a useful tool in improving accessibility, but it has limitations and risks. Advocating for AI ethics and accessibility, collecting data, and addressing specific accessibility concerns are important. Khan Academy is working on improving accessibility in education.

In this Talk, the speaker discusses how to build smart user interfaces using React and AI. They explore using AI to enhance data interaction in a table component and demonstrate how AI can be used to pre-fill calendar slots in a booking engine. The speaker also explains how to create dynamic UIs with React by adjusting language prompts and specifying UI elements. They introduce LangChain, an open-source framework for building language model applications, and explain how it can be used to generate UIs. The speaker concludes by discussing how JSON data can be used to generate UIs and provides documentation and examples for building UI components.

Hi everyone, I want to share a story about my experience as a junior developer working with a client who had strict technical standards. They required high performance for web and mobile apps, but lacked a measurable way to determine performance. To address this, I created a tool called Flashlight, similar to Lighthouse, that measures mobile app performance. Flashlight provides performance scores and can run end-to-end tests to explore the app. The speaker's dream is to have an easy way to determine app performance without extensive testing. Using AI to automate the tedious task of exploring an app is the speaker's proposal. The speaker explains that the repository for the app exploration tool is open source and will be shared later. The tool requires commands to be run, such as logging in and performing a search. The AI tool helps identify performance issues, suggests using React DevTools for debugging, and recommends using FlashList instead of FlatList in React Native. The AI can continue after errors occur, and controlling the AI's completion of goals can be achieved by adding a human or another AI agent manager into the loop. AI in end-to-end testing is gaining popularity, with companies launching AI-based testing tools. The goal is to automate app exploration using AI and ensure quality checks for performance, accessibility, and security. AI can help QA engineers by automating repetitive tasks and allowing them to focus on more important aspects. The cost of performing one test case using the 4.0 Mini model is usually less than one cent. AI testing with Flashlight is still experimental, but the non-AI part is reliable. Future improvements include running on CI and easy iteration on multiple projects.

tldraw is a canvas and a whiteboard SDK that can be integrated into React apps. It allows for high-performance rendering using the DOM and React. tldraw's canvas is powerful and can be enhanced with Vision models for image creation and AI utilization. It enables easy iteration and updating of designs based on annotations. tldraw can be used to create UIs, complex applications, interactive things, calculators, ASCII art, and more. It explores the use of AI on the canvas and can generate images, code, flowcharts, and content. tldraw focuses on performance optimization, AI model exploration, security, usability, and accessibility. The Talk also discusses the business model, future plans, and the vision of building a plugin community for the canvas.

Today's Talk discusses empowering NX with AI and building an AI-powered documentation system. NX is a powerful build system with smart features like project graph analysis and dependency management. The AI features include an assistant for streamlined navigation of documentation, AI error explainer, and resource allocation optimization on NX Cloud. The AI-powered documentation system uses embeddings and vector matching to find relevant Docs, utilizing tools like OpenAI, GPT, Superbase, and Vercel's AI SDK.

AI offers opportunities for React developers to code faster and automate tasks. Generative AI is a crucial area for developers to focus on. Working with AI APIs and RAGS can open up new possibilities for projects. Orchestration frameworks and tools like Lanchain and relevance help chain tasks together and work with different AI models. AI is a supplement to human capabilities and learning to code with AI can help developers push boundaries and become better.

AI in front-end development empowers developers to take on more ambitious projects and innovate at a faster pace. Natural language is a new programming language that can be used for coding, learning, and automating complex tasks. However, it is important to remember that AI is a supplement to human capabilities, not a replacement. Developers need to evolve their skills and stay ahead of emerging technologies to work effectively with AI. The demand for AI engineers is high.

Today, I'll be talking about GspotJS and Gaussian Splatting, a revolutionary graphics pipeline that can render high-fidelity scenes at 144 FPS. Gaussian Splatting is a technique that converts data directly into an image using Gaussians. GspotJS is a lightweight JavaScript library for Gaussian Splat rendering, with features like 4D rendering. The library aims to provide a simple and speedy way to view Splats on the web, while more advanced applications can use Mackellog Gaussian Splats 3D. Both Gaussian Splatting and gSplotJS are open-source.

This Talk explores the world of coding assistants powered by language models (LLMs) and their use cases in software development. It delves into challenges such as understanding big code and developing models for context in LLMs. The importance of ranking and code context is discussed, along with the use of weak supervision signals and fine-tuning models for code completion. The Talk also touches on the evaluation of models and the future trends in code AI, including automation and the role of tasks, programming languages, and code context.

This Talk explores the role of mindset in software development and the use of AI assistants. It emphasizes the importance of training the AI assistant and the potential impact of outdated beliefs. The conscious mind is discussed as the gatekeeper to thoughts and feelings, influencing our actions and results. Mindful media consumption and prioritizing mental health are also highlighted, along with the need to support team well-being. The Talk concludes with the significance of fitness in supporting mental health.

The Talk emphasizes the importance of maintaining a second brain, a curated collection of digital notes, to enhance memory retention in software engineering. Building a second brain helps in recalling information, problem-solving, and retention. It is easy to create your own second brain using various tool options like Notion, Obsidian, Reflect, Rome Research, and Tana. Starting with small, self-contained notes and gradually expanding to form a mesh of related information is recommended for effective learning and retention.

May 25th, 1977, the first Star Wars movie inspired generations with its space travel and lightsaber fights. Goran Vukicic built replicas of the P-Droid and found a community of designers who sell 3D designs for printing. Bringing the droid to a conference is challenging but rewarding. The NVIDIA Jetson Ori Nano is a powerful device for running AI models. Building Star Wars droids at home is possible with open-source development.

This talk explores the use of AI in the interviewing process for software engineering. It discusses the history of interviewing and the skills needed for future interviews. The speaker questions the relevance of traditional coding challenges and highlights the shift towards evaluating specific programming languages and debugging skills. The talk also emphasizes the importance of understanding the uses and limitations of AI and the value of communication skills in technical interviews.

AI can help developers become 10x more efficient by leveraging powerful GPUs. Codium is an AI developer tool that can accelerate learning, analyze dependencies, and provide personalized coding experiences. It abstracts away complexity and allows developers to focus on building user experiences. Codium aims to transform the software industry and empower developers to become 10x engineers.

Today, we explore DJ mixing and how deep learning revolutionizes the art by discussing sound processing, extracting features, and using machine learning. Deep learning allows for efficient extraction of audio features and high-resolution track separation. Neural networks can achieve source separation by converting audio to spectrograms and applying convolutional and recurrent neural networks. This has immediate impact on industries such as karaoke and music transcription.

This talk explores the ways AI is being used to shape the future of applications. It emphasizes the importance of an AI-first approach and the potential for AI to enhance various industries, such as aviation. The talk also contrasts the limitations of the AI-on-top approach with the continuous learning and user-centric focus of the AI-first approach. It discusses the importance of building trust through safety, transparency, and browser-based processing, and highlights the potential of AI to address user experience issues and improve accessibility.

Maya Chavin, a senior software engineer at Microsoft, discusses generative AI and the core model for LM. The flow of a document Q&A service and the importance of prompts in enhancing it are explored. The injection and querying phases of document Q&A are explained, emphasizing the need for efficient storage, indexing, and computing relevant prompts. The talk also covers the use of embedding models, optimization strategies, and the challenges of testing and validating AI results. Creative uses of LLMs and the impact of AI on job security are mentioned.

AI is a revolutionary change that helps businesses solve real problems and make applications smarter. Vectors enable semantic search, allowing us to find contextually relevant information. We'll build an AI-powered documentation site that answers questions, provides contextually relevant information, and offers links for further exploration. To enable vector search with MongoDB, we use the LingChain method to connect to MongoDB, create vector embeddings for user queries, and find related documents using maximal marginal reference. Join the workshop for a complete start-to-finish guide and integrate MongoDB Vector Search into your next React-based AI application.

AI testing with generative AI is revolutionizing JS testing by automating test creation and improving software test processes. Key technologies like natural language processing and neural networks, as well as quality data, play a crucial role in AI testing. The benefits of AI testing include speed, efficiency, adaptability, bug detection, and limitless potential. Generating JavaScript tests can be tailored to different tools like Selenium, and there are popular tools available for automating test automation. AI tools like Datadog, RecheckWeb, and Applitools Eyes offer powerful capabilities for anomaly detection, visual regression testing, and code list testing. The horizon for AI in testing continues to expand with evolving capabilities, and understanding AI's role in testing revolution and machine learning is crucial for practical application and continuous learning.

This Talk discusses integrating machine learning into software testing, exploring its use in different stages of the testing lifecycle. It highlights the importance of training data and hidden patterns in machine learning. The Talk also covers generating relevant code for test automation using machine learning, as well as the observation and outlier detection capabilities of machine learning algorithms. It emphasizes the use of machine learning in maintenance, bug management, and classifying bugs based on severity levels. The Talk concludes with the results of classification and bug management, including the use of clustering.

Today's Talk introduces the concept of building a digital AI-powered sommelier using the Bracel.ai SDK. The speaker emphasizes the role of developers in shaping the impact of AI, particularly generative AI, on our work. The Talk showcases a simple digital sommelier built using the Resell AI SDK and OpenAI API, highlighting the ease of implementation and the potential of open source tools. The speaker encourages users to explore the possibilities of generative AI responsibly and recommends checking out And Why, a design and technology studio from Munich.

Today's Talk discusses the application of GenreBI in apps, using Docker to make ChatGPT work on any machine, challenges with JSON responses, testing AI models, handling AI API and response issues, counting tokens and rate limits, discovering limitations and taking a reactive approach, reliability and challenges of AI APIs, and the use of GPT and AI Copilot in software development.

This Talk discusses the use of AI in API testing and provides a step-by-step strategy for incorporating artificial intelligence with chat.dpt. It emphasizes the importance of analyzing documentation and creating test cases using tools like Swagger and Cypress. The Talk also addresses the role of human involvement in testing, the balance between manual work and AI assistance, and the need for validation of AI-generated tests. Overall, AI can significantly speed up the testing process, but human analysis and vigilance are still necessary for accurate results.

TypeChat is an open-source library that uses TypeScript types to guide and validate responses from language models. It allows for the creation of complex responses and provides a way to repair errors in the model. TypeChat programs enable better flow of data and the ability to refer to individual steps. The math example demonstrates the use of a program translator and metaprogramming techniques for type safety. Language models trained on both code and prose perform well in this context.

Today we're going to build a React application with just our voice using GitHub Copilot, an AI peer programmer powered by OpenAI Codecs. It's important to be specific in your comments to get accurate suggestions from Copilot. Prompt engineering tips can be used to create different applications, such as a basic markdown editor and a simple to-do app. The application was tested successfully by adding and deleting to-do items using voice commands.

This talk explores JavaScript's role in distributed machine learning at scale, discussing the lack of tooling and the accessibility of machine learning deployments. It also covers cloud-based machine learning architecture, machine learning at the edge, and the use of HarperDB for simplified machine learning deployment. The concept of iterative AI and model training is also discussed.

GitHub Copilot is an auto-completion tool that provides suggestions based on context. Research has shown that developers using Copilot feel less frustrated, spend less time searching externally, and experience less mental effort on repetitive tasks. Copilot can generate code for various tasks, including adding modals, testing, and refactoring. It is a useful tool for improving productivity and saving time, especially for junior developers and those working in unfamiliar domains. Security concerns have been addressed with optional data sharing and different versions for individuals and businesses.

The Talk discusses the use of machine learning in search engines, specifically focusing on semantic search and vector embeddings. It explores the integration of JavaScript and machine learning models, using Weaviate as an open-source vector database. The Talk demonstrates how to connect to Weaviate, query data, and perform machine learning queries. It also highlights the benefits of Weaviate, such as its superior developer experience and performance. Additionally, the Talk addresses customization options, data privacy concerns, and the varying effectiveness of different machine learning models.

With AI and web GPU, it's an exciting time to be a developer. The speaker's journey involves combining programming and design, leading to the creation of Pure Blue, a powerful programming environment. Adding AI to the mix, the speaker discusses the potential of AI in the creative process and its impact on app development. The talk explores the role of React components and WebGPU in enabling fine-grained editing and running AI models locally. The future of app development is discussed, emphasizing the need to stay ahead of the curve and leverage the power of JavaScript.

This Talk explores controlling apps with the mind and the future of UI and UX. It discusses the integration of VR and AR into UI and UX, the understanding of neurons and EEG headsets, connecting to Muse via Bluetooth, measuring brain waves and blink detection, feeding data to machine learning, and mind control with AR. The speaker emphasizes the importance of learning React Native, AR, React, Bluetooth, and drones for those interested in exploring these topics.

AutomL simplifies the complexity of building machine learning models, allowing engineers to focus on the hard problems and applications. It enables the solving of problems that wouldn't be feasible otherwise. The three-step AutomL approach by MathWorks includes wavelet scattering for feature extraction. AutoML also enables feature selection and model optimization for memory and power-limited embedded systems. MATLAB can translate to low-level code for deployment.

The Talk discusses the current state of AI and the challenges faced in educating developers. Google's mission is to train 10 percent of the world's developers in machine learning and AI. They have developed specializations and training initiatives to make AI easy and accessible. The impact of AI education includes rigorous certification exams and partnerships with universities. The Talk also highlights the growth trends in the tech industry and the importance of AI skills. TensorFlow is recommended for its deployment capabilities, and practice is emphasized for building a career in machine learning.

Today's talk explores intelligent prefetching in React, including code splitting, lazy loading, and prefetching to improve performance. The use of neural networks for sequence prediction and training with actual user behavior is discussed. React context is used to link UI handlers with predictions and prefetching, enabling dynamic content import and improved user experience. The combination of AI and UI development is showcased in this personal project.