#artificial intelligence

Today we will discuss React server components with AI and how to build a better search experience using them. We will learn how to make a Next.js app AI-enabled using the Vercel AI SDK. The Vercel AI SDK's streamUI function with the GPT 4.0 model will be used to make suggestions interactive. We will explore the use of history and conversation in AI and how to continue the conversation and read the result. The concept of generative UI with the vector database will be introduced, along with querying the database for movies. We will process user queries and return movies based on them. The power of React server components in enhancing UI will be demonstrated. In summary, the Talk covers vector embeddings, natural language search, and generative UI.

AI design challenges include bias, safety, and security. Trust and transparency are important in AI. Design principles for AI include user control, fighting bias, and promoting good decision-making. AI can enable the design process and investors expect to see it included in products. AI empowers individuals to create and share ideas, but managing expectations is crucial.

In this Talk, the speaker demonstrates how to create an AI chat bot that can answer questions based on information it was never trained on. They build a basic RAG pipeline in just five minutes using live coding. The speaker also shows how to create embeddings and a vector database, set up a vector search index and endpoint, and modify the chat route to enhance the chat bot's capabilities. The program is run and tested, and the Talk concludes with an invitation to join a workshop for more information.

AI in game development has evolved rapidly, with generative AI being a focus. However, game developers like Romero Games have concerns about ethics and prefer using AI to automate processes and make creative work easier. AI has been used in games for decades, from path-finding AI to decision trees. Procedural world building and advanced AI technology are pushing the boundaries of FPS games. Different teams within a company have different approaches to the use of AI, depending on their specific needs and requirements.

Web AI in JavaScript allows for running machine learning models client-side in a web browser, offering advantages such as privacy, offline capabilities, low latency, and cost savings. Various AI models can be used for tasks like background blur, text toxicity detection, 3D data extraction, face mesh recognition, hand tracking, pose detection, and body segmentation. JavaScript libraries like MediaPipe LLM inference API and Visual Blocks facilitate the use of AI models. Web AI is in its early stages but has the potential to revolutionize web experiences and improve accessibility.

Codium is a generative AI assistant for software development that offers code explanation, test generation, and collaboration features. It can generate tests for a GraphQL API in VS Code, improve code coverage, and even document tests. Codium allows analyzing specific code lines, generating tests based on existing ones, and answering code-related questions. It can also provide suggestions for code improvement, help with code refactoring, and assist with writing commit messages.

Join Christoph from Nakazawa Tech in building the AI for Athena Crisis, a game where the AI performs actions just like a player. Learn about the importance of abstractions, primitives, and search algorithms in building an AI for a video game. Explore the architecture of Athena Crisis, which uses immutable persistent data structures and optimistic updates. Discover how to implement AI behaviors and create a class for the AI. Find out how to analyze units, assign weights, and prioritize actions based on the game state. Consider the next steps in building the AI and explore the possibility of building an AI for a real-time strategy game.

This Talk discusses building a voice-activated AI assistant using web APIs and JavaScript. It covers using the Web Speech API for speech recognition and the speech synthesis API for text to speech. The speaker demonstrates how to communicate with the Open AI API and handle the response. The Talk also explores enabling speech recognition and addressing the user. The speaker concludes by mentioning the possibility of creating a product out of the project and using Tauri for native desktop-like experiences.

This talk explores the use of AI in web development, including tools like GitHub Copilot and Fig for CLI commands. AI can generate boilerplate code, provide context-aware solutions, and generate dummy data. It can also assist with CSS selectors and regexes, and be integrated into applications. AI is used to enhance the podcast experience by transcribing episodes and providing JSON data. The talk also discusses formatting AI output, crafting requests, and analyzing embeddings for similarity.

In this Talk, the speaker discusses how to build smart user interfaces using React and AI. They explore using AI to enhance data interaction in a table component and demonstrate how AI can be used to pre-fill calendar slots in a booking engine. The speaker also explains how to create dynamic UIs with React by adjusting language prompts and specifying UI elements. They introduce LangChain, an open-source framework for building language model applications, and explain how it can be used to generate UIs. The speaker concludes by discussing how JSON data can be used to generate UIs and provides documentation and examples for building UI components.

Hi everyone, I want to share a story about my experience as a junior developer working with a client who had strict technical standards. They required high performance for web and mobile apps, but lacked a measurable way to determine performance. To address this, I created a tool called Flashlight, similar to Lighthouse, that measures mobile app performance. Flashlight provides performance scores and can run end-to-end tests to explore the app. The speaker's dream is to have an easy way to determine app performance without extensive testing. Using AI to automate the tedious task of exploring an app is the speaker's proposal. The speaker explains that the repository for the app exploration tool is open source and will be shared later. The tool requires commands to be run, such as logging in and performing a search. The AI tool helps identify performance issues, suggests using React DevTools for debugging, and recommends using FlashList instead of FlatList in React Native. The AI can continue after errors occur, and controlling the AI's completion of goals can be achieved by adding a human or another AI agent manager into the loop. AI in end-to-end testing is gaining popularity, with companies launching AI-based testing tools. The goal is to automate app exploration using AI and ensure quality checks for performance, accessibility, and security. AI can help QA engineers by automating repetitive tasks and allowing them to focus on more important aspects. The cost of performing one test case using the 4.0 Mini model is usually less than one cent. AI testing with Flashlight is still experimental, but the non-AI part is reliable. Future improvements include running on CI and easy iteration on multiple projects.

tldraw is a canvas and a whiteboard SDK that can be integrated into React apps. It allows for high-performance rendering using the DOM and React. tldraw's canvas is powerful and can be enhanced with Vision models for image creation and AI utilization. It enables easy iteration and updating of designs based on annotations. tldraw can be used to create UIs, complex applications, interactive things, calculators, ASCII art, and more. It explores the use of AI on the canvas and can generate images, code, flowcharts, and content. tldraw focuses on performance optimization, AI model exploration, security, usability, and accessibility. The Talk also discusses the business model, future plans, and the vision of building a plugin community for the canvas.

Today's Talk discusses empowering NX with AI and building an AI-powered documentation system. NX is a powerful build system with smart features like project graph analysis and dependency management. The AI features include an assistant for streamlined navigation of documentation, AI error explainer, and resource allocation optimization on NX Cloud. The AI-powered documentation system uses embeddings and vector matching to find relevant Docs, utilizing tools like OpenAI, GPT, Superbase, and Vercel's AI SDK.

AI offers opportunities for React developers to code faster and automate tasks. Generative AI is a crucial area for developers to focus on. Working with AI APIs and RAGS can open up new possibilities for projects. Orchestration frameworks and tools like Lanchain and relevance help chain tasks together and work with different AI models. AI is a supplement to human capabilities and learning to code with AI can help developers push boundaries and become better.

AI in front-end development empowers developers to take on more ambitious projects and innovate at a faster pace. Natural language is a new programming language that can be used for coding, learning, and automating complex tasks. However, it is important to remember that AI is a supplement to human capabilities, not a replacement. Developers need to evolve their skills and stay ahead of emerging technologies to work effectively with AI. The demand for AI engineers is high.

Today, I'll be talking about GspotJS and Gaussian Splatting, a revolutionary graphics pipeline that can render high-fidelity scenes at 144 FPS. Gaussian Splatting is a technique that converts data directly into an image using Gaussians. GspotJS is a lightweight JavaScript library for Gaussian Splat rendering, with features like 4D rendering. The library aims to provide a simple and speedy way to view Splats on the web, while more advanced applications can use Mackellog Gaussian Splats 3D. Both Gaussian Splatting and gSplotJS are open-source.

This Talk explores the role of mindset in software development and the use of AI assistants. It emphasizes the importance of training the AI assistant and the potential impact of outdated beliefs. The conscious mind is discussed as the gatekeeper to thoughts and feelings, influencing our actions and results. Mindful media consumption and prioritizing mental health are also highlighted, along with the need to support team well-being. The Talk concludes with the significance of fitness in supporting mental health.

This Talk explores the world of coding assistants powered by language models (LLMs) and their use cases in software development. It delves into challenges such as understanding big code and developing models for context in LLMs. The importance of ranking and code context is discussed, along with the use of weak supervision signals and fine-tuning models for code completion. The Talk also touches on the evaluation of models and the future trends in code AI, including automation and the role of tasks, programming languages, and code context.

The Talk emphasizes the importance of maintaining a second brain, a curated collection of digital notes, to enhance memory retention in software engineering. Building a second brain helps in recalling information, problem-solving, and retention. It is easy to create your own second brain using various tool options like Notion, Obsidian, Reflect, Rome Research, and Tana. Starting with small, self-contained notes and gradually expanding to form a mesh of related information is recommended for effective learning and retention.

May 25th, 1977, the first Star Wars movie inspired generations with its space travel and lightsaber fights. Goran Vukicic built replicas of the P-Droid and found a community of designers who sell 3D designs for printing. Bringing the droid to a conference is challenging but rewarding. The NVIDIA Jetson Ori Nano is a powerful device for running AI models. Building Star Wars droids at home is possible with open-source development.

This talk explores the use of AI in the interviewing process for software engineering. It discusses the history of interviewing and the skills needed for future interviews. The speaker questions the relevance of traditional coding challenges and highlights the shift towards evaluating specific programming languages and debugging skills. The talk also emphasizes the importance of understanding the uses and limitations of AI and the value of communication skills in technical interviews.

AI can help developers become 10x more efficient by leveraging powerful GPUs. Codium is an AI developer tool that can accelerate learning, analyze dependencies, and provide personalized coding experiences. It abstracts away complexity and allows developers to focus on building user experiences. Codium aims to transform the software industry and empower developers to become 10x engineers.

Today, we explore DJ mixing and how deep learning revolutionizes the art by discussing sound processing, extracting features, and using machine learning. Deep learning allows for efficient extraction of audio features and high-resolution track separation. Neural networks can achieve source separation by converting audio to spectrograms and applying convolutional and recurrent neural networks. This has immediate impact on industries such as karaoke and music transcription.

This talk explores the ways AI is being used to shape the future of applications. It emphasizes the importance of an AI-first approach and the potential for AI to enhance various industries, such as aviation. The talk also contrasts the limitations of the AI-on-top approach with the continuous learning and user-centric focus of the AI-first approach. It discusses the importance of building trust through safety, transparency, and browser-based processing, and highlights the potential of AI to address user experience issues and improve accessibility.

Maya Chavin, a senior software engineer at Microsoft, discusses generative AI and the core model for LM. The flow of a document Q&A service and the importance of prompts in enhancing it are explored. The injection and querying phases of document Q&A are explained, emphasizing the need for efficient storage, indexing, and computing relevant prompts. The talk also covers the use of embedding models, optimization strategies, and the challenges of testing and validating AI results. Creative uses of LLMs and the impact of AI on job security are mentioned.

AI is a revolutionary change that helps businesses solve real problems and make applications smarter. Vectors enable semantic search, allowing us to find contextually relevant information. We'll build an AI-powered documentation site that answers questions, provides contextually relevant information, and offers links for further exploration. To enable vector search with MongoDB, we use the LingChain method to connect to MongoDB, create vector embeddings for user queries, and find related documents using maximal marginal reference. Join the workshop for a complete start-to-finish guide and integrate MongoDB Vector Search into your next React-based AI application.

AI testing with generative AI is revolutionizing JS testing by automating test creation and improving software test processes. Key technologies like natural language processing and neural networks, as well as quality data, play a crucial role in AI testing. The benefits of AI testing include speed, efficiency, adaptability, bug detection, and limitless potential. Generating JavaScript tests can be tailored to different tools like Selenium, and there are popular tools available for automating test automation. AI tools like Datadog, RecheckWeb, and Applitools Eyes offer powerful capabilities for anomaly detection, visual regression testing, and code list testing. The horizon for AI in testing continues to expand with evolving capabilities, and understanding AI's role in testing revolution and machine learning is crucial for practical application and continuous learning.

This Talk discusses integrating machine learning into software testing, exploring its use in different stages of the testing lifecycle. It highlights the importance of training data and hidden patterns in machine learning. The Talk also covers generating relevant code for test automation using machine learning, as well as the observation and outlier detection capabilities of machine learning algorithms. It emphasizes the use of machine learning in maintenance, bug management, and classifying bugs based on severity levels. The Talk concludes with the results of classification and bug management, including the use of clustering.

Today's Talk introduces the concept of building a digital AI-powered sommelier using the Bracel.ai SDK. The speaker emphasizes the role of developers in shaping the impact of AI, particularly generative AI, on our work. The Talk showcases a simple digital sommelier built using the Resell AI SDK and OpenAI API, highlighting the ease of implementation and the potential of open source tools. The speaker encourages users to explore the possibilities of generative AI responsibly and recommends checking out And Why, a design and technology studio from Munich.

Today's Talk discusses the application of GenreBI in apps, using Docker to make ChatGPT work on any machine, challenges with JSON responses, testing AI models, handling AI API and response issues, counting tokens and rate limits, discovering limitations and taking a reactive approach, reliability and challenges of AI APIs, and the use of GPT and AI Copilot in software development.

This Talk discusses the use of AI in API testing and provides a step-by-step strategy for incorporating artificial intelligence with chat.dpt. It emphasizes the importance of analyzing documentation and creating test cases using tools like Swagger and Cypress. The Talk also addresses the role of human involvement in testing, the balance between manual work and AI assistance, and the need for validation of AI-generated tests. Overall, AI can significantly speed up the testing process, but human analysis and vigilance are still necessary for accurate results.

TypeChat is an open-source library that uses TypeScript types to guide and validate responses from language models. It allows for the creation of complex responses and provides a way to repair errors in the model. TypeChat programs enable better flow of data and the ability to refer to individual steps. The math example demonstrates the use of a program translator and metaprogramming techniques for type safety. Language models trained on both code and prose perform well in this context.

Today we're going to build a React application with just our voice using GitHub Copilot, an AI peer programmer powered by OpenAI Codecs. It's important to be specific in your comments to get accurate suggestions from Copilot. Prompt engineering tips can be used to create different applications, such as a basic markdown editor and a simple to-do app. The application was tested successfully by adding and deleting to-do items using voice commands.

This talk explores JavaScript's role in distributed machine learning at scale, discussing the lack of tooling and the accessibility of machine learning deployments. It also covers cloud-based machine learning architecture, machine learning at the edge, and the use of HarperDB for simplified machine learning deployment. The concept of iterative AI and model training is also discussed.

GitHub Copilot is an auto-completion tool that provides suggestions based on context. Research has shown that developers using Copilot feel less frustrated, spend less time searching externally, and experience less mental effort on repetitive tasks. Copilot can generate code for various tasks, including adding modals, testing, and refactoring. It is a useful tool for improving productivity and saving time, especially for junior developers and those working in unfamiliar domains. Security concerns have been addressed with optional data sharing and different versions for individuals and businesses.

The Talk discusses the use of machine learning in search engines, specifically focusing on semantic search and vector embeddings. It explores the integration of JavaScript and machine learning models, using Weaviate as an open-source vector database. The Talk demonstrates how to connect to Weaviate, query data, and perform machine learning queries. It also highlights the benefits of Weaviate, such as its superior developer experience and performance. Additionally, the Talk addresses customization options, data privacy concerns, and the varying effectiveness of different machine learning models.

With AI and web GPU, it's an exciting time to be a developer. The speaker's journey involves combining programming and design, leading to the creation of Pure Blue, a powerful programming environment. Adding AI to the mix, the speaker discusses the potential of AI in the creative process and its impact on app development. The talk explores the role of React components and WebGPU in enabling fine-grained editing and running AI models locally. The future of app development is discussed, emphasizing the need to stay ahead of the curve and leverage the power of JavaScript.

This Talk explores controlling apps with the mind and the future of UI and UX. It discusses the integration of VR and AR into UI and UX, the understanding of neurons and EEG headsets, connecting to Muse via Bluetooth, measuring brain waves and blink detection, feeding data to machine learning, and mind control with AR. The speaker emphasizes the importance of learning React Native, AR, React, Bluetooth, and drones for those interested in exploring these topics.

AutomL simplifies the complexity of building machine learning models, allowing engineers to focus on the hard problems and applications. It enables the solving of problems that wouldn't be feasible otherwise. The three-step AutomL approach by MathWorks includes wavelet scattering for feature extraction. AutoML also enables feature selection and model optimization for memory and power-limited embedded systems. MATLAB can translate to low-level code for deployment.

The Talk discusses the current state of AI and the challenges faced in educating developers. Google's mission is to train 10 percent of the world's developers in machine learning and AI. They have developed specializations and training initiatives to make AI easy and accessible. The impact of AI education includes rigorous certification exams and partnerships with universities. The Talk also highlights the growth trends in the tech industry and the importance of AI skills. TensorFlow is recommended for its deployment capabilities, and practice is emphasized for building a career in machine learning.

Today's talk explores intelligent prefetching in React, including code splitting, lazy loading, and prefetching to improve performance. The use of neural networks for sequence prediction and training with actual user behavior is discussed. React context is used to link UI handlers with predictions and prefetching, enabling dynamic content import and improved user experience. The combination of AI and UI development is showcased in this personal project.