Rise of the Robots

Rate this content
Bookmark

Discover the future of automated mobile application testing with a JavaScript-powered mechanical arm. During this talk we will explore the design, prototyping, and implementation of this cutting-edge solution, optimizing testing efficiency and precision on real mobile devices. We will also discuss the challenges of building a hardware solution for the real world, and how to overcome them.

This talk has been presented at TestJS Summit 2023, check out the latest edition of this JavaScript Conference.

FAQ

Proxima Analytics is an open-source, ethical-first analytics platform founded by Theodor, a software engineer from Athens, Greece.

Isaac Asimov's three laws of robotics are a set of rules designed to govern the behavior of robots, ensuring they act ethically and safely around humans.

Mobile end-to-end testing faces challenges such as the need to test on real devices, the sandboxing of applications which limits OS interactions, and disruptions like phone calls and notifications.

A Delta robot consists of a base with three or four motors attached, using pairs of arms connected to an actuator. It's known for its fast, efficient, and natural movements.

Jason Huggins is one of the creators of Appium and Selenium, and he founded a company that developed Tapster Bot, an open-source device used for automating tests on real devices.

JavaScript is effective for programming microcontrollers in robotics due to its natural handling of events and callbacks, making it suitable for tasks like automating robotics tests.

Calibration in robotics is crucial for ensuring accuracy and reliability in tasks, as it helps the robot identify precise positions of objects it interacts with, such as mobile phones in testing scenarios.

Push notifications can be tested by attaching a camera to the robot for visual verification or by intercepting the notifications through software tools that emulate or capture the notification events.

Theodore Vorillas
Theodore Vorillas
27 min
07 Dec, 2023

Comments

Sign in or register to post your comment.
Video Summary and Transcription
This Talk discusses the possibility of robots taking over based on Asimov's three laws of robotics. It explores the use of automation robots for testing, including building and controlling them. The Talk also covers implementing interfaces, conducting math game challenges, and the capabilities of automation testing. It addresses questions about responsive design, camera attachment, and the future roadmap. The affordability of the setup and the potential for future automation are also discussed, along with a rapid fire Q&A session.
Available in Español: El Ascenso de los Robots

1. Introduction to the Question of Robots Taking Over

Short description:

Hi, everyone. I'm Theodor, a software engineer and founder of Proxima Analytics. Let's discuss if robots will take over based on Isaac Asimov's three laws of robotics.

Hi, everyone. I hope you're enjoying the conference so far. Okay. So I'm Theodor. I'm a software engineer based in Athens, Greece. I'm also the founder of Proxima Analytics. It's an open source ethical first analytics platform that you should definitely check this out. And you can also find me online via the alliance wordless. So if there's a question that we can make for 2023, it's going to be are there robots going to actually take over? Like, is going your freeds going to take you as a hostage, or are you going to lose our jobs as software engineers? And truth to be told, this question is, like, rather too old, if we can say so. So this is Isaac Asimov, one of the most famous science fiction authors. And in one of his books, which is called iRobot, in 1953, I think, he came up with the three laws of robotics. So basically, this is a manual when robots are going to actually take over. And today, we're going to challenge this question, and we're going to try to find out if this is true or not.

2. Exploring Automation Robots for Testing

Short description:

In 2018, while working for a company, I experimented with different aspects of software engineering. Mobile end-to-end testing is challenging due to sandboxed applications and external interruptions. I had the idea to build a robot for automating tests on real devices. There are three categories of automation robots based on movement: Cartesian robots, robot guns, and Delta robots.

So fast forward, in 2018, I was working for a company. And besides that, I was also trying to experiment with different aspects of software engineering, trying to mix up things like 3D printing, electronics, and stuff.

And I was working for a company where we had three major products. Like, the first one was a web application and two mobile applications, one for Android and another one for iOS. And I can really tell you this. So mobile end-to-end testing is pretty damn hard. It really is.

So if you have ever tried to run end-to-end testing in mobile applications, we're basically more or less stuck to the emulator. On the other hand, we also need to test mobile applications on real devices, right? But it's quite tricky, actually. So for security reasons, most of the applications are sandboxed. That means that we cannot actually test how our applications are interacting with the operating system. We cannot test sharing links between applications. We cannot test a workflow where we want to authenticate users using the email client in the mobile phone.

Moreover, we have external interruptions like phone calls. Mobile phones are like living organisms. So we have phone calls, notifications, push notifications, and so on. And then I had that weird idea, like, what if we could actually try and build a robot for automating tests in real devices? And you know what? I know that most people would actually think something scary or big like the Terminator or so. But truth be told, we can categorize automating automation robots in three big categories based on their movement.

So the first category is they're called Cartesian robots. We have three axes. And basically, the actuator moves into the three-dimensional space using bolts and wheels. 3D printers and CNC machines are working in that way, in that format, actually. But in our case, it does not work because you have a limited area to work with. And also, the movement does not feel that natural.

Next up, we have robot guns, which are basically made with separate motors and three or four attached parts. They are a needless standard for the car industry and medical operations as well. But they're pretty hard to operate. And moreover, they're pretty expensive and mostly used for repetitive tasks. So our pickup for today are called Delta robots. We have a base on the top, three or four motors attached.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Scaling Up with Remix and Micro Frontends
Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
This talk discusses the usage of Microfrontends in Remix and introduces the Tiny Frontend library. Kazoo, a used car buying platform, follows a domain-driven design approach and encountered issues with granular slicing. Tiny Frontend aims to solve the slicing problem and promotes type safety and compatibility of shared dependencies. The speaker demonstrates how Tiny Frontend works with server-side rendering and how Remix can consume and update components without redeploying the app. The talk also explores the usage of micro frontends and the future support for Webpack Module Federation in Remix.
Network Requests with Cypress
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Cecilia Martinez, a technical account manager at Cypress, discusses network requests in Cypress and demonstrates commands like cydot request and SCI.INTERCEPT. She also explains dynamic matching and aliasing, network stubbing, and the pros and cons of using real server responses versus stubbing. The talk covers logging request responses, testing front-end and backend API, handling list length and DOM traversal, lazy loading, and provides resources for beginners to learn Cypress.
Testing Pyramid Makes Little Sense, What We Can Use Instead
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Top Content
Featured Video
Gleb Bahmutov
Roman Sandler
2 authors
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
Full Stack Components
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
RemixConf EU discussed full stack components and their benefits, such as marrying the backend and UI in the same file. The talk demonstrated the implementation of a combo box with search functionality using Remix and the Downshift library. It also highlighted the ease of creating resource routes in Remix and the importance of code organization and maintainability in full stack components. The speaker expressed gratitude towards the audience and discussed the future of Remix, including its acquisition by Shopify and the potential for collaboration with Hydrogen.
Debugging JS
React Summit 2023React Summit 2023
24 min
Debugging JS
Top Content
Watch video: Debugging JS
Debugging JavaScript is a crucial skill that is often overlooked in the industry. It is important to understand the problem, reproduce the issue, and identify the root cause. Having a variety of debugging tools and techniques, such as console methods and graphical debuggers, is beneficial. Replay is a time-traveling debugger for JavaScript that allows users to record and inspect bugs. It works with Redux, plain React, and even minified code with the help of source maps.
Making JavaScript on WebAssembly Fast
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
Top Content
WebAssembly enables optimizing JavaScript performance for different environments by deploying the JavaScript engine as a portable WebAssembly module. By making JavaScript on WebAssembly fast, instances can be created for each request, reducing latency and security risks. Initialization and runtime phases can be improved with tools like Wiser and snapshotting, resulting in faster startup times. Optimizing JavaScript performance in WebAssembly can be achieved through techniques like ahead-of-time compilation and inline caching. WebAssembly usage is growing outside the web, offering benefits like isolation and portability. Build sizes and snapshotting in WebAssembly depend on the application, and more information can be found on the Mozilla Hacks website and Bike Reliance site.

Workshops on related topic

Master JavaScript Patterns
JSNation 2024JSNation 2024
145 min
Master JavaScript Patterns
Top Content
Featured Workshop
Adrian Hajdin
Adrian Hajdin
During this workshop, participants will review the essential JavaScript patterns that every developer should know. Through hands-on exercises, real-world examples, and interactive discussions, attendees will deepen their understanding of best practices for organizing code, solving common challenges, and designing scalable architectures. By the end of the workshop, participants will gain newfound confidence in their ability to write high-quality JavaScript code that stands the test of time.
Points Covered:
1. Introduction to JavaScript Patterns2. Foundational Patterns3. Object Creation Patterns4. Behavioral Patterns5. Architectural Patterns6. Hands-On Exercises and Case Studies
How It Will Help Developers:
- Gain a deep understanding of JavaScript patterns and their applications in real-world scenarios- Learn best practices for organizing code, solving common challenges, and designing scalable architectures- Enhance problem-solving skills and code readability- Improve collaboration and communication within development teams- Accelerate career growth and opportunities for advancement in the software industry
Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
How to Start With Cypress
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
Filip Hric
Filip Hric
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
Integrating LangChain with JavaScript for Web Developers
React Summit 2024React Summit 2024
92 min
Integrating LangChain with JavaScript for Web Developers
Featured Workshop
Vivek Nayyar
Vivek Nayyar
Dive into the world of AI with our interactive workshop designed specifically for web developers. "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" offers a unique opportunity to bridge the gap between AI and web development. Despite the prominence of Python in AI development, the vast potential of JavaScript remains largely untapped. This workshop aims to change that.Throughout this hands-on session, participants will learn how to leverage LangChain—a tool designed to make large language models more accessible and useful—to build dynamic AI agents directly within JavaScript environments. This approach opens up new possibilities for enhancing web applications with intelligent features, from automated customer support to content generation and beyond.We'll start with the basics of LangChain and AI models, ensuring a solid foundation even for those new to AI. From there, we'll dive into practical exercises that demonstrate how to integrate these technologies into real-world JavaScript projects. Participants will work through examples, facing and overcoming the challenges of making AI work seamlessly on the web.This workshop is more than just a learning experience; it's a chance to be at the forefront of an emerging field. By the end, attendees will not only have gained valuable skills but also created AI-enhanced features they can take back to their projects or workplaces.Whether you're a seasoned web developer curious about AI or looking to expand your skillset into new and exciting areas, "Hands-On AI: Integrating LangChain with JavaScript for Web Developers" is your gateway to the future of web development. Join us to unlock the potential of AI in your web projects, making them smarter, more interactive, and more engaging for users.
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
React Day Berlin 2022React Day Berlin 2022
86 min
Using CodeMirror to Build a JavaScript Editor with Linting and AutoComplete
Top Content
WorkshopFree
Hussien Khayoon
Kahvi Patel
2 authors
Using a library might seem easy at first glance, but how do you choose the right library? How do you upgrade an existing one? And how do you wade through the documentation to find what you want?
In this workshop, we’ll discuss all these finer points while going through a general example of building a code editor using CodeMirror in React. All while sharing some of the nuances our team learned about using this library and some problems we encountered.
Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
WorkshopFree
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop