React in the Autonomous Robotics Industry

Rate this content
Bookmark

Too many people are wondering what a JavaScript developer is doing in a robotics company , but In an era where the intersection of software development and robotics is rapidly expanding, it's not uncommon for JavaScript developers to find themselves in the midst of the autonomous robotics industry .

During this talk I'll show some use cases of react in the robotics industry , showcasing how this versatile JavaScript library can revolutionize the way we approach robotic development from user interfaces for controlling robotic systems to VR teleoperation capabilities . Robotics often demands real-time responsiveness and efficient resource utilization we'll discuss the careful considerations and decisions we've made to ensure optimal performance. We'll delve into techniques such as virtual component memoization, and minimizing re-renders .

This talk has been presented at React Summit 2024, check out the latest edition of this React Conference.

FAQ

Hamza Hawi is a software engineer at the Autonomous Robotics Research Center at TII.

A video overlay in Leaflet is an image or video layer placed on top of a map to show additional information, such as a weather forecast with wind direction.

The main focus of the Autonomous Robotics Research Center is to research, develop, and innovate new technologies for robots to make them fully autonomous or perform specific tasks individually or in combination.

The center works with a mixed fleet of robots, including autonomous cars, drones, boats, and racing cars.

JavaScript is used to develop mission planning software, which optimizes routes or sets specific goals for robots to achieve autonomously. The software includes maps, multimedia streaming, and control components.

The team chose React because of its performance and responsiveness, its ability to handle multiple data messages per second, memoization, component lifecycle methods, and the large support community with numerous libraries and packages.

Leaflet is an open-source package used for maps. It was chosen due to its flexibility and multiple plugins, which help in operating various types of vehicles like drones, cars, and boats.

The react-leaflet-draw package provides drawing tools for creating paths, zones, and waypoints on maps. It includes callbacks for creation, deletion, and editing events.

The team uses the react-leaflet-offline package to cache map images and tiles for offline use, reducing internet dependency.

The team uses the strategy design pattern to handle different types of joysticks for various robots, optimizing control based on the specific requirements of each robot.

Hamza Haoui
Hamza Haoui
11 min
18 Jun, 2024

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Hamza Hawi, a software engineer at the Autonomous Robotics Research Center, discusses the use of React and Leaflet in robotics. The research center utilizes a mixed fleet of robots and relies on React for mission planning software, while Leaflet is used for mapping and custom layers. Leaflet offers flexibility for different types of vehicles and supports the creation of custom layers like image and video overlays. Additionally, the talk mentions the use of video overlays for weather forecasts and optimizing joystick usage with a strategy design pattern.

1. Introduction to JavaScript and Robotics

Short description:

Hello everyone. I'm Hamza Hawi, software engineer at the Autonomous Robotics Research Center at TII. We research, develop, and innovate new technologies for robots. We have a mixed fleet of robots, including autonomous cars, drones, boats, and racing cars. React is crucial for our mission planning software due to its performance, memoization, and community support. We use leaflet for maps.

Hello everyone. I'm Hamza Hawi, software engineer at the Autonomous Robotics Research Center at TII. And what led me to do this talk today is that I got too many questions about what JavaScript developer is doing on the Autonomous Robotics Research Center, or how can we combine JavaScript and robotics, or what's the relationship between JavaScript and robotics. And in order to answer this question, I will start by defining who we are.

So I'm part of the Autonomous Robotics Research Center at TII, and what we are trying to do is to research, develop, and innovate new technologies for robots in order to make them fully autonomous or perform some specific tasks individually or all together combined. And we have a mixed fleet of robots, which can be autonomous cars, drones, or even boats. And we have also racing cars. And in order to put some context to the discussion, I will show like an architectural overview. So we have a mixed fleet of robots that are sending data continuously to the server, and that data will be sent to the client side via web sockets. And for the client side, we may have control stations, laptops, tablets, or wherever is needed to take control over that robot.

So now React comes into play because we are trying to develop what's called a mission planning software. A mission planning software is a software that lets you, let's say, optimize a road or draw a specific path or goal for the robot to go to autonomously. And if we try to see the generic screenshot that I have on the left side, we can see that we have three core parts of the system. First will be the maps and the geographical information. And in order to have clear view about what's happening, we need to have some streaming coming from the robot. That's why we need multimedia components. And if we want to, let's say, drive that robots and the operation mode, we will still need to read some data from the joysticks that we have. And that's why we developed some control components to take control of that robot.

Now why we decided to choose React. As we know, robotics applications need good performance and responsiveness that will let you operate with, let's say, operate easily those robots. And if we try to see and analyze our software, we will see that we are receiving at minimum 10 messages each second for a robot from data messages or health of the sensors. And we also have multiple vehicles operating on the same time. So at this point, performance is very crucial. And that's why we decided to use React because React, due to its memoization and the component lifecycle methods, will help us to avoid unnecessary renders by choosing exactly when each component should re-render. And also, because of the memoization, the memoization will give us, let's say, the ability to reduce some computational power just by caching the operations that we did before. And one of the key things also is that the huge community support that React has, because as you know, React is here for many years, and the community is growing and you will have a large choice between libraries and packages that you can use. And this will help us to avoid to re-invent the wheel at each time.

Now let's start with the first part, which are the maps. As you know, when you want to choose a map to start working with it on React, we have multiple choices. But for us, we decided to choose leaflet.

2. Using Leaflet for Maps and Custom Layers

Short description:

Leaflet is an open source package with multiple plugins that provide flexibility for operating on different types of vehicles. We use react-leaflet-draw to send goals and paths to vehicles, and react-leaflet-offline to reduce internet usage. Leaflet also allows the creation of custom coordinate reference systems and the addition of custom layers, such as image overlays and video overlays for weather forecasts and other applications.

Leaflet, as you know, is an open source package and it has multiple plugins and it gives you the flexibility that you need to operate on multiple types of vehicles, drones, cars or boats or whatever. So that flexibility helped us to cover maximum use cases that we can.

Now let's start by the first functionality that we have. As you know, we need to send goal or path to vehicle and if we are operating under drones, we need to define the zone that we are operating to. For that, we use the package called react-leaflet-draw. So it's a package that came with a long list of, let's say, drawing tools from, I don't know, from lines, circles, rectangles, markers, that you can use. And it came with multiple set of callbacks for each event, for the creation or even when you delete or edit one of the paths that you draw before. And it will give you the vector that contains all the waypoints drawn before and you can use them whatever you want based on your use case and your application.

Now, what if we want to reduce the amount of internet that we are using or what if we are operating without internet? For that, we decided to use a package called react-leaflet-offline, but before talking about that package, let's just try to see and know how leaflet is rendering the tiles, which is the layout that we put on the top of the map. And if we see this, we will find that React is not rendering the whole map on the same, is not bringing the full layout on the same time, but it's bringing them as a set of rectangular images and loading them. And this will let us avoid to render a full map at the same time. So if we want to cache those images, one possible approach is to create your own server that will send those images based on X, Y, and Z, which is the zoom level, and send them to your frontend. And another approach also is by using what's a package called react-leaflet-offline. As you see here on this example, when using react-leaflet-offline, you will have the choice to save a specific area that you are zooming in right now or delete that area and save it again. The map is updated, if there is any update on the map, and on the blue, those rectangles on the blue, as we see here, those are the areas that we save at different zoom levels.

And for the next thing that made us choose Leaflet is that on some use cases, maybe you are not operating on Earth. Maybe you are creating some aerospace application where you are operating on different map than the map that we are using to. So React-Leaflet gives us the ability to create our own, let's say, coordinate reference system based on the requirements that we have. So for this, you can also define how many pixels you will have by each meter and put your origin of the map. You can choose the origin, define it as you want. Even the coordinate system, you can put your own coordinate system based on your use case. The second thing is that after defining exactly your coordinate and the reference system, we can put on top of it our custom layer. For example, if we are operating on a warehouse or a room or any indoor area, we will have to provide the map with our own tile. For that, you can put what's called on Leaflet, image overlay.

Now if you are wondering what we can do with a video overlay, for example, if we are on some weather forecast things and we want to show the direction of the wind. So for this, you can put in a video overlay that you will specify exactly the bounds, like the top right and the bottom left of the video. And it can be as we see on the right side of the slide. This is like a video overlay of a weather forecast where it will be moving and you can see the wind's direction and all those things.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

A Guide to React Rendering Behavior
React Advanced 2022React Advanced 2022
25 min
A Guide to React Rendering Behavior
Top Content
This transcription provides a brief guide to React rendering behavior. It explains the process of rendering, comparing new and old elements, and the importance of pure rendering without side effects. It also covers topics such as batching and double rendering, optimizing rendering and using context and Redux in React. Overall, it offers valuable insights for developers looking to understand and optimize React rendering.
Building Better Websites with Remix
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Top Content
Remix is a web framework built on React Router that focuses on web fundamentals, accessibility, performance, and flexibility. It delivers real HTML and SEO benefits, and allows for automatic updating of meta tags and styles. It provides features like login functionality, session management, and error handling. Remix is a server-rendered framework that can enhance sites with JavaScript but doesn't require it for basic functionality. It aims to create quality HTML-driven documents and is flexible for use with different web technologies and stacks.
Don't Solve Problems, Eliminate Them
React Advanced 2021React Advanced 2021
39 min
Don't Solve Problems, Eliminate Them
Top Content
Kent C. Dodds discusses the concept of problem elimination rather than just problem-solving. He introduces the idea of a problem tree and the importance of avoiding creating solutions prematurely. Kent uses examples like Tesla's electric engine and Remix framework to illustrate the benefits of problem elimination. He emphasizes the value of trade-offs and taking the easier path, as well as the need to constantly re-evaluate and change approaches to eliminate problems.
React Compiler - Understanding Idiomatic React (React Forget)
React Advanced 2023React Advanced 2023
33 min
React Compiler - Understanding Idiomatic React (React Forget)
Top Content
Watch video: React Compiler - Understanding Idiomatic React (React Forget)
Joe Savona
Mofei Zhang
2 authors
The Talk discusses React Forget, a compiler built at Meta that aims to optimize client-side React development. It explores the use of memoization to improve performance and the vision of Forget to automatically determine dependencies at build time. Forget is named with an F-word pun and has the potential to optimize server builds and enable dead code elimination. The team plans to make Forget open-source and is focused on ensuring its quality before release.
Using useEffect Effectively
React Advanced 2022React Advanced 2022
30 min
Using useEffect Effectively
Top Content
Today's Talk explores the use of the useEffect hook in React development, covering topics such as fetching data, handling race conditions and cleanup, and optimizing performance. It also discusses the correct use of useEffect in React 18, the distinction between Activity Effects and Action Effects, and the potential misuse of useEffect. The Talk highlights the benefits of using useQuery or SWR for data fetching, the problems with using useEffect for initializing global singletons, and the use of state machines for handling effects. The speaker also recommends exploring the beta React docs and using tools like the stately.ai editor for visualizing state machines.
Routing in React 18 and Beyond
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Top Content
Routing in React 18 brings a native app-like user experience and allows applications to transition between different environments. React Router and Next.js have different approaches to routing, with React Router using component-based routing and Next.js using file system-based routing. React server components provide the primitives to address the disadvantages of multipage applications while maintaining the same user experience. Improving navigation and routing in React involves including loading UI, pre-rendering parts of the screen, and using server components for more performant experiences. Next.js and Remix are moving towards a converging solution by combining component-based routing with file system routing.

Workshops on related topic

React Performance Debugging Masterclass
React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Top Content
Featured WorkshopFree
Ivan Akulov
Ivan Akulov
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
Concurrent Rendering Adventures in React 18
React Advanced 2021React Advanced 2021
132 min
Concurrent Rendering Adventures in React 18
Top Content
Featured WorkshopFree
Maurice de Beijer
Maurice de Beijer
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?

There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.

Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Hooks Tips Only the Pros Know
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Top Content
Featured Workshop
Maurice de Beijer
Maurice de Beijer
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.
React, TypeScript, and TDD
React Advanced 2021React Advanced 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Web3 Workshop - Building Your First Dapp
React Advanced 2021React Advanced 2021
145 min
Web3 Workshop - Building Your First Dapp
Top Content
Featured WorkshopFree
Nader Dabit
Nader Dabit
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn