Remote Rendering with Web Workers

Hi, everyone. My name is Trish Thao. Thanks for joining me for this talk about remote rendering with web workers. Today, I will walk you through how we built Argo, an extensible framework that allows rendering of remote content inside of Shopify's apps.

Here's what we'll be covering today. I'll talk about the core concepts. How we built Argo, the end-to-end flow, how we can swap out React. And I'll follow up by having a quick demo of all the pieces working together.

Here are the core concepts behind remote rendering. First, we need to create a JavaScript sandbox. This sandbox allows us to securely execute code in a background thread. The communication to the main thread can be achieved by sending messages. This is implemented as a web worker on web. On iOS and Android, this is implemented as a headless JavaScript engine. Next, we need to implement a remote procedure call layer, RPC for short. This is a protocol that enables one program to call the service in another program. Finally, we make use of proxies. A proxy is a wrapper that intercept and redefines the target's core operations, such as invoking functions or accessing properties.

I'll talk about how we built Argo. First of all, what we were trying to achieve is to enable remote rendering. What this means is allowing an external script to define the UI, and then let the main application render it natively. The framework allows the external script to exchange data with the main application. That results in UI being rendered and custom behavior to be executed. So the objectives we were trying to achieve are to provide a good user experience. Injected UI content is indistinguishable from Shopify's own native content on all platforms. We wanted to provide a good developer experience. External scripts can be built using familiar languages like JavaScript and TypeScript, and frameworks like React or Vue, and we also wanted to provide a security layer. The external script can only access and execute APIs provided by Shopify. At a high level, this isn't what we were trying to do. We have a JavaScript sandbox that loads an external script inside of it.

Through an RPC proxy, the script can call the RenderUI inside the main application. When a user interacts with the UI, for example, clicking on a button, if the handler is defined inside the external script, the main application can call the handler through the RPC layer.

Argo was built on top of a library called RemoteUI. This library provides the following. A communication layer between the JavaScript sandbox and the main application. A remote component tree that the external script can operate on and the host can render. A proxy layer to allow the host to call functions defined in the JavaScript sandbox and vice versa.

Now I'll talk to you about the end to end flow of how rendered, how rendered, how the content is rendered. On web, first the host does some set up steps to set up the worker. Then we load the external script. And then the external script constructs UI using our libraries. Then it calls the render. This results in the host receiving a message with the serialized component tree, which is finally rendered as UI.

Digging into the whole setup, we are building an extension component that can communicate with a web worker. We start by setting up the worker and the RPC layer. We create a worker with a worker.js file. I'll dive deeper into what this is later. And then we call create endpoint to create a communication channel between the host and a worker. This create endpoint function is provided by Remote UI. And then we return the extension. The endpoint call method. This is a proxy that allows functions from inside the worker to be executed by the host asynchronously via message passing.

Next we set up an instance of the remote receiver. This receiver manages an internal component tree that can be operated on in response to messages. We also set up a controller which specifies the implementation to use when rendering UI. In this example, we are allowing buttons to be rendered using the button component from the Polaris UI library. The host gets to decide which components will be used. This makes it possible to swap out components depending on what's needed.

The last step is to render the remote renderer component from the remote UI library and pass it the receiver and controller. The remote render is responsible for converting the component tree it receives into UI using the implementation specified by our controller.

Now let's look at what's inside the worker script. Inside the worker, we define a render function the host will be able to call with a script URL, extension point, and a message handler. The first step is to call import scripts with the external script URL to load it. Then we create a remote route and pass our message handler. This sets up the connection between the host receiver and the workers remote root. We also create a map of registered extension callbacks. We rely on the external script to populate the map. When render is called, we get the registered callback for the extension point and call it with the remote root. Here we can also pass in additional data or APIs so that the external script can get access to it. But this example only shows us passing in the remote root.

Next, we set up the workers RPC layer by creating an endpoint to the worker. This is the counterpart to the endpoint created on the host. We then call to expose the render function. Internally, the RPC layer automatically managed storing the function in memory and then responding to a message from the host to trigger the render function when necessary. This seamlessly allows the worker and host to exchange data and make proxy calls to each other.

Another thing that's inside the worker file is to set up globals available for the external script. Here is where we expose the extend function under the namespace Shopify. The external script will be able to call this function to register a callback to be saved in the map we saw earlier. And then we are also able to restrict globals available to the external script. We are removing the ability to call import scripts so that additional scripts cannot be loaded. Finally, once the render function is set up inside the worker, the host just needs to call render with the external script and the extension point. The extension point is set up through a string playground. This is just a representation of a point available on the host which external scripts can execute custom behavior on. We are also passing through the receiver.receive method that will serve as the handler for messages coming in from the remote route. Let's jump into the external script inside the worker. Once it's loaded, custom behavior can be executed. Here's an example of an external JSX script rendering custom UI. First, the script imports a button component from our React library. The script then creates an app component that renders the button with a title and a callback assigned through the onpress property. Writing the script is exactly the same as writing any other React app.

Let's dive into the button component from the Argo React library. It is constructed by calling the create remote React component provided by Remote UI. This function follows the same interface as a React functional component. One thing to note is that the onpress is now converted to a function that returns a promise. This is because all functions passed through the RPC layer always return a promise, as they are executed asynchronously via message passing.

If you're using an editor with TypeScript enabled, you get access to static typing and code completion, just like any other React component. Once the script sets up the app component, it needs to call to actually render the UI. The Argo library provides two functions, extend and render. Extends provide a way to call back for a particular extension point. In this example, we are setting the call back for the playground extension point, which is the same one enabled by the host. Then we set the callback to be calling render and return our app component we defined earlier.

If we dig into the extend method provided by the library, this simply calls Shopify extend, the method that we've defined earlier in the worker script. The library takes care of the implementation details by providing the extend function. Similarly, the Argo library also provides the render method that takes care of the implementation details. At a high level, this function takes in a render callback and returns a function that can be triggered with a remote root. It then sets up a custom React reconciler. To keep things simple, I have omitted our custom convicts that's passed into the reconciler, but you can think of this reconciler as a similar reconciler to React DOM or React Native. However, it's hooked up to our remote root, and this allows the remote root to manage the internal component tree and communicate with the host in order to actually do the UI rendering.

So once the reconciler is set up, we call the external scripts render callback. And then we append the results to the remote root container. And finally, we call mount on the remote root. On the host side, we get a mount message and the children from the remote root as the payload. Digging into the render payload, we get an array of objects representing each node in the tree. Recall that we are setting the props as follows, in the external script. We have a button with a title and onPressedCallback. This gets converted into a node that has a unique ID, the type that represents its implementation, and the props as specified. You can see that onPress is now converted to a proxy function. When the proxy is called, the RPC layer takes care of calling the right handler with the matching proxy ID from inside the worker, and then the SayHi callback is triggered. We also receive children as an array of objects. In this case, we don't have any children, so the array is empty.

Finally, the renderer on the host side converts the payload into UI. Internally, the renderer takes care of converting each node in the component tree by recursively calling React CreateElement with the implementation and the props for each child. Now that we've covered how UI is rendered, I'll walk you through how UI gets updated based on user inputs.

Here, we have an example of an external script rendering a text field with the value prop managed internally using state. When a user types into the text field, the onChange proxy is called, which triggers the setTextValue method to update the state inside the worker. The text field value prop is updated with a new text value. Internally, React Reconciler calls the remote route to handle updating their component tree. This in turn sends an updated message through the receiver on the host, which then triggers an update to its internal route. Finally, the renderer is called with the updated route that contains the updated prop for the text field, resulting in a UI being rendered.

Now that we've covered how Argo is built on web using React, I'll talk about how React can be swapped out. Looking back at the end to end flow for rendering UI, a few pieces can be replaced. Different client libraries can be used by the script to construct the UI. For example, the Argo React library can be replaced with vanilla JS or Vue.js. Similarly, we can swap out the renderer on the host side with another implementation. React DOM can be replaced with Swift or Kotlin or React Native. Swapping out the pieces is possible as long as the contract between the JavaScript sandbox and the host is maintained. The same render payload with the serialized component tree is sent no matter which library the external script is using to construct the UI. Similarly, no matter how the UI is rendered on the host, when a user interaction on the UI results in calling a handler on the external script, the host triggers the handler via a proxy.

Now, I'll show a quick pre-recorded demo of how all the pieces work together. Here, I have an external script rendering a form and a banner into Shopify on web and iOS. For this example, I've enabled live reload, which will allow us to see the updated UI as we change our code. Now, I'll demonstrate how we can pull in and utilize additional APIs provided by Shopify. The library provides useful hooks in order to do that. We'll call the use extension API hook. And from the API, we'll pull out the Toast object, which contains a method to call to show a Toast message. Now, all we want to do is output the content of our form in a Toast message as a JSON string. Once this refresh, we can test out our new callback. So I'm filling out the form. And now if I hit submit, I get a Toast message with my form data. I can also do the same on iOS.

Notice that iOS has rendered the Picker as a native Picker component. And when we submit, we get a Toast message, just like on web. The same external script is used to render content on both iOS and web. And the content is exactly like how native Shopify content appears. So that concludes the presentation. Our goal was built with flexibility in mind and security in mind. I hope this talk allowed you to learn more about how it works. Thanks for watching.

Good to see you again. So, people, Trisha has asked which hobbies you've picked up during the pandemic and it looks like 50% of you have started playing video games or I'm thinking maybe played more video games. 44% exercise, good to see people are staying healthy and working out. Reading, of course, great. Great to see board games, oh I love that. And some house plans. Really good to see people investing in the fresh air in their home. How about you, Trish? What have you been doing? As you can see from my background, mostly house plans. I picked up like maybe 50 house plans since COVID started. Also video games, I wish I picked up more exercise. That's on my to-do list. Well, if you got 50 plans, that's one a week. And that's a lot of walking and biking to get the plans, right? So that's kind of exercise. Oh yeah, and walking around the house, watering them, bringing them down, you know, clean them and water them. Yeah, it's a lot of work. Carrying all that water. Exactly. No, great job. So I myself have been picking up woodworking. I wouldn't say I'm good, but hey, it's something else in development. So we're going to go to the audience questions and yeah, hopefully you can give some insights to the, well, the things they want to know.

So we're going to go to the audience questions and yeah, hopefully you can give some insights to the, well, the things they want to know.

For me, web workers have always felt like black magic. So really cool to see that you've managed to make them your own.

First question is from Sasha. In what context would you use this strategy? This is a completely new concept to me, so it would be nice to understand why that's something we would want to use.

Okay, so the concept here is allowing any external code to render inside a main application. So at Shopify, we have different applications that can be extended by third party developers. So developers can build on top of our platform. So that means we want to be able to run their code securely and then have them execute custom behavior, render custom UI in our own apps. And an additional challenge is that our apps are written in many different languages. So it could be built in React Native, it could be written in Kotlin or just like a web application. We want the same external code to be written in a familiar language. So let's say an external developer can write in React and that same React code can be rendered natively on all of our platforms. So the strategy, I guess, would be like flexibility for both sides. The host can choose the technology it used to render and the external script can also choose the technology it wants to write the code in. I hope that clears it up. If Sascha doesn't think so, then she can join you in your facial chat and have a chat. Yes.

Next question is from Thorne. As a follow-up to Sascha's question, actually, this seems overkill for this example. What situation is this useful for? I guess you kind of already touched on that. Or do you want to extend your answer? Yeah. So I guess the example I gave may be simplistic. But I guess I kind of already covered it. We want flexibility. So we want the host to use whatever technology it needs to render UI and have kind of built a contract. So I mentioned that in my talk. We can interchange the pieces and technology use can be interchanged as long as the contract is followed. Cool.

Next question is from our user, visitor, Solal.

Why is Argo over federated modules if we can use a federated module? Right. So for our particular case, the code is external, provided by different developers. And it has to be run securely. So we cannot just pull in modules. It has to be an additional requirement is that we built up a platform to allow developers to submit code to us, but then they can also version it and roll back to previous releases. So they manage that part themselves. And on Shopify, we just render their code. So it's kind of like a additional separation basically. So we're not pulling in code that we wrote ourselves. This is all external.

All right. Next question is from P. Tarek. What's the point of using a remote renderer? And what's the motivation behind it? Is it gaining performance on the main thread? So the motivation is mainly security and flexibility. OK, well, fair enough. So there's a sandbox that the external script is executed inside of that's separate from the main thread. And then, basically, we can expose different APIs as we see fit to that external sandbox. So there's a separation that we have created for security. And sometimes I would guess that performance is a free bonus you're getting while it's implemented for security. Yeah. That's the motivation behind it. Mostly it's security. Yeah. I can imagine that that's driving force at your employer for most of the tech decisions, security. Yeah. As I mentioned, it's all external code. We want to allow flexibility, but also maintain control over what that code could do. So this is the way we've been able to make that work. Great. Awesome.

Next question is from Nippuna777. What are the advantages of using the worker here instead of doing it in the app itself? Yeah. So I think I mentioned this. We want a separation from the main app versus the sandbox basically that the external script can be run inside of. Cool. So again, security. Again, security. Next question. This looks the same question. This one is from Sosia.

Would one ever use it in React Native or web only? Yeah. So that's a good question. The beauty of about building this framework is that React Native is just a technology that we can choose to implement the host in. So depending on how the main Shopify app has been created, we can start developing in React Native. So some of our apps are written in React Native, the shop app is, so that's one place where if we wanted to offer extensions, we can write the implementation using React Native.

Nice. This is really powerful. Next question is from John B. He's asking, I know that Angular or Blazor allow server side renderings, is this a React implementation of server rendering? So actually, everything happens in the browser, like on the client. Sorry, not in the browser. I wouldn't say that, it's happening on the client. So it's not really server side render. So the UI, basically a payload comes in from the sandbox and then the host takes that payload and render actual UI. So it's all happening on the client side.

Okay, and then Paolo Henrique is asking, but what if the host is down? Yeah, so in this case, the host being the main app, if the host is down then the entire app would be down and nothing would be rendered. So yeah, this is kind of like, think of it as external scripts plugging into an existing app. So if the existing app doesn't run, then basically all the children or external pieces of it would not run as well. Either side.

Okay, we have time for a few more questions. So let's go to the next one from Werner Bafa. What was the effort to set up all of this architecture? That's gonna be a crazy answer, I guess. Yeah, so actually I would like to give a shout out to Kris Helvé. So he's the author of Remote UI. This is an open source library. You can visit the repo on GitHub. So Kris Helvé set up kind of like the foundation and our team built the pieces around it. So it was a really big effort because we have to think about like a whole bunch of things like making sure what we expose to third-party developers can be extended and customized depending on the app that they're extending into. So as I mentioned before, Shopify has a whole bunch of different apps for different purposes. So we needed to make this flexible. But yeah, the effort, it's hard to quantify, but like Kris Helvé wrote the foundation and then a whole bunch of people use that foundation to build all these pieces.

And was this an open source project or is Kris an employee of Shopify that then after building it for Shopify, open sourced it? I think, I don't know, Kris would be the best person to answer this, but it was always like built with open source in mind. Like anybody can go to the site and pull the library and build stuff around it. But I mean, he is an employee of Shopify. Yes, sorry, Kris Helvé works at Shopify. Yeah, a lot of our libraries, we try to contribute back to the open source community. And this is one of them.

Okay, great, great. Thank you and Shopify. The next question is from Zex, is this approach already in production and Shopify's production apps and for how long? Yeah, so it is in production. We are offering a certain extent of stability in some of our products and then we're gonna offer more and more in the future. It is live. We launched this for, I guess it was last October and there will be many more, we call them Argo extensions to come in different places on Shopify. So more to see next year. Maybe you'll give a talk again next year on how this works. Or someone else can have a chance. We have a big team building this. It's not just me. Yeah, okay. For us, you're the face now but of course there's a team.

Next question. And the last question, unfortunately we have time for. It's a scary question. It's from Nikhil Bittekar. Any learnings when using this with the IE browser? Oh, I see for IE browser, we only support Edge and so for any learning. So I guess the rendering side, it's the same as building a website. You have to make sure it works. Luckily for us, the extensibility we chose to offer is in one of our app called Shopify admin, which is built using Polaris. It's a UI framework with UI components that's been tested across browser. So we kind of bought that all for free. But it's building the whole side of it depending on where it's surfaced.

You have to make sure it works. So the whole side of the web, part of it has to work cross browser, basically. And Android and iOS has different challenges because they have to work on the phone. So yeah, it depends. We have to make it all work.

But luckily we were able to use Polaris. It saves a lot of work, of course.

Well, unfortunately, that's all the time I can give you here at this lovely stage. But people, if you have more questions for Trish or want to go deeper into Web Workers, she's going to be in her spatial chat. So Trish, thanks a lot for joining us and hope to see you again soon. And have fun in your speaker room on spatial chat.