Figma to React With AI, Are We There Yet?

Hey, everyone. Today we're going to talk about AI and in particular, how the recent advancements in AI technologies are going to empower us in the near future to write better React code, faster while also automating away some of the boring parts of our job.

So before we start a short presentation, I'm Federico, I'm a software architect at Anima and our mission is to automate as much as possible the boring parts of front-end development. So if you're interested, visit animaapp.com. Enough said, let's get started.

As a front-end developer, I used to focus a lot on how to write good React code. But that's not what we are going to discuss today. Today we won't be developers trying to write React code, but rather developers trying to create a machine that writes React code for us automatically. And we're going to start this journey with a story.

I worked for about a year as a front-end developer. And my day-to-day job involved many of the problems I'm sure you're all very familiar with, like interfacing with APIs, state management, performance optimizations. But there was one particular task I found extremely tedious. Every time we had to implement a feature, a designer would come to me with a design, usually in Figma, and then asked me to make it happen as an actual feature. But before implementing the actual functionality, the business logic, what I really liked, my first job was to take the design and then painfully recreate it into code, with all the styling, structure, elements, and so on. And because recreating a pixel-perfect design with code is not easy, the process often requires some back and forth between developers and designers. Because I'm sure you know, designers are really good at spotting slight differences between their designs and the production code. Anyway, very tedious. To the point where many developers started to ask themselves a question, can we automate it? And if we go from a high-level perspective, the question is, can we translate the information in a Figma design into a code representation? And at first, it might seem like an impossible task, as the two representations look quite different.

So, let's dive a bit into what's possible in Figma. First of all, in a Figma design, nodes are structured in a hierarchy, which is very similar to the DOM structure in an HTML page. Figma also supports components, and those components support properties, and those properties can define things like text content, visibility, instances, and many other things. Figma also supports subcomponents. So we can have a button component instead of a car component. And finally, Figma even supports actions and events. So for example, I can say, when I click on this button, open this web page. And by now you might have realized that this information is not that different from our React code. It's kind of the same information, but in a totally different representation. And so our mission today is to translate the Figma representation into a React representation. Before we dive into this translation process, we briefly need to discuss how to actually access the information contained in a Figma design. And it turns out that Figma makes it really easy, because it not only provides an API to access the information about a design, but it provides two.

One that is REST-based, and another that is JavaScript-based. If you're interested, I highly recommend checking out the documentation, but for the sake of this talk, you can basically think of a Figma design as a JSON document. This JSON structure contains all the design nodes, along with their style properties and other useful information. It's like HTML, but in JSON and using Figma-specific properties. The contained information is roughly the same.

So, we have this JSON document representing a design, right? How do we convert it into code? And there are multiple possible approaches, so let's see some of them. We start from the most basic one, heuristics. And you might have heard of the term heuristic before. For the sake of this talk, we'll refer to heuristics as a collection of rules that define the behavior of our system. In simple words, imagine some code that says, if this happens, do this, otherwise do that. Pretty straightforward.

And we can actually convert a Figma design into code by just using heuristics. In fact, here we've prepared a small JavaScript function which is less than 20 lines of code that can convert any Figma design into somewhat working HTML code. So, let's see what it does step by step. First of all, the function takes as input a Figma node. Then we need to decide which HTML tag the current node corresponds to and we use a very simple rule. If it's a text node, make it a span, otherwise make it a div. Then we use a Figma method called getCSSAsync which basically gives us an approximate CSS representation of the node styles. And finally, we basically call the function recursively for all the children. We can then put together all this information in an HTML string and if I open up Figma, I have a video here to show you. So basically if you open Figma and go to Plugin, Development, Open Console, this opens up a console you can paste any JavaScript you want to execute on. If I paste the function we just created, this produces some HTML code. Now if we get that HTML code and we paste it into CodePen, give it a sec, this is the card. That's pretty impressive.

And the thing is, let me go next, the thing is this is the resulting code and for sure it's not production ready. It's ugly, it's verbose, it uses inline styles, but the thing is, it works and it took us less than 20 lines of code. So you can imagine that even though it's not perfect, you can extend these heuristics to handle more and more cases and produce higher quality code. But even though heuristic can get us pretty far in creating high quality code, it's not a bulletproof solution and have its problems. For example, let's say that you want to generate code that uses CSS classes rather than inline styles. One of the problems you need to face is that you need to come up with good CSS class names for each node.

For example, the body node on Figma was, we need to come up with a class for that node inside the code. The first approach could be to use the Figma node name as a CSS class name. So, for example, you can see that the body node in Figma becomes a div with class body, the content becomes a div with class content and so on.

Now, in theory this works, but I'm not sure how familiar you are with real world Figma designs, but usually designers don't really care about proper layer naming. Instead of good names like body, content, title, you will see things like frame 23, frame 32, button 3 and so on. And if we feed this to our heuristics, the resulting code will have terrible class names.

And to expand on this, I prepared one of my favorite quotes, there are two hard problems in computer science, caching validation, naming things and off by one errors. And I love this quote because it's funny, but it's also really true. Naming things is hard also for humans. And we might not always realize it, but our job requires a high degree of creativity to solve problems. And it turns out that heuristics lack creativity. It's really hard to encode as a heuristic, a task that requires creativity. Like coming up with good CSS class names. Five years ago, this would have been a very hard problem to solve. You would have needed an entire team of machine learning engineers and weeks to tackle it. But things have changed.

You might have heard of this transformative recent technology that is, among other things, really good at creative tasks. And of course, I'm talking about large language models like Chadjupt, Claude, Lama, or many others. We won't be able to go in depth into large language models during this talk, but in a large language models are trained on enormous datasets, including open source projects on GitHub, Wikipedia, Stack Overflow, in general, a good chunk of the Internet. And their job, in a nutshell, is to always predict what comes next. So, for example, if I feed the model with, the CAD is on, it might decide that the table is what comes next. And because it has been trained on open source code, as well, it also knows the structure and conventions of popular frameworks like React. And so, if we go back to our previous example with bad CSS class names, it turns out we can solve the problem in 20 seconds by just asking, charge a bit. And here, I ask it to rewrite this React component to use better CSS names, and 20 seconds later, I have code that is using much better CSS class names. For example, user car container, user profile, user name heading, and so on.

And so, thanks to the recent advancements in AI, what used to take an entire team weeks now takes seconds. But this is just the tip of the iceberg of what's possible with these technologies. So, let's see another use case. At Anima, our goal is to go from any Figma design into code you would actually put in production. And that's a very difficult problem to solve.

One of the common complaints we got from our users is that even though we generated high-quality code, they couldn't put it in production because it didn't follow their team's conventions. And that's one of the problems of heuristic-based code generation. It would be extremely hard to make a heuristic that is general enough to handle all the possible variations in conventions, code style, styling frameworks, state management, and so on.

So how do we actually generate code that follows every team's conventions? A few years ago, you might have seen an image like this. This is called neural style transfer, and it's a deep learning-based technique. The idea is you start from an image, in this case a Lamborghini, and a reference, in this case a Van Gogh painting. You feed them into a deep neural network, and you get as output a Lamborghini painted in Van Gogh style. In a nutshell, this process transfers the style of the reference, the Van Gogh painting, to the original image.

You might be wondering why are we even discussing this at a React conference. The reason is that, from a high-level perspective, we could frame our problem in a similar way. Let's say we have a heuristic that generates high-quality code, but unfortunately it's not personalized, it does not follow the user's conventions. What if there could be a system that takes as input both the high-quality heuristic-generated code and your team conventions, combining them, basically taking the original code and refactoring it to follow your team's convention?

Now, let's see an example. This is some basic React code that one of our heuristics generated. The code works fine, but our hypothetical user is not very happy about it because it does not follow their team's convention. Let's say they want to use arrow functions, they want to use named exports and many other things. Now, how can we solve that? We could extend our heuristics to be more and more and more complex, to handle more and more cases, but that's not scalable because you know how variable the conventions and styling and state management can be among teams.

What we can do today is, again, ask charging PT. I won't go too much into the details of the prompt we use, but let's see the key points. I start by framing the program. You are an expert frontend developer. You should refactor a React project to follow the given guidelines. Then there is some boiler plates, and then finally the guidelines. You should use Arrow functions to define components, and then components should use named exports. And then we provide the component code, and after about 30 seconds, we have React code that uses Arrow functions, that use named exports, and that's exactly what the user wanted.

So again, to reiterate on this, we started from a high quality heuristic-generated code that was generic, and then we used large language models to customize it to actually follow your team's convention. And by the way, these techniques are also extremely useful if your team is going through a migration or a rewrite. Those things can save weeks of time. And this is not the most exciting thing I want to show you today. I want to close the talk with what I think is a sneak peek into the future, because even though the technology is not there yet, I'm confident that in two or three years most of us will integrate something similar in their workflow. So, so far we discussed how you can turn a Figma design into React code.

But that's a one-time thing. And what we need to do, what we really need to do, is updating existing components and screens rather than creating new ones. And that's especially relevant if your company uses a design system.

For example, a designer might add an icon on a bottom component in Figma and then some developer has to actually update the code to support that icon. And so I prepared a small proof of concept in one of our internal hackathons at Anima And I want to share it with you. So we start from a button in Figma, you can see here, and here's the corresponding code in our code base. We have the JSX, we have the CSS and let's pretend we are designers and we update this design. We changed the border radius to five. We changed the background color. We removed the icon.

Now, imagine if the designer could click, send changes, and then after a few seconds, our code base would be automatically updated. As you can see, the icon has been removed and the styles have been updated. The border radius, the background color, now all of this was done automatically using large language models. And the thing is, those large language models are so powerful that the process can also be done in reverse. And so, for example, I can go inside the style, update the radius from 20 to 1, and then a designer on the other side can hit pull changes. And after a few seconds, here we go, the border radius reflects the code.

Now the first time I saw this, I was mind blown. Imagine how much you can automate with a solution like this, like all the tedious back and forth between designers and developers. Even if you could automate 30, 40 percent of those small updates, that would be such a time saver. I'm going to show you the next slide. So here we go. Some might argue that the example we discussed so far was too trivial. How does it work with more complex scenarios? And to answer that question, let me discuss another key feature of large language models. So let's say you have this card in Figma will be the one on the top left and on the bottom right, you have the corresponding React code. Let's say that our designer changes the default description of the card from this is a test description to, I have changed the description. If I feed this design to the previous code that I showed the demo before, this is the result. As you can see, the AI, instead of updating the default description constant as I expected, basically updated the text in line. And that's not what I expected. I wanted it to update a constant. Now, let's pretend for a second that this code was not AI generated, but rather it was made by a new intern at the company.

What would you do? Well, if there are problems in a pull request, you usually write review comments, and in this case, I suggest you should not remove the constant. And the thing is, large language models are trained to be conversational, which means they can handle suggestion and iterate on their answer based on those suggestions.

And if we rerun the system with my comment, this is the result. Bingo. The AI updated the constant exactly as we expected. And what makes, again, large language models so transformative is that they can adapt to very different scenarios and code bases while also being able to iterate on their answer based on suggestions in natural language, which is something that we are very used to do in pull requests.

And what makes me so excited today is that those technologies have the potential to automate away many of the boring tasks of our job so that we can actually focus on the really important things like building features and delivering value to the users. With that being said, thank you so much for listening. If you want to know more, you want to chat, you can reach out to me on Twitter, YouTube, LinkedIn and also my website. And if you want to know more about Anima and what we do, you can visit the website animaapp.com. Keep in mind that many of the things I've shown today are actually usable products that you can use today. So if you're interested and you want to know more, just reach out to me. I'll be happy to answer all of your questions.