AI-Powered Frontend Development: Building Better UIs Faster

What's up, everybody? Let's talk about building better UIs faster with AI. These first two are mine. I'm co-founder and CEO of a company called Builder.io. I've also made some cool open source projects. So I want to step back for a second. The crazy part about our current wave of AI is that it's easy to forget that LLMs are literally just predictive text models. You give them a chunk of text and they predict what text is coming next. Like that's it. But of course, they're behind them is a very, very large neural network loosely modeled after how our brain works. Which means they're surprisingly smart sometimes. And the question always becomes, how can we make best use of the smart parts, coding and development especially for UIs, and minimize the not smart parts?

And so I don't want to bore you with much slides. I want a demo. So let's jump into a couple examples. I want to talk about using a few different mediums here. But I'm going to start in cursor. So in my case, this is just cursor IDE. I want to start with the basic, here, we're going to be in an IDE, and we're going to edit text. And so here's just a plain Vite app. I'll show you my running app. Let's npm run dev. And cool. Let's do some stuff. There's two things that I think are really valuable with AI tools for coding. Specifically in frontends. So the first one is what you would just call vibe coding. The second one is a trick you probably don't use much, if at all. So I'm excited about that one. But let's start with the basics. You already probably know that AI can do autocomplete. What you may not be using it for, realizing the full potential of, is just telling it to make updates to the UI.

So let's pop to app.jsx. In cursor, by default, the file I have open here will be included as context here. Of course, I can add additional files if those would be relevant as well, like app.css. Maybe it would be a good one. But cursor is pretty good at automatically figuring that out. You don't have to manage the context so tightly as you did in the olden days, in my opinion.

Let's go in and let's say, when I click the count button, launch a modal and show the modal in the count. So I'm just going to vibe, I didn't even spell it right. I'm just going to vibe some UI updates I want to make. In a lot of cases, I've had people, as much as like senior engineers on our team, basically say like, he doesn't write code any. He types what he wants and then he lets the AI do it. And look, my code update is done already.

So let's try our count. There we go. Cool. Now, in this case, it doesn't really match the dark mode. I'm guessing the white text here I can't see. Yeah. So that's super easy. Let's take a screenshot and be like, hey, cool, but I can't see the white text on a white modal, bud. I like to talk to AI in a friendly, joking way. You are, so you're welcome to talk to AI however you want to. I wouldn't recommend being mean if you've seen sci-fi movies. You never know what's going to come with regard to our AI overlords. Cool. I mean, I updated that crazy fast. I'm not even speeding this up. This is real-time. Great. And every time I click it, the count increases.

This is all bound. Beautiful. You can continuously give it any feedback you want. It's good at updating UIs and replicating patterns it already sees in the UIs. Awesome. A lot of cases, you want to add a button. You want to do a thing. Blah, blah, blah. I just vibe code it. And I only get in the details when I really care to.

If you need to add additional instructions, you can add custom instructions. Let's talk about a more advanced use case and a more advanced workflow that I definitely personally don't see people use enough. So you saw me here where I did a thing that I had to check it myself and I had to tell it what was wrong. That's a little bit slow, right? I'm basically now a QA tester. I'm kind of like a product manager plus QA tester. Product manager says what they want. The QA tester verifies if we got what we wanted and then repeat.

But that can be really tedious. It can be really annoying to prompt, test manually, prompt, test manually. You feel like you're in this infinite loop of death, especially with more complex things that require a lot of iterations. So what I found great success with is actually using a flavor of test-driven development. Let me show you. So let's go in here and I'll just continue. It's not even Git repo. Let's make this a Git repo really quickly. Git init. Awesome. Git add. And by the way, another cool feature of Cursor and VS Code is click a button. It'll generate a commit message for you.

Bam, it's not too bad. It used to be verbose in the past. It's not that verbose anymore. I like it. Good.

So let's do something more advanced. Let's go in here, and I'll make this prompt a little bigger because we're going to get a little fancy with our prompting tricks right now. I'm going to say write me a function that converts a markdown string to a single line of HTML string out. Here's the important part. If you did that, nine out of 10 times, AI will not get it right on the first pass. Maybe 95 out of 100 times, honestly. Even the latest grade is super great. Your favorite model. I don't care. It doesn't do well with this. But there's a trick.

What we're going to say is write tests first, write the code, run the tests, and then update the code until it tests us. Bam. Now you need to turn on autorun for this in Cursor first. So I'll let it go. It's going to start doing some stuff. Great. Now let's make sure our settings are correct here for anybody who is setting this up from scratch. We're going to go to features. I think it is. Enable autorun mode. So you can just turn this toggle on. Autorun mode means it can autorun bash commands. You can also give it very specific allow and deny list, etc. And you can also control it as it goes too.

So let's go back to our chat. And then I will say one thing, Cursor is doing this really annoying thing lately, where it used to just do stuff. And now it always asks, do you want me to do this stuff? And I just always have to say yes. Just do the thing, please. I don't know what happened. This happened in the last like week or two. And it's really annoying.

But this is great. It's going ahead and installing dependencies. I do like the choice of VTest. So let's go ahead and do that. It added react route or DAW. Or what was that? No, it added react testing library. So here, it started by generating my tests. This looks pretty good. Pretty nice. I will say if these tests pass, then I have a good sense as to what I wanted.

So again, if we're working on something, especially that's not inherently quite as visual, might be front end, but it might not be purely visual in the UI. It's nice to not have to be kind of a test dummy where I have to run the code and be like, oh, did I do that right? Here, I have, bam, automated tests. Right now. Hey, at least these pass. It's doing something right. And as you can see, it's going through and it's going ahead and running the tests on its own. It's using the test output to then make subsequent updates. It ran the tests and I saw one was failing. There's our one failed. A second time, they failed and then, bam, good timing. They all pass now. This is really nice because I have guarantees. The thing does the thing and it finds it itself.

I don't have to tell it every time. I don't have to test it every time. It'll just continue to work through the problem with clear feedback. That's what the AI generally needs, is clear feedback. That's the big secret to LLM success in my opinion. Clear feedback. And if you can write code that gives the feedback, even better.

So let's make this more complex. Let's see how well our AI commit does. I love it. It's beautiful. Updated packages and padded. Markdown. Each and every generation of thing. Great. So let's go over here and tell it, please handle more advanced use cases. Add more elaborate tests and make sure they all pass.

So in this case, I'm telling it like do more complicated stuff. I don't see a lot of nesting. Do I even see lists here? Oh, I see lists, but I'm not seeing something crazy advanced. We're going to see if we can get 15 out of 15 tests passed, including all this more nuanced, stuff, including whitespace handling, multiple things, etc. The one thing I do want to share as a tip, though, is while it's exciting that I'm not using any hands right now, I wouldn't just go walk away from your desk or your computer and move on with your life and assume that AI is going to figure everything out.

There are problems. AI can get stuck in weird loops of death where it just can never solve something. So my opinion is you want to watch this stuff. If it's clearly stuck in a loop, in a lot of times you just want to pause it. If it got the same result a bunch of times, it might start doing weird, sketchy stuff in the code.

Let's go back to focusing more on the front end. In the front end, we're dealing with UIs. We're dealing with design systems. We're dealing with Figma designs. We're dealing with a lot of stuff that's inherently visual. And I found this pure text environment is not the perfect thing for everything when you have those kind of considerations.

So I want to show you something, a sneak peek at something we're going to be launching soon that we currently are calling Fusion. This is sort of trying to take the environment of Cursor but make it as visual as possible. Make it connect to tools like Figma deeply, more than just like a simple MCP integration would do. And make it feel like a design, a more visual environment.

Let me show you what I mean. And you all can tell me, you can tweet at me if you think this is a good thing or a piece of junk. So Fusion is this new projects tab in Builder. And again, this is trying to take LLMs and use them what they're best for, updating code, but make it a more visual experience. And I'll kind of show what I mean.

So here you can connect any repo or just build any app, live code it, whatever. And then within these repos, I've got a few. Let's launch a couple. So I've got this one. This is our Builder Academy site. This is a platform we have for showing courses, custom next.js code base, etc. And I can go in and this is sort of a visual cursor. Now we can do more visual stuff.

So maybe we want to do something like duplicate this and put these side by side. And maybe we want two of this. And maybe we want a little bit of space. And maybe we want to change the layout and direction of this. All this stuff is stuff that we can do in a way very similar to other design tools. Now, that's cool and all, but actually, that's not what I want to do, right? I want to start pulling updates we have in Figma. Let me show you that and show you how like a more visual interface to your code and through an LLM can be handy.

Let me show you that and show you how like a more visual interface to your code and through an LLM can be handy. So let's go over to Figma and let's go over to here we go. So here, this is a design that pretty much mirrors this I'm missing some sections, but I'm going to bring them in. So let's go and let's just grab this and let's copy. Actually we've got to launch the builder plugin and we hit export design and then we can just cool. We'll just paste this in. Awesome. New section added. And the cool part is because it's visual, I can make some visual tweaks too. So maybe I want a little bit more space here, maybe less or whatever. I can do whatever I want. Maybe I want a little bit of difference amount of space here. Like this margins larger than I expected. Maybe let's just pull it in tighter. Wonderful. I can do additional things as well. For instance, let's add a little bit of context in here as well. And let's pull in some additional stuff. So let's say I want to add a charis below this and I want multiple slides in it. So I can not just copy and paste to the canvas here. I can also copy and paste to the context. So let's go over here and say, let's grab our first slide, export design, copy and say also add a carousel with two slides below this slide one and just paste the Figma content. And then let's go slide two and paste. And now we can apply these all at once. So the visual changes I made, any UI changes, copy, pasted, tweaked, et cetera, and additional context. Like, I also want this additional stuff that I want you to make into a carousel. You can use this for multi-step flows that you can clean up afterwards, multiple pages, build a whole apps. That's all fine and dandy. While this is running, let's do a couple of other things. Let's do some prompting with the UI directly.

So let's go into another branch I made. Oops, that's a different design here. Another branch of the same app. I'll show you other stuff too. And in this one, I want to just use the current stuff we have to make something new. So I'm just going to prompt and say add a contact us page and add a link in the net. I'm just going to hit go. The agent here can work for a while if you have something sufficiently advanced, a very complex application with a million things. It'll scan your code base. It'll do a lot of the stuff you used to cursor.

But again, we're in a very visual environment, one where you could bring your designers into the tool and get pull requests from them. I'll show you that in a second. And one where you could get your PMs in here. You could all be collaborating on one canvas effectively. While this runs, let me go check on, okay, we're still opting our carousel. Let's show some stuff with some other kind of design systems and APIs. So here I've got this cloud scape design system example.

And here, why don't we do this? Say add a new page at slash weather and make it a weather dashboard powered by the OpenMatteo API. You can use your own design systems. You can use your own APIs. But what's cool is this agent can also look up stuff on the web, look up the docs, figure things out or reference local documentation or private docs that you have. Let's do one other cool example. Let's go to this Mui dashboard and let's switch to light mode so you can see it a little better. And let's go in here and let's say, oh, there's nothing in this customers tab.

Let's say, update the contents of the customers tab to have a table and populate it with users from random user dot me add edit buttons where when I hit edits, there is a modal with the user fields I can edit. Let's just run that. So a bunch of different things we can do using different design systems and different setups and stuff like that.

Here's that contact page now we are generating. And why don't we go in and look at that and the other contact page using all our different stuff. And here's where we can go in and make additional edits. So why don't I go in here and say like, oh, there's a specific thing I want. I'd actually rather this be above this one. And let's say, also make the FAQs an accordion. So I made my visual changes.

You can use all the kind of CSS settings and like a Figma style editor and then let it rock and roll. Back over here. Boom. Here's the other updates we asked for a minute ago where we made some changes. We also so we brought in these two new sections. We also added in this carousel. And again, it was from the context.

I pass in design, sold it to make a carousel with these slides. And it did. Multiple changes in one. But if let's go do two more things. Let's go in here and this kind of width got shrunk up a little bit. So I really want these things to fill the total width so we can go select in the layers menu. Let's get to like the root of the carousel. And it looks like these things are width 100 percent, but actually filling 100 percent of the width. This one as well. This one is sort of like a grid. We got a max width. I don't want that max width here. Get rid of that. Boom. That's more like what I'm really looking for. And I'm gonna guess we've got a max width on something here. Yeah, I don't necessarily want that. Oh, I might break our carousel. Why don't we get a little bit less fancy for a second. But the idea here is you have a complete, unlimited canvas to do whatever that you want to apply changes however you want. Let me show you one other really cool use case, too, that is really helpful when you have this visual canvas is design updates. It's very common that you go over here. And let's say we're like, oh, I don't like how we got a purple background versus a gray one. So let's go in here and I'll update the design to make it so that there we go. We trimmed up some padding and spacing and we gave it the gray background. And now what I'm going to do is I'm going to take this updated layer and I am going to export that. And this time I'll add in the context. But this time what I'm gonna do is a style transfer. So I'm going to take existing code we have, whether builder generated it or it was made by a human. And I'm going to jump in here and I'm going to select this and I'm going to say, update my selection to have the design changes I just made.

Paste. So I'm pasting the famous context and I'm bringing in design changes. So you can use this to very granularly make updates. We found a visual interface where you can specifically select, modify, and pull in designs. Apply this here, this there. Can be much more impactful than just a general kind of like YOLO kind of figure it out.

Let's go maybe check out our dashboards really quickly. So here I think we wanted like a weather page just tell the AI to fix it, and then it can go and start finding a fix. And then why don't we jump over. We had one other design system. We had the material UI. Let's let those run for a second and fix. And while that's happening, let me show you one last thing that's really cool is when you're happy with the changes, all you do is click the send PR button.

Let's do a couple so we can look at it. We'll start with this one. So this is one using material UI. And let's do one more. Let's do the cloud, cloud. So here's our weather page. Let's see if our errors are fixed. Did AI do everything for me. Let's type in San Francisco. And this is 100%. The AI just came up with this. Beautiful. I'm getting a little bit of not a number, but I got all this stuff. I could feed in a screenshot and say, oops, fix those nones. But you get the idea. This is quite cool. And we have quite a number of stats here.

All dynamic. Let's go to New York, New York. Bam. And maybe we can see if our other updates are applied. One more error to fix. Again, let the AI work through it. This is an example of auto-detecting errors and applying the fixes.

But while we do that, let's go take a look at the PR for our weather dashboard, as well as our material table, perhaps. So here's our pull request for our weather dashboard. You can see it knows to update the route. It's adjacent configuration. It's using open Mateo API. It's using types. It's doing all kinds of useful, interesting things.

The other cool thing is it's using CloudScape components through and through end to end. Reusing existing components using CloudScape. I mean, there's almost not a single div to be found here. It is CloudScape from top to bottom. So this is cool. And the coolest thing, too, in this world is what we can do is let's say some of these let's find a component that's a little big, you know, too big and we want to break it up.

Let's say we can tag the builder bot here in a comment. And say builder bot, move this to be in its own class and hit comment. We can tag the builder bot and it can make updates and push updates to the PR so we can continuously give it feedback and it can update the code to be how we want it. Why don't we go back and we can also see one other thing is it takes a second, but we added or we should have added when we go to our other PR and our other PR. We can see we added a really nice PR description and title. Boom. Customer management. Bam, going back to this PR.

It takes a second. The AI is going to generate all this stuff. That'll maybe take one more moment. But anyway, you get the idea. This is brand new and something we're just coming out with. And I'd love to see your feedback in a moment. Well, we're going to run out of time for my talk, but in a moment we'll see that the agent said it'll get right back to work and it'll push updates. Oh, it did push an update already. Oh, I'll show you that now before I have to run out of time.

This PR is already outdated. The agent made an update. Let's go and look at its commit. And ba-bam. It pulled these out and into their own file, just like what I asked. So, perfect. Really, really awesome. I would love any feedback you all have. Try this out. Use those cool cursor tips. Let me know how they're working for you. Try Builder for a more visual interface. You can have your whole team collaborate on and do more kind of precise, granular UI generation and Figma design, precise transferring, etc. Thanks so much for listening to me talk.