Agentic Interfaces: Tools, Skills, Generative UI and Web MCP

Hey, everybody, thank you so much for coming out. Stoked to be here. Like Scott said, my name is Wes. I'm half of, or part of Syntax, actually not half, we have a big team now. I'm here today to talk to you about agentic interfaces. Don't leave yet, because I know how obnoxious some of these hucksters are, and I'm not here to sell you anything today, but you've probably heard at least one of these things today, right? UI is dead. Chat is the final interface. Just give me a skill, bro. All of these things. The web is dead. My grandma has vibe coded all kinds of stuff. And we're kind of sitting here being like, well, I paid all this money for this conference and I'm leaving kind of bummed out. Do we even need websites anymore? Yes. That's the talk. Thank you for coming, but the way that we interact with the web certainly is changing.

So let's take a look at some ways that we actually interface with software. Sometimes I don't want a UI at all. This is a piece of software called Handbrake that I use for video tasks. Sometimes I just want to use my voice, like yelling at Google or Siri. Other times, I want the UI tailored to my needs, like using FreshBooks for expenses. Sometimes I want my apps to have interconnected capabilities, centralizing control.

We have various ways to implement interfaces as developers. One approach is generative UI, where AI adjusts in real-time based on user input. Generative UI has three levels: basic tool calls, declarative UI composition, and fully generative code mode. Each level offers increasing autonomy in UI generation. Declarative UI involves the AI composing building blocks to fulfill requests like rescheduling a flight. Fully generative UI goes all out in creating interfaces, sometimes referred to as code mode.

Sometimes I want my apps to kiss, right? This is the explosion of skills and tools and CLIs and all of that stuff that we're seeing right now. This is probably the best use case that we have right now, which is I have one agent and then I have all of these different things connected to it via MCP or tools or skills or agentic browsers, all of the different stuff, and it's like one little command center where I can run everything from. We have all of these different ways.

How do we as developers implement all of these interfaces? Let's take a look at one, which is generative UI. Generative UI is when the AI dynamically will create, transform, or simply just select the AI in real time in response to a user input as well as contextual clues. We saw this slide earlier today where there's three ways you can do it. The first one is the very basic. How many of you have built an MCP server with tool calls before? A handful of you. In that case, you're going to declare what the tools are, get whether, you may be able to pass some params, and the server is going to return data.

With a static UI, you might also return a static component that then renders out to some weather thing, and you render it out inside the chat or on your web page or whatever it is that you want. Then, a step further, you have declarative UI, which you may have a whole bag of components that you use for your website and you'll give all of those to an LLM. You ask the agent or the LLM to figure out which of these building blocks will you compose together to make it.

Then you say, I need to reschedule my flight, and it's going to return to you some sort of spec. It's often JSON. Sometimes it's streaming text. That's going to return to you and say, all right, given I know that you have these components, I'm going to click them together for you, and then we're going to render out what I think is going to be a nice UI for yourself. The third one is fully generative, like whole hog all the way out. I like to refer to this one or you've probably heard this term as code mode.

Just let the AI generate it all, give it a spot to run some code. In order to understand what these things are, let's take a look at this application right here. Pretty simple grocery list. You got multiple stores. You're able to add stores. You can add items. You can drag and drop. You can mark it. You get the point, right? You add stuff to a list. A simple example of inlining components where you may go to the chat and say, hey, I want to add some items to my thing, but I don't want to just use the chat. I want to be able to see the components.

Instead of running the tool call, maybe it's going to return to you a component inside of there. Same thing when you were to view the list, like show me my Walmart list, it's going to return to you a list, but you can interact with that list in there. You can check things off. You don't have to explicitly tell it what to do. Code mode is kind of cool because, yes, the LLMs are good at returning tool calls and JSON specs for the UI, but you know what LLMs are really good at? Maybe you guys have heard, LLMs are good at writing code, right?

I said give me a map of all of the items on my grocery list on a map overlaid on it. Then it ran get items, and now it's making a map for me, and then it renders out probably the worst map of the world that I've ever seen. I left it in there. That's not very good, but the point is that that's not an existing component that was in there, obviously.

This is just a LLM, or I say give me a wacky shopping list, right? We'll see what it builds here. Oh, a silly, wacky shopping list where everything is ... and it's still usable because it's still calling the tools and integrated with everything that is available there, right?

Some thoughts on that. There's probably middle ground between the two. If you work for a bank, you probably don't want the wacky list showing up. Be like, show me my wacky banking accounts and stuff like that. You want a little bit of control over the components that are shown. There's probably somewhere in between that you want to hit, which is dynamic components, things that are known, versus being able to click them together and fully generative.

This also just isn't about MCP. I don't think we're anywhere, maybe close, but we haven't landed on what the final thing is. I don't think anybody's totally happy with where we're at. These specs are still evolving. This also isn't just about chat interfaces, because you probably have seen some of these examples today, and that sounds awful, to use that as the interface. You're telling me that no more web browser. We're just going to work with these little squares inside of here. This is one of the grocery stores in Canada. If you want to increment the amount of Limes in your cart, you have to type, add another Lime, and it takes 20 seconds to respond, and then come back and render out the UI. How did we get from worrying about 20 millisecond clicks to 20 second things? Not to say that you can't do that, but a lot of these current MCP interfaces kind of suck.

How did we get from worrying about 20 millisecond clicks to 20-second interactions? Many current MCP interfaces are sluggish and offer subpar experiences, like Starbucks' slow and watered-down ordering app. The journey towards better interfaces extends to adaptive and reactive systems, such as Smart Home setups. These systems should be context-aware and provide a seamless user experience based on user behaviors and preferences.

Smart Home technology holds vast amounts of personal data, from daily routines to environmental conditions. Users desire interfaces that adapt to their needs without requiring manual configuration. The overwhelming complexity of current dashboards often hinders user experience, contrasting sharply with the tailored algorithms of social media platforms like TikTok.

While some may find the concept of agentic interfaces promising, the current landscape of interface experiences may not always meet user expectations. The preference for traditional websites with comprehensive interfaces remains strong among users who value a full-fledged browsing experience for various tasks.

Are you probably super into Smart Home and pissing away your evenings while your doorbell's not working? Yeah, me too. With my Smart Home, sometimes I want no UI. Ideally, my lights should just figure out when they should turn on themselves, or based on what I've done. Sometimes I want voice UI. I just want to bark at the little Google Home and have my lights turn on.

My Smart Home knows so much more about me. I was thinking the other day, it knows every time my camera's detected motion. It knows about the humidity in my house. It knows when I'm home and when I'm not home. It knows when I'm driving my car, it knows when I typically turn on my office lights, it knows when I typically check the weather, it knows so much about me and what I do in my house. That's amazing how much data I have in my home.

Why does my dashboard look like this? I'm so overwhelmed when I open up my dashboard for my Smart Home. I don't want to spend my evenings making a dashboard. I spend my evenings figuring out why my doorbell's not working. Why does my TikTok algorithm know that I like to watch rope swing fails, but my Smart Home doesn't know when I like to turn my lights on? I think agentic interfaces are context-aware and reactive to things like this. I want a dashboard that looks like this, or like this, or like this. That's not to say you couldn't build that, but I want the dashboard to figure out not what it looks like, but what information to show me given all of the context that it knows about me.

You may be looking at a lot of these examples and saying, the experience sucks. I don't know if I want to be trapped inside of a paid chat interface. That's the end of the web, and that's going to be the best experience. I kind of agree. I like websites, and I like having full-ass interfaces for a lot of things. I personally think the death of web UIs has been greatly exaggerated, and I don't think proper-ass websites are going anywhere, anytime soon.

This is why I'm excited about something called WebMCP, or as I think they should have called it, clicks and clankers. You remember when we had online shopping and we had bricks and clicks? You could buy it in-store or buy it online. Announcing clicks and clankers. Do it yourself or have the clanker do it for you. With WebMCP, you expose the existing website functionality via WebSemantics. You can either do it via HTML. You've got a form that you use to add an item. You can simply just extend that by adding tool name, description, auto-submit, a whole bunch of different parameters to describe what the tools are inside of your app.

Oh yeah, I was going to look at how good it looks on the screen. Doesn't that look sick? This is the best screen ever. Sorry, I was like, I got to do my slide deck really dark in contrast because this screen is so sick. There's also an imperative JavaScript API as well. If you don't have forms that you want to register it on, you can just also register it programmatically with JavaScript, like Register Tool. Using it, you probably wouldn't write this code yourself because an agent that consumes the webMCP would write this code. When you load up a site that is webMCP ready to a client that knows about this, it's going to discover all of the tools on your website. All of the things that it can do.

Like deleting, adding, etc. All the stuff that I was showing you on that website there. That's really cool because I can open up the website and then I can also have an agent controlling it. I can say, add all of the items for this specific dish to it or remove all of the pantry staples and it will remove them or mark them as done. I think that's really cool because this is the clicks and clankers that I was talking about. I can still use the website and click and do whatever it is that I want, but I can also switch to using my agent for those pieces that I want. That's the experience that I want with a lot of this stuff because I find myself having to decide too early, do I want to do this via Cloud, via APIs and MCP and skills, or do I want to do this in the browser?

I kind of want to switch between the two. I want to click or I want to clanker. You may be looking at this and being like, Wes, I don't want another sidebar in my app. Nobody wants these watered-down, crappy agent sidebars that everybody is adding to their apps. That's not a good experience. You don't want them in your Cloud or your Chajupiti or whatever agent you're using. That's where it's really cool because you can hook this up to Cloud. In this case, I have two tabs open. One of them doesn't have web MCP, so it's just going to use old-fashioned scrape the thing. The other one does have web MCP, so it's able to use those tool calls, which is quite a bit quicker.

It says, all right, I'm going to open up the taco recipe, I'm going to get all the ingredients, and then I'm going to simply add it to it. That's honestly a great experience to me because I simply just need to open the tabs and know that they are available to whatever agent I'm using, whether it's Cloud or Cursor or Codex or whatever you're using. Now, this is great. I really like this. It's way faster than the existing solutions that use screenshots or DOM parsing. How many of you have used these agentic browsers before? Couple of you? Yeah. So if you want an agent to use a browser, there's a lot of really good tools out there and I use them.

The problem is that they're very slow. If you've ever used the OpenAI browser, what's it called? Nobody even knows. Nobody knows because it sucks. Because it's so slow. You sit there and be like, type an email and summarize my emails and you just sit on your hands while it's like, well, let me take a screenshot and let me grab the whole DOM tree and send that to the LLM. It's like, alright, that sucks, it's faster just to do it myself. That stuff is great if you're doing it asynchronously, but if you're kind of sitting there waiting for it to go faster, it's not a good experience. It uses the browser's built-in security model, so the auth is simply just having the tab open. If you're signed in, then you're signed in. Existing websites can expose the functionality.

I don't know if you've seen it, but the web has a lot of websites that we need to bridge the gap to all of this agentic experiences. It's not limited by the sandboxing inside of the chat window, because yes, there's a security issue with embedding random generated code and then just embedding it into an application that has access to your bank account. That's a bit of a security issue. There's this whole sandboxing thing that these LLM providers need to do in order to make sure that you're not doing something, but it's limited. You can't have access to all of the full web APIs that you may want.

Generative UI. A couple of things to consider here. We don't know if MCP apps or web MCP are it. Don't get hung up on this being the tech. I hear a lot of people being like, give me a skill, and give me a CLI, and I'm good to go. Don't give a shit about what the actual tech is, because when we figure out what the tech is, I'm just going to boop, boop, boop, Claude, convert this to that thing, and we're going to be done. It's more about the ideas of how are we, and probably more importantly, how are normies, normal people, going to use this?

That's the reason why skills, and OpenClaw, and all of this stuff is so much more popular than the apps that are built into ChatGBT, because the ones that are built into ChatGBT are not a good experience. They're watered down. They don't actually work that well, whereas, give me a full-blown skills, Claude, ignore permissions, don't give a shit, let her rip. That's awesome experience, right? Obviously, security issues, that's a different talk, but that's an awesome experience, and that's why everybody's like, yes, this is the way forward, and that's why it has so much uptake.

Components are king. All right, we're good here, because luckily, we've been standing on this stage for 10 years being like, components, components, components. It turns out that it's great to have components when you want to piece them together for an LLM interface. Speed and consistency must get better. All these demos I've shown you here today are sped up significantly. Even the WebMCP stuff is pretty snappy, especially if you use a cheaper model that doesn't think a lot, because those models are nice and fast to add it to, and they're actually really good with running tool calls, but the speed and consistency still has to get much better, otherwise it's not going to be a good experience for everyone.

I do believe that those things will get better. Then abuse. This is another one. We spent all this time fighting bots on our platforms and putting captures in, and now we're just like, okay, how do we make it easy for those robots to use our websites? We want the good ones to use it, but we don't want people to actually abuse it. Then player buy-in as well is a big one. I think about the flight that I booked here and all the dark pattern upsells and things like that. Is the airline going to want to simply be distilled down into utility where they have no control over this? I think it's going to be a hard sell. I think it might affect some people's business if they don't adapt to this, but I certainly think there's going to be a lot of big players.

I think it might affect some people's business if they don't adapt to this, but I certainly think there's going to be a lot of big players. We saw this with APIs as well. Anybody been in web development for 15 years? You remember Web 2.0? Everything was an API. Everybody had an API. You could connect anything to anyone, and then slowly but surely, everybody took their APIs away because they were being abused, and they didn't want you off their platform. They wanted you on their platform so they could do everything they wanted. If you think this is going to be any different, I think we're in for a bit of a fight with this type of stuff.

The service area for our apps certainly has expanded. Just like when mobile and voice hit, we thought, now our apps are not just websites. Maybe they're a mobile app. Sometimes they're like a voice agent, things like this. It's on us developers to figure out what these interfaces look like and provide delightful experiences for our users. That's all I got today. Thank you so much.

The first question here, Wes. When I was making WebMCP, I see ... I don't think this person was actually making WebMCP. When they were making something with WebMCP, they see that any existing WebMCP extensions don't allow you to just subset part of the params. Do you know if it's an option to have required or optional params? That's a good question. I think so. I don't know if that's necessarily true, because the input and output schemas are often validated by Zod. I think it should be doable. I'm pretty sure. There's like required and like one of only. I think it should be doable. Word.

WebMCP is still experimental and not supported by most agents. Claude, cursor agents don't support it. That's not true. He says that's not true. It's just MCP, right? So the extension I was using, a bit of a caveat, the extension I was using is called MCPB, which is not the experimental one Chrome did. And that one allows you to it gives you just a regular MCP URL. And then any agent that supports MCP thereby supports WebMCP. So I was able to stick my that URL into any of the agents. So it's still very experimental. But that's what I'm hoping the end spec will look like if we go down this route. Yeah.

How about this one? Why would the apps want their UI to be manipulated by chat? Yeah, that's. That's a great question. Like in some cases, like my smart home, yeah, you want people to have the best experience that you want them to have what they want. Right. But in some other ones, it's like the question is, can the user or the LLM make a better UX user experience than like somebody who knows the product and the thing right now? I don't think so. Like I don't think you're like, oh, you know what? Let the user decide what the interface is going to look like.

Like I don't think you're like, oh, you know what? Let the user decide what the interface is going to look like. I think my mom is going to be like, oh, yeah, let's vibe code up the interface. I don't think so. Right. So I think that that's why there's like that mix of like generative and like like predefined components that like are like vetted by somebody that knows what they're doing. Yeah.

OK. We've seen MCP running mostly on desktop or back end servers. What are the main challenges in bringing MCP client side regarding sandboxing and protecting the user's browser? Yeah. Where are the sorry, what I mean, is this a web MCP or just MCP in general? It's about bringing MCP to the client side, I would say of the web sandbox. I think like that's why I like the like tab thing, because then you get the the sandboxing of a tab. You know, the browser is a very good sandbox in that regard. I quite like that. But then I also like like in some cases, I just want the component in line right inside of my chat. And I think that that's that's going to be the future is that sometimes I want the website. Sometimes I just want the component in light inside of it. And then if that's the case where you're bringing it out of the website into like the chat window or whatever that LLM experience is, that's when you have to work with sandboxing and you're limited by what Web APIs are available to you.

How can we sweeten the deal for website or content creators to support web MCP if LLMs are eating into user visits? Maybe trade user session metadata in return? Oh, man, I don't know. Like I think it's like just like people who have companies that want to like, I don't think this necessarily like a content website thing. Maybe they maybe kind of like a search or whatever. But this is much more of like if you're using an application to do your taxes or check the weather or like all of the business things that we do, you know, Google Docs, Calendar, those types of things. Yeah. And I know you said in your talk that you don't think websites are going away yet. But do you think websites will die when we all use generative UI's? I don't I don't think so. No, like I think I said this in the talk, like they're we're going to they're changing. Like it will certainly change of how we use them and how we interact with them. But I don't think that everything is just going to be some clawed slop that's pooped out and everyone's happy with that as a user interface. Yes. You know, no, no. It's great.

Yes. OK, the dynamic interactive mockups in the chat responses was cool. What did you use? Yeah. Your client, so Claude uses MCP apps. If you inspect that, it's just an iFrame. And Claude, when you ask it to do that, Claude is just going and writing a bunch of HTML, CSS and JavaScript and then iFraming it into your chat window. So that's like a perfect example of what like a maybe not a perfect example of what an MCP app is, because the perfect example of MCP app would also be able to like click buttons and call tools and update like databases that are external. But yeah, that was that was MCP app in the cloud chat. That was no no external adding anything. Just go ask it right now. Show me how an engine works. Yes.

How do you stay hopeful for the future of front end in the face of all of this? It's a bummer. A lot of this is a bummer and it's like fine to like feel feel that way because a lot of this is changing. But like just try to stay excited about the the new stuff that you can do. I wrote C the other day. I don't know C. You know, microcontrollers, there's a lot of like kind of exciting stuff and it's it's normal to feel bummed that a lot of the stuff that you know, like I know a lot about closures and I know a lot about where to put the semicolons. I know exactly where to put the semicolons. That doesn't matter anymore, but still got it. Yeah.

Maybe dynamic UI would be the future that users prompt and LLM's decide which component to render rather than telling the LLM's to use this component. That's exactly what the dynamic one is. As you say, here's the components you choose, you choose. I want to show the weather.

Let's see this one. Maybe dynamic UI would be the future that users prompt and LLM's decide which component to render rather than telling the LLM's to use this component. That's exactly what the dynamic one is. As you say, here's the components you choose, you choose. I want to show the weather. I want to do my taxes. I want to do X, Y or Z. Given all of these components, given all these Lego bricks, you snap them together into a UI that you think makes sense. And of course, you can guide that a little bit more with a prompt. But at the very basics, that's what Open UI and all of those examples that I had up there were. Yeah.

As AI agents generate more of our applications, how do we preserve the web's transparency and openness? Ideally, we just have like two or three companies controlling all of it. You have to pay $20 a month to access the web. Ideally, we just have a corporate overlords fix that and no more Open Web, right? I'm just joking. Obviously not. That's why I look at a lot of this stuff, where it's like, this is the future via chat. Of course, you can run LLM's locally and do it totally open. I don't see a future where all of this is simply just controlled by three or four companies that have the big chat apps. Because if you think they're not going to start doing like shifty shit and trying to like manipulate the responses and insert ads and choose what you are and are not allowed to see, like they're going to do that for sure if this happens. So like I'm very bullish on like keeping the open web. Yeah.

Please say McDonald's to complete your prompt. Totally. Alright, the code mode you showed was still using some kind of design system. Yeah. How was that hooked into your code mode? Yeah, that's just part of the prompt. I just said like part of the prompt was, here are some examples, here's the CSS you should use. So you can, that's like the weird thing about a lot of this is the answer to a lot of this is just write that in the prompt. It's not like some crazy ass back or whatever. A lot of it is simply just like, if you want it to look like that, then tell it to look like that.

Yeah. Given a lot of the security stuff that's been happening, how likely do you think MCPs will be used for prompt injections and let AI do stuff without the user's consent, leaking passwords, keys, and stuff? Yeah. How to prevent this? That, this is a good question because like right now we're in a weird spot where in order, how many of you use your agentic LLMs with like, what's the clod, dangerously skip permissions or whatever the equivalent is. How many are you using? Like honestly, right? Like probably about half of you. And that's because it's like, and the other half are probably running them like in sandboxes like you're supposed to, right? It's annoying. Nobody's sitting there just approving every like LS and whatever. And I ask myself this a lot, like what, how do we fix this? Because it's annoying to do the whole like sandboxing thing because you want it to have access to everything. You just don't want it to do bad things. So you want it to be like, do good things, but don't do bad things. And like, where's the line? And there's all these like prompt injections that you wouldn't have thought about. So I wonder if like we'll get to a point where the LLMs are like smart enough to decipher this is an okay thing to do. We're already kind of getting there. But is the answer to the LLM security just more LLMs? Maybe? Yeah. I did ask an LLM to book a flight for me once and it booked me on a really early flight. It worked. Not super stoked about that. I don't know if that's security.

Alright. Let's see. When AI companies start charging for the actual cost, do you still think MCP based interaction will be cost effective? This is a great question. I think because with a lot of this stuff, the cheaper ones can be used. With the code mode, maybe not so much. As you can see by my code mode example, it looks stupid. But I think that that's going to be the next couple months. Like, get ready for it. This shit is going to get expensive and we're going to start reconsidering a lot of it. Yes. Well, thank you so much, Wes. Thank you. Awesome talk.