How We Used AI to Build TanStack AI

Hello, everyone. My name is Zalim Tuzlak. And today I'm going to be explaining how we built TenStack AI using AI. So before I go into how we did that, I just wanted to show you what it is and explain briefly how it works. And TenStack AI is your AI SDK that allows you to talk with LLM providers, and it easily integrates AI functionalities into our app. So if you want to add a chatbot, if you want to generate images, generate something like a video, transcription, summarization, speech, songs, whatever, you can use TenStack AI to achieve this.

And it's pretty easy and above all, type safe. So the interesting thing about TenStack AI is our main hook, and also its server counterpart called the chat hook. So the chat hook is really a true powerhouse, and it has a lot of features that come integrated into it. And the main point of this hook is to connect to any LLM provider. So for example, if you want to connect to OpenAI, you're going to use OpenAI text, you're going to put it in the chat method, it's going to talk to OpenAI. If you want to use the Entropic adapter, you can just swap it out. It's instead of OpenAI text, it's Entropic text, and you've got it working.

Messages are sent from the client, ingested in the chat method, and everything happens for you magically. And we have a bunch of features. So first of all, we support a lot of model options and all of these are type safe based on the model you use. So for example, if you want to use GPT 5.5, GPT 5.5 has its own functionalities like reasoning, tool calling, and stuff like that. And if a model doesn't support some of the functionalities, you can send it in the model options. So that part is fully type safe.

Another cool feature that we have is middleware. Middleware is one of the coolest things in the chat method as it allows you to hook into any point of the life cycle of the chat. It enables you to interact with streaming chunks, user statistics, transform streaming chunks, change and cache tool calls, and more. Implementing middleware offers capabilities like logging, rate limiting, persistence, and many other features. Additionally, if a specific feature is not supported by TenStack AI at the moment, middleware provides a flexible solution for custom implementations.

The next notable feature is the MCP servers integrated into the chat method. By providing connected MCP servers to the chat method, automatic connection and disconnection functionalities are facilitated. Under the hood, MCP servers offer functionalities like lazy tool calling. A significant aspect related to tools is the introduction of tool code mode. Unlike traditional tool usage, code mode allows the LLM to write typescript scripts efficiently calling available tools, leading to optimized interactions and self-healing capabilities.

It has a lot of functionalities under the hood like lazy tool calling. And when it comes to tools, one feature that's really cool is tool code mode. And what code mode is, if you think about LLM models, it's very inefficient for us to give it a huge list of tools and have it call a tool one by one. So for example, it has to fetch all the products, then it goes back to the LLM. The LLM is like, okay, I have a list of 30 products. I want to recommend this single one. I'm going to call this other tool. It's going to send back what it wants to do. It's going to call that tool. It's going to send the results back to the LLM. And then it's back and forth between the LLM and your server.

Well, code mode is a really interesting concept where you actually don't use tools in a traditional sense, but you actually allow the LLM to write typescript syntax. And what happens is it writes a script that's going to call all your available tools. So, for example, if you have get all products, and get product by ID, and recommend product functions inside of your app, it's going to use code mode. It's going to write the script that calls the products, calls a specific one, and then sends that into the recommend. And that gets a result back. And it's going to call that script and efficiently get the results back. And the cool thing is it's also self-healing.

So, for example, if you send back something to the LLM, like the script failed because of reason X and Y and Z, the LLM is going to be, okay, I messed up by adding a semicolon or something like that. It's going to fix the script and send it back again. And that's really powerful if you're using AI models because it kind of gets around their non-deterministic nature. And then you just plug it into your client side. So, we support five frameworks, and the hooks themselves are pretty similar across all frameworks. You just have the use chat hook, and you just connect to it, to your server. And that's it. It works.

But today my talk is not really about how cool TenStack AI is, but it's about taking you behind the scenes and telling you how we built it. And it all started last year when Tanner and Jack Harrington were talking about building a library for AI at ReactConf. And incidentally, I, at the same time, wanted to build an AI product in TenStack, and I asked Tanner, hey, should we even build this? And the reason I asked him that is because we are a very small team, and at that time only Jack and I were supposed to build this with the help of Tanner, but he wasn't as involved in the project. So it was like two or three people.

And then you have every major provider like OpenAI, Entropic, Grok, Grok with the DQ, not the K. And there's a lot of providers out there. And then you've got to support all of the frameworks, like Svelte, React, Solid. There's a lot of features that come out every day. And AI is a new and constantly evolving technology. So it's very hard to keep up, and we weren't sure how we are going to build at the scale of all of these technologies.

Last year, around November, I think, I call it the Opus 4.5 moment because back then in open source, whenever I tried to use models to write open source code, they were horrible at it. They wrote subpar code, and it was pretty much unusable. And you could get away with some smaller apps and stuff like that. And this was even before skills became a thing. So it was really hard to write good code with AI. And then Opus 4.5 came out, and it changed everything. That was the first model that I tried after trying 20 different models before it that actually produced good enough code where I can get it to write 90 to 95% of the code.

And then I just need to refine it to get it to a point where it fits my quality standards. It was always funny to me because I hated using AI for development, and I thought it was never going to be a thing. And then I tried Opus, and I loved it. And I kept telling my open source friends, like the maintainer of Nux is a close friend of mine, and I kept telling him, you have to try coding with Opus 4.5. And he was like, yeah, it doesn't work in open source. The models are just not good enough. And I was like, just try it. And he did, and he loved it as well.

So how did we actually build this? Well, first, what we wanted to do is figure out the requirements. Luckily, because we weren't the first people doing this, we already had examples in the industry, like the AI SDK from Marcel, Mastra, Langraph, Langchain, there were a lot of companies that were already doing this. So we had a lot of references to fall back to and figure out what the final APIs are supposed to be. And our main goal, because we're 10 stack, is type safe end to end. So we wanted the model options to be type safe per model. And how we did that, we manually mapped every supporting feature per model to the types. So I spent like seven days just reading the docs and mapping the features one by one.

The second requirement is that it was three-shakeable by activity. And the reason why we needed this is because we were a very small team. And the issue we faced is if you use a huge adapter that does like chat, image, video, transcription, if you wanted to add, for example, we supported only OpenAI and Tropic and Olama back then. If you wanted to add something like ROK, we would have to implement all of those functionalities that are supported. Otherwise the class wouldn't match. So what we decided to do is split it by activity. First of all, what you get by this is three-shaking. And second of all, what you get is it's really easy to write an activity. For example, we want to add images. We just add images in without supporting video and stuff like that. And it made it a lot easier for us to maintain in the long run. And then you can swap providers at runtime. That's another cool thing you can do. It's all type safe. And the next final bit was isomorphic tools where you can define a tool definition and then after a while you can just use this.

It's very similar to server functions from start where you can define a server implementation and the client implementation of a tool and then send it to the provider and everything else is handled by the library itself. So we built a huge alpha and it had a lot of features. And for example, if you think about good architecture, it usually comes from a lot of mistakes and a lot of learnings through the years with experience and stuff like that. And because code became a little bit cheaper to actually write, we decided to build a huge alpha, get things wrong, get things right and then figure out what the right things were. And that's what we did.

Because the alpha we built was too big, we built like all the functionalities like video image and stuff like that. We realized where the mistakes were and we wanted to basically cut all of the features down and release a small alpha so people can play with it. And we can also get the right APIs from the beginning. And that's what we did. So we basically built a huge alpha and then constrained it down to a small chat method and we released that as the alpha. And we learned a lot of lessons through doing that which we applied later on as we went through the alpha phases.

This here is the first behind the scenes prototype of 10-Stack AI. And the video here on the left is what we had early on. So we had this AI class that would basically allow you to call anything. It was AI.chat, AI.image and stuff like that. It was a really cool concept and it was all type safe. And we also had this concept of fallbacks where if one LLM responds and dies, you can fall back to another one. And it kind of died because of tree shaking. And if we include all of this by default, you would be using the whole library. And for example, if you need, I don't know, just the chat method, you're pulling in the image, the video, the audio and all of this stuff you don't need into your bundle.

And for example, if you need, I don't know, just the chat method, you're pulling in the image, the video, the audio and all of this stuff you don't need into your bundle. And what we did next is we had basically live chats every so often. And we basically talked through ideas. So, for example, this is an example from the other day where we were adding multimodality support to image generation. So you can like send reference images and get it to work. And we basically just debated on the API. And because the 10-Stack group has a lot of smart people, they all chime in. Give their opinions. And then we can just refine the API. And for example, this is the idea for the final API, which went through like three or four revisions, which kind of helps us build a great API because if you have a great team behind it, that powers a lot of the cool stuff.

Finally, the came skills. So, for example, after a while, you realize that the output from LLMs is good, but it's not good enough. So what we basically did was write a bunch of skills to help us get the consistency and the quality that we are after. So, for example, if we had tests, what we did was, okay, we don't want the tests to use casts as any because that's just like kills the type safety in the test and you have no idea if it's correct or not. And then, for example, don't overabstract tests, don't skip tests and stuff like that. And then you just write this library of skills that improve the codebase over time. And for example, what we did was we wrote a skill for dogs and whenever we add a new feature, this skill is a quality gate that stops us from shipping it until we actually write really good dogs that cover the new features.

Another problem that we ran into while shipping features and making sure that it all works is that we had no budget for marketing. We are an open-source community and we rely on our sponsors and we don't have a dedicated designer team who can create cool videos for us to promote our new features. So what I did was I used a library called HyperFrames. So it's a remotion alternative. And I created a skill that generates these videos for us. So it helps us promote our new features that we're building and not make it just a footnote on X or whatever. So nobody is like aware of all the cool features that you're shipping. And this in of itself helps us ship these features rather than spending time on like creating these videos because it shortens the amount of time you have to spend on marketing, which we like to spend more time writing code. And finally, the last thing that is kind of cool here is that new problems require new tools. And for example, our docs, when we used LLMs, they would often hallucinate APIs and it was very frustrating. So what I did was I created a small library that basically type checks your documentation, code snippets, and markdown. And it basically checks if, for example, if you've got an import or if your documentation, you change some API, but you forgot to update the documentation, this is going to catch it in CI and fix it.

And it basically checks if, for example, if you've got an import or if your documentation, you change some API, but you forgot to update the documentation, this is going to catch it in CI and fix it. So small teams get big leverage by using AI because it allows us to have guardrails on code quality. It helps us shape what we want to ship, and it allows us to iterate cheaply on problems. Before, if we wanted to, for example, explore a new API, it would probably take us like a month to first implement the POC. And then everybody has to test it and then give feedback. And then we realized, OK, this API doesn't work, we have to throw it in the trash. But we already spent a month on it. So this AI allows us to iterate quickly on an idea. We get a POC out of the day. We try it out. We like it. We don't like it. We throw it in the trash if it just doesn't work. And we we just move on.

And then finally, tools beget new tools. So whenever we run into an issue or a problem, we just create a tool for it that helps us ship this with more confidence. And what the final point of my presentation is, is that you should build and dream bigger than before, because AI allows us to quickly iterate on code, quickly get POCs out and test new ideas. And you don't have to get locked into a single API just because it took like a month to write it. You can just write it in a day, see if you like it and then throw it in the trash because it's become easier than ever to try new things out and figure new APIs out.

And one more thing before I end my speech, and that is I'm not sure if you follow the Ten Stack Twitter, but we announced something that's going to happen on June 11th, and that is that Ten Stack AI is going into the beta release. We have finalized and stabilized the API and it's ready for public use. And thank you for the presentation. And if you are interested, you can follow me on Twitter. And I'm also doing a workshop on Ten Stack AI. That's the QR on the right. And thank you very much. We saw React, Svelte, Vue. Where was Angular in all of this? Yeah, so there's no reason not to support Angular. We just haven't done it yet. I personally am not an Angular guy, so I probably have to find a maintainer that's going to do it for us because I don't feel comfortable just having AI write the Angular adapter and then hoping it works.

So it's either going to be a community contribution, if anybody's interested in that, or we're going to find somebody in our Ten Stack maintainer group who can actually do that. Seems like a good job for whoever asked that question, right? Yeah, exactly. I should mark it as good first issue. Yeah. All right. Oh, my gosh, I keep accidentally dismissing these. Let's see.

Did you run AI in auto mode or did you have to approve every single action it executed? Yeah, so that's a very interesting question. Before, I didn't trust it much. So basically I just ran it in the normal mode and then I permitted everything that it needed to do. Later on, as the models and the guardrails got better, I actually just switched it to use to run with auto permissions. And because I work at a company called CopilotKit, and our CTO really loves writing skills for AIs to enhance the guardrails. We have a whole skill system that just makes the output very, very good. And it gets you 95% there with the guardrails of it not deleting your whole hard drive. All right. Here's the one I accidentally dismissed.

Did you use Opus 4.5 all along or changed or other harnesses? Yeah, so I've been switching between Entropic and Codex a lot, OpenAI. I basically started with Opus 4.5 back when it came out. Then I went to 4.6, 4.7. Then I went back to GPT 5, I think it was 2 and then Codex and the GPT 5.5. So I keep switching back and forth. From my personal experience, Entropic is better at creative work, like creating those videos I just showed you. Where Codex is better at coding tests in general. I'm personally curious. Is it all vibes for you or what are you using personally to evaluate? Because I know me, I'm always switching and trying new things, but it's often just vibes. Do you have a framework for understanding what you like the best? For me, it's mostly like I run the agent on usually the same stuff, and then I compare the outputs and see what works better. And it also comes down a lot to time. So, for example, if it takes seven hours for Opus to get there and gets it done in two hours, that's a lot more valuable for me because I can then start testing it and iterating on it five hours earlier. So I can actually like test it out.

So I can actually like test it out. And, for example, for creative work, it's a lot easier because you visually see the output and you're like, Oh my God, this looks horrible. And you're like, this model is clearly better at creative work. And that's how I figure out the Opus stuff, because the videos I generated with GPT are like horrible. All right. Next one here.

I'm using the AI SDK from Vercell. Why should I switch to TanStack AI? Yeah, so we offer quite a few features that are lacking in Vercell. One of them is LazyTools, which is a really cool concept where you basically can mark a tool as lazy and not send it to an LLM. And basically, it allows you to save a bunch of tokens. And another reason is that we have code mode, which I explained earlier how it works. Not only does it save tokens, but I think, according to Benchmark, it completes 90% faster than the regular tool calling loop. So that's another really great reason. OK, let me check this one off, too. OK. Oh, and I just saw the question.

So the AI gateway is not... So AI SDK is the open source alternative to TanStack AI. But AI gateway is just a vendor for models, so it's not different in that term. The AI gateway is something like open router alternative, where AI SDK itself is the alternative to TanStack AI. Awesome. Thank you. OK, next one here.

I noticed that TanStack has been shipping skills for pretty much all of your packages. I've noticed your skills files sometimes contain more information than your regular documentation. Are there any harnesses to ensure the quality of your docs? I can't speak for all the libraries because I don't maintain all of them. I maintain dev tools and AI. For what I can see in AI and dev tools, there are a lot of guardrails to make sure that we write the documentation that we need. I have a skill called doc writing and I enhance like a call at every time when I'm done with a feature and basically writes docs for every user persona. So for beginners, for advanced users, for somebody in the middle.

And that's how we make sure that the quality of the docs are on par on TanStack AI. So that's what we do at TanStack AI. I can't say for the other libraries. All right. Let's do this one.

What do you find is the new bottleneck for your development with all of the new gains from AI development? Yeah, this one is pretty easy to answer. The newest bottleneck is me. Because, for example, I can sometimes spin up like eight terminals and then it asks me a lot of questions from all sides. And then I have to like think about every answer because I don't like just click recommend the recommended.

I have to think about every answer. And then I have to manually test it when it's done and make sure it works and then check all the code. And I think that's the slowest part, because I sometimes create like three, four features that are supposed to go to TanStack AI. And then I create it in like four or five hours. And then it takes me like four or five days to manually verify it, make sure it's a good quality, the API is right and stuff like that. And to actually refine it further with AI. So the biggest bottleneck is me, myself.

Yeah, how do you feel about your stress levels with all of this? I know for me, it can feel the decision fatigue. It can feel a little bit more stressful all the time. Yeah, I've been talking with my CTO about this, and it's like entering a whole new era of stress. And I think we're not aware of it yet, like what it's going to bring in the long term side effects. But I also think a lot about that because, for example, before when I used to like manually write code, I would write from 9 to 5. And then after I'm done, I would like go to the grocery store, think about a problem.

How can I solve it? Like most of the things I created came from my showers. And now I'm just like after 9 to 5, I'm just too exhausted to think. My brain is just fried and I just can't think and I have no idea what's going to do with the long run. I've been hearing that more and more.

What was the biggest architectural mistake you repeatedly see developers make when building AI powered React applications? That's an interesting question. I think, well, it's not really an architectural mistake, but rather like a UX mistake. But it's not the developers themselves that make it, but more of a manager problem where like AI tries to get shoved into like products that really don't need AI. And then developers have to shove it into their projects. But now that I think about it more, another mistake is that people think that in order to add AI into your projects, you need this whole new approach and stuff like that.

How about this? This is even a little bit related. What parts of TanzDec AI were the hardest to build with AI assistance? I think middleware was the hardest one because middleware is like scary powerful because it can hook into anything inside of the agent loop, which is really cool. So the API itself needed to be perfect. And it usually takes me a few days to figure it out. But for middleware itself, it took me like three or four weeks to get it perfect.

So the API itself needed to be perfect. And it usually takes me a few days to figure it out. But for middleware itself, it took me like three or four weeks to get it perfect.

Here's one I'm personally interested in. What do you use to document important decisions in your team, such as tree shaking on the activity level? We don't really document those. They live in my head, as the usual experience. The good thing is I can't leave the company as the usual way. So everybody just loses the experience. So that's the good part. But yeah, I think so. I've been wanting to. The React Router team, I used to be a big React Router guy, a remix guy. And one thing I loved that they did was they wrote decisions down on, for example, why they went with A instead of B. And I think I want to start doing that because it's like a great context for LLMs as well to make decisions. But I just haven't done it yet. Yeah, I think some people do like an ADR system for that too, right? Where you're documenting, feed that back in.

Yeah, in my personal experience, I found it to be pretty effective. Yeah. But maybe you are just, you know, obsolescence there. Yeah, I have a skill for self-improving workflows where it's like whenever I tell it, hey, you made a mistake, it like auto-documents that in the repository so it doesn't make it in the future. And that's been really cool. And that's basically what ARs are. So I think that's the right way in the long run to do these kinds of things. Yeah. And along with the skills, what kind of process does your team have for creating skills in projects? Are there approval steps, refinement sessions, quick feedback? What types of things are you using? Yeah. So mostly for, it's funny because I've kind of become the skills guy in the 10-Sec Maintainers Channel because I keep writing skills for everything. And basically the usual flow is, let's say I want to write the TypeScript skill that's going to solve stuff like casts and things like that. I basically dump all of my TypeScript experience and knowledge into a prompt and then I tell it to write a skill. And then I start testing it on running it on some bad TypeScript code and I see the output and then I'm like, OK, these parts are good, but I forgot to refine these parts. And then I just keep iterating on it until it's good enough to use and one-shot whatever I'm after.

And that's the usual flow I do. But we don't really have an internal process of improving skills because we're not a company. So we just like, if somebody thinks that this will improve the skill, we just merge it.

Here's an interesting one. What makes Tanstack AI, oh no, sorry. What is the end goal for Tanstack AI and what is its best selling point? Yeah, so the end goal of Tanstack AI is to basically be something similar to Mastra. But instead of like offering, so because we don't sell products, we are not going to be opinionated on stuff like persistence, where you host stuff and things like that. The idea is we're fully headless. You, for example, bring your own database, you bring your own hosting provider and stuff like that. But when it comes to features, we are on par with something like an agent framework similar to Mastra, Lankchain and stuff like that.

Awesome. Well, thank you so much, Olam. This has been awesome. And I thought your talk was incredible. Thank you. And thanks everybody for coming.