Video Summary and Transcription
Today's Talk explores architectural shifts in transactions, highlighting the dominance of JavaScript in the modern transactional stack and the elimination of the frontend-backend divide. The use of serverless and component-driven toolings allows for scalable infrastructure provisioning and removes the need for DevOps teams. Additionally, workflow engines play a crucial role in orchestrating async functions in the serverless native world.
1. Architectural Shifts in Transactions
Today, I will share interesting architectural shifts in the context of transactions. JavaScript is winning in the old world architecture, where microservices are divided between the client and server. In the modern transactional stack, serverless and JavaScript alone power the whole application. There are no longer frontend and backend, and platforms like Superbase, Convex, and Upsash eliminate the need to manage backend infrastructure.
Hi, everyone. My name is Yoko. I'm a tech cartoonist, developer, product manager, and lastly partner at H16z investing in the JavaScript ecosystem and developer tools. So today I would like to share a few interesting architectural shifts I have seen after talking to hundreds of developers and tech companies. I'm putting the architectures in the context of transactions since they're so core to our daily lives. They power things from plane tickets building to getting food deliveries to our doors to booking cat sitters and dog sitters. And we're seeing a major shift in how developers use and reason about technologies here.
Well, spoiler alert, JavaScript always wins. So let's talk about why and how. Now, here is the architecture you will see in the old world. As you can see here, there are a bunch of microservices divided between client and the server because they're retaining different languages, also because front end and back end developers can't possibly agree with each other. So when you see the request path, when a request comes in, for example, placing an order. It gets authenticated and then sent to an API. The API endpoint will need to orchestrate hundreds of microservices to deliver a business transaction to the user. And then it will go through things like inventory management service, order processing service, payment processing service. This is when developers use queues and caches as you can see on the lower half of the picture to store what we call application states. Things like order received, order paid. The developers will also hand code the retry logic to get the data to the databases and cross their fingers and hope for the best. We found that as the system scales so does the complexity of managing queues and caches and corner cases. And now we actually see many companies start to build their own new projects very differently.
Now enter the modern transactional stack where you no longer need to provision and manage hundreds and thousands of microservices. Instead, we have got to a point that serverless and JavaScript alone is good enough to power the whole application. So you may be wondering what is the modern transactional stack? First, it's entirely JavaScript and backend is taken care of by a vendor. Sometimes the vendor is serverless. There's no longer what we call frontend and backend. Everyone is a full stack engineer and there are no longer APIs since it's written in the same programming language. And then, now you have things like server components and TRPC just calling a function. We are seeing more and more companies embracing this pattern as they go. What we traditionally call the backend, now we have platforms like Superbase, Convex, Upsash and others that can entirely eliminate the need for the app developers to directly manage backend infrastructure. Trust me, you don't want to work on AWS.
2. Modern Transactional Stack and Workflow Engines
I used to provision infrastructure all the time and it's not fun. In this architecture, you will also utilize services that can scale down to 0 and scale up based on demand quickly. Removing the need for DevOps teams altogether. Another trait of the modern transactional stack is that it's powered by component-driven toolings. For authentication, you no longer need to deal with lower-level tools backend engineers traditionally use. Now you can just pick up tools like Clerc and use a signing React component. For object storage, you can use something called UploadThing natively in React instead of using AWS S3 directly. Lastly, as businesses use more APIs and accept more webhooks, they have to deal with async functions at scale. Workflow engines play a central role in the serverless native world, orchestrating async functions. If you want to chat more, feel free to send me a DM on Twitter at staff Yoko Draws. Thank you.
I used to provision infrastructure all the time and it's not fun. In this architecture, you will also utilize services that can scale down to 0 and scale up based on demand quickly. Removing the need for DevOps teams altogether. Upsash is a great example of this.
Second, another trait of the modern transactional stack is that it's powered by component-driven toolings. For authentication, you no longer need to deal with lower-level tools backend engineers traditionally use. Now you can just pick up tools like Clerc and use a signing React component. Another good example here is an open-source project called React Email. You no longer have to code vanilla HTML for transactional emails and now you can just use React and Tailwind directly in your email. For object storage, this is always something that a lot of engineers tell me and even myself, how to learn AWS for. But now you can use something called UploadThing natively in React instead of using AWS S3 directly.
Lastly, one highlight of this stack is that as businesses use more APIs, more functions and accept more webhooks, they essentially have to deal with async functions at scale. So when this happens, there needs to be a central place to orchestrate these things since handcrafted retries are hard. Now we start to see workflow engines playing a central role in the serverless native world. And the way to understand them is that they are the orchestrator of async functions. Here's the intuition. If your app handles hundreds of user signups and then sends a bunch of welcome emails to the users, you can use tools like ingest to write a few lines of TypeScript code to define what functions should be triggered when a signup event happens. It will also retry this part of the logic automatically with exponential backoff if that process fails. And on the client side, if you want, you can use the authoring layer of the state machine called xState. I hook it up with the workflow engine. It's a super cool project, makes things much easier to reason about compared to Redux which I used to use all the time.
There is probably hours and days of things to talk about in a stack we see over and over again here in the serverless era. If you want to chat more, feel free to send me a DM on Twitter at staff Yoko Draws. And that's all I have today. Thank you.
Comments