The Rise of Modern Transactional Stack

Databases and transactional backends are the heartbeats of our everyday lives - they power nearly every transaction in the modern world, from booking airplane tickets to ordering food on Doordash. The status quo of powering these transactions today is building on top of a microservices-heavy architecture -- by leveraging caches and queues to preserve transaction state and OLTP that guarantees idempotency and consistency. The next evolution of this stack is replacing cache/queues with a central orchestrator -- workflow engines like cadence and conductor. This all happened in what we traditionally call "the backend", but a new stack is emerging for the Javascript world: more and more greenfield projects are written only in Javascript/Typescript, and this group of projects share a unique stack unlike what we have seen previously. The presentation will dive into the history and evolution of platform shifts, and focus on the most exciting transactional problems presented in the modern monoliths world that's edge-native, 100% Javascript -- and argue why the word "serverless" is out of date :)

This talk has been presented at React Summit 2023, check out the latest edition of this React Conference.

Watch video on a separate page

FAQ

Yoko is a tech cartoonist, developer, product manager, and partner at H16z, investing in the JavaScript ecosystem and developer tools.

Yoko's talk focuses on the architectural shifts in technology, particularly in how developers use and reason about technologies in the context of transactions.

The traditional architecture involves a bunch of microservices divided between client and server, using different languages. The request path involves multiple steps like authentication, API orchestration, inventory management, order processing, and payment processing, with developers managing queues and caches for application states.

The modern transactional stack is entirely based on JavaScript with backend managed by a vendor, often serverless. There are no distinct frontend and backend roles, and tools like Superbase, Convex, and Upsash eliminate the need for direct backend infrastructure management.

Examples include Clerc for authentication, React Email for transactional emails, and UploadThing for object storage, all of which work natively with React.

Workflow engines orchestrate async functions, handling tasks like user signups and welcome emails with automatic retries and exponential backoff. Tools like Ingest and xState are used for this purpose.

The modern transactional stack simplifies development by eliminating the need for microservices, direct backend management, and DevOps teams. It leverages serverless architecture, component-driven tools, and workflow engines for efficient scaling and orchestration.

Upsash is an example of a service that can scale down to 0 and scale up based on demand quickly, removing the need for DevOps teams.

Transactional emails are handled using tools like React Email, which allows developers to use React and Tailwind directly in their emails, eliminating the need to code vanilla HTML.

Developers can manage object storage using tools like UploadThing, which work natively with React and eliminate the need to use AWS S3 directly.

Yoko Li
Yoko Li
7 min
06 Jun, 2023

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Today's Talk explores architectural shifts in transactions, highlighting the dominance of JavaScript in the modern transactional stack and the elimination of the frontend-backend divide. The use of serverless and component-driven toolings allows for scalable infrastructure provisioning and removes the need for DevOps teams. Additionally, workflow engines play a crucial role in orchestrating async functions in the serverless native world.

1. Architectural Shifts in Transactions

Short description:

Today, I will share interesting architectural shifts in the context of transactions. JavaScript is winning in the old world architecture, where microservices are divided between the client and server. In the modern transactional stack, serverless and JavaScript alone power the whole application. There are no longer frontend and backend, and platforms like Superbase, Convex, and Upsash eliminate the need to manage backend infrastructure.

Hi, everyone. My name is Yoko. I'm a tech cartoonist, developer, product manager, and lastly partner at H16z investing in the JavaScript ecosystem and developer tools. So today I would like to share a few interesting architectural shifts I have seen after talking to hundreds of developers and tech companies. I'm putting the architectures in the context of transactions since they're so core to our daily lives. They power things from plane tickets building to getting food deliveries to our doors to booking cat sitters and dog sitters. And we're seeing a major shift in how developers use and reason about technologies here.

Well, spoiler alert, JavaScript always wins. So let's talk about why and how. Now, here is the architecture you will see in the old world. As you can see here, there are a bunch of microservices divided between client and the server because they're retaining different languages, also because front end and back end developers can't possibly agree with each other. So when you see the request path, when a request comes in, for example, placing an order. It gets authenticated and then sent to an API. The API endpoint will need to orchestrate hundreds of microservices to deliver a business transaction to the user. And then it will go through things like inventory management service, order processing service, payment processing service. This is when developers use queues and caches as you can see on the lower half of the picture to store what we call application states. Things like order received, order paid. The developers will also hand code the retry logic to get the data to the databases and cross their fingers and hope for the best. We found that as the system scales so does the complexity of managing queues and caches and corner cases. And now we actually see many companies start to build their own new projects very differently.

Now enter the modern transactional stack where you no longer need to provision and manage hundreds and thousands of microservices. Instead, we have got to a point that serverless and JavaScript alone is good enough to power the whole application. So you may be wondering what is the modern transactional stack? First, it's entirely JavaScript and backend is taken care of by a vendor. Sometimes the vendor is serverless. There's no longer what we call frontend and backend. Everyone is a full stack engineer and there are no longer APIs since it's written in the same programming language. And then, now you have things like server components and TRPC just calling a function. We are seeing more and more companies embracing this pattern as they go. What we traditionally call the backend, now we have platforms like Superbase, Convex, Upsash and others that can entirely eliminate the need for the app developers to directly manage backend infrastructure. Trust me, you don't want to work on AWS.

2. Modern Transactional Stack and Workflow Engines

Short description:

I used to provision infrastructure all the time and it's not fun. In this architecture, you will also utilize services that can scale down to 0 and scale up based on demand quickly. Removing the need for DevOps teams altogether. Another trait of the modern transactional stack is that it's powered by component-driven toolings. For authentication, you no longer need to deal with lower-level tools backend engineers traditionally use. Now you can just pick up tools like Clerc and use a signing React component. For object storage, you can use something called UploadThing natively in React instead of using AWS S3 directly. Lastly, as businesses use more APIs and accept more webhooks, they have to deal with async functions at scale. Workflow engines play a central role in the serverless native world, orchestrating async functions. If you want to chat more, feel free to send me a DM on Twitter at staff Yoko Draws. Thank you.

I used to provision infrastructure all the time and it's not fun. In this architecture, you will also utilize services that can scale down to 0 and scale up based on demand quickly. Removing the need for DevOps teams altogether. Upsash is a great example of this.

Second, another trait of the modern transactional stack is that it's powered by component-driven toolings. For authentication, you no longer need to deal with lower-level tools backend engineers traditionally use. Now you can just pick up tools like Clerc and use a signing React component. Another good example here is an open-source project called React Email. You no longer have to code vanilla HTML for transactional emails and now you can just use React and Tailwind directly in your email. For object storage, this is always something that a lot of engineers tell me and even myself, how to learn AWS for. But now you can use something called UploadThing natively in React instead of using AWS S3 directly.

Lastly, one highlight of this stack is that as businesses use more APIs, more functions and accept more webhooks, they essentially have to deal with async functions at scale. So when this happens, there needs to be a central place to orchestrate these things since handcrafted retries are hard. Now we start to see workflow engines playing a central role in the serverless native world. And the way to understand them is that they are the orchestrator of async functions. Here's the intuition. If your app handles hundreds of user signups and then sends a bunch of welcome emails to the users, you can use tools like ingest to write a few lines of TypeScript code to define what functions should be triggered when a signup event happens. It will also retry this part of the logic automatically with exponential backoff if that process fails. And on the client side, if you want, you can use the authoring layer of the state machine called xState. I hook it up with the workflow engine. It's a super cool project, makes things much easier to reason about compared to Redux which I used to use all the time.

There is probably hours and days of things to talk about in a stack we see over and over again here in the serverless era. If you want to chat more, feel free to send me a DM on Twitter at staff Yoko Draws. And that's all I have today. Thank you.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Scaling Up with Remix and Micro Frontends
Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
This talk discusses the usage of Microfrontends in Remix and introduces the Tiny Frontend library. Kazoo, a used car buying platform, follows a domain-driven design approach and encountered issues with granular slicing. Tiny Frontend aims to solve the slicing problem and promotes type safety and compatibility of shared dependencies. The speaker demonstrates how Tiny Frontend works with server-side rendering and how Remix can consume and update components without redeploying the app. The talk also explores the usage of micro frontends and the future support for Webpack Module Federation in Remix.
Understanding React’s Fiber Architecture
React Advanced 2022React Advanced 2022
29 min
Understanding React’s Fiber Architecture
Top Content
This Talk explores React's internal jargon, specifically fiber, which is an internal unit of work for rendering and committing. Fibers facilitate efficient updates to elements and play a crucial role in the reconciliation process. The work loop, complete work, and commit phase are essential steps in the rendering process. Understanding React's internals can help with optimizing code and pull request reviews. React 18 introduces the work loop sync and async functions for concurrent features and prioritization. Fiber brings benefits like async rendering and the ability to discard work-in-progress trees, improving user experience.
Full Stack Components
Remix Conf Europe 2022Remix Conf Europe 2022
37 min
Full Stack Components
Top Content
RemixConf EU discussed full stack components and their benefits, such as marrying the backend and UI in the same file. The talk demonstrated the implementation of a combo box with search functionality using Remix and the Downshift library. It also highlighted the ease of creating resource routes in Remix and the importance of code organization and maintainability in full stack components. The speaker expressed gratitude towards the audience and discussed the future of Remix, including its acquisition by Shopify and the potential for collaboration with Hydrogen.
The Eternal Sunshine of the Zero Build Pipeline
React Finland 2021React Finland 2021
36 min
The Eternal Sunshine of the Zero Build Pipeline
For many years, we have migrated all our devtools to Node.js for the sake of simplicity: a common language (JS/TS), a large ecosystem (NPM), and a powerful engine. In the meantime, we moved a lot of computation tasks to the client-side thanks to PWA and JavaScript Hegemony.
So we made Webapps for years, developing with awesome reactive frameworks and bundling a lot of dependencies. We progressively moved from our simplicity to complex apps toolchains. We've become the new Java-like ecosystem. It sucks.
It's 2021, we've got a lot of new technologies to sustain our Users eXperience. It's time to have a break and rethink our tools rather than going faster and faster in the same direction. It's time to redesign the Developer eXperience. It's time for a bundle-free dev environment. It's time to embrace a new frontend building philosophy, still with our lovely JavaScript.
Introducing Snowpack, Vite, Astro, and other Bare Modules tools concepts!
Composition vs Configuration: How to Build Flexible, Resilient and Future-proof Components
React Summit 2022React Summit 2022
17 min
Composition vs Configuration: How to Build Flexible, Resilient and Future-proof Components
Top Content
Today's Talk discusses building flexible, resilient, and future-proof React components using composition and configuration approaches. The composition approach allows for flexibility without excessive conditional logic by using multiple components and passing props. The context API can be used for variant styling, allowing for appropriate styling and class specification. Adding variants and icons is made easy by consuming the variant context. The composition and configuration approaches can be combined for the best of both worlds.
Remix Architecture Patterns
Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Remix Architecture Patterns
Top Content
This Talk introduces the Remix architecture patterns for web applications, with over 50% of participants using Remix professionally. The migration from single page applications to Remix involves step-by-step refactoring and offers flexibility in deployment options. Scalability can be achieved by distributing the database layer and implementing application caching. The backend for frontend pattern simplifies data fetching, and Remix provides real-time capabilities for collaborative features through WebSocket servers and Server-SendEvents.

Workshops on related topic

AI on Demand: Serverless AI
DevOps.js Conf 2024DevOps.js Conf 2024
163 min
AI on Demand: Serverless AI
Top Content
Featured WorkshopFree
Nathan Disidore
Nathan Disidore
In this workshop, we discuss the merits of serverless architecture and how it can be applied to the AI space. We'll explore options around building serverless RAG applications for a more lambda-esque approach to AI. Next, we'll get hands on and build a sample CRUD app that allows you to store information and query it using an LLM with Workers AI, Vectorize, D1, and Cloudflare Workers.
High-performance Next.js
React Summit 2022React Summit 2022
50 min
High-performance Next.js
Workshop
Michele Riva
Michele Riva
Next.js is a compelling framework that makes many tasks effortless by providing many out-of-the-box solutions. But as soon as our app needs to scale, it is essential to maintain high performance without compromising maintenance and server costs. In this workshop, we will see how to analyze Next.js performances, resources usage, how to scale it, and how to make the right decisions while writing the application architecture.