From Fragile to Future-Proof: Solving Web Monetization Chaos with Payment Systems That Scale

Hey, everyone. Thank you for coming to my JS Nation talk. I'm really excited to be introducing a topic that's been really close to my heart the past couple of years, and it's about monetization and how to bring payment systems that scale to your frontend systems. And today, we're going to be talking a little bit about how to bring that into the realm of React. And the talk is specifically called From Fragile to Future Proof, How Do You Solve for Web Monetization Chaos with Payment Systems That Scale?

And the way I like to start this is a little bit with a story time. And so I work at SmallPDF. And SmallPDF is a platform where we've got tens of million users per month and we process payments in multiple different countries across the globe. And so with one of the payment providers that we have, and one of those providers is called Braintree, we had an outage a couple of months ago. And in that outage, we actually just lost the integration with Braintree because one of our API keys was removed by mistake.

And when those API keys was removed, all of a sudden we started receiving a bunch of 401 errors, and we didn't know what was going on, and every single payment was failing. And Braintree essentially helped us process all of our PayPal and credit card payments in the majority of our countries. So that means that no users were able to process any payments, create any subscriptions, because we were heavily reliant on Braintree as a processing system. And so that's really chaotic. And this issue happened really, really late at night, and it took really long to figure it out. So that took long for the alerting system to tell us something that was going on.

And so when we have a platform that's running 24 seven, and something starts going wrong, things start getting bad pretty quickly. So this is actually a screenshot from our TrackJS dashboards. This is the equivalent of your Sentry dashboard, which uses a different platform for that. And you can see that all of a sudden, there's a massive spike, a spike in these Braintree token fetch errors. And that's exactly those errors that we're facing because the API key was removed. And that lasted for a long period of time. And it was really hard to discern anomalies, because there was so much of the sea of errors and logs coming through, it was very hard to be able to determine what was going on.

Moving forward, you can see over here in our data doc dashboard, there's now a log for the amount of subscriptions that we create. And that same amount of time, that same slot of time, where we saw all those errors climb up with TrackJS, you can now see that all the subscription creations tacked. And so it shows the direct correlation where the errors happening meant that we weren't able to process subscriptions. That makes sense. Now you'll see that there's still some small spikes over there. That still show we were able to process some subscriptions. And I'll show you why that's the case. Because Braintree wasn't the only gateway we're using at the time, we're also using Adgen and some other gateways to process some other payments. But it still wasn't the majority of our volumes, we saw massive tank.

Now how bad was this incident? It lasted from 9pm till 10am. So working hours, right? It was actually, yeah, clearly after working hours, we all work Swiss times in small PDF, the main office is based in Zurich. And so people are probably in the middle of their dinner time or getting ready for bed. And whoever received that alert was not having a great night. If you're good at math, that translates to around about 12 hours of downtime, which is no small amount of time. So for 12 hours, nobody was able to create a subscription. And beyond that, that translates to five to $10,000 in lost subscriptions. That's no small amount of money. And we're going to explore something that's gonna change that.

So who am I, my name is Faris, I'm a software engineer at small PDF. So we do everything in terms of PDF stuff, in terms of merging PDFs, compressing stuff with a really cool suite of tools, everything is on the web. And we've got like a ton of users, as I previously mentioned. So when something goes wrong, it goes wrong really, really quickly. Some of my previous experience is in the connected TV world, having worked on applications such as Eurosport, GCN, Discovery Plus, worked in growth and FinTech teams are really interested around payment systems in particular, and also web performance and engineering leadership. From time to time, I work on some open source projects, including some Raycast extensions, I'm actually in the middle of revamping the Stripe extension on Raycast, something I have a lot of fun doing, because Raycast is really incredible. And they have also I think, recently come to Windows and iOS. So I'm really happy to have seen that expand beyond the Mac ecosystem. And then also the co founder of Zurichess, which is the JavaScript meetups that we run in the Zurich region.

What are the struggles with payment systems? If you're building any SaaS platform, it's very likely that you're going to want to charge users either on a one-time basis or on a subscription basis. And just because I'm working on developing my applications in Switzerland, that doesn't mean I don't want to process payments across the globe. In Switzerland, we use Swiss francs, whenever I'm traveling somewhere else, whenever I'm traveling to France or Germany, I'm using euro. So a lot of us, for example, have Revolut cards or Monzo cards, because the exchange rate is really favorable when you use those rather than your credit cards that have a lot of fees. You know, those are the things that start to come into play when you want to expand your payment opportunities for your SaaS platform.

So as I mentioned, there's that currency and FX fee issue, where if I am trying to purchase, you know, a product in Switzerland, or I want my Netflix subscription, essentially in Switzerland, I want to be paying Swiss francs for that, I don't want to be paying an exchange rate or on top of that credit card fees to be able to process that. And there's also an association with trust. And a business depending on if you're offering localized pricing or not. And localized pricing doesn't just mean that I'm giving you a currency that is local to wherever you are, such as you know, US dollars in the US, it also means that in other countries, developing countries, for example, you offer lower prices based on what the earning power of that country. So for example, in India, it's very typical to offer Indonesia lower prices over there, which means that the local market has an easier time subscribing to a product rather than just giving a blanket price for everyone.

Then there's also the subject of optimizing payment methods per country. Now in Switzerland, we have a payment method called twint. Anybody that's not based in Switzerland will have almost never heard of that payment method. And if you go to China, there's Alipay. If you go to Thailand, there's QR code based payments. And these are the systems that are used to process payments on a daily basis, which means they're frictionless systems. And so whenever I am paying for a service in Switzerland, using twint is 10 times faster than inputting on my credit card details. And so I have a much higher chance of converting and subscribing to a product if I'm using a local payment method. And so what's really interesting about that is that they can have a really big impact from a conversion perspective on your product if you're able to support payment methods that are just local for the region that you're operating in.

So as I mentioned, we've got twint in Switzerland. In Brazil, there's PIX Bank Transfers. In India, there's UPI. And you know, credit card is used very little 80 to 90% of the payments are run through UPI, it's dominant over there. For the Netherlands, for example, it's ideal payments. And so you can tell that every single country and region has their own unique way of paying for things. So it starts to make a little bit of sense in the story if we want our SaaS product to be able to also support things for the local regions that they want to operate in.

Now earlier this year, at small PDF, we introduced Alipay into our product. And Alipay just allowed for users in China to have a much easier time subscribing to the platform rather than inputting their credit card information. And that actually led to a 3.5x increase in annual recurring revenue. Just by not changing the prices, but only changing the payment method used by people in China, it was much easier to subscribe that platform. And so it really shows you the power of introducing regional based payment methods that are optimized for that. There's another part to the story, which is also regulation and compliance.

So for example, with UPI payments in India, you generally need an entity over there. And so for a lot of payment processors, you need an entity in the region you want to operate in. So if all of a sudden your business in Switzerland or your business in the UK and you want to operate in other countries, you're not going to now start opening entities all over the place. This is where merchant of records become a really interesting solution because they take on the entity management part of the solution. However, they have much higher fees associated with them, and sometimes there's some limiting factors there. But there's not only the entity issue, there's also sanctioned countries.

So if there are countries and currencies that from a legal perspective or political perspective, they're a little bit politically heated at this at that at the point in time, and you no longer want to be able to operate in them. Otherwise, let's say you can't operate in country X, and Stripe is going to now cut your payments as a result of you operating in country X, or you know, providing service Y, you want to be able to have a really easy way to be able to reroute payments away from that country or disable that. So having configurability now becomes something really important and part of a business's liability to be able to operate in a compliant manner.

So when it comes to regulation and compliance, there's also the topic of sanctioned countries and restricted transactions. There are just certain, you know, transactions that you're not able to do from a financial perspective, and each country and each payment provider has their own risk threshold, whether that's for chargebacks, disputes, whether that's for, you know, particular types of transactions based on maybe your credit card and location, you know, the fraud rate is particularly high, or just for politically heated countries where you don't want to be able to operate there, because then your payment provider may cut your ability to operate. And that means you suffer in terms of your revenue in other countries. So having the ability to reroute payments or being able to manage your liabilities really important when you're operating in over 100 countries.

So having the ability to reroute payments or being able to manage your liabilities is crucial when operating in over 100 countries. The topic of merchant records is intriguing when avoiding the need for local business entities everywhere. Compliance varies by country, with the US having 11,000 tax jurisdictions, requiring intricate configuration. Services offering tax management globally come with trade-offs in flexibility and control.

The issue of chargebacks and disputes arises when rates exceed thresholds, leading to potential service disruptions by providers like Adgen or Stripe. Reliability and uptime are critical, with providers striving for high availability levels like the five nines standard. However, individual payment methods may not always align with the overall system availability, impacting service reliability and uptime.

Ensuring system availability is challenging, especially when specific payment methods experience downtime, affecting the overall service reliability. Managing taxes at a global scale involves complexities due to varying tax jurisdictions, even down to street levels. Dealing with disputes and chargebacks can lead to service disruptions if not managed efficiently. Providers maintain stringent risk thresholds, impacting service continuation based on dispute rates.

If your chargeback and dispute rate gets too high, you may face consequences with payment providers like Adgen or Stripe implementing risk thresholds that lead to payment processing termination. Reliability and uptime targets, such as the five nines standard, are crucial, although individual payment methods may not always align with these standards, affecting overall service reliability. Providers may discontinue services if disputes are not efficiently managed, impacting service continuation based on dispute rates.

Reliability and uptime are essential considerations for payment providers aiming for high availability levels, with some targeting the five nines standard for system uptime. However, individual payment methods may not consistently meet this high availability requirement, potentially affecting service reliability. Payment outages from specific payment methods can disrupt payment processing, emphasizing the importance of payment orchestration for directing transactions efficiently.

Payment orchestration, likened to air traffic control for payments, involves directing transactions to optimize cost, speed, and reliability by managing multiple payment providers like Stripe, Revolut, and Adgen. Orchestrating these providers allows for efficient transaction routing based on various factors like cost, success rate, and region. While some providers offer automatic optimization of payment methods, it's essential to customize optimization based on individual needs rather than relying solely on default settings.

Even if we look at just the three nines 99.9% availability, that means around about eight hours and 45 minutes of failure in a year of a system. Now what's interesting about some of these payment providers is that while the overall system availability may be the five nines, individual payment methods may not have that five nine availability. So if you ever see over here, these are the past issues that have happened, or the uptime of certain services offered by Stripe. So this is their status page. And it's the history of outages. There were, for example, elevated PayPal hours for an hour and a half. So that already, for example, in the PayPal scenario kills the five nine availability metric, you're no longer meeting the SLA, there are for 45 minutes elevated bank, bank contact errors, and then two hours of elevated API errors for other stuff. So individual payment methods, you know, have their own uptime. It's not just the entire gateway being available to individual payment methods accessible. That's also really important.

We look at Braintree and PayPal, there was an eight-hour outage there. And so there will be nothing you'd be able to do. Even whether that's just the gateway itself, or the payment method, it's unavailable eight hours, a long time to not be able to process payments. And this is where the subject of payment orchestration comes into play. And then I really like to think of this as air traffic control for payments, we're able to direct transactions to the best route for cost, speed, and reliability. And so this payment orchestration system is this idea of having one system that manages multiple payment providers. Stripe is a payment provider, Revolut is a payment provider, Adgen is a payment provider. So orchestrating these together become really, really powerful. And so you're able to, again, you have route transactions based on cost, success, or region. For example, Stripe automatically optimizes the order of your payment methods if you use their Stripe elements UI. But that doesn't mean the default optimization is the right optimization, optimization for you. For example, if in China, they're showing Google Pay as the primary option, but in the other seven or eight payment methods available, there's Alipay, and you've determined through A, B test that Alipay is the better payment method, you may want to switch the order yourself and have your own optimization that's best for your SaaS and your use case. The default doesn't always work for everyone.

This also naturally lands us in a place where we are reducing reliance on a single provider to avoid downtime, which means that if we have that, for example, eight-hour outage with PayPal, we're able to redirect to another gateway to be able to support us during that time. Even if that gateway has a lower conversion rate, you're still able to operate. And a big part of that is being able to optimize payment methods per country for a higher success rate. And not every single gateway provides you all the payment methods, mixing and matching gateways gives you a lot of flexibility and being able to provide a much larger array of methods. And if we look at this example here, if we're comparing a Revolut Pay and Stripe Pay, so these two gateways, if we look at overlapping countries like the United Kingdom and France, you can see that Stripe fees are actually 1.5% and two pence 2.5% and 2.2 euros. And in Revolut, it's much lower with a 1% fee for both countries. And they're able to process payments in the same country.

So why would you go for Stripe when in this case, the fees are lower, you can go for Revolut. In the example of Cyprus and Estonia, Stripe does not operate in those two jurisdictions. So you're just unable to expand your platform into those two jurisdictions, if you're operating with just Stripe. If you had Revolut, you'd be able to take advantage of the fact that Revolut does operate in those two countries, and you're able to then expand into them, which means that you're able to reach more customers. In the case of Stripe, Revolut does not operate in the US or Canada. So even though there are slightly higher fees, you have no choice but to go down that route to be able to expand into those high ARR countries. So then you can see where different gateways start to provide different advantages one way or another.

What's the front end architecture of how this looks like? Now, payment orchestration is predominantly a back end concept, something that's done on the back end. So the front end just says I want to make a payment and the back end figures out what to do. But what can get really interesting is being able to react to the states that we develop on the front end side. And I'll explain what this means in a second, and being able to show the right orchestration, the right gateway, depending on the user and their context. So a user may open a checkout in our front end platform. And what we do is we initialize an orchestration layer. And this orchestration layer will have its own set of rules of deciding what to render at what time. It'll consume a global state, the global state, whether that's your Redux state, or your context providers, whatever it may be, and then a rules configuration.

What I meant by consuming global state earlier is that your front end knows the country users in, the front end may be tracking the state of A, B tests, maybe you're running a test in Japan, maybe you're running a test in Germany, testing two different payment methods, and then a list of currencies that you want to support. And so maybe you'll pick a gateway that matches one that you already use, or you'll pick gateways based on experiments, other countries, all this sort of stuff. What you'll do with that information is now you'll then take it into a decision tree. And once it's taken to a decision tree, it comes out with a prioritization of payment providers. And why do we go for prioritization and not a singular selection? Because if gateway one is down, you want to be able to fall back to gateway two. And so if we look at how this prioritization may come about, we have three payment method, payment gateways. In this example, we've got Stripe, which supports Alipay and Chinese Yuan. And we're going to take the China example in this case, we've got Revolu that supports credit card and USD and we've got Agilent that supports credit card and Chinese Yuan. So the best option in China is Stripe naturally. But what if Stripe is having downtime right now? The second best option is one that still supports the local currency. So we're going to go for Agilent as a secondary option if Stripe is down. And then worst case scenario, you fall back onto Revolu. This means that they're prioritized in this order based on what matches our criteria best. And then we render that array and we decide what to show to the user.

And then we render that array and we decide what to show to the user. Now in terms of a coding perspective, what does this look like when you're dealing with multiple payment gateways, you're generally dealing with a different JavaScript SDK? So Stripe comes with their old Stripe.js, Agilent comes with one, Recurly comes with one. And so what ends up happening is that you receive a lot of checkout components, and you receive a lot of context providers, you've got your Stripe provider, you've got your Agilent provider, Revolu provider, you end up sandwiching all of those to pass all of that context down into your checkout. And that can be a very overwhelming from a bundle perspective, from a state management perspective, you don't want to have to sandwich all of that.

And so when you're dealing with a payment orchestration system, ideally, based on the user state in this example of being in China and having selected a currency of Chinese yuan, you want to be able to just render the provider that's necessary based on the state that we acquired. So okay, we figured out that Stripe was best in the scenarios under the hood, we're rendering just the Stripe context provider. That's what this is doing. It's minimizing and loading the minimum viable code just to execute the checkout work. And we'll see that this has actually had a really significant change to our bundle size.

In terms of then actually processing the checkout you receive from these SDKs, again, a bunch of different hooks, you'll maybe have used Stripe use Agilent or you have created your own hooks to process payments, and they'll all have their own way of processing payments, there are different keys and parameters that they accept. So maybe Stripe accepts an amount and a currency, whereas Agilent accepts a value and a currency code. semantically, they all mean the same thing. But syntactically, they're differently represented. And your front end, again, should only care that needs to process the payment, not how it's processing that gateway is using under the hood.

So again, that layer of abstraction that the orchestration layer is providing, it's just caring that it needs to process the payment, it's going to use an adapter under the hood to adapt the right gateway. When it comes to that rules configuration that we talked about, we can see actually over here what a rules configuration looks like for a Stripe payment gateway. So you can see that Stripe is defined, and we have something called an enablement criteria. And this has an SQL like type syntax, or, you know, in terms of understanding how it operates, where you can have or an and operators to decide what to include and exclude and how to define your conditions for what should work, what should render in your orchestration layer. And so you can see, for the country's enablement criteria, you have only in China, so only render this payment method in China, and only in test Stripe gateway, which is a feature flag. So if I want to be able to enable this in the US, I can enable a feature flag in the US. And it will also be enabled via an experiment, I don't just have to rely on the country alone. And then there's no currency limits, we put null for currencies. We've also defined AliPay as a payment method.

Just because you've selected a gateway in China, as Stripe, that doesn't mean all of the payment methods are supported in that particular country. So we've defined AliPay. And we've said that, hey, only render this in China. And then, as we explained a little bit before, in terms of the context providers, you're passing that Stripe provider that's provided by the Stripe SDK in as the payment gateway context provider so it knows what provider to render so that we can render that checkout and provide it the right state. If we wanted to add a new payment method, such as PayPal, we just need to define PayPal as a payment method, say that it's enabled in the US, for example, add us as a country to our enablement criteria. And in seven lines of code, you've added a payment method.

Now, if we look a little bit in terms of numbers, if you've got a 10 million ARR business and you're operating 24 7, for example, you can estimate even though you have peaks and lows that you're making around about $1,140 per hour. And if you're running at a 99.9% availability or SLA, that's around about eight hours, 41 minutes as downtime, as we've seen before that you could have as a potential loss, which translates to nearly $10,000 a year, no small amount again. Not only does being able to switch to different gateways give you that advantage of being able to reroute during downtime, but you also get an uplift from the payment methods. So using, for example, PICS payments in Brazil versus credit card can increase conversion by five to 15%. But not only that, credit cards have fees. And so you can actually reduce the amount that you have to pay in terms of fees, because you're no longer using credit card using PICS payments instead, and you don't pay the two to 4% fees on top of that. So you're optimizing at multiple different levels. It's not just the availability. It's not just the payment method. It's the combination of these that actually have a significant effect on how you're able to monetize and optimize everything in your SaaS platform, especially when you get to a certain scale. And this all has benefits as you scale. This is going to apply as you go from a 10 million to a 20 million ARR business. Now, let's check out a demo of how this works with SmallPDF. So I've opened up the SmallPDF platform. And what we're going to do is we're going to open up our checkout.

And we're going to initially see is that we've got Braintree and Recurly Loaded as a primary option in the UK, in Great Britain. We can go and see that everything is in British Pounds over here, we've got PayPal and credit card as options. But what I'm going to do next, I'm going to switch the country over to China, I'm going to refresh everything, and China doesn't have PayPal supported. So I'm only rendering credit card here. But we want to also run an A-B test to test if Stripe works better with Alipay. So clicking on that Stripe gateway automatically starts rendering now Alipay and I can test that out over here. Again, if we switch that out to another country like the US, we have different payment methods available, Alipay is no longer supported. So we now show Google Pay and a bunch of different options.

And then we can switch that flag back off to go back to our recurring Braintree setup with credit card and PayPal. And then in Germany, SEPA payments really popular so you have direct debit as an option. So you can see every little configuration completely changes the user experience and optimizes it for the country and for the A-B test that we want to try it. For example, in India, even though you want to process with credit card payments, we actually use Adjin under the hood instead of Braintree because different, even though you have a processing via credit card, different gateways have different success rates, even though it's the same payment method. So even that can make a significant difference.

Now, with all of that stuff loading, you saw we've refreshed every single time what the orchestration layer does is that it figures out what the right gateways is to load and then starts lazy load loading that code in preparation for the checkout. So what this actually did in reality was drop our bundle size by 70% by dynamically loading just what we needed to to process the payment. So let's take an example of what about when things go wrong, we started this conversation with, you know, big outage that happened for 12 hours, which is frankly unacceptable. And so what does this orchestration layer lend to us? What we can start doing is also now anomaly detection. And you can set that up with Datadog or most of your observability platforms these days. And based on the seasonality, so you'll have, you know, maybe Black Friday sales go really high, but then you'll have some downtime during Christmas. And so based on seasonality and being able to detect anomalies, using those rules that the gateway has to decide what to what to use to decide what to render, like the country, the currency and etc, you can start segregating your alerts based on gateway country and currency.

So if I know specifically, that for Indian rupee in India, Agents are working, I know the three pieces of information, I need to, to be able to reconfigure the gateway to switch to process to processing via stripe. And we actually had to do this two weeks ago, payments in India weren't working. So we had to switch to two different gateways at the same time. And we ran a 5050 AB test experiment. And that was done incredibly fast.

So you're 15 minutes away from detecting any metric deviation. And so why is this segregation also really interesting, because when you segregate all these logs and these metrics, spikes and anomalies become a lot clearer to be able to detect rather than if you were to combine all these together, you wouldn't see those spikes as much. And so 15 minutes is a lot better than the 12 hours we previously saw. And I think it was four hours for us to detect if something had gone wrong.

So let's take an example. We got an alert, we've got failing payments in brain with brain tree in Switzerland. So let's reroute things to stripe. Opening up the small PDF website, we've got brain tree showing up right now we're in Switzerland. What I'm going to do is I'm going to move myself over to the code, I'm going to go and add Switzerland as a supported country for stripe. So now I'm going to rerender everything. And we've got stripe available to us in just one line of code.