Type-Safe App Configuration: A Powerful, Emerging Way to Accelerate Product Development

Today, I'm going to talk about type-safe app configuration, an emerging best practice that can help you accelerate your software development. Now, if you've heard of feature flags, you can think of them as the tip of the app configuration iceberg.

Now, the best way to understand what app configuration is is to just see it in action. So let's imagine that we're a typical SaaS company called Potion, and this is the plans page of our app where users can subscribe to one of our paid plans.

Now, let's say the marketing team wants to run a 20% discount across all of our paid plans. So they ask us, the engineering team, for help. So we'll go into our code base and we'll write some code to implement the discount. And then we'll deploy this to a staging environment and then send the marketing team a link to the staging environment so they can have a look and see if it looks good. They say it looks good. So they tell us that they're happy for this to go to production. So we deploy this to production, and now the discount is live.

But then after some time, the marketing team gets back to us and says that they want to pause the discount. So we go back into the code base, we write some code to pause the discount, and then we deploy to production. And now the discount is paused. But then after some time, they get back to us and they say they want to resume it. Now this back and forth is getting a bit tedious. So we decide to put the discount behind a feature flag, so they can control when it's on or off themselves.

Now we've actually set up a feature flag system in our app already called HyperTune. So we can go into the HyperTune UI and then create a new flag called isDiscountEnabled. And it's a boolean because it's just on or off. And by default, it's off. So we'll save that. And then we'll go into our code base and run npxHyperTune to regenerate our type safe client to access all of our flags. And then we'll use the flag in our code with HyperTune.isDiscountEnabled. And we'll pass a fallback value false. We'll deploy this to production. And then we'll send the marketing team a link to the flag. So now they can control whether it's on or off themselves. So right now it's off. So if they go into the app, there's no discount.

But if they flip it on and then hit Save, then once that propagates, the discount appears in the app. Now this is great because they can turn the discount on or off themselves without having to ask us, and then we don't have to wait for them to tell us when they want the discount to be live. So both of our work streams are now decoupled and we can move a lot faster and independently.

They also realized that they only want to run the discount for UK users. They can turn it off here and add a specific rule to check for the user's country, and if it's the UK, then the discount is on. They also realized that they want to A-B test the discount to see if it actually improves the number of upgrades, total revenue, and total profit. So instead of it just being on for all UK users, they drop in an A-B test over here, and they set it on for the test group, which will be 50% of users, and then off for the control group, which will be the other 50%. And then they'll be collecting analytics on both of those groups, and they can visualize that in a funnel to see how the discount drove new sign-ups, upgrades of revenue and profit, and so on. So they're pretty happy with the system, and we don't hear from them for a while, but then one day they come to us and they say that they want to run a 40% discount here instead of a 20% one. Now the issue is that the 20% was hardcoded in our codebase. Now we could just change this to 40% for them, but then we'd end up in that same slow back and forth process that we had before we had this feature flag. So let's see if we can do something similar to give them control over this percentage as well.

Now one thing we can do is rather than returning a simple boolean flag over here, we could return a whole discount object with subfields to say whether the discount is enabled and then what the percentage should be. So the first thing we'll do is define a new discount object type in our schema in HyperTune. And we'll add a subfield called isEnabled, which will be a boolean just like before, and another subfield called percentage, which will be a float. And then we'll deprecate our simple boolean feature flag, and we'll add a new flag called discount, which instead of returning a boolean, returns a whole discount object. And then we'll save that, and we'll go back into our codebase and run npx-hypertune to regenerate our type-safe client. Then we'll see a type error where we're still using that deprecated feature flag. So we'll replace its instances with our new flag which is hypertune.discount.isEnabled. And we'll also replace this hard-coded percentage here with our new percentage flag. So that's hypertune.discount.percentage, and then we'll pass a full back value of 0. Now we'll deploy this to production, and then we'll tell the marketing team that as well as being able to control whether the discount's on or off just like before, they can now also control the percentage. So for now, they turn it on for everyone, and they set the percentage to 40%. So we'll save that, and then once that's propagated, they'll see that new 40% discount in the app. Now this is great, and they're pretty excited. And in their excitement, they ask whether they can control the copy of the discount over here as well. So that rather than saying, summer sale, they can experiment with different messaging. Now what we can do is go back to the discount type in Hypertune and just add a new field called name, which we'll set to a string. And we'll use this to control the copy of the discount. So we'll save that, and we'll run npx hypertune again, and then we'll replace this hard-coded copy over here with our new string flag.

So that's hypertune.discount.name, and we'll pass a fallback value of just discount. Now we'll deploy this to production. And so now the marketing team can experiment with different messaging here. So rather than summer sale, they'll just have summer discount. And once they save that, they'll see that update in the app.

Now they actually decide they want to experiment with different messaging. And so instead of just having one string here, they drop in an A-B test to test three different discount messages. So they'll have summer sale, and then summer sale ends soon, and then summer sale ends today. Then they decide they don't want to go through all the steps of a traditional experiment, like waiting for it to run and then collecting data until they have statistical significance, and then picking a winning variant, and then shipping that to 100%, and then cleaning up the test. They want to automate all of those steps.

So they go to the experiment page in hypertune, and they convert it from an A-B-C test into an AI loop. And they'll automatically optimize for upgrades. So now hypertune will automatically learn what the best variant is to maximize the number of upgrades. And it can take into account user attributes to also learn if the best variant is different for different types of users. So they're pretty happy with this system now. They can target the discount to different user segments. They can A-B test different percentages. They can even use AI to personalize the copy of the discount.

But then one day they come to us with an even more complex use case. So as well as being able to run a 20% discount for everyone, they also want to run an additional 10% discount for users who enter a special referral code. So they need a way to run multiple overlapping discounts at the same time.

Now the problem is we were only returning a single discount object from our discount flag over here. And so instead we need to return a list of discounts. So what we'll do is we'll deprecate this flag. And then we'll add a new flag called discounts, which will this time would be a list of discount objects. And we'll save this. But before we do, we'll also go into our schema. And on that discounts flag, we'll also add some arguments to pass into this flag specifically. So we'll pass in the plan which the user has selected or is looking at. And also the code which they've entered.

So we've set that up here, where we're calling hypertune.discounts. We're passing in the code the user entered, the plan they're looking at, and then we filter the discounts by whether they're enabled or not. So we'll deploy a list of production. And now the marketing team can add and remove as many discounts as they like from this list. So they can have as many discounts overlapping as they want.

So by evolving our simple Boolean feature flag into this flag which returns a list of discount objects, we've empowered our marketing team with this really flexible, powerful system to manage and optimize discounts by themselves.

So they can have as many discounts overlapping as they want. So the first thing they do is they add a discount which is on for all users, and that's for 20%, and that's the summer sale. And so if we save that, then they see that appear in the app. And then they add another discount, but this time it's only enabled if the user has entered a code which is 10 off. And then otherwise it's disabled. And the percentage of the discount is 10%. And this is our special discount. So we'll save that. And so we don't see that 10% discount at the moment, but if we enter code equals 10 off, then we see that extra discount applied. So now, just like before, the marketing team can A-B test discounts. They can target discounts to different user segments, A-B test different percentages, and even personalize the copy of each discount with AI, and they can have multiple discounts running at the same time.

So by evolving our simple Boolean feature flag into this flag which returns a list of discount objects, we've empowered our marketing team with this really flexible, powerful system to manage and optimize discounts by themselves. And we've actually done a similar thing with other parts of our app. So here the marketing team can actually control the prices of all our different plans. And here they can control which features are in which plans and the feature grid that we display to users in the app. Here they can actually control all of our landing page content like the headline and the layout, and they can A-B test different headlines and even use AI to personalize headlines to different visitors. And here, we're actually managing the configuration of our AI assistants which we have in our app. So when users visit our app, we have an AI assistant which suggests a feature for them to use based on what we know about them. And so here we're controlling which AI model to use, what prompt to pass in. We're injecting details of the user into the prompt. So this is more of a prompt template. And we're controlling the other parameters that we pass in like the temperature, the max tokens. And so this means we can tweak all of these parameters very quickly without having to update our code and do another deployment, but it also means we can optimize these parameters. So we can A-B test different prompts and temperatures like we're doing here to see what drives the best metric uplift. And we can even use AI to decide what the best temperature is. So in that case, we would even be using AI to optimize our AI. So we actually used an app configuration system like this. It's very similar to this at Meta. And it was called Configurator. And Meta actually released a paper on this called Holistic Configuration Management at Facebook and it dives into how Configurator was used for lots of different use cases in a similar way to platformize products and back-end services.

So as well as rolling out features, it was used to configure feature permissions and usage limits like API rate limits, pricing plans and billing logic, in-app copy, CMS content, internationalization and localization, forms, page layouts and funnels, all of our AI models, prompts, temperatures, weights, chains, prediction thresholds, rules to automate approvals and detect spam and fraud, redirect maps, timeouts, magic numbers, third-party integration settings, and automated back-end jobs, and a large long tail of other kinds of configuration. But across all of these different configuration use cases, we got the same high-level benefits. So number one, we got instant updates decoupled from code deployments and the release cycle. Number two, we decoupled dependencies in our organization. So we empowered others to self-serve and make changes by themselves without that slow back-and-forth process with us. And then it also meant that we could focus on shipping code to production without being blocked by them. And then number three, once we had all of this rich configuration, it became easy to optimize it with A-B testing and automated AI loops. And then number four, we simplified our system by extracting all this configuration logic, like user-specific code, out of different code bases and into a single source of truth. And then we also separated the concerns of code and configuration so we could focus on the high-level logic in our code and then worry about configuration details later or give someone else control over that. And then number five, we got more flexibility so we could adapt to changing requirements without needing to make code changes.

So app configuration is great, but how do we go about building a flexible app configuration system that can scale to all of those complex use cases that we went through? Well, first we need a flexible schema language and a type system to define the inputs going into our system, like the current environment or user, and the outputs coming out of our system, like our feature flags or the pricing plans that we offer. Second, we need a flexible configuration language to define the logic that maps the inputs to the outputs, and we should be able to embed A-B tests and AI loops into our logic too. And then we need a UI for non-technical people to edit that logic visually. And then third, we need an analytic system to log events and then see the results of our A-B tests and then collect training data for the AI loops. So when we combine those three ingredients, a flexible schema language, flexible configuration language, an analytic system, we can scale to all the different use cases that we went through earlier. So this is really powerful, but it's also a bit scary as we're moving business logic out of our code base and into our configuration system. So if the system fails or we make an incorrect change, it could cause major issues. And we also don't want to impact the performance or efficiency of our app by querying an external configuration system. So how can we make the configuration as safe, reliable, performant, and efficient as our own code?

Well, first, we can design the SDK to locally evaluate our configuration logic. So it fetches all the configuration logic from the server upfront in one initial network request. And then after that, there's no dependency on the server to evaluate configuration. So it's super reliable, performant, and efficient. But we still have that one initial network request that could fail. So to make that more robust, we can include a snapshot of our configuration logic in our app bundle so the SDK can use that as a fallback in case it can't initialize from the server. And we can also set up a webhook to trigger a new build and deploy of our app whenever the configuration logic changes, so we can keep that snapshot up to date. And then if we do that, we don't even need to initialize from the server at all, and we can use the SDK in a local only offline mode. But then we do lose the benefit of being able to update the configuration instantly from the server. But if we combine those approaches where we have the up to date build time snapshot and the instant server initialization, then we get the best of both worlds. The next thing we can do is that when configuration is evaluated, we can pass hard coded fallback values for the case when the SDK hasn't initialized and there's no snapshot. We can also ensure that configuration is evaluated with full end-to-end type safety to avoid typos, catch errors at compile time rather than runtime, and enable a modern developer experience with IDE integration for code completion, find all references and js.comments. And then finally we can version control all of our configuration in a git-style history so we can see exactly what changed and when with clean diffs of our changes and branches the test and preview changes before merging them into the main branch for the pull request. We can also layer on team roles and permissions to require changes are first reviewed on a pull request to safely onboard non-technical team members. So to recap with local evaluation, build time snapshots, type safety and git-style version control we can get a similar level of safety, reliability, performance and efficiency as our own code. There is no tool or platform out there like this so we built Hypertune and there are some links to check us out. And thank you for listening today. I hope you enjoyed the talk.