Real-Time Feature Toggles with AppConfig

There’s a common and critical trap that solo founders, indie hackers, startups, and even large enterprises tend to fall into: they tend to build features that users do not actually need or want.

Mostly, this happens because of the "build it and they will come" fallacy.

This fallacy occurs when companies and stakeholders assume that simply creating a feature or product guarantees user adoption and success, or when they consider something as a "must-have" without any validation.

This mindset overlooks the importance of user feedback and validation. Companies often invest significant resources into development without verifying demand. This leads to wasted effort and potential failure when the product doesn’t meet actual user needs or solve real problems.

But there’s a solution for this problem: feature flags and A/B testing. And this comes cheap, easily, and without much operational effort with the help of AWS AppConfig.

As always, we're providing a sample repository so you can easily experiment with AppConfig without having to do all the hard work initially.

Introduction to AWS AppConfig

AWS AppConfig, a feature of AWS Systems Manager, enables you to adjust application behavior in production environments quickly and securely without full code deployments.

It also supports Feature flags, which allow for the gradual release of new software increments, so you can test and validate feature before you roll them out to the whole user base or even before you fully build them. This also allows you to adopt trunk-based development, avoiding large and complex merges and allowing pre-release code to be safely rolled out to production.

As the feature flags are remotely controlled through AppConfig, you can enable and disable them post-deployment without rolling back code.

Feature Flags as the Main Building Block

At the core of AppConfig are feature flags. These flags are mostly used to control the visibility of a feature in an application

This is done by hiding the new feature behind an if-else statement that can be activated at any point after it is already deployed. This if-else query can also be targeted to a specific user segment, device type, or any other context you define.

With this approach, you can gradually increase the percentage of users that receive the new feature.

Why Feature Flags Really Enable Trunk-Based Development

Before we start with our AppConfig deep-dive, let’s have a look at some software development history. This will help us get a clear picture of why feature flags are so important.

There are different approaches to iterate and deploy software. These include feature branch and trunk-based development.

Each of them is a distinct approach to managing code integration and deployment. How do they differ?

• Long-Living Branch (Feature Branch) Based Development: Isolates feature development in separate branches, reducing the risk of incomplete features affecting the main codebase. However, it often leads to significant merge conflicts as changes accumulate. It also results in a slower feedback loop, as developers tend to work in isolation for longer periods.

• Trunk-Based Development: Developers are committing small, frequent changes directly to the main branch. This doesn’t mean that there are no feature branches at all, but they are short-lived and merged back into the main branch as soon as possible. This simplifies integration and reduces merge conflicts due to fewer changes but requires a disciplined team to manage the fast pace.

These are the two most common approaches to software development that every experienced developer is familiar with.

But there’s a third approach that is gaining popularity in recent years:

• Feature Flag-Based Development: Enhances trunk-based development by allowing incomplete features to be merged into the main branch and controlling them via feature flags. This means one or multiple parts of a feature are already implemented and in the main branch, but not yet visible to or activated for the user. This supports continuous delivery and reduces the risk of deploying unfinished features to users. Though it introduces additional complexity in managing the flags.

When we look at the three approaches, feature flag-based development is the most flexible and agile. Nevertheless, it requires a good strategy and discipline to manage the flags properly.

We’ve now learned about the different development approaches and why feature flags are so important. We’ve also learned how feature flags work and what they can be used for.

In the further paragraphs, we’ll dive into the details of AppConfig and how it can help us implement feature flags in our application.

Benefits of Using AppConfig

Using a feature-flag based, iterative approach in software development isn’t just a nice gimmick or trend. It’s a game-changer - not just for the development team, but also for the whole organization.

Let’s have a look at some of the benefits that come with feature flags:

• Reduced Production Deployment Risks: Deploying a new release to production can be a stressful experience. Depending on the release cycles, which are especially infrequent at the enterprise level (e.g., every two weeks or longer), it can be hard to predict the impact of a new release. Even with a pre-production environment (or maybe multiple), high test coverage, and automated regression tests, there is always a risk of introducing a bug that was not caught beforehand. Software is becoming more complex, and the number of dependencies is increasing, making it harder to predict the impact of a change. With an iterative approach, where new features are hidden behind feature flags, this risk is drastically reduced. Also, you will never be locked in "we can’t deploy because this feature is not ready yet" situations. If you’ve been working in software development for a while, you’ve probably heard this sentence a lot. More on that later.

• Granular Rollout of New Features: With feature flags, you can enable or disable features without deploying any part of your application. This means that the feature release is not coupled to the actual deployment anymore. Your new feature will be hidden behind an if-else statement that can be activated at any point after it is deployed. This activation is done via a call to a remote service. The remote service, in this case CloudWatch, is also able to decide which percentage of users will receive this new feature. With a defined rollout plan for the feature flag, you can automatically enable the feature for a certain percentage of users at a certain point in time.

• Minimized Impact: A "Dark Launch" helps you minimize the impact of a new feature. This is due to the fact that you can safely evaluate the results on a fraction of users or just for a short period of time. As it’s hard to replicate production environments, especially in terms of scale, load, and network latency, this is a great way to test new features in a real-world scenario without the risk of breaking the application for each and every user.

• Rollbacks without Code Deployments: This rollout can happen gradually, all while keeping an eye on technical and business metrics. If everything goes as expected and there are no error spikes or other anomalies, the percentage of users that receive the new feature can be increased. If something goes wrong, there is no need for a deployment rollback, but just a click to go back to the old and working application state. All of this can be done in a matter of seconds, not minutes.

This is just a small excerpt of the benefits that come with feature flags and Evidently. Working with feature flags instead of long-living feature branches is definitely revolutionary. It drastically reduces the risk of introducing bugs and allows for a more agile development process.

How AppConfig Works

What do we need to set up so that we can actually use AWS AppConfig inside of our application?

1. Identifying Our Configuration Values And Feature Flags - This is an ongoing process because your application will change over time, and configuration values and flags will come and go.

2. Creating an Application Namespace - AppConfig uses the concept of namespaces, which are simply organizational structures, like folders.

3. Creating Environments - An AppConfig configuration is always linked to an environment, such as production or preview. This also refers to a logical grouping of targets, like your backend, web frontend, or mobile app. It can also be more detailed, down to sub-components.

4. Creating a Configuration Profile - A configuration profile holds your configuration data and a profile type. AppConfig offers two profile types: freeform and feature flags. Configuration profiles can also include validators to prevent developers and other stakeholders from providing invalid values.

5. Deploying our Configurations - To apply your configuration, you need to create a deployment. A deployment links to an application, profile, configuration version, and environment. The deployment also requires selecting a deployment strategy, which includes the rules for how the configuration will be rolled out to the environment.

6. Retrieving the Configuration Within Our Application - When your configuration changes take effect, you need to fetch it from within your application.

7. (Optionally) Attaching CloudWatch Alarms to Our Environment - With AppConfig, we can automatically monitor the deployment process of our configurations using CloudWatch by attaching one or more alarms. If an alarm is triggered during deployment, the configuration will automatically roll back. This is a great feature and is easy to set up; we just need to add the ARNs of the alarms and a dedicated role that can describe the alarms.

The AWS AppConfig Agent Lambda Extension

Before we really jump into the hands-on part, let’s discuss one further point that is important when using AppConfig with AWS Lambda.

With Lambda, we have an almost unlimited number of execution environments because AWS manages when new instances start and stop. At first glance, this means we need to make many calls to the AppConfig service, and of course, we are billed for each call.

Fortunately, there's the AWS AppConfig Agent Lambda Extension, which we can use right away to help us easily reduce the number of calls we make.

This layer will internally cache our configurations fetched from AppConfig and will automatically keep the cache updated without any action needed from us. This means it doesn't matter how often we fetch the configuration in our code, as these calls don't actually result in calls to AppConfig.

Putting It All Together

Let's apply what we've learned by creating an application that fully integrates with AWS AppConfig.

As mentioned at the start, you can also check out the open-source code provided with the full article. The explanations will not cover every detail.

The example application is bootstrapped with SST, which enabled us to set up and deploy everything with just a few lines of code.

Creating our AppConfig Application

Let’s jump into the AWS AppConfig console and click on Create application. Here, we can enter our application name.

After we’ve done that, we can click into our new application and continue with creating an environment.

Setting up an Environment

In the Environments section, we can click on Create environment and set the name for our environment. In our example application, we're using only one environment, but in a real-world scenario, we would have at least two environments to separate production from development/testing stages.

Let’s continue with creating our configuration profile.

Creating our Configuration Profile

As we’ve learned earlier, we can choose from two types: either feature flag or freeform.

Each type is used for a different purpose:

• Feature Flag - The feature flag type is used for if/else statements in your code. This is useful for testing MVP (minimum viable product) features before releasing them to all users or even before fully developing the feature itself.

• Freeform - The freeform configuration can be used to store any configuration data within various AWS services, including AppConfig itself, S3, CodePipeline, Secrets Manager, or the Parameter Store of Systems Manager.

In our example application, we want to create a feature flag configuration. For this, when we click on Next after entering the profile name, we’ll be prompted for the feature flags itself.

Each profile can contain multiple feature flags, and each feature flag can have multiple attributes. The attributes are not limited to boolean values; they can also be strings, numbers, or even arrays of strings or numbers.

At the end, we’ll be asked if we want to set the flag to be enabled or disabled.

In our example application, we've set up a feature flag that determines whether a modal component is displayed to the user and what title and description it shows.

Defining a Deployment Strategy

Now we've successfully set up everything needed from the configuration side. The last step is to handle the actual deployment of our configuration data!

We can either do this by using the predefined deployment strategies that come out of the box, or we create our own ones. The predefined strategies look like that:

• AppConfig.AllAtOnce - Deploys all changes at once with a bake time of 10 minutes and a 100% rollout.

• AppConfig.Linear50PercentEvery30Seconds - Deploys 50% of changes every 30 seconds with a bake time of 1 minute.

• AppConfig.Canary10Percent20Minutes - Gradually rolls out 10% of changes over 20 minutes with a bake time of 10 minutes.

• AppConfig.Linear20PercentEvery6Minutes - Deploys 20% of changes every 6 minutes with a total deployment time of 30 minutes.

When we create our own deployment strategy, we need to define the following important parameters:

• Deployment Type - How the deployment should progress through the steps.

• Step Percentage - The additional percentage of users targeted in each step.

• Deployment Time - The total time for the deployment.

• Bake Time - The period during which the deployed changes are monitored to ensure stability before finishing the rollout.

In our example, we created an "instant deployment" that rolls out immediately and validates the deployment for just one minute.

Deploying our Configuration

Everything is set up - let’s actually deploy our configuration by going to our AppConfig profile and clicking on Start deployment after selecting a specific configuration version.

We'll be asked to choose the hosted configuration version again and select the environment we want to target.

After that, we can select Start deployment and the process will start.

In this overview we can watch how the deployment proceeds and if automated rollbacks are triggered.

Bonus: Setting up CloudWatch Alarms for Automated Rollbacks

Yes, you read that right. You can easily set up automated rollbacks using CloudWatch Alarms. This is done by attaching a CloudWatch Role (needs to have the cloudwatch:DescribeAlarm action allowed) ARN and one or several CloudWatch Alarm ARNs to the environment.

If one Alarm is triggered, AppConfig will immediately stop the rollout and rollback to the previous configuration.

Summary

After working with AppConfig for a while, we don't really know why AWS didn't just add the Analytics part of Evidently to CloudWatch as an extension for AppConfig.

AppConfig has so many similar features and it works almost identically (even though some things are not quite as fancy as in Evidently), but having two services that do want to do the same thing doesn't sound right.

TL;DR: AppConfig is great and you should use it! 😊

Don't forget to read the ​full post​ which also includes every step on how to set up AppConfig with a simpel feature flag - code included! 🤖

FAQ

1. What is the main purpose of AWS AppConfig? AWS AppConfig is designed to help manage application configurations and feature flags, allowing for quick and secure adjustments to application behavior in production (and testing) environments without code deployments.

2. How do feature flags benefit software development? Feature flags enable developers to control the visibility of features, allowing for gradual rollouts, A/B testing, and minimizing deployment risks. They support trunk-based development by allowing incomplete features to be merged into the main branch without being visible to users.

3. What are the different types of configuration profiles in AppConfig? AppConfig offers two types of configuration profiles: feature flags, used for controlling feature visibility, and freeform, which can store any configuration data across various AWS services.

4. How does the AWS AppConfig Agent Lambda Extension help reduce costs? The AWS AppConfig Agent Lambda Extension caches configurations fetched from AppConfig, reducing the number of calls made to the service. This helps lower costs as it minimizes the need for repeated requests.

5. What is a "Dark Launch" and how does it minimize impact? A "Dark Launch" involves releasing a new feature to a small subset of users to evaluate its performance and impact without affecting the entire user base. This approach helps identify potential issues in a real-world scenario while minimizing the risk of widespread disruptions.