To make terraform truly easy to use for developers, and eliminate the need for developers to have any Terraform knowledge, we created an abstraction layer, by defining Terraform modules. These introduce governance by the person who creates the module, through locking certain input values, and making others configurable (within some limits) by the users who will attach the Terraform modules to the environments.

We also adhere to GitOps practices, by updating the module within Bunnyshell whenever Git changes occur. Environments are able to auto-update Terraform resources (by choice), and offer protection against incompatible changes made in Git - compared to the configuration you defined for the module when you attached it. This prevents environments from becoming unusable through updates, even if auto-update is turned on.

Storing dynamic values is also something that is pretty straightforward, but you still need to store all of these encrypted, and keep some obfuscated from the eyes of the UI and API users. This involves creating and maintaining a highly-available infrastructure for encryption/decryption.

You may already have a pipeline building your images, but an image build system that is designed for a large volume of environments requires different considerations.

Handling updates for a fixed (and reduced) number of images is different from creating & deleting + updating a variable (and considerable) number of environments.

Keeping track of environments' state - components and additional resources, created through Terraform - as well as parallelizing the create/update operations can quickly become complex, as many situations may arise by combining various scenarios.

Generating, storing and injecting secrets into environments is also a must, in order to achieve full automation when creating.

Applications are usually linked by setting environment variables which point to the various domains used by the environment’s services.

Creating a build for every service, for every deployment is time- and resource-consuming.

To reduce deployment time you need to create some form of a cache system, to skip builds for which you already have images, and to build only images for components which have indeed changed.

Running builds in parallel will greatly decrease the deployment time, and is also something that is a definitive must.

Once your environments are deployed, you will need to create DNS records for them and configure Ingress routing rules, so their exposed services can be accessed. Furthermore, you will need to handle SSL certificates, by either allowing wildcard certificates or generating individual ones.



Renewing certificates might also be a challenge, depending on the solution you choose to implement.

When having sustained activity, you will probably need to scale and be able to run multiple Terraform processes for different environments in parallel.

To reduce the time needed to run Terraform, you would need to implement a distributed caching system for modules. Also, you would need to build a fully async process for both apply and destroy. We handle this by creating pods in a K8s cluster, and have them notify our platform of the end result.

Also, versioning the state will become mandatory, to be able to trace & possibly revert (unintended) changes.

Maybe you already have some of these pieces built into your pipelines. And you clone pipelines to create new environments. But this becomes messy to update & maintain really fast when going at scale, not to say propagating it towards already existing environments.

To ensure that users operate securely with our platform and API, we have HTTPS enforced for all public-facing network communication. Every time you communicate with the Bunnyshell - - platform or APIs, you will be redirected through a secure connection using HTTPS.
Authentication is done using JWT and OAuth 2.0.
All communication with internal services is done through a secured VPN connection.