Introduction
I have been experimenting with layered Terraform or OpenTofu ephemeral environments for about 2 years.
Here is a brief introduction to a layered approach. Assume, you need a single EC2 VM for your ephemeral. Being “a good citizen”, you first set up a VPC, networking and other shared components. Then in the same script you create your VM. With that, every time you run the script, it creates a new VPC with the desired VM inside it. But is it really what you want?
If all you need is a VM, then the lifecycle for all those additional resources is wasteful. Particularly, it takes a lot of time to set up a VPC and similarly long to destroy it. It probably would be better to pre-create a VPC and then create my ephemeral VMs inside it.
So I ended up creating 2 scripts – which is a layered approach in a nutshell – the first layer creates a VPC, the second layers depends on the first one and creates a VM inside the VPC.
However, it turned out that the actual implementation was a little non-obvious because the second script needs to pull some data from the first one. Also we needed to allow for some ID randomization in order to not duplicate resources between runs for separate ephemerals. That required some new tooling.
Introducing REEF
I first assembled a very tightly coupled proprietary tool which ran a hard-coded to script to pre-create a VPC and then allowed for ID randomization in the second layer. That solved my existing problem for the day but was in no way scalable or extensible for the future.
The tool I built was operational and handling Reliza Hub ephemeral logic, when I discovered a similar project on Hacker News promoting layered approach to ephemerals. The project was called Layerform and it has been archived since, but the overall community response was positive.
Unlike Layerform designers, I believe that 2 layers are enough for any real-world scenario. There is no real reason to go into higher dimensionality of layers, which introduces much greater complexity.
Another big related project would be Crossplane – it has some aspects of layered approach, but makes everything revolving about Kubernetes APIs which in turn brings a lot of extra complexity – and that is hard to justify for me. Also, I was not trying to build a comprehensive infrastructure management project, the scope was solely to handle ephemerals in the best way possible.
After considering all the above, I decided to make my ephemeral project a separate Open Source entity – and this is how REEF (Reliza Ephemeral Framework) was born.
What REEF Can Do Now
REEF currently allows to spawn ephemeral infrastructure on AWS or Azure using tf templates stored in a Git repository. It is using OpenTofu by default as we don’t want to deal with recent Terraform licensing changes. However, it is possible to change this to Terraform if needed.
As discussed before, REEF implements 2-layered infrastructure approach. Base layer is called “Silo”. Actual ephemeral layer is called “Instance”. Any output variables from the Silo script are fed into the input variables of the Instance script. In example, a Silo layer can build an AWS VPC, and then an Instance layer can build a VM inside it using security group id and subnet id from the parent layer.
REEF has GraphQL API and UI – both UI and non-UI workflows are possible.
What are major limitations of REEF for now
REEF is currently lacking any user management, authentication or authorization functionality. We are first focusing on introducing those as part of Reliza Hub integration but will welcome any open source contributions in this regard.
REEF also does not do any management of resources within Kubernetes outside of tf scripts. Specifically, if your ephemeral lives in a namespace of an existing Kubernetes cluster – this is not what REEF is concerned with. We have recently introduced functionality to support it in Reliza Hub as part of Reliza CD – so it is unlikely we will be doing any work to introduce this separately in REEF. If you are looking for a fully open source implementation of this, it can be achieved with Argo CD – although I believe Reliza Hub / Reliza CD approach is much nicer.
Next Steps for REEF
For now, we mainly plan to add support for more cloud provider accounts to REEF and further work on improving integration between REEF and Reliza Hub.
Another point of development is improving interactivity and debug-ability of the tf scripts used by REEF.
Finally, community suggestions are always welcome and may shape decisions on the priorities.