Monorepos: How we do it with Reliza Hub

Monorepo is a software structure which assumes multiple projects in the same repository.

What is good about monorepos? Developers work with only one repository and don’t have to remember or care about multiple repositories for the product. Essentially, everything is in one place – that’s the primary goal.

It is also assumed that monorepo is releasable from a single commit. In other words, a single git commit hash provides a snapshot of the whole product.

What is bad about monorepos? Mainly, the fact that builds are messy. Modern CI strategies assume building on every push. However, rebuilding everything in the monorepo on every push becomes prohibitively expensive. So the problem becomes to identify what components need to be built when.

Existing tooling, such as Lerna, is usually only catered towards node.js based projects. Below I will describe how we are using Reliza Hub to resolve build logic in the monorepo context regardless of the type of the project.

Sample Monorepo

I’m going to describe our sample monorepo project – which is our Dockerfile collection used to build various utilities we are using.

The reason we decided to use monorepo for this project is because the footprint of each component is small and it would be wasteful to use polyrepo strategy here.

Set up on Reliza Hub

On Reliza Hub we define separate project per each component, but just a single Git repository for all of them. Refer to my video tutorial of Reliza Hub to see how this can be set up.

Also the same tutorial shows steps to set up GitHub Actions link to Reliza Hub. Note that we can either define separate credentials for each project or define global read-write key and in that case specific project UUID when engaging Reliza CLI from GitHub Actions.

Our sample repository is using separate project credentials for every project.

Resolving What Projects To Build

Now comes the core part of this post: how we know which projects to build?

The idea is the following – we need to do a diff on the content of each project directory and only build if this diff contains something. To do the actual diff we have git diff command handy. The only question is how to figure out which commits to diff.

Here we are going to leverage Reliza Hub. Since on every build Reliza Hub gets metadata about associated commit of that build, we can query Reliza Hub for that information.

Note, that an alternative way (without using Reliza Hub) would be to store that information in the git itself. However, this would require additional commit (or at least tag), but it quickly becomes messy.

Actual Implementation

So here is the actual code we use to resolve last known build. For complete working sample refer to the GitHub Actions build file in our Dockerfile repository:

        dir=maven-postgresql
        dobuild=false
        last_commit=$(docker run --rm relizaio/reliza-cli getlatestrelease -i ${{ secrets.RELIZA_MAVEN_POSTGRESQL_API_ID }} -k ${{ secrets.RELIZA_MAVEN_POSTGRESQL_API_KEY }} --branch $GITHUB_REF | jq -r ".sourceCodeEntryDetails.commit")
        if [ ! -z "$last_commit" ]
        then
          echo -n "--commits $(git log $last_commit..$GITHUB_SHA --date=iso-strict --pretty='%H|||%ad|||%s' -- $dir | base64 -w 0) " > commit_list
          difflines=$(git diff $last_commit..$GITHUB_SHA $dir | wc -l)
          if [ "$difflines" != "0" ]
          then
            dobuild=true
          fi
        else
          dobuild=true
        fi
        if [ "$dobuild" == "true" ]
        then
### Actual Build Logic Goes Here ###

Let us dissect this code. Essentially, we first define directory of our project. In the above case this would be maven-postgresql.

Then we initialize dobuild variable as false. This variable would control whether we actually need to build this specific project.

Next we retrieve last known release details from Reliza Hub via Reliza CLI using getlatestrelease command. Then we extract latest commit from the response.

If latest commit is not found, it automatically means we need to do the build.

If latest commit is found, we do git diff between latest commit and current commit over project directory. If the diff returns something, it means that the difference exists and we proceed with the build.

Essentially, this solves the problem of only building monorepo components which have changed. This also works for any project types, not only node.js and does not require any node.js dependencies.