Getting Started with Acorn for Kubernetes

In early August 2022, Acorn has been released into the Open Source world by Acorn Labs. It is still in beta, but the Acorn team is heavily working on the 1.0 milestone.

Acorn itself is an abstraction on top of Kubernetes to improve packaging and deployments of apps to k8s. The development team signalized that more backends than just Kubernetes might be introduced in the future.

But before introducing another tool into our daily life, let's step back and have a look.

The Problem

The world of Kubernetes is not perfect and there are many challenges to solve. From Acorn's point of view there are mostly three problem areas: the deployment phase, different environments and the complexity.

First, let's examine on the deployment phase. Whenever we want to deploy an application into our k8s cluster, we need to apply all our YAML resource files in a specific order. For example we need to make sure to start by creating all namespaces followed by config maps, secrets, persistent volume claims and many more.

If we remove resources, we also need to eventually delete unused resources like load balancers, deployments, ingresses from production once they are not needed any more. Otherwise we would simply waste node capacity and ultimately money. What we don't want to delete necessarily are persistent volume claims for example: we might want to store a backup first.

Once we decided how to apply our deployments we hit the next bummer: different environments. We have our production and staging systems. Of course we try to keep them as similar as possible, but nonetheless they still differ. They have a different ingress configuration to reflect the different domains, different secrets and we even change some hand-selected configs like changing the main color to provide a visual feedback to our testers. And as there is barely no traffic on staging compared to production, our nodes are less, smaller and cheaper.

But if we have a look on the development environment, the world becomes much more complex. Here we really want support for debugging tools and a fast live reload, especially for our frontend developers. In practice there are many approaches to tackle this: especially mobile app developers tend to work on your staging system or on a separate development system. Most other developers need your services to be running locally. Many teams either highly customize the local cluster or fall back to docker-compose. Both resulting in lots of work.

This situation is also a hint on the next pain point: Kubernetes is a complex tool to solve a complex problem. Especially beginners struggle with understanding the problem domain and the solution to that. But even for more experienced k8s experts, this complexity doesn't vanish. Most challenges have been solved by another layer of indirection. You always need to handle that, even in cases where a simpler solution would be sufficient.

Acorn's Approach

Now that we know the pain points Acorn tries to solve, let's dive into its approach.

To reduce the complexity from the beginning, Acorn is specifically designed to address the needs of standard applications. Acorn applications are not able to dynamically spawn new deployments like a CI/CD-pipeline would need to. This focus allows more expressiveness and convenience while still being able to scale your services up - unlike the restricted docker-compose or the verbose Kubernetes YAML file resources.

Acorn also forces you to declare your whole application within one single file: the Acornfile. As you will see, it is easily readable and even less experienced users can implement small changes. And everything is centralized. No need to keep dozens of resources in mind.

Once you compile your Acornfile you will receive one single image called the Acorn Image. Beside the resources it references all container images, that are being used within the Acornfile. So if you push your Acorn Image to a registry of your choice, it will also push all missing images, too. With one push you have all of them. You do only need to set up the credentials and firewall rules to one single OCI container registry!

To tackle multiple environments, Acorn replaces the use of the markup language YAML for the configuration by a configuration language based on CUE. In contrast to YAML it supports if statements, for loops and much more. You are even able to pass arguments to your Acornfile to further customize your application. And all that is being type-checked. You will actually get compile errors on invalid configurations!

Prerequisites

Let's get started by ticking all the boxes Acorn requires. First of all, you need to install the Acorn CLI along with an installed default ingress controller and a default storage class.

We will start by creating a local development Kubernetes cluster. There are many different alternatives.

Docker Desktop

Using Docker Desktop to create your local development cluster is the easiest way to get started. It is available for Windows, Mac and Linux. You can download it here.

Enable the Kubernetes feature in the Docker Desktop settings and you are ready to go. Acorn will install missing components for you automatically.

Minikube

If you are using minikube, you need to enable the nginx ingress controller by running the following command:

minikube addons enable ingress

In order to access your local services, you also need to tunnel the traffic using:

minikube tunnel

Kind

If you prefer using kind, you need to add extraPortMappings when creating your cluster:

❯ cat <<EOF | kind create cluster --name acorn-example --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF

Make sure it works by running:

kubectl cluster-info --context kind-acorn-example
kubectl get pods -A

If everything is up and running, install the nginx ingress controller:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

As this might take a while, you can check the status by running:

kubectl wait --namespace ingress-nginx \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/component=controller \
  --timeout=90s

Is everything up and running? Great!

Installing Acorn Into Your Cluster

Now that we prepared our cluster, we can install Acorn itself by a simple:

acorn install

And that's it! In practice there are many options you can pass during the installation as changing the default ingress with --ingress-class-name and many more.

Preparing the Project

But before getting started with migrating an application, we need to actually need to create an example project. For the sake of simplicity, you can clone the repository from my KCD Munich 2022 talk about Acorn.

git clone https://github.com/vknabel/kcd-munich-2022-acorn.git

Let's have a look at the project itself: it is a standard Rails application with a Postgres database. I also wrote the application specific Dockerfile we will use later. We also made a few changes to whitelist our development mode domain name and for being able to pass the database credentials via environment variables:

• DATABASE_HOST
• DATABASE_NAME
• DATABASE_USERNAME
• DATABASE_PASSWORD

If you want to have a look at the changes, feel free to scan through the README.md file and the project sources.

The Acornfile

Now that we have Acorn installed up and running on our development cluster, we are ready to start creating our very first Acornfile and app. The Acornfile consists of multiple sections and we will have a look on some of these step by step.

If you are using Visual Studio Code, you can install the Acorn extension to get syntax highlighting and auto-completion.

First we are going to declare our Ruby on Rails application server in the containers section.

// Acornfile

containers: {
    server: {
        build: {
            context: "."
        }
        dependsOn: ["db"]
        ports: publish: "3000/http"

        env: {
            DATABASE_HOST: "db"
        }
    }
}

As you can see we declare a new container named server and we want to build the container image in the context of the current folder that contains the Acornfile and in this case also the Dockerfile. As our server requires our database, we add a dependsOn and decide to access our Postgres through db.

Now let's have a closer look on the ports. In Acorn you always need to declare all ports. If you don't the ports won't be accessible by any participant. There are three different scopes for ports. Here our scope is publish and is available from outside the cluster, expose-ports can be accessed cluster-wide and the default internal restricts port visibility to the Acorn image.

In CUE, you can abbreviate nested key/value-pairs with only a single field by dropping the curly braces. Hence ports: publish: "3000/http" is equal to ports: { publish: "3000/http" }.

In our case, Ruby on Rails listens by default on port 3000 and we want our server to be accessible through a custom domain via HTTP and eventually including an HTTPS certificate. So as port we pick 3000/http. Under the hood this will create ingress rules for us.

In the end we pass the database host db as environment variable to our container.

Next we want to add our Postgres database db:

// Acornfile

constainers: {
    // server: ...

    db: {
        image: "postgres"
        ports: "5432/tcp"

        dirs: {
            // mount volume to directory
            "/var/lib/postgresql/data": "volume://pg-data"
        }
    }
}

// declare volumes
volumes: {
    "pg-data": {}
}

In this case we want to use a prebuilt image of postgres from the Docker Hub. We want our database to only be accessible from within our Acorn image and declare the port as an internal 5432/tcp port. Picking the tcp suffix will not create an ingress rule for use.

As we don't want to loose our precious data on restarts or updates, we need to mount a volume to the default Postgres data directory /var/lib/postgresql/data. We reference the volume using a volume://-scheme URL followed by its name pg-data. And thereafter we configure it to stick to the default storage class of our Kubernetes cluster.

Now that we have both containers in place and the server knows how to access our database, we are only missing the secrets. Let's create those.

// Acornfile

// containers, volumes...

secrets: {
    "db-name": type: "token"
    "db-username": type: "token"
    "db-password": type: "token"
}

Here in the secrets section we create three secrets db-name, db-username and db-password. All three of them are secrets of type token. When running your Acorn image on your cluster for the first time with these secrets, Acorn will automatically create random character sequences for you. So your secrets are different than mine. This feature is great as no insecure passwords are getting introduced and there is no opportunity for developers to commit secret information into version control.

If you are migrating from an existing container already running in production, with acorn secret create you can manually create secrets and pass them to your Acorn image. With acorn secret expose you can read the secret value from your terminal.

Now, let's pass our secrets to our containers:

// Acornfile

containers: {
    server: {
        //...
        env: {
            DATABASE_HOST: "db"
            DATABASE_NAME: "secret://db-name/token"
            DATABASE_USERNAME: "secret://db-username/token"
            DATABASE_PASSWORD: "secret://db-password/token"
        }
    }
    db: {
        //...
        env: {
            POSTGRES_DB: "secret://db-name/token"
            POSTGRES_USER: "secret://db-username/token"
            POSTGRES_PASSWORD: "secret://db-password/token"
        }
    }
}

Similarly to using volumes, you can reference secrets with the secret://-url-scheme, followed by the name. As we are using a token secret, we also need to unwrap it by appending /token. Now we are ready to run our application!

We simply execute acorn run . to build the Acorn image from our Acornfile, push it to our cluster and run the Acorn image.

Tip: You can run these steps independently with acorn build, acorn push and acorn run.

❯ acorn run .
[+] Building 1.6s (5/5) FINISHED
[+] Building 40.6s (10/10) FINISHED
[+] Building 0.5s (5/5) FINISHED
lingering-cherry
STATUS: ENDPOINTS[] HEALTHY[] UPTODATE[]
STATUS: ENDPOINTS[] HEALTHY[0] UPTODATE[0] pending
STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[0/1] UPTODATE[1] [containers: db ContainerCreating; lingering-cherry is not ready]
STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[0/1] UPTODATE[1] [containers: lingering-cherry is not ready]
STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[1] UPTODATE[1] pending
STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[1/2] UPTODATE[2] [containers: lingering-cherry is not ready; server ContainerCreating]
STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[1/2] UPTODATE[2] [containers: lingering-cherry is not ready]
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
| STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[2] UPTODATE[2] OK |
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
STATUS: ENDPOINTS[http://server-lingering-cherry-1eeeca59.local.on-acorn.io => server:3000] HEALTHY[2] UPTODATE[2] OK

If you have a closer look on the logs, you can spot a name: in this case lingering-cherry. As we did not name our application, Acorn generated a random name for us! Later below, we see that Kubernetes will spin up the Postgres db container and wait for it to become healthy before it starts creating our server container.

It will also print us an url endpoint for our HTTP server. Acorn provides a DNS name server to loop back to your local machine. So you can access your application from your browser by visiting the printed URL. Though, depending on your local network setup and settings of your local machine, this doesn't work. In this case you can simply add the domain name to your /etc/hosts file.

127.0.0.1       server-lingering-cherry-1eeeca59.local.on-acorn.io

Once you open the link, you should see the rails logo and no errors in your console. Great!

How Does Acorn Work?

First of all, we have the Acorn CLI locally installed on our dev machine. Through it we access, control and prepare our Kubernetes cluster. Similar to the kubectl command, it relies on the same $KUBECONFIG. It provides commands to build, push and run our Acorn image. But you are also able to gather the logs of your application, get a shell into your container or even delete your application.

The Acorn CLI talks to the Acorn API Server of our cluster. It has many responsibilities like observing the build process of images. When trigggered by a request of our CLI, it will generate Acorn custom resources to represent the deployment of our applications.

The Acorn Controller observes these generated custom resources and converts these into standard k8s resources like pods, deployments, services, persistent volume claims and secrets.

The Buildkit and the Internal Registry are currently packed together in one pod. The buildkit is responsible for building our Acorn image, while the Internal Registry acts as a cache for Acorn images.

If you want to gain more insight into this, have a look in the Ten-Thousand Foot View on docs.acorn.io.

So what actually happened here? We told the API Server to observe the build of our Acornfile in the Buildkit through our CLI. Once the build finished, the Acorn image was pushed to our Internal Registry and the API Server generated some Acorn resources to represent the configuration of our Rails application and our Postgres database. Once the controller spotted these new resources, it converted them into standard k8s resources and deployed them to our cluster.

In the end, the API Server reported all this information back to our CLI and we were able to access our application.

Local Development

Now we were able to deploy and briefly understood the workflow of Acorn. But what about local development? How can we work on our application and see the changes immediately?

For that purpose, Acorn provides a specific flag when running our Acorn images: --dev or -i in short. Whenever you change your Acornfile, it will automatically rebuild your Acorn image, push and run it to your cluster. This command line flag will also be passed to our Acornfile and can be used to configure our application for development.

Our goal is to mount our local source code into our container in development mode to gain a much faster development experience. For this we will make use of a language feature of Cue that JSON or YAML wouldn't be able to provide: if statements.

// Acornfile

containers: {
    server: {
        //...

        if args.dev {
            dirs: {
                "/app": "./"
            }
        }
    }

    //...
}

//...

Paraphrased, if the --dev flag is passed to the Acorn CLI, we want to mount the local directory ./ into the container directory /app.

Let's start our application with the --dev flag:

acorn run --dev .

Now poke around your application and change some files. Within your console, you should see in the logs that files are copied around. If you reload the website, you should see your changes immediately.

Let's summarize what we accomplished here:

• When we change our Acornfile, Acorn will automatically rebuild and redeploy changed parts of our application.
• Once we change our Dockerfile, Acorn will automatically rebuild and redeploy this container.
• All file changes of our application will be copied into the container. As most Ruby files will always be re-evaluated, we don't need to restart our application.
• Only some changes like adding dependencies to the Gemfile don't trigger a reinstall.

In case of the Gemfile or when using servers written in NodeJS for example, we need to pay special attention. Here we would be required to replace the entry command of our container with a watch task, which automatically install dependencies or restart our application. When having a look on compiled languages like Go, we typically use multistage builds to strip development tools from our final production image. To improve our development experience we would swap the container image to the previous stage and recompile our application with every code change. In Go we could use gow for this.

Summary

Now we had a brief look on how Acorn works and how we can use it to deploy our Rails application to Kubernetes. We also saw how we can use the --dev flag to improve our development experience. We saw how powerful Acorn can be and how it can help us from the development phase to the production phase. And the Acornfile is much more approachable than a Kubernetes manifest. These benefits are possible from the decision to build Acornfiles on top of the Cue language and to focus on applications rather than services.

In the end, I'd like to drop you some additional reading materials that might help you finding out wether Acorn fits your needs:

• GitOps: acorn install and acorn run have a flag -o yaml which prints the Acorn custom resources instead of applying them. So you can connect them with your GitOps workflow. There is an Online Meetup recording on Youtube.
• Beside the args.dev argument, you can also define custom arguments in your Acornfile. And with profiles, you can override these arguments for different environments. Have a look at the docs for more information.
• You can easily scale containers if they are non-stateful. For stateful services, there is also an Online Meetup recording on Youtube.
• Acorn can automatically generate TLS certificates using Let's Encrypt. But you are always in control of the certificates. Have a look at the docs for more information.
• Even ingress routing based on paths is possible.
• You can create jobs, sidecars and liveness probes
• To bridge the gap between Kubernetes and Acorn, you have control over the labels
• I.e. in production, you can replace specific containers like Postgres with a production ready version, centrally created in a separate Acorn app and then link both apps together.

I hope you enjoyed this article. If you have any questions, feel free to reach out to me using the linked contact information above.