Published: Apr 4, 2023 by Isaac Johnson

Ran Cohen, CTO/Co-Founder of ConfigU asked me to take a look at their offering; ConfigU, “Configuration Management as a Service”. I’m always game to check out new DevOps, SCM and SRE tools and if there is a Community/Free tier, I’m even more excited to dig in and see it can play a part in my work streams.

Today, we’ll setup ConfigU and follow a basic “Getting Started” flow. We’ll expand it to try new “schemas” and play with their built-in webhooks. We’ll integrate it into a CICD Flow using Azure DevOps Pipelines and Token based auth. Lastly, we’ll look at the pros/cons and compare with what we might achieve in source (GIT) and using AKV.

Company background

ConfigU is a Tel Aviv based started founded in late 2020 or early 2021 (the oldest archive I see is from Sept 2021). It was started by Peleg Porat and Ran Cohen, Peleg starting it after being a Software Developer at F5 Networks. Ran then joined him having been a Full Stack Engineer at Tricentis Testim. ConfigU is a private company that’s raised at least $2.8m in seed funding from private investors and Cardumen Capital.

I saw official “we’re live” announcements on Peleg’s LI as recently as five months ago and he demoed at EISP’s 2021 Demo day in Oct of 2021.

I tried to get an idea where Ran and Peleg overlapped and best I can tell, the only past shared experience was in the IDF where they overlapped in Military Intelligence between 2014 and 2016.

Getting Started

We can click “Get Started” and signup with an IdP like Github, Google or Microsoft

The signup wizard lets us set a name and theme (I went with light)

I’ll set a few more details

then get dropped into a “Getting Started” page

The next steps come from following that guide:

$ curl https://cli.configu.com/install.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2420  100  2420    0     0  10852      0 --:--:-- --:--:-- --:--:-- 10852
This script requires superuser access.
You will be prompted for your password by sudo.
[sudo] password for builder:
Installing CLI from https://cli.configu.com/channels/stable/configu-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 79.8M  100 79.8M    0     0  30.3M      0  0:00:02  0:00:02 --:--:-- 30.3M
v18.15.0
configu installed to /usr/local/bin/configu
@configu/cli/0.8.0 wsl-x64 node-v18.15.0

I’ll init my workspace

✔ /home/builder/Workspaces/jekyll-blog/get-started.cfgu.json generated with 3 records

The login is actually pretty slick. It lets me use a token, or if set to user, pops up a browser window I can use to login right then

$ configu store upsert --type 'configu'
? select authentication type User
press any key to open up the browser to login or press ctrl-c to abort.
you should see the following code: 629 170 429. it expires in 15 minutes.:
? select organization Freshbrewed
ℹ configu upserted to cli configuration

I’ll set the config

$ configu upsert \
> --store 'configu' --set 'example' --schema './get-started.cfgu.json' \
> --config 'GREETING=hey' --config 'SUBJECT=Isaac Johnson'
ℹ configs upserted successfully

Then I can pull it back down to save in a local JSON

$ configu export \
--from 'store=configu;set=example;schema=./get-started.cfgu.json' \
--format 'JSON' \
> 'greeting.json'

$ cat greeting.json
{
  "MESSAGE": "hey, Isaac Johnson!",
  "SUBJECT": "Isaac Johnson",
  "GREETING": "hey"
}

In the WebUI, I can see settings I added

Installations

We did Linux without trouble. We can see there are plenty of other options

Let’s try Windows next

I created a quick directory in which to test

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\Users\isaac> mkdir ConfigUTest


    Directory: C:\Users\isaac


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----          4/2/2023  11:38 AM                ConfigUTest


PS C:\Users\isaac> cd .\ConfigUTest\
PS C:\Users\isaac\ConfigUTest>

It was a pretty slow install and left the admin PS window open. It didn’t seem to work in the source window - likely because this tool installs NodeJS 18 and then the app

I could get it working, however, in a fresh prompt

C:\Users\isaac\ConfigUTest>"C:\Program Files\configu\bin\configu" init --get-started
√ C:\Users\isaac\ConfigUTest\get-started.cfgu.json generated with 3 records

C:\Users\isaac\ConfigUTest>"C:\Program Files\configu\bin\configu" store upsert --type "configu"
? select authentication type User
press any key to open up the browser to login or press ctrl-c to abort.
you should see the following code: 422 306 180. it expires in 15 minutes.:
? select organization Freshbrewed
i configu upserted to cli configuration

Where I could export as well

C:\Users\isaac\ConfigUTest>"C:\Program Files\configu\bin\configu" export --from "store=configu;set=example;schema=./get-started.cfgu.json" --format "JSON" > greetings.json

C:\Users\isaac\ConfigUTest>type greetings.json
{
  "MESSAGE": "hey, Isaac Johnson!",
  "SUBJECT": "Isaac Johnson",
  "GREETING": "hey"
}

CICD

Here I was plesantly suprised how many CICD tooks they supported

Let’s try AzDO to start.

I’ll create a new Azure Pipeline

I’ll choose Azure Repos and pick a Repo to use. Then I’ll ask for a starter pipeline

At this point, you should see something similar

I’ll setup the pipeline to just download the config and show it

trigger:
- master

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    (curl https://cli.configu.com/install.sh || wget -qO- https://cli.configu.com/install.sh) | sudo sh
  displayName: Install Configu CLI
- script: |
      echo "Configu Export"
      export --from 'store=configu;set=example;schema=./get-started.cfgu.json' \
      --format 'JSON' > 'greeting.json'
  displayName: "Export ConfigU"
  env:
    CONFIGU_ORG: $(CONFIGU_ORG)
    CONFIGU_TOKEN: $(CONFIGU_TOKEN)

- script: |
    echo "Now show the config"
    cat ./greeting.json
  displayName: 'Show the Configu Config'

The two variables we need are CONFIGU_ORG and CONFIGU_TOKEN.

For the time being, we can just set that in the “Variable” section

Click “New Variable” and we can define the org (for me, that is “freshbrewed”)

Back in ConfigU, I’ll want to create a Token. I can do that in Settings/Tokens:

I’ll name it

Then it will show it for me to copy the one time

Back in Azure Pipelines, I’ll make that New Variable also a secret, but one that could also be overridden

We can now see both are listed

Lastly, I’ll save and run

I’ll use a new branch, and in my case, not start a PR

This kicks off a build

It didn’t like the export option

Maybe you caught the issue already - but I neglected to set the binary in the export command. I also missed a few other steps like init.

I fixed that, set a local store, and then updated trigger branch

trigger:
- azure-pipelines-configu

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    (curl https://cli.configu.com/install.sh || wget -qO- https://cli.configu.com/install.sh) | sudo sh
  displayName: "ConfigU: Install"

- script: |
      echo "Configu Init"
      configu init --get-started
  displayName: "ConfigU: Init"

- script: |
      set +x
      echo "Configu Store Set"
      configu store upsert --label "my-store" --connection-string "store=configu;org=$(CONFIGU_ORG);token=$(CONFIGU_TOKEN)"

      echo "Configu Store Use"
      configu export --from "store=my-store;set=example;schema=./get-started.cfgu.json" \
      --format 'JSON' > 'greeting.json'
  displayName: "ConfigU: Store Set and Export"

- script: |
    echo "Now show the config"
    cat ./greeting.json
  displayName: 'Show the Configu Config'

We can now see it works just dandy

Webhooks

I saw an “Integrations/Webhooks” area, but little documentation around usage.

I Created a new webhook

I used webhook.site to see what the payload would be

I ran a test

and saw this was sent

{
  "webhook": {
    "_id": "rrv6pajkN3acESPJW94hXxBdjN14h5dG",
    "contentType": "application/json",
    "createdAt": 1680458749836,
    "events": [],
    "isActive": true,
    "name": "InvokePipeline",
    "orgId": "freshbrewed",
    "updatedAt": 1680458749836,
    "url": "https://webhook.site/697160e1-c271-4f39-886a-2271b4b69a3a"
  },
  "entityId": "rrv6pajkN3acESPJW94hXxBdjN14h5dG",
  "text": "Webhook 'InvokePipeline' has been invoked"
}

I saw that the webhook fires on configu exports, such as when I ran the pipeline

And an update (upsert) looks like

{
  "webhook": {
    "_id": "rrv6pajkN3acESPJW94hXxBdjN14h5dG",
    "contentType": "application/json",
    "createdAt": 1680458749836,
    "events": [],
    "isActive": true,
    "name": "InvokePipeline",
    "orgId": "freshbrewed",
    "updatedAt": 1680458749836,
    "url": "https://webhook.site/697160e1-c271-4f39-886a-2271b4b69a3a"
  },
  "entityId": "UYyt2MFfuDtxJst5pCpD0IpbUsWmRnKn",
  "text": "Configs have been upserted to the following sets: example"
}

Schema

We can look at the ConfigU types to see some options

The Default “Hello World” Schema that was created at init looks like

$ cat get-started.cfgu.json
{
  "GREETING": {
    "type": "RegEx",
    "pattern": "^(hello|hey|welcome|salute|bonjour)$",
    "default": "hello"
  },
  "SUBJECT": {
    "type": "String",
    "default": "world"
  },
  "MESSAGE": {
    "type": "String",
    "template": ", !",
    "description": "Generates a full greeting message"
  }
}

We can see that take effect if we try and push up a value outside the schema:

$ configu upsert --store 'configu' --set 'example' --schema './get-started.cfgu.json' --config 'GREETING=wassup' --config 'SUBJECT=Isaac Johnson'
    Error: invalid config value "wassup" for key "GREETING" at UpsertCommand > run, value "wassup" must be of type "RegEx"

Let’s create a different config and use it to see an example

Here I’ll create a JSON around football:

$ cat NFL.cfgu.json
{
  "BESTTEAM": {
    "type": "RegEx",
    "pattern": "^(vikings|packers)$",
    "default": "vikings"
  },
  "FANLOYALTY": {
    "type": "Number",
    "default": "1"
  },
  "COUNTRY": {
    "type": "Country",
    "default": "US",
    "description": "Country of Origin"
  }
}

Now if we tried to set a config with say, the Chargers

$ configu upsert --store 'configu' --set 'realfootball' --schema './NFL.cfgu.json' --config 'BESTTEAM=chargers'
    Error: invalid config value "chargers" for key "BESTTEAM" at UpsertCommand > run, value "chargers" must be of type "RegEx"

It would fail, both because the Regular Expression failed the test and the Chargers are dead to me after they left San Diego.

But if we set it to my Vikings

$ configu upsert --store 'configu' --set 'realfootball' --schema './NFL.cfgu.json' --config 'BESTTEAM=vikings'
ℹ configs upserted successfully

We can see it works.

We can also see that it’s been added in the ConfigU UI

If I wish to edit the cfgu JSON, I can use the editor in the Web UI

Which I could use to test the schema validation

We can see a few small limitations here.

For one, if I update a value, there is no history stored on the configs, at least as exposed in the UI

$ configu upsert --store 'configu' --set 'realfootball' --schema './NFL.cfgu.json' --config 'BESTTEAM=packers'
ℹ configs upserted successfully

I can really only see the field was updated by me at a time

with a resulting webhook

{
  "webhook": {
    "_id": "rrv6pajkN3acESPJW94hXxBdjN14h5dG",
    "contentType": "application/json",
    "createdAt": 1680458749836,
    "events": [],
    "isActive": true,
    "name": "InvokePipeline",
    "orgId": "freshbrewed",
    "updatedAt": 1680458749836,
    "url": "https://webhook.site/697160e1-c271-4f39-886a-2271b4b69a3a"
  },
  "entityId": "6SLUuVSxzKKmcjtomegH14s6Cq9K23wE",
  "text": "Configs have been upserted to the following sets: realfootball"
}

ConfigU also would not be a good place to put secrets.

If I were to add a password field, for instance

$ cat NFL.cfgu.json
{
  "BESTTEAM": {
    "type": "RegEx",
    "pattern": "^(vikings|packers)$",
    "default": "vikings"
  },
  "FANLOYALTY": {
    "type": "Number",
    "default": "1"
  },
  "COUNTRY": {
    "type": "Country",
    "default": "US",
    "description": "Country of Origin"
  },
  "PASSWORD": {
    "type": "String"
  }
}

And then use it

$ configu upsert --store 'configu' --set 'realfootball' --schema './NFL.cfgu.json' --config 'PASSWORD=deflategate'
ℹ configs upserted successfully

It would be plain text in the UI

This isn’t a deal breaker, however.

We could make a reference to a Key Vault, for instance, and fetch it in a second pass with a pipeline;

$ configu upsert --store 'configu' --set 'realfootball' --schema './NFL.cfgu.json' --config 'PASSWORD=akv:mysecret'
ℹ configs upserted successfully

Then fetch it

$ configu export --from "store=my-store;set=realfootball;schema=./NFL.cfgu.json" --format "JSON"
{
  "PASSWORD": "akv:mysecret",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "packers"
}

So let’s say I want to use a secret in AKV

I could make a reference to it in the config

$ configu upsert --store 'configu' --set 'realfootball' --schema './NFL.cfgu.json' --config 'PASSWORD=akv:idjtestsecret'
ℹ configs upserted successfully

Then in use, I could export the config

$ configu export --from "store=my-store;set=realfootball;schema=./NFL.cfgu.json" --format "JSON" > t.json
$ cat t.json
{
  "PASSWORD": "akv:idjtestsecret",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "packers"
}

Then use something like a perl script

$ cat t.pl
#!/usr/bin/perl
#

my $filen = $ARGV[0];

open(FILEH,$filen);
@filec = <FILEH>;
close(FILEH);

foreach my $line (@filec)
{
   if ($line =~ /^(.*)"akv:([^"]*)"/)
   {
           my $prefix=$1;
           my $newValue=`az keyvault secret show --name $2 --vault-name idjakv -o json | jq -r .value | tr -d '\n'`;

           if ($line =~ /,$/)
           {
               print "$prefix\"$newValue\",\n";
           } else {
               print "$prefix\"$newValue\"\n";
           }
   }
   else
   {
           print $line;
   }
}

Now I could use it to fetch that on the fly in the config

$ perl ./t.pl t.json
{
  "PASSWORD": "this is a test 2",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "packers"
}

Plans

I can save up to 50 “configs” (I might call them just settings) and have up to 5 users in the free community edition.

I think that’s a pretty reasonable start to kick the tires

To upgrade, it is a conversation. There is no specified pricing anywhere on the site.

Reaching out to ConfigU, I was given the range of a “a few dozen to a few hundred dollars per month”, depending on usage. The Enterprise plan would then extend beyond that.

Alternate approaches

This presents a bit of a quandary on config storage.

EVEN if I desired to separate config, why would I externalize it?

That is, if my config is arguably non-secret data (or references to secret data), then why not just check it in to source?

$ cat dev.json
{
  "PASSWORD": "akv:dev-idjtestsecret",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "packers"
}
$ cat qa.json
{
  "PASSWORD": "akv:qa-idjtestsecret",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "vikings"
}
$ cat prod.json
{
  "PASSWORD": "akv:prod-idjtestsecret",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "packers"
}

Alternatively, I could have “settings.json”, but on different branches in source (with different values). This would allow merging between and branch rules on update;

builder@DESKTOP-72D2D9T:~/Workspaces/tfAnsibleAzure$ cat settings.json
{
  "PASSWORD": "akv:idjtestsecret",
  "COUNTRY": "US",
  "FANLOYALTY": "1",
  "BESTTEAM": "packers"
}
builder@DESKTOP-72D2D9T:~/Workspaces/tfAnsibleAzure$ git checkout qa-main
Switched to branch 'qa-main'
builder@DESKTOP-72D2D9T:~/Workspaces/tfAnsibleAzure$ cat settings.json
{
  "PASSWORD": "akv:qa-idjtestsecret",
  "COUNTRY": "US",
  "FANLOYALTY": "2",
  "BESTTEAM": "packers"
}

We could store all our config and secrets in AKV and prefix them by environment to keep them seperate

- task: AzureKeyVault@2
  inputs:
    azureSubscription: 'Pay-As-You-Go'
    KeyVaultName: 'idjakv'
    SecretsFilter: 'qa-*'
    RunAsPreJob: false

Lastly, in a similar fashion to source, we could always make an AKV per environment as our Access Policies are at the Key Vault level and bring them in at each stage

- task: AzureKeyVault@2
  inputs:
    azureSubscription: 'Pay-As-You-Go'
    KeyVaultName: 'settings-qa'
    SecretsFilter: '*'
    RunAsPreJob: false

Which could substitute values in a file with the FileTransformer

# Update appsettings.json via FileTransform task.
- task: FileTransform@1
  displayName: 'File transformation: settings.json'
  inputs:
    folderPath: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    targetFiles: '**/settings.json'
    fileType: json

Or the often used replace tokens task

- task: qetza.replacetokens.replacetokens-task.replacetokens@3
  displayName: 'Replace tokens'
  inputs:
    targetFiles: |
      **/*.config
      **/*.json => outputs/*.json

I know this is an Azure specific implementation, but the end results are that my secrets are entirely under my control in my own subscription. I don’t have to worry about access controls on a new vendor. I also have variable history - I know what changed, when, and by whom

I can also set metadata tags, descriptions (notes) and expiration dates, if that affects our data per-secret

Summary

I’m always cautious on new tooling that offers SaaS only and no obvious way to dump all my data. It’s not that vendor lock-in is always bad, but I like to know my options. Arguably you can view your full Configs on a page so one can always screen-scrape.

I also have a general bent against companies with opaque pricing. However, in this case, the offering is new enough they likely are working out what the specifics on prices ought to be.

As far as new products go, the documentation is great. They cover multiple deployment types, tools and languages with pretty easy to follow guides

e.g. Looking at Schema and Keys:

Additionally, not every company can easily use cloud services so having something not married to Azure, AWS or GCP might be valuable in specific situations.

Overall, I found ConfigU to be an interesting offering with a fairly fast free tier and worthy of consideration.