Exploring the awesome Bicep Test Framework

Introduction

Bicep test framework is still in early developmental phase being an experimental feature. It was first presented by Bicep team on the August community call and peaked the interest from the community, myself included.

The goal of the test framework is a focus on client-side tests without the need to send off to ARM, this will help validate earlier in the CI/CD lifecycle and bring more confidence come the time of actual deployment.

Sounds great and I’m a huge fan of this direction.

How to enable the Bicep test framework?

Adding the two experimental flags in your bicepconfig.json file is required. This sits in your root directory, you can find out more from the Microsoft documentation here.

{
    "experimentalFeaturesEnabled": {
        "testFramework": true,
        "assertions": true
  }
}

Make sure your Bicep version is updated (Azure CLI):

az bicep upgrade
az bicep --version

Also make sure your Bicep VSCode extension is up to date.

Assertions

Assertion statements are Boolean expressions that help validate the expectations of the file, which we can then reference to a corresponding tests, file for example:

// Rios Engineer - Bicep Test Framework
targetScope = 'resourceGroup'

param location string
param kind string

// Storage Account
module storage 'br/public:storage/storage-account:3.0.1' = {
  name: 'riosengineerlongchar001'
  params:{
    kind: kind
    location: location
  }
}

// Test location is valid for UK regions
assert locationUk = contains(location, 'uk')

// SKU assert
assert sku = contains(kind, 'StorageV2')

We’re assessing that the storage code block returns a true or false values for each of our assertions listed above. In this example, we are testing that the region contains ‘uk’ (so either uksouth or ukwest regions), and that the kind returns a true value for StorageV2.

Assert statement syntax:

• assert keyword
• Name of your assertion statement, in our case locationUk
• Must be Boolean for this to work – as mentioned above

bicepparam file example:

using './main.bicep'

param location = 'northeurope'
param kind = 'StorageV2'

Test File

Now we’ve got some assertions defined we can run some tests against these to check their values represent what is intended.

Creating a tests.bicep file like the below to run the tests against:

// Rios Engineer - Bicep Test Framework
// Pass test 
test mainTest 'main.bicep' = {
  params: {
    kind: 'StorageV2'
    location: 'uksouth'
  }
}
// Fail test 
test mainTestFail 'main.bicep' = {
  params: {
    kind: 'BlobStorage'
    location: 'northeurope'
  }
}

Test syntax:

• test keyword
• Name of the test code block (e.g. mainTest in our example)
• Reference the file to run tests against
• Parameters object mocking from the main Bicep file

Running the test

We should have files like so now:

.
└── bicep/
    ├── main.bicep
    ├── main.bicepparams
    └── tests.bicep

Which means we can now run the tests cmdlet to review if it passes or fails the tests file we want to validate with.

bicep test .\filename.bicep

 ~  bicep test .\tests.bicep
WARNING: The following experimental Bicep features have been enabled: TestFramework. Experimental features should be enabled for testing purposes only. Do not enable these settings for any production usage, or you may be unexpectedly broken at any time!
[✓] Evaluation mainTest Passed!
[✗] Evaluation mainTestFail Failed at 2 / 3 assertions!
        [✗] Assertion locationUk failed!
        [✗] Assertion sku failed!
Evaluation Summary: Failure!
Total: 2 - Success: 1 - Skipped: 0 - Failed: 1

We can see the first test block passed successfully as the parameters being passed through all match the true values we want to test for.

However, as we can see the second test block fails because this test block looks for a different SKU type, which the parameter is not matching. The assertion returns a false value and this test fails. Awesome right?

Some limitations & observations

• You must pass in a parameter object, you cannot reference an existing parameter file currently
• Test blocks must be in a separate file from the .bicep template you want to test
• Test result output summary is not standardised
• strings are case sensitive, e.g. StorageV2 would pass, but storagev2 would not if you don’t match your param case sensitivity throughout

CICD Unit Testing

With the introduction to this new test framework we can now start to look at how this can be integrated to CI/CD pipeline flows, as another linting tool in the arsenal to validate code pre-deployment.

I hope with time the team provide support for some Sarif output so we can have some nice Azure DevOps reporting on the tests output for reviewing.

Conclusion

This is a fantastic step forward for Bicep, shifting left some unit testing and giving us another useful linting tool. However, as it’s still in experimental I expect things to change and more features to be added (I believe the file will move to .biceptest instead).

Yes it does potentially overlap what-if + Azure Policy but I think it’s a good way to help bring another useful validation & linting tool to the table for teams to validate their code during development and pre-deployment.

I think it’ll also further increase adoption of Bicep with more and more useful features like this.

Check out my blog on testing Bicep modules here: https://rios.engineer/bicep-modules-with-psrule-testing-documentation-ci-pipeline-examples/