Introduction
Bicep test framework is still in early developmental phase being an experimental feature. It was first presented by Bicep team on the August community call and peaked the interest from the community, myself included.
The goal of the test framework is a focus on client-side tests without the need to send off to ARM, this will help validate earlier in the CI/CD lifecycle and bring more confidence come the time of actual deployment.
Sounds great and I’m a huge fan of this direction.
How to enable the Bicep test framework?
As of 2024, it looks like the Bicep team are not proceeding with taking this out of the experimental phase. If you think this feature would be a good GA addition, please join the Bicep community calls and/or raise on the Bicep GitHub repo the feedback.
Adding the two experimental flags in your bicepconfig.json
file is required. This sits in your root directory, you can find out more from the Microsoft documentation here.
{
"experimentalFeaturesEnabled": {
"testFramework": true,
"assertions": true
}
}
JSONMake sure your Bicep version is updated (Azure CLI):
az bicep upgrade
az bicep --version
Also make sure your Bicep VSCode extension is up to date.
Assertions
Assertion statements are Boolean expressions that help validate the expectations of the file, which we can then reference to a corresponding tests
, file for example:
// Rios Engineer - Bicep Test Framework
targetScope = 'resourceGroup'
param location string
param kind string
// Storage Account
module storage 'br/public:storage/storage-account:3.0.1' = {
name: 'riosengineerlongchar001'
params:{
kind: kind
location: location
}
}
// Test location is valid for UK regions
assert locationUk = contains(location, 'uk')
// SKU assert
assert sku = contains(kind, 'StorageV2')
BICEPWe’re assessing that the storage
code block returns a true
or false
values for each of our assertions listed above. In this example, we are testing that the region contains ‘uk’ (so either uksouth or ukwest regions), and that the kind returns a true value for StorageV2
.
Assert statement syntax:
assert
keyword- Name of your assertion statement, in our case
locationUk
- Must be Boolean for this to work – as mentioned above
bicepparam file example:
using './main.bicep'
param location = 'northeurope'
param kind = 'StorageV2'
BICEPTest File
Now we’ve got some assertions defined we can run some tests against these to check their values represent what is intended.
Creating a tests.bicep
file like the below to run the tests against:
// Rios Engineer - Bicep Test Framework
// Pass test
test mainTest 'main.bicep' = {
params: {
kind: 'StorageV2'
location: 'uksouth'
}
}
// Fail test
test mainTestFail 'main.bicep' = {
params: {
kind: 'BlobStorage'
location: 'northeurope'
}
}
BICEPTest syntax:
test
keyword- Name of the test code block (e.g. mainTest in our example)
- Reference the file to run tests against
- Parameters object mocking from the main Bicep file
Running the test
We should have files like so now:
.
└── bicep/
├── main.bicep
├── main.bicepparams
└── tests.bicep
Which means we can now run the tests cmdlet to review if it passes or fails the tests file we want to validate with.
bicep test .\filename.bicep
~ bicep test .\tests.bicep
WARNING: The following experimental Bicep features have been enabled: TestFramework. Experimental features should be enabled for testing purposes only. Do not enable these settings for any production usage, or you may be unexpectedly broken at any time!
[✓] Evaluation mainTest Passed!
[✗] Evaluation mainTestFail Failed at 2 / 3 assertions!
[✗] Assertion locationUk failed!
[✗] Assertion sku failed!
Evaluation Summary: Failure!
Total: 2 - Success: 1 - Skipped: 0 - Failed: 1
BICEPWe can see the first test block passed successfully as the parameters being passed through all match the true values we want to test for.
However, as we can see the second test block fails because this test block looks for a different SKU type, which the parameter is not matching. The assertion returns a false value and this test fails. Awesome right?
Some limitations & observations
- You must pass in a parameter object, you cannot reference an existing parameter file currently
- Test blocks must be in a separate file from the .bicep template you want to test
- Test result output summary is not standardised
- strings are case sensitive, e.g. StorageV2 would pass, but storagev2 would not if you don’t match your param case sensitivity throughout
CICD Unit Testing
With the introduction to this new test framework we can now start to look at how this can be integrated to CI/CD pipeline flows, as another linting tool in the arsenal to validate code pre-deployment.
I hope with time the team provide support for some Sarif output so we can have some nice Azure DevOps reporting on the tests output for reviewing.
Conclusion
This is a fantastic step forward for Bicep, shifting left some unit testing and giving us another useful linting tool. However, as it’s still in experimental I expect things to change and more features to be added (I believe the file will move to .biceptest instead).
Yes it does potentially overlap what-if + Azure Policy but I think it’s a good way to help bring another useful validation & linting tool to the table for teams to validate their code during development and pre-deployment.
I think it’ll also further increase adoption of Bicep with more and more useful features like this.
Check out my blog on testing Bicep modules here: https://rios.engineer/bicep-modules-with-psrule-testing-documentation-ci-pipeline-examples/
Latest Posts
Eliminate old API versions in your Azure Bicep templates
Azure DevOps Agent: Azure Container Instance with Private Azure Container Registry
Compliance with Azure Bastion: Session recording