Fixing ARM deployment errors for Linux disk encryption

When running ARM templates to deploy Linux with disk encryption on Azure I encountered a few errors. The errors where coming when I rerun the same template multiple times. In this post I explain the errors and how I fixed them.

Error: … is not a valid versioned Key Vault Secret URL
The first error I ran into was related to the return message of the encryption extension (.instanceView.statuses[0].message). The result message contains the Key Vault Sercet Url if the encryption is successful. When running the same template over and over again the following messages popped up:

"message": "{rn  "error": {rn    "code": "InvalidParameter",rn    "target": "encryptionSettings.diskEncryptionKey.secretUrl",rn    "message": "Encryption succeeded for all volumes is not a valid versioned Key Vault Secret URL. It should be in the format https://<vaultEndpoint>/secrets/<secretName>/<secretVersion>."rn  }rn}
"message": "{rn  "error": {rn    "code": "InvalidParameter",rn    "target": "encryptionSettings.diskEncryptionKey.secretUrl",rn    "message": "Encryption succeeded for data volumes is not a valid versioned Key Vault Secret URL. It should be in the format https://<vaultEndpoint>/secrets/<secretName>/<secretVersion>."rn  }rn}"

The cause of the message is that the return message is used for two different purposes:

1. returning the Key Vault Secret URL
2. returning verbose messages

The first time the extension runs the return value is the Key Vault Secret URL. Then when the encryption is finished the extension will return the verbose message that the Encryption succeeded. The verbose message will break the next ARM template that applies the encryption information to the VM configuration.

Idempotent provisioning and configuration
Creating an idempotent provisioning and configuration for provisioning will enable you to rerun your releases at any time. ARM Templates are idempotent. This means that every time they will be executed the result will be exactly the same. The configuration is set to what you have configured in your definitions. Because the definitions are declarative, you do not have to think about the steps on how to get there; the system will figure this out for you.
See Infrastructure as Code and VSTS

What I want the extension to do is give me the same result when I run the template multiple times (idempotent). Because of the verbose message I have to find a solution to be able to reapply my templates without having error messages about successful disk encryption. The solution turned to set the sequenceVersion parameter for the disk encryption extension to a new value every time you run the ARM template.
sequenceVersion: Sequence version of the BitLocker operation. Increment this version number every time a disk encryption operation is performed on the same VM (documentation)
Set the sequenceVersion
The documentation on the sequenceVersion suggests that it is a number, however it can be anything (as long different from the last string). Unfortunately you can’t generate a random string in a ARM template, that would break the idempotent property. To solve this I use a little trick to generate a unique string in my deployment template without adding an extra parameter. Each deployment in our VSTS pipeline get an unique name. I used that name to generate a unique string:

"value":"[uniqueString(resourceGroup().id, deployment().name)]"

The deployment name is different on each deployment (resource group deployment task VSTS), for example: xxxx-20161208-1436, this can be utilized to generate unique strings. The function deployment().name will return the name of the current template, so you have to be in the top level template when using linked templates and pass the it from there.
Error: Failed to get status file [Errno 2] No such file or directory: …
When you rerun the template with a new sequenceVersion before you have restarted the VM (when a restart is needed). You will get a new error message:

"message": "VM has reported a failure when processing extension 'AzureDiskEncryptionForLinux'. Error message: "Failed to get status file [Errno 2] No such file or directory: '/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999239/status/1.status'"

To solve this error, as the message imply, you need to restart the VM. Go to this post to see how to accomplish this.
Conclusion
After applying the unique string to the sequenceVersion, the deployments keep working. The only thing you have to be keen on is to restart you VMs when they are in the VMRestartPending state.