How To Execute PowerShell And Bash Scripts In Terraform

The first thing to know is what Terraform expects of the scripts it executes. It does not work with regular command line parameters and return codes. Instead, it passes a JSON structure via the script’s standard input (stdin) and expects a JSON structure on the standard output (stdout) stream.

The Terraform documentation already contains a working example with explanations for Bash scripts.

#!/bin/bash
set -e

eval "$(jq -r '@sh "FOO=\(.foo) BAZ=\(.baz)"')"

FOOBAZ="$FOO $BAZ"
jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'

I will replicate this functionality for PowerShell on Windows and combine it with the OS detection from my other blog post.

The trick is handling the input. There is a specific way, since Terraform calls your script through PowerShell, something like this echo '{"key": "value"}' | powershell.exe script.ps1.

$json = [Console]::In.ReadLine() | ConvertFrom-Json

$foobaz = @{foobaz = "$($json.foo) $($json.baz)"}
Write-Output $foobaz | ConvertTo-Json

You access the C# Console class’ In property representing the standard input and read a line to get the data Terraform passes through PowerShell to the script. From there, it is all just regular PowerShell. The caveat is that you can no longer call your script as usual. If you want to test it on the command line, you must type the cumbersome command I have shown earlier.

echo '{"json": "object"}' | powershell.exe script.ps1

Depending on how often you work with PowerShell scripts, you may bump into its execution policy restrictions when Terraform attempts to run the script.

│ Error: External Program Execution Failed
│
│   with data.external.script,
│   on main.tf line 8, in data "external" "script":
│    8:   program = [
│    9:     local.shell_name, "${path.module}/${local.script_name}"
│   10:   ]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ Program: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
│ Error Message: ./ps-script.ps1 : File
│ C:\Apps\Terraform-Run-PowerShell-And-Bash-Scripts\ps-script.ps1
│ cannot be loaded because running scripts is disabled on this system. For more information, see
│ about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
│ At line:1 char:1
│ + ./ps-script.ps1
│ + ~~~~~~~~~~~~~~~
│     + CategoryInfo          : SecurityError: (:) [], PSSecurityException
│     + FullyQualifiedErrorId : UnauthorizedAccess
│
│ State: exit status 1

You can solve this problem by adjusting the execution policy accordingly. The quick and dirty way is to allow all scripts as is the default on non-Windows PowerShell installations. Run the following as Administrator.

Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope LocalMachine

This is good enough for testing and your own use. If you regularly execute scripts that are not your own, you should choose a narrower permission level or consider signing your scripts.

Another potential pitfall is the version of PowerShell in which you set the execution policy. I use PowerShell 7 by default but still encountered the error after applying the unrestricted policy. That is because the version executed by Terraform is 5. That is what Windows starts when you type powershell.exe in a terminal.

PowerShell 7.4.1
PS C:\Users\lober> Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope LocalMachine
PS C:\Users\lober> Get-ExecutionPolicy
Unrestricted
PS C:\Users\lober> powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\Users\lober> Get-ExecutionPolicy
Restricted
PS C:\Users\lober> $PsVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.22621.2506
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22621.2506
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Once you set the execution policy in the default PowerShell version, Terraform has no more issues.

A screenshot that shows the Windows Terminal output of the Terraform plan command.

And for completeness sake, here is the Linux output.

A screenshot that shows the Linux terminal output of the Terraform plan command.

You can find the source code on GitHub.

I hope this was useful.

Thank you for reading

How To Detect Windows Or Linux Operating System In Terraform

I have found that Terraform does not have constants or functions to determine the operating system it is running on. You can work around this limitation with some knowledge of the target platforms you are running on. The most common use case is discerning between Windows and Unix-based systems to execute shell scripts, for example.

Ideally, you do not have to do this, but sometimes, you, your colleagues, and your CI/CD pipeline do not utilize a homogeneous environment.

One almost 100% certain fact is that Windows addresses storage devices with drive letters. You can leverage this to detect a Windows host by checking the project’s root path and storing the result in a variable.

locals {
  is_windows = length(regexall("^[a-z]:", lower(abspath(path.root)))) > 0
}

output "absolute_path" {
    value = abspath(path.root)
}

output "operating_system" {
    value = local.is_windows ? "Windows" : "Linux"
}

The output values are for demonstration purposes only. All you need is the regex for potential drive letters and the absolute path of the directory. Any path would do, actually.

The regexall function returns a list of all matches, and if the path starts with a drive letter, the resulting list contains more than zero elements, which you can check with the length function.

You could also check for “/home” to detect a Linux-based system or “/Users” for a macOS computer. In those instances, the source code must always be located somewhere in a user’s directory during execution. That may not be the case in a CI/CD pipeline, so keep that in mind. Here is the result on Windows.

A screenshot that shows the Windows Terminal output of the Terraform plan command.

And here on Linux.

A screenshot that shows the Linux terminal output of the Terraform plan command.

You can find the source code on GitHub.

I hope this was useful.

Thank you for reading

Terraform Azure Error SoftDeletedVaultDoesNotExist

I just ran into a frustrating error that seemed unexplainable to me. My goal was to replace an existing Azure Resource Group with a new one managed entirely with Terraform. Besides a few other errors, this SoftDeletedVaultDoesNotExist was incredibly confusing because no more Key Vaults were found in the Resource Group’s list of resources.

Error: creating Vault: (Name "my-fancy-key-vault" / Resource Group "The-Codeslinger"): 
keyvault.VaultsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- 
Original Error: Code="SoftDeletedVaultDoesNotExist" 
Message="A soft deleted vault with the given name does not exist. 
Ensure that the name for the vault that is being attempted to recover is in a recoverable state. 
For more information on soft delete please follow this link https://go.microsoft.com/fwlink/?linkid=2149745"

with module.base.azurerm_key_vault.keyvault,
on terraform\key_vault.tf line 9, in resource "azurerm_key_vault" "keyvault":
    9: resource "azurerm_key_vault" "keyvault" {

That is because it was soft-delete enabled. And it was the Key Vault from the other Resource Group that I previously cleared of all resources, not the new Resource Group.

Using the az CLI you can display it, though.

> az keyvault list-deleted
[
    {
        "id": "/subscriptions/<subscription-id>/providers/Microsoft.KeyVault/locations/westeurope/deletedVaults/my-fancy-key-vault",
        "name": "my-fancy-key-vault",
        "properties": {
            "deletionDate": "2021-08-02T09:39:29+00:00",
            "location": "westeurope",
            "purgeProtectionEnabled": null,
            "scheduledPurgeDate": "2021-10-31T09:39:29+00:00",
            "tags": {
                "customer": "The-Codeslinger",
                "source": "Terraform"
            },
            "vaultId": "/subscriptions/<subscription-id>/resourceGroups/My-Other-ResourceGroup/providers/Microsoft.KeyVault/vaults/my-fancy-key-vault"
        },
        "type": "Microsoft.KeyVault/deletedVaults"
    }
]

And finally delete it.

> az keyvault purge --name my-fancy-key-vault

After that, it is gone.

$ az keyvault list-deleted
[]

Another option seems to be the Azure Portal, but I discovered this only after removing it on the command line.

Terraform Azure Error: parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1; Failed to load token files

There are some instances where I have managed to screw up my Azure CLI configuration file with Terraform. It must have something to do with parallel usage of Terraform or Terraform simultaneously with the az tool. Either way, I ran into the following error.

$ terraform refresh
Acquiring state lock. This may take a few moments...

Error: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error aiting for the Azure CLI: exit status 1

  on main.tf line 16, in provider "azurerm":
  16: provider "azurerm" {

I wondered: "What might block the Azure access? Am I maybe not logged in?" So, I went ahead and tried to log in.

$ az login
Failed to load token files. If you have a repro, please log an issue
at https://github.com/Azure/azure-cli/issues. At the same time, you 
can clean up by running 'az account clear' and then 'az login'. 

(Inner Error: Failed to parse /home/rlo/.azure/accessTokens.json with exception: Extra data: line 1 column 18614 (char 18613))

The error probably comes from parallel access to my Azure CLI configuration file. When I opened the /home/rlo/.azure/accessTokens.json, I found some dangling garbage at the end of it that broke the JSON format.

Here’s a snippet of the last few lines.

        "refreshToken": "0.A...",
        "oid": "<oid>",
        "userId": "<userId>",
        "isMRRT": true,
        "_clientId": "<clientId>",
        "_authority": "https://login.microsoftonline.com/<uid>"
    }
]bc1"}]

I took out the trash bc1"}], saved the file, and it worked again. Many access to resources. Such joy 😉