Learned a few things today, thought I’d post them online as code snippets.
I was using Terraform to manage some infrastructure, and some resource was giving me grief - breaking the program and giving an unbeatably vague error message when I tried to do a plan involving it. Unfortunately the TF state file was over a thousand lines, and so it was hard to track down the resource in question. It was hard to isolate individual resources in such a big file.
Each domain name creates four kinds of resources from each zone, so that meant working with four separate arrays in the JSON.1 So: I needed to manipulate the 4 kinds of resources created from each zone, directly in tfstate, and remove everything except the broken part to know what’s causing the problem. Processing JSON? Sounds like a task for the sublime jq.
Given a tfstate file like:
Back up the original file:
and then run the following examples like this:
Let’s get into it.
Binary search functions vs an array of numbers
Make your own functions for a binary search by chaining these:
The example starts with the array [1..18], then takes the right half. It slices that up and takes the left half of what’s left, returning [10, 11, 12, 13].
With these functions in hand, you can apply them to do your own binary search in tfstate. Narrow down and see what’s breaking it.
Binary search on the Terraform state
Try modifying the state by removing half of it, then run Terraform again, and see if it breaks:
If that works fine (no error), try the right half instead:
Say that showed the error, then you’d subdivide it:
If that shows the error, great, add another
| left_half to keep narrowing it down. If not, try
| right_half again. Keep doing this until you’ve found one set of resources corresponding to one domain. Tada, you’ve done it.
Once you know what you’re looking for, or only a couple, you can try another approach to only keep specific domains’ resources and discard the rest.
Another approach: filtering elements based on sub-elements
This syntax is super useful, using the update operator with
You can filter members of the resources array by whether their index_key has one of the zones you want.
This is not the best explanation I can write (promise!) but I hope these code samples come in handy for you.
The jq manual is packed full of useful information. It’s usually a better place to start than StackOverflow.