How to Troubleshoot Terraform

When working with Terraform, you might run into four types of issues: language, state, core, and provider errors. Understanding these error types will allow you to troubleshoot terraform issues and keep your infrastructure running smoothly. Here’s what each one means and how they can affect your workflow:

– Language Errors:
Terraform uses HashiCorp Configuration Language (HCL) to define infrastructure.

If there’s a mistake in your configuration, like a syntax error, Terraform will highlight the problem, show you the line number, and provide an explanation to help you fix it.

– State Errors:
Terraform keeps track of your infrastructure using a state file. This file maps resources to your configuration and stores important metadata. If the state gets out of sync, Terraform might unexpectedly change or even destroy resources.

If you’re experiencing issues, once you’ve checked your configuration, take a look at your state. You may need to refresh, import, or replace resources to get everything back in sync.

– Core Errors:
The core of Terraform handles everything from interpreting your configuration to managing state and resource dependencies.

If something goes wrong at this level, it might be a bug in Terraform itself.

– Provider Errors:
Providers are what connect Terraform to cloud services, handling authentication and API calls.

If there’s an issue with a provider, it could be due to a misconfiguration or a bug in the provider plugin.

I wrote this tutorial with a real word use case where you can see these errors and how to troubleshoot them.

In this tutorial, you’ll clone a repository with a faulty Terraform setup designed to deploy an EC2 instance along with its networking. The configuration has intentional errors, so you can get hands-on experience troubleshooting Terraform.

Prerequisites

Before diving in, make sure you have:

Terraform installed
An AWS account with credentials properly set up for Terraform

Step 1: Format the configuration file

Change into the repository directory.

cd learn-terraform-troubleshooting

Go ahead and open the main.tf file in your editor.
You might notice some issues right away (they’re there on purpose), but don’t make any changes just yet. In the next section, you’ll run a command to help identify and highlight these errors for you.

The terraform fmt command helps clean up and standardize your Terraform configuration files. It scans your directory, finds any formatting issues, and fixes them automatically.

Go ahead and run terraform fmt in your terminal. You’ll get two errors, one about an invalid character and another about an invalid expression, both on line 49. Don’t worry, we’ll tackle those next.

Terraform ran into some issues it couldn’t automatically fix, so it’s letting you know where to make adjustments.

Step 2: Fix the Variable Interpolation Error

Open main.tf and find the tags section of the aws_instance.web_app resource on line 46. The Name tag is set up wrong, it’s trying to add a string to the name input variable but missing the right syntax to do so.
Just update the Name tag with the correct syntax, and that should fix the issue.

Run terraform fmt again to ensure your variable name meets the formatting requirements.

You’ve resolved the invalid character and expression errors, and they’re gone for good.
Now, Terraform is properly reading ${var.name}-learn as your variable name, adding the -learn string to create a custom value just the way you intended.

Step 3: Validate Your Configuration

terraform fmt only checks for syntax issues like interpolation errors or incorrect resource definitions, which is why you should run terraform validate after formatting. This ensures your configuration meets the expectations of the providers you’re working with.
Before doing that, make sure to initialize your Terraform directory so it can download the necessary providers for your configuration.

Run terraform validate in your terminal. The output contains a cycle error that highlights a mutual dependency between two security group resources.

Step 4: Fix Cycle Errors

Cycle errors occur when there’s circular logic in the Terraform dependency tree. Terraform looks at how resources depend on each other to figure out the right order for its operations. When there’s a cycle, it can’t determine which resource to create first.

Let us fix this issue in your dependency graph.

Step 5: Correct a Cycle Error

The aws_security_group resources in your configuration reference each other in their security_groups attributes. AWS can’t create the security groups because they’re each waiting on the other to be created first.

To fix this, remove the circular references between the security groups, leaving both groups without any ingress rules. This will resolve the cycle and allow Terraform to proceed.

Before changes:

After changes

Instead of putting the rules directly into the aws_security_group configuration, you can use the aws_security_group_rule resource and reference the security group IDs instead. This approach avoids the cycle error, as Terraform will create both security groups first, without any interdependent rules, and then create and attach the rules afterward.

Go ahead and add the new, separate rule resource configurations to main.tf to fix the issue.

Terraform does not continue validating once it catches an error.
Run the terraform validate command to catch new errors: an invalid reference a the for_each attribute because of a splat expression (*) in its value.

The each attribute in the vpc_security_group_ids cannot return the IDs because of the for_each error above it. Terraform did not return any security group IDs, so the each object is invalid.
Review the construction of the instance resource.

Terraform can’t automatically convert types without using extra functions, so you’ll need to make some adjustments.

Step 6: Fix the for_each Error

The for_each attribute in Terraform allows you to create multiple similar resources based on the criteria you set.

In this case, you’re trying to create a set of instances, each assigned to a different security group. However, Terraform can’t process aws_security_group.*.id in this context because the splat expression (*) only works with lists, while for_each requires a map. A local value can return a map type, which will solve the issue.

In main.tf, go to line 36 and replace the for_each value with a local value. Then, on line 39, update the vpc_security_group_ids value to use the value from the for_each attribute.

Finally, make sure to update the tags attribute so each instance has a unique name.

Define the local value in your main.tf file. This converts the list of security groups to a map.

After editing your configuration files, they may not be formatted correctly. Format your configuration.

The next time you run terraform validate, you’ll see errors because of the changes you made to the for_each value. Specifically, your outputs aren’t capturing the multiple instances in the aws_instance.web_app resources.

Run the validation again to see these output errors and address them.

In the next section, you’ll fix these errors by using a for expression to define outputs that capture the lists of instance IDs, IP addresses, and names.

Step 7: Correct Your Outputs

To resolve this, you’ll use a for expression to capture the details from all your resources.

The for expression loops through each aws_instance.web_app and stores the elements in a temporary variable called instance. Terraform will then return the specific values for each instance like instance.id, instance.public_ip, and instance.tags.Name—for every instance you created.

Open outputs.tf and update the output values using the for expression to return the information you need.

Before:

After

Now format your configuration.

$ terraform fmt

Validate your configuration

$ terraform validate

Step 8: Apply Your Changes

You’ve fixed the configuration, so it’s time to apply those changes. Run terraform apply to create your resources, and when prompted, type “yes” to confirm and make the changes take effect.

Bravo! You’ve troubleshoot Terraform successfully.

What You Did:

Formatted Configuration: Used terraform fmt to clean and standardize files.
Fixed Interpolation Errors: Resolved incorrect syntax in variable assignments.
Validated Configuration: Ran terraform validate to detect dependency cycles.
Fixed Cycle Errors: Removed circular dependencies in security group rules.
Resolved for_each Issues: Converted lists to maps for correct iteration.
Corrected Outputs: Used for expressions to capture instance details properly.
Applied Changes: Successfully deployed the infrastructure using terraform apply.

Key Technologies & Tools Used:

Terraform: Infrastructure as Code for defining and managing resources.
AWS EC2, VPC, Security Groups: Cloud infrastructure components.
Git & .gitignore: Prevented committing sensitive data.

What You Learned:

How to troubleshoot and fix Terraform syntax, state, and dependency errors.
Best practices for structuring Terraform configurations to avoid cycles.
How to handle provider-specific issues and state inconsistencies.
The importance of validating and formatting Terraform code before applying changes.

Cloud with DJ