Blog

Module Parameter Defaults with the Terraform Object Type

25 Jun, 2022

In this post, I will walk you through the challenges I’ve faced when adopting the object type in my Terraform 0.12 modules and the solutions I came up with to work around the caveats. As an example, I will use the google_storage_bucket resource, as this is part of one of the modules I’ve built.

The Terraform Object Type

With the launch of Terraform 0.12 back in May 2019, many new cool features are introduced. Compared with Terraform 0.11, where you would find yourself repeating a lot of code, you can now utilise the new for_each functionality and object type to write cleaner code. If you haven’t discovered the new object type yet, you may be surprised by its potential. It is one of the two complex types Terraform provides and gives you the possibility to describe object structures. While this new type has many advantages, such as mixing types and defining multi-layered structures, it also has a few caveats that I will explain further down.

The content below is taken from the Terraform docs itself: object(...): a collection of named attributes that each have their own type. The schema for object types is { <key> = <type>, <key> = <type>, ... }</type></key></type></key> — a pair of curly braces containing a comma-separated series of <key> = <type></type></key> pairs. Values that match the object type must contain all of the specified keys, and the value for each key must match its specified type. (Values with additional keys can still match an object type, but the extra attributes are discarded during type conversion.)

Example object structure:

<span class="token property">my_object</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
  <span class="token property">a_string</span>  <span class="token punctuation">=</span> example,
  <span class="token property">a_number</span>  <span class="token punctuation">=</span> <span class="token number">1</span>,
  <span class="token property">a_boolean</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>,
  <span class="token property">a_map</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
    <span class="token property">type1</span> <span class="token punctuation">=</span> <span class="token number">1</span>,
    <span class="token property">type2</span> <span class="token punctuation">=</span> <span class="token number">2</span>,
    <span class="token property">type3</span> <span class="token punctuation">=</span> <span class="token number">3</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
HCL

Provide Module Default Params with Object

As promised, in this blog post I will explain how I used the object type in my custom Terraform module. In the example below I’ve used the object type to define the supported settings of the GCP storage bucket.

<span class="token keyword">variable<span class="token type variable"> bucket_settings </span></span><span class="token punctuation">{</span>
  <span class="token property">type</span> <span class="token punctuation">=</span> object(<span class="token punctuation">{</span>
    <span class="token property">location</span>           <span class="token punctuation">=</span> string
    <span class="token property">storage_class</span>      <span class="token punctuation">=</span> string
    <span class="token property">versioning_enabled</span> <span class="token punctuation">=</span> bool
    <span class="token property">bucket_policy_only</span> <span class="token punctuation">=</span> bool
    <span class="token property">lifecycle_rules</span> <span class="token punctuation">=</span> map(object(<span class="token punctuation">{</span>
      <span class="token property">action</span> <span class="token punctuation">=</span> map(string)
      <span class="token property">condition</span> <span class="token punctuation">=</span> object(<span class="token punctuation">{</span>
        <span class="token property">age</span>                   <span class="token punctuation">=</span> number
        <span class="token property">with_state</span>            <span class="token punctuation">=</span> string
        <span class="token property">created_before</span>        <span class="token punctuation">=</span> string
        <span class="token property">matches_storage_class</span> <span class="token punctuation">=</span> list(string)
        <span class="token property">num_newer_versions</span>    <span class="token punctuation">=</span> number
      <span class="token punctuation">}</span>)
    <span class="token punctuation">}</span>))
  <span class="token punctuation">}</span>)
<span class="token punctuation">}</span>
HCL

Previously, when writing your Terraform module, you would need to create a variable for each setting you want to pass to your resource. Sure, you would have maps and lists, but a map could only contain values of the same type, limiting its use significantly. When using the object type, we can combine these settings in a complex structure. I can use this to populate my bucket, providing all these settings in a variable (bucket_settings in the listing below).

<span class="token property">bucket_settings</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
  <span class="token property">location</span>           <span class="token punctuation">=</span> europe-west4
  <span class="token property">storage_class</span>      <span class="token punctuation">=</span> REGIONAL
  <span class="token property">versioning_enabled</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>
  <span class="token property">bucket_policy_only</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>
  <span class="token property">lifecycle_rules</span>    <span class="token punctuation">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span>
<span class="token punctuation">}</span>
HCL

Since I have certain settings that I want to have applied to all of my buckets, I want to have most of these settings to be set for me by default. To do this, we can provide a default value for the bucket_settings object in the variable definition itself:

<span class="token keyword">variable<span class="token type variable"> bucket_settings </span></span><span class="token punctuation">{</span>
  <span class="token property">type</span> <span class="token punctuation">=</span> object(<span class="token punctuation">{</span>
    <span class="token property">location</span>           <span class="token punctuation">=</span> string
    <span class="token property">storage_class</span>      <span class="token punctuation">=</span> string
    <span class="token property">versioning_enabled</span> <span class="token punctuation">=</span> bool
    <span class="token property">bucket_policy_only</span> <span class="token punctuation">=</span> bool
    <span class="token property">lifecycle_rules</span> <span class="token punctuation">=</span> map(object(<span class="token punctuation">{</span>
      <span class="token property">action</span> <span class="token punctuation">=</span> map(string)
      <span class="token property">condition</span> <span class="token punctuation">=</span> object(<span class="token punctuation">{</span>
        <span class="token property">age</span>                   <span class="token punctuation">=</span> number
        <span class="token property">with_state</span>            <span class="token punctuation">=</span> string
        <span class="token property">created_before</span>        <span class="token punctuation">=</span> string
        <span class="token property">matches_storage_class</span> <span class="token punctuation">=</span> list(string)
        <span class="token property">num_newer_versions</span>    <span class="token punctuation">=</span> number
      <span class="token punctuation">}</span>)
    <span class="token punctuation">}</span>))
  <span class="token punctuation">}</span>)

  <span class="token property">default</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
    <span class="token property">location</span>           <span class="token punctuation">=</span> europe-west4
    <span class="token property">storage_class</span>      <span class="token punctuation">=</span> REGIONAL
    <span class="token property">versioning_enabled</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>
    <span class="token property">bucket_policy_only</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>
    <span class="token property">lifecycle_rules</span>    <span class="token punctuation">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
HCL

Now, if I create my bucket, I can just import the module like this:

<span class="token keyword">module<span class="token type variable"> my_bucket </span></span><span class="token punctuation">{</span>
  <span class="token property">source</span> <span class="token punctuation">=</span> path.to/my/module
  <span class="token property">name</span>   <span class="token punctuation">=</span> bucket-with-defaults
<span class="token punctuation">}</span>
HCL

And since I’ve set the defaults, I don’t need to provide the bucket_settings variable at all. ## Change One Setting, and The Defaults Disappear

But what if I want to overwrite just one of the defaults and keep the rest? In the following example, I try to set a lifecycle policy for the bucket using my module. Notice how I am passing the bucket_settings as a parameter.

<span class="token keyword">module<span class="token type variable"> my_bucket </span></span><span class="token punctuation">{</span>
  <span class="token property">source</span> <span class="token punctuation">=</span> ./modules/bucket
  <span class="token property">name</span>   <span class="token punctuation">=</span> bucket-with-defaults

  <span class="token property">bucket_settings</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
    <span class="token property">lifecycle_rules</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
      delete <span class="token property">rule</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
        <span class="token property">action</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span> <span class="token property">type</span> <span class="token punctuation">=</span> Delete <span class="token punctuation">}</span>
        <span class="token property">condition</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
          <span class="token property">age</span>        <span class="token punctuation">=</span> <span class="token number">30</span>
          <span class="token property">with_state</span> <span class="token punctuation">=</span> ANY
        <span class="token punctuation">}</span>
      <span class="token punctuation">}</span>
    <span class="token punctuation">}</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
HCL

Unfortunately, this will result in multiple errors: Missing attributes

The given value is not suitable for child module variable bucket_settings
defined at modules/bucket/main.tf:6,1-27: attributes bucket_policy_only,
location, storage_class, and versioning_enabled are required.
Shell

Since I provided the bucket_settings for this module, it will overwrite the entire defaults I’ve set for the variable. I will need to provide the keys like location, storage_class, versioning_enabled and bucket_policy_only. But even after providing these settings, I’m getting a new error:

The given value is not suitable for child module variable bucket_settings
defined at modules/bucket/main.tf:6,1-27: attribute lifecycle_rules: element
delete rule: attribute condition: attributes created_before,
matches_storage_class, and num_newer_versions are required.
Shell

For the lifecycle_rule condition, I’ve only provided the age and with_state since I don’t care about created_before, matches_storage_class or num_newer_versions. But since I’ve defined them in the object definition, Terraform will complain and tell me I need to provide them as well. So to make this work with the module setup described above, I will need to provide all of the settings again. As you can see, especially when you want to create multiple buckets with only a few differences in the bucket configuration, this will become quite cumbersome. # Solution, using a defaults variable and merge

Because I only want to provide the settings that differ from the defaults, I’ve developed a solution for this. When I create a separate variable for the defaults, I can merge the provided bucket_settings with this bucket_defaults variable before calling the actual bucket provider. The result looks like this:

Module code snippet

<span class="token keyword">variable<span class="token type variable"> name </span></span><span class="token punctuation">{</span>
  <span class="token property">type</span> <span class="token punctuation">=</span> string
  <span class="token property">description</span> <span class="token punctuation">=</span> <span class="token string">"The name of the bucket"</span>
<span class="token punctuation">}</span>

<span class="token keyword">variable<span class="token type variable"> bucket_defaults </span></span><span class="token punctuation">{</span>
  <span class="token property">type</span> <span class="token punctuation">=</span> object(<span class="token punctuation">{</span>
    <span class="token property">location</span>           <span class="token punctuation">=</span> string
    <span class="token property">storage_class</span>      <span class="token punctuation">=</span> string
    <span class="token property">versioning_enabled</span> <span class="token punctuation">=</span> bool
    <span class="token property">bucket_policy_only</span> <span class="token punctuation">=</span> bool
    <span class="token property">lifecycle_rules</span> <span class="token punctuation">=</span> map(object(<span class="token punctuation">{</span>
      <span class="token property">action</span> <span class="token punctuation">=</span> map(string)
      <span class="token property">condition</span> <span class="token punctuation">=</span> object(<span class="token punctuation">{</span>
        <span class="token property">age</span>                   <span class="token punctuation">=</span> number
        <span class="token property">with_state</span>            <span class="token punctuation">=</span> string
        <span class="token property">created_before</span>        <span class="token punctuation">=</span> string
        <span class="token property">matches_storage_class</span> <span class="token punctuation">=</span> list(string)
        <span class="token property">num_newer_versions</span>    <span class="token punctuation">=</span> number
      <span class="token punctuation">}</span>)
    <span class="token punctuation">}</span>))
  <span class="token punctuation">}</span>)

  <span class="token property">default</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
    <span class="token property">location</span>           <span class="token punctuation">=</span> europe-west4
    <span class="token property">storage_class</span>      <span class="token punctuation">=</span> REGIONAL
    <span class="token property">versioning_enabled</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>
    <span class="token property">bucket_policy_only</span> <span class="token punctuation">=</span> <span class="token boolean">true</span>
    <span class="token property">lifecycle_rules</span>    <span class="token punctuation">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>

<span class="token keyword">variable<span class="token type variable"> bucket_settings </span></span><span class="token punctuation">{</span>
  <span class="token property">description</span> <span class="token punctuation">=</span> <span class="token string">"Map of bucket settings to be applied, which will be merged with the bucket_defaults. Allowed keys are the same as defined for bucket_defaults."</span>
<span class="token punctuation">}</span>

<span class="token keyword">locals</span> <span class="token punctuation">{</span>
  <span class="token property">merged_bucket_settings</span> <span class="token punctuation">=</span> merge(var.bucket_defaults, var.bucket_settings)
<span class="token punctuation">}</span>
HCL

I can now use the module by providing only the lifecycle rule that I want to set for my bucket, and all other defaults will still be applied. Using the module

<span class="token keyword">module<span class="token type variable"> my_bucket </span></span><span class="token punctuation">{</span>
  <span class="token property">source</span> <span class="token punctuation">=</span> path.to/my/module
  <span class="token property">name</span>   <span class="token punctuation">=</span> bucket-with-defaults

  <span class="token property">bucket_settings</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
    <span class="token property">lifecycle_rules</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
    delete <span class="token property">rule</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
        <span class="token property">action</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span> <span class="token property">type</span> <span class="token punctuation">=</span> Delete <span class="token punctuation">}</span>
        <span class="token property">condition</span> <span class="token punctuation">=</span> <span class="token punctuation">{</span>
          <span class="token property">age</span>          <span class="token punctuation">=</span> <span class="token number">30</span>
          <span class="token property">with_state</span> <span class="token punctuation">=</span> ANY
        <span class="token punctuation">}</span>
      <span class="token punctuation">}</span>
    <span class="token punctuation">}</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
HCL

Conclusion

I like the fact that I can now define objects with mixed types to create more complex structures, but there are a couple of things I think that could be improved:

  • The way variable defaults are handled. It would be great if these would be merged with the provided value, or at least have a setting to allow for this behaviour.
  • You should be able to define optional keys for your objects. However, the need for this will become less urgent when the first point is taken care of.

Hopefully, this blog post will help you understand the object type and its limitations while giving you an idea of utilising it to its best potential. In my next blog post, I will tell you more about the for_each functionality and how to use this in your Terraform modules.

Explore related posts