Using Converge to Unarchive a Compressed File

March 06, 2016 // Allison Richardet // Converge

Converge 0.6.0 adds an unarchive resource that handles archive formats such as zip and tar.gz. This new feature allows you to verify archive sources and manage the unarchiving process on a system.

We will discuss the configuration settings and how the unarchive resource works. Examples will cover fetching, verifying, and extracting a file.

Data Used to Perform the Unarchive

Converge requires the source and destination be provided. Additionally, a few other fields are supported as optional parameters to be used for the unarchive process.

• Source: Path to the local compressed source.
  • Valid types: tar.gz, tgz, tar.bz2, tbz2, zip, gz, bz2.
• Destination: Path to the existing destination directory.
• Checksum (optional): Checksum of the compressed source; both the checksum hash and hash type must be provided.
  • Valid hash types: md5, sha1, sha256, sha512.
• Force (optional): Boolean used to indicate whether existing files in the destination should be replaced if their checksum differs from the same file in the compressed source.

How Unarchive Works

Converge’s unarchive resource fetches the local compressed source file. If a checksum has been provided, a verification is performed to ensure the source file has the same checksum. The force option is used to indicate how existing files in the destination should be handled.

Based on the value of the force option, there are several outcomes:

1. force = true: The unarchived data is saved to the destination. It is possible to overwrite files.
2. force = false: If files exist within the destination, determine whether there are checksum mismatches with the files contained within the unarchived source.
   1. Mismatch: Any mismatch will result in aborting the unarchive.
   2. No mismatch: Save the unarchived data to the destination.

In the event a source is not local, Converge’s file.fetch resource can be used to fetch the remote source prior to unarchiving.

Unarchive Examples

Fetch a Remote Source

The following example can be found in Converge’s samples/unarchive.hcl.

This example was designed specifically to handle setup for the user by:

• Creating the destination if it does not exist. This is handled by the file.directory "consul" block.
• Ensuring the source is local. file.fetch "consul.zip" fetches a remote compressed source.

Additional details:

• We have declared two params: zip and destination. These params give us power in two ways:
  • Allow us to use them throughout our HCL like this "{{param `zip`}}" .
  • The value of a param can be indicated when running Converge by using the --params or -p flag, and is illustrated below when we run this example.
• Note the depends used in the unarchive "consul.zip" block. This requires file.directory "consul" and "file.fetch.consul.zip" to be completed prior to executing the containing block, unarchive "consul.zip".

param "zip" {
  default = "/tmp/consul.zip"
}

param "destination" {
  default = "/tmp/consul"
}

file.directory "consul" {
  destination = "{{param `destination`}}"
}

file.fetch "consul.zip" {
  source      = "https://releases.hashicorp.com/consul/0.6.4/consul_0.6.4_linux_amd64.zip"
  destination = "{{param `zip`}}"
}

unarchive "consul.zip" {
  source      = "{{param `zip`}}"
  destination = "{{param `destination`}}"

  depends = ["file.directory.consul", "file.fetch.consul.zip"]
}

Run the Example

Let’s plan our changes, along with use of the -p flag, by running:

converge plan --local -p zip=/tmp/test.zip -p destination=/tmp/test samples/unarchive.hcl

Your output should indicate changes for both file.directory.consul and file.fetch.consul.zip. However, you will notice an error for unarchive.consul.zip indicating there is no such file or directory for /tmp/test.zip. This is ok and should be expected. We have not applied our changes yet, so file.fetch "consul.zip" has not been executed to fetch our remote source.

Next, apply the changes by running:

converge apply --local -p zip=/tmp/test.zip -p destination=/tmp/test samples/unarchive.hcl

The output should indicate no errors and three changes. We have just fetched a remote compressed source and unarchived it! Take a look in /tmp, and you will see test.zip and test/ which contains the unarchived executable.

Provide Checksum for the Source

For this next example, let’s add two lines to the HCL we’ve been using. The lines starting with hash_type and hash should be added to the unarchive "consul.zip" block.

unarchive "consul.zip" {
  source      = "{{param `zip`}}"
  destination = "{{param `destination`}}"
  hash_type   = "sha256"
  hash        = "abdf0e1856292468e2c9971420d73b805e93888e006c76324ae39416edcf0627"

  depends = ["file.directory.consul", "file.fetch.consul.zip"]
}

We have indicated the compressed source should have a checksum that matches hash and be of the hash_type sha256. This gives us extra security that the source we are unarchiving is what we expect.

Run the Example

Let’s plan our changes for the unarchive, using the default values for the zip and destination params:

converge plan --local samples/unarchive.hcl

You should see two changes again for file.directory.consul and file.fetch.consul.zip, and an error for unarchive.consul.zip.

Next, apply the changes:

converge apply --local samples/unarchive.hcl

Again, the output should indicate no errors and three changes. We have just fetched a remote compressed source, verified it using the provided checksum, and unarchived it. Take a look in /tmp, and you will see consul.zip and consul/ which contains the unarchived executable.

Using Force - Checksum Mismatch

This next example illustrates the force option. Here, we have changed the source in file.fetch "consul.zip" to a different version, and updated hash in unarchive "consul.zip". We are also using a new default for the param "zip".

param "zip" {
  default = "/tmp/consul_0.7.5.zip"
}

param "destination" {
  default = "/tmp/consul"
}

file.directory "consul" {
  destination = "{{param `destination`}}"
}

file.fetch "consul.zip" {
  source      = "https://releases.hashicorp.com/consul/0.7.5/consul_0.7.5_linux_amd64.zip"
  destination = "{{param `zip`}}"
}

unarchive "consul.zip" {
  source      = "{{param `zip`}}"
  destination = "{{param `destination`}}"
  hash_type   = "sha256"
  hash        = "40ce7175535551882ecdff21fdd276cef6eaab96be8a8260e0599fadb6f1f5b8"

  depends = ["file.directory.consul", "file.fetch.consul.zip"]
}

Run the Example

Let’s plan our changes for the unarchive:

converge plan --local samples/unarchive.hcl

The output should indicate there is one change for the file.fetch.consul.zip task. One error should be output for the unarchive.consul.zip task that the compressed source to be fetched /tmp/consul_0.7.5.zip does not exist. Again, this error is expected.

Next, apply the changes:

converge apply --local samples/unarchive.hcl

The output shows the compressed source has been successfully fetched. You will notice an error associated with unarchive.consul.zip:

root/unarchive.consul.zip:
 Error: will not replace, "/consul" exists at "/tmp/consul": checksum mismatch
 Messages:
  use the "force" option to replace all files with checksum mismatch
 Has Changes: yes
 Changes:
  unarchive: "/tmp/consul_0.7.5.zip" => "/tmp/consul"

There is a checksum mismatch with the executable in the destination for our unarchive, /tmp/consul. We are attempting to unarchive a different version here, so we should expect a checksum mismatch. Let’s force replace this consul executable.

The Force Option

To force replace the consul executable in our unarchive destination, add the force = true line to the unarchive "consul.zip" block.

unarchive "consul.zip" {
  source      = "{{param `zip`}}"
  destination = "{{param `destination`}}"
  hash_type   = "sha256"
  hash        = "40ce7175535551882ecdff21fdd276cef6eaab96be8a8260e0599fadb6f1f5b8"
  force       = true

  depends = ["file.directory.consul", "file.fetch.consul.zip"]
}

Run the Example

Plan the changes for our unarchive:

converge plan --local samples/unarchive.hcl

You will see there is one change for unarchive.consul.zip.

Finally, let’s apply our changes for the unarchive:

converge apply --local samples/unarchive.hcl

Success! We have just force replaced the consul executable in /tmp/consul.

Conclusion

Converge’s unarchive resource provides the user with a powerful way to unarchive a compressed source. Providing a checksum gives the user assurance regarding the integrity of the source. The force option ensures the user is aware files in the destination may be overwritten, putting the power in their hands. Take a look at the unarchive docs for more details.

We’d love to hear from you! Reach out to us on the Converge Slack or open a GitHub Issue.