Hevo Terraform Provider

Terraform is an infrastructure as code tool. You describe the resources that you want in configuration files, and Terraform creates, changes, or deletes them to match your configuration. Terraform manages the resources of a product through a provider, a plugin built for that product.

The Hevo Terraform provider lets you create and manage your Destinations, Pipelines, and Pipeline objects as code.

With the Hevo Terraform provider, you can:

  • Track every change: Keep your Destination and Pipeline configurations in version control, alongside the rest of your infrastructure code.

  • Review changes before they apply: Run the terraform plan command to preview each change, and review it in a pull request before you apply it.

  • Repeat a setup: Create the same Destinations and Pipelines in another Hevo account or region from the same configuration.

  • Automate the setup: Create and update your Destinations and Pipelines from your Continuous Integration and Continuous Delivery (CI/CD) system, such as GitHub Actions or GitLab CI/CD. This makes them part of your deployment process, so you do not need to set them up manually in the Hevo UI.

For example, suppose your team replicates data from a MySQL database to Amazon Redshift, and you need the same Pipeline in a test account and in your production account. You describe the Destination and the Pipeline once, in a configuration file that you store in your Git repository. When a teammate wants to replicate another table, they add the table to the file and open a pull request. Before anyone applies the change, the reviewers see it in the terraform plan output, for example, one object added to the Pipeline. After the pull request is merged, your CI/CD system runs the terraform apply command, and Hevo updates the Pipeline. To create the same Pipeline in your production account, you apply the same configuration with the API credentials of that account.

The provider runs on the machine or CI/CD runner where you run Terraform. All requests go to the Hevo API version 2 and use your API credentials. As a result:

  • Each request counts toward the rate limits of your team.

  • Hevo runs the same checks as it does for an API request, such as the connection tests for a Source or a Destination.

Note: The provider works with Terraform version 1.5 or higher.

To use the provider, specify hevoio/hevo as the source address in the required_providers block of your configuration. Terraform uses this address to find and download the provider.

Resources

A resource is an object in your Hevo account that Terraform creates and manages. The provider offers the following resources:

ResourceDescription
hevo_destinationA Destination, the system that your Pipelines load data into, such as Snowflake or Databricks.
hevo_pipelineA Pipeline and its Source.
hevo_pipeline_objectsThe objects that a Pipeline replicates, such as tables or SaaS objects, and the settings for each object, such as its load mode and the fields that the Pipeline does not replicate.
hevo_pipeline_stateThe state of a Pipeline, enabled or disabled.

The provider creates every Pipeline in the disabled state, so you can select its objects before the Pipeline starts to replicate data. You enable the Pipeline with the hevo_pipeline_state resource.

Data Sources

A data source reads information from your Hevo account without changing it. You can use this information elsewhere in your configuration, such as the ID of an existing Destination. The provider offers the following data sources:

Data sourceDescription
hevo_connectorsThe Source or Destination connectors available to your team.
hevo_connector_metadataThe configuration settings that a connector accepts, such as the settings that you must specify and the settings that are credentials.
hevo_destinationThe details of a Destination.
hevo_destinationsAll the Destinations of your team.
hevo_pipelineThe details of a Pipeline, such as its settings, state, and health.
hevo_pipelinesThe Pipelines created by your team. You can filter them by Destination, Source connector, or state.
hevo_pipeline_objectsThe objects of a Pipeline, with the names that you specify in the hevo_pipeline_objects resource.
hevo_pipeline_jobsThe jobs of a Pipeline, and the objects that a job processed.

Read Reading Data with Data Sources for examples of each data source.

Limitations

The provider creates and changes your Destinations and Pipelines. It does not run tasks on them, such as resyncing a Pipeline or triggering a sync. Perform the following actions in the Hevo UI or with the Hevo API: