Reading Data with Data Sources

A data source reads information from your Hevo account without changing it. Terraform reads each data source again every time you run the terraform plan or terraform apply command, so the values are always current. You can use the values elsewhere in your configuration, or display them as outputs.

Use the data sources of the Hevo Terraform provider to:

All timestamps that the data sources return are in the Request for Comments (RFC) 3339 format, for example, 2026-10-29T04:04:57Z.

Prerequisites

Finding Connectors

A connector is the integration that Hevo uses to replicate data from a type of Source, such as PostgreSQL, or to load data into a type of Destination, such as Snowflake. You specify the connector by its ID when you create a Destination or a Pipeline.

Two data sources help you find connectors. The hevo_connectors data source lists the connectors available to your team, and the hevo_connector_metadata data source reads the settings that a connector accepts.

hevo_connectors

This data source lists the Source or Destination connectors available to your team. Connectors that your team does not have access to are not listed. A connector in beta is listed only if it has been enabled for your team.

The following example lists the IDs of all the Destination connectors:

data "hevo_connectors" "destinations" {
  kind = "destination"
}

output "destination_connector_ids" {
  value = data.hevo_connectors.destinations.connectors[*].connector_id
}

To find the connectors for one type of Source, specify its type. More than one connector can ingest data from the same type of Source. For example, type = "POSTGRES" returns SOURCE_CONNECTOR_POSTGRES and postgres_xmin, among others.

Arguments

The following table lists the arguments of the hevo_connectors data source:

ArgumentDescription
kind (String, Required)Specifies whether to list Source connectors or Destination connectors. Possible values: source, destination.
type (String, Optional)Lists only the Source connectors of this type. For example, HUBSPOT. Specify it only when kind is source. Otherwise, Terraform displays an error.

Attributes

The following table lists the attributes of the hevo_connectors data source:

AttributeDescription
connectors (List of Object)The connectors available to your team.
connectors[*].connector_id (String)The ID of the connector. Specify this value as connector_id in a hevo_destination resource, or as source_connector_id in a hevo_pipeline resource. For example, snowflake.
connectors[*].type (String)The type of the Source or Destination. For example, SNOWFLAKE. A Source connector is listed once for each of its Source types. For example, SOURCE_CONNECTOR_MYSQL is listed with the types AURORA, AWS_RDS_MYSQL, AZURE_MYSQL, GCP_MYSQL, and MYSQL.
connectors[*].display_name (String)The name of the connector in the Hevo UI.
connectors[*].description (String)A short description of the connector.
connectors[*].documentation_url (String)The link to the Hevo documentation for the Source or Destination.
connectors[*].in_beta (Boolean)Indicates whether the connector is in beta.

hevo_connector_metadata

This data source reads the configuration settings that a connector accepts, including the settings that are required and the settings that are credentials. Use it to build the config of a Destination or the source_config of a Pipeline.

The following example lists the settings that a Snowflake Destination always requires:

data "hevo_connector_metadata" "snowflake" {
  kind         = "destination"
  connector_id = "snowflake"
}

output "snowflake_required_keys" {
  value = data.hevo_connector_metadata.snowflake.required_config_keys
}

The required_config_keys attribute lists only the settings that are always required. A setting that is required only for a particular choice, such as the password for one authentication type, is described in schema_json. Read schema_json with the jsondecode function. The settings of the connector are described under properties.config.properties, and the settings that are always required are in properties.config.required:

locals {
  snowflake_settings = jsondecode(data.hevo_connector_metadata.snowflake.schema_json).properties.config.properties
}

This schema has the same structure as the response of the Get Configuration Details of a Source Connector and Get Configuration Details of a Destination Connector endpoints.

Arguments

The following table lists the arguments of the hevo_connector_metadata data source:

ArgumentDescription
kind (String, Required)Specifies whether the connector is a Source connector or a Destination connector. Possible values: source, destination.
connector_id (String, Required)The ID of the connector, as the hevo_connectors data source returns it. For example, snowflake.

Attributes

The following table lists the attributes of the hevo_connector_metadata data source:

AttributeDescription
display_name (String)The name of the connector in the Hevo UI.
type (String)The type of the Destination, or the default Source type of a Source connector. For example, SNOWFLAKE for a Destination, or MYSQL for a Source.
source_types (List of String)The Source types of a Source connector, which you can specify as source_type in a hevo_pipeline resource. For example, AWS_RDS_MYSQL. This list is empty for a Destination connector.
required_config_keys (List of String)The settings that are always required.
write_only_config_keys (List of String)The settings that are credentials. Hevo accepts them when you create or change a resource, and masks them in every response.
schema_json (String)The JSON schema of the request that creates a resource with this connector, as a JSON string.

Reading Destinations

Two data sources read your Destinations. The hevo_destination data source reads the details of one Destination, and the hevo_destinations data source lists all the Destinations of your team.

hevo_destination

This data source reads the details of one Destination by its ID. Hevo returns the credentials in config masked as ********.

The following example reads an existing Destination and creates a Pipeline that loads into it:

data "hevo_destination" "warehouse" {
  id = "7"
}

resource "hevo_pipeline" "orders" {
  destination_id = data.hevo_destination.warehouse.id
  # ...
}

Arguments

The following table lists the argument of the hevo_destination data source:

ArgumentDescription
id (String, Required)The ID of the Destination.

Attributes

The following table lists the attributes of the hevo_destination data source:

AttributeDescription
name (String)The name of the Destination.
connector_id (String)The ID of the Destination connector. For example, snowflake.
type (String)The type of the Destination. For example, SNOWFLAKE.
status (String)The status of the Destination. For example, CREATED.
config (Dynamic)The configuration settings of the Destination, with credentials masked.
created_at, updated_at (String)The time when the Destination was created and last changed.
created_by, updated_by (String)The user who created and last changed the Destination.

hevo_destinations

This data source lists all the Destinations of your team. The list does not include the configuration settings. To read the settings of a Destination, use the hevo_destination data source.

The following example lists the names of all the Destinations:

data "hevo_destinations" "all" {}

output "destination_names" {
  value = data.hevo_destinations.all.destinations[*].name
}

Attributes

The following table lists the attribute of the hevo_destinations data source:

AttributeDescription
destinations (List of Object)The Destinations of your team. Each Destination has the id, name, connector_id, type, status, created_at, created_by, updated_at, and updated_by attributes, as described for the hevo_destination data source.

Reading Pipelines

Three data sources read your Pipelines. The hevo_pipeline data source reads the details of one Pipeline, the hevo_pipelines data source lists the Pipelines of your team, and the hevo_pipeline_objects data source lists the objects of a Pipeline.

hevo_pipeline

This data source reads the details of one Pipeline by its ID, including its state and health, which the hevo_pipeline resource does not include.

The following example reads a Pipeline and displays its state:

data "hevo_pipeline" "orders" {
  id = "42"
}

output "orders_state" {
  value = data.hevo_pipeline.orders.state
}

Arguments

The following table lists the argument of the hevo_pipeline data source:

ArgumentDescription
id (String, Required)The ID of the Pipeline.

Attributes

The following table lists the attributes of the hevo_pipeline data source:

AttributeDescription
name (String)The name of the Pipeline.
state (String)The state of the Pipeline. For example, Initialized or Disabled.
disabled (Boolean)Indicates whether the Pipeline is disabled.
health (String)The health of the Pipeline. For example, HEALTHY.
source_name (String)The name of the Source.
source_connector_id (String)The ID of the Source connector. For example, SOURCE_CONNECTOR_MYSQL.
source_type (String)The type of the Source. For example, AWS_RDS_MYSQL.
destination_id (String)The ID of the Destination that the Pipeline loads data into.
destination_prefix (String)The prefix that Hevo adds to the names of the tables that it creates in the Destination.
replication_type (String)The data that the Pipeline replicates. Possible values: HISTORICAL_AND_INCREMENTAL, INCREMENTAL_ONLY.
load_mode (String)The load mode of the Pipeline, which determines how Hevo writes data into the Destination. Possible values: MERGE, APPEND, HISTORY_TRACKING.
schema_evolution (String)The changes to the Source schema that Hevo replicates. Possible values: ALLOW_ALL, ALLOW_COLUMN_LEVEL, BLOCK_ALL.
schedule (Object)The schedule of the Pipeline. It contains the following:
  • sync_type: The type of schedule. Possible values: SCHEDULED, ON_DEMAND.
  • frequency_minutes: The interval, in minutes, at which a SCHEDULED Pipeline ingests data, or null for an ON_DEMAND Pipeline.
failure_handling_policy (Object)The action that Hevo takes when objects fail to load. It contains the following:
  • object_failure_level: The failure handling level. Possible values: PARTIAL, ALL_OR_NONE.
  • object_failure_threshold: The number of objects that can fail when object_failure_level is PARTIAL.
latency_alert (Object)The alert that Hevo sends when replication falls behind. It contains the following:
  • enabled: The status of the alert. Possible values: true, false.
  • threshold_minutes: The latency, in minutes, that raises the alert.
created_at, updated_at (String)The time when the Pipeline was created and last changed.
created_by, updated_by (String)The user who created and last changed the Pipeline.

hevo_pipelines

This data source lists the Pipelines created by your team. You can filter the list by Destination, Source, and state.

The following example lists the Pipelines that load data into a Destination and are disabled:

data "hevo_pipelines" "disabled_in_warehouse" {
  destination_id = data.hevo_destination.warehouse.id
  states         = ["Disabled"]
}

output "disabled_pipeline_names" {
  value = data.hevo_pipelines.disabled_in_warehouse.pipelines[*].name
}

Arguments

The following table lists the arguments of the hevo_pipelines data source:

ArgumentDescription
destination_id (String, Optional)Lists only the Pipelines that load data into this Destination.
source_connector_id (String, Optional)Lists only the Pipelines with a Source that uses this connector.
states (List of String, Optional)Lists only the Pipelines in these states. Possible values: Created, Created Source, Created Destination, Configured Pipeline, Configured Object, Initializing, Initialized, Failed, Disabling, Disabled, Enabling, Restarting. A Pipeline that is enabled and replicating data is in the Initialized state. For example, ["Initialized", "Disabled"].

Attributes

The following table lists the attribute of the hevo_pipelines data source:

AttributeDescription
pipelines (List of Object)The Pipelines of your team that match the arguments. Each Pipeline has the id and the attributes described for the hevo_pipeline data source. The health attribute is empty in this list, as Hevo returns the health of a Pipeline only when you read the Pipeline by its ID. To read the health of a Pipeline, use the hevo_pipeline data source.

hevo_pipeline_objects

This data source lists the objects of a Pipeline, with the fully qualified names that you specify in the objects of the hevo_pipeline_objects resource. The data source and the resource have the same name.

The following example lists the names of the objects of a Pipeline:

data "hevo_pipeline_objects" "orders" {
  pipeline_id = hevo_pipeline.orders.id
}

output "object_names" {
  value = data.hevo_pipeline_objects.orders.objects[*].fully_qualified_name
}

By default, the list contains only the objects that you can select or deselect, in the statuses ACTIVE, INACTIVE, RESETTING, BLOCKED, and INCONSISTENT. For example, an object that Hevo cannot access, with the status INACCESSIBLE, is not listed.

Note:

  • Right after a Pipeline is created, the list can be empty for a few minutes, until Hevo reads the objects of the Source.

  • The order of the objects can change between reads. If you display the list as an output, the terraform plan command can show a change to the output even when the objects have not changed.

Arguments

The following table lists the arguments of the hevo_pipeline_objects data source:

ArgumentDescription
pipeline_id (String, Required)The ID of the Pipeline.
statuses (List of String, Optional)Lists only the objects in these statuses. For example, ["ACTIVE"] lists only the objects that the Pipeline replicates. Default value: ["ACTIVE", "INACTIVE", "RESETTING", "BLOCKED", "INCONSISTENT"].

Attributes

The following table lists the attributes of the hevo_pipeline_objects data source:

AttributeDescription
objects (List of Object)The objects of the Pipeline.
objects[*].fully_qualified_name (String)The name of the object to specify in the hevo_pipeline_objects resource. For example, employees.departments for a MySQL table.
objects[*].name (String)The name of the object in the Source. For example, departments.
objects[*].selected (Boolean)Indicates whether the Pipeline replicates the object.
objects[*].status (String)The status of the object. For example, ACTIVE for a selected object or INACTIVE for an object that is not selected.
objects[*].load_mode (String)The load mode of the object. Possible values: MERGE, APPEND, HISTORY_TRACKING.

Monitoring Pipeline Jobs

A job is one run of a Pipeline, such as an ingestion, a historical load, or a resync. The hevo_pipeline_jobs data source reads the jobs of a Pipeline, so that you can check the progress of the Pipeline from Terraform.

hevo_pipeline_jobs

This data source reads the jobs of a Pipeline, newest first. Set job_id to read one job, along with the details of each object that the job processed.

The following example displays the status of the latest job of a Pipeline:

data "hevo_pipeline_jobs" "recent" {
  pipeline_id = hevo_pipeline.orders.id
  limit       = 5
}

output "last_job_status" {
  value = try(data.hevo_pipeline_jobs.recent.jobs[0].status, "no jobs yet")
}

Arguments

The following table lists the arguments of the hevo_pipeline_jobs data source:

ArgumentDescription
pipeline_id (String, Required)The ID of the Pipeline.
job_id (String, Optional)The ID of a job. If specified, Hevo returns only this job, and fills the objects attribute.
limit (Number, Optional)The number of the newest jobs to read, from 1 to 1000. Default value: 20.
statuses (List of String, Optional)Lists only the jobs in these statuses. For example, ["IN_PROGRESS"].
types (List of String, Optional)Lists only the jobs of these types. For example, ["HISTORICAL"].

Attributes

The following table lists the attributes of the hevo_pipeline_jobs data source:

AttributeDescription
jobs (List of Object)The jobs of the Pipeline, newest first.
jobs[*].id (String)The ID of the job.
jobs[*].type (String)The type of the job. Possible values: HISTORICAL, INCREMENTAL, REFRESHER, RESYNC, RESYNC_WITH_DROP_AND_LOAD.
jobs[*].status (String)The status of the job. For example, COMPLETED or IN_PROGRESS.
jobs[*].events_ingested, events_loaded, events_failed (Number)The number of Events that the job read from the Source, loaded into the Destination, and failed to process.
jobs[*].objects_success, objects_failed, objects_queued, objects_skipped (Number)The number of objects that completed, failed, are still queued, and were skipped.
jobs[*].duration_ms (Number)The duration of the job, in milliseconds.
jobs[*].latency_mean_ms, latency_max_ms (Number)The mean and maximum replication latency, in milliseconds.
jobs[*].created_at, updated_at (String)The time when the job started and last changed.
objects (List of Object)The objects that the job processed. Returned only when you specify job_id.
objects[*].object_name (String)The name of the object.
objects[*].status (String)The status of the job for this object.
objects[*].events_ingested, events_loaded, events_failed (Number)The number of Events that the job read, loaded, and failed to process for this object.
objects[*].offset (String)The position up to which the job read the object.