Managing Objects in Pipelines

An object is an entity in your Source application or database that contains the data Hevo must fetch. For example, a table in your PostgreSQL database forms an object. After a Pipeline is created, Hevo identifies the objects in your Source, and you select the ones you want to replicate and set the load mode for each of them.

Refer to the following sections to control what a Pipeline replicates:

• Listing Objects: Read the objects Hevo identified in your Source, and the status of each one.
• Changing the Object Selection: Choose which objects the Pipeline replicates and how each one is loaded.
• Load Modes: Understand how Hevo writes an object's records into its Destination table.
• Object Groups: Understand the Sources that group objects together, and how a group decides which of its members you can select.
• Viewing the Field Mapping: Read the Source fields of an object and the Destination columns they land in.
• Tracking Replication Progress: Read the point in your Source that each object has been replicated up to.

Prerequisites

• An active Hevo account. You can start with a 14-day full-feature, free trial account.
• You are assigned the required role for the endpoint you want to use.
• You have generated your API credentials. Read Authentication for the steps to generate them.
• An existing Pipeline. Read Creating a Pipeline with Hevo API for the steps to create one.

Listing Objects

To list the objects Hevo identified in your Source, you need to call the List All Pipeline Objects endpoint. The response includes each object's name, its location in the Source, the Destination table it loads into, and its current status.

Perform the following steps to list the objects in a Pipeline:

1. Navigate to the List All Pipeline Objects endpoint.

2. In the Path Params section, specify the pipeline_id. This is the unique identifier of the Pipeline with the objects that you want to read.

   

3. (Optional) In the Query Params section, specify status to return only the objects in the statuses you want.

   

4. The API request is simultaneously generated in the cURL Request block to the right. Click Try It! to make the API request and list the objects.

   

For example, the following request lists the objects in the Pipeline with the id 12345:

GET /api/v2/pipelines/12345/objects

Once the request is complete, you will get a response code 200 with one page of the Pipeline's objects.

Object Statuses

Each object in the list carries one of the following statuses:

Replication Statuses

The replication field reports whether the structure of the object has been replicated to your Destination, not how its last data load went. This field may not be present for an object.

The group, child_selection_mode, selectable, resync_allowed, allow_load_mode_selection and allowed_load_modes fields may not be present for an object.

The reason field explains the status. It is always present, and is null when there is nothing to report.

Hevo identifies the objects in your Source after the Pipeline is created, and the list is empty until it does. Read the list again after a short interval.

Changing the Object Selection

You can change which objects a Pipeline replicates, and how each one is loaded into the Destination.

Following are the scenarios in which you may want to change the object selection:

• You created the Pipeline in a disabled state and are choosing the objects to replicate before enabling it.
• A new object was added to your Source and you want the Pipeline to replicate it.
• You no longer need an object and want to stop replicating it to reduce the Events consumption.
• An object requires a different load mode, such as changing it to MERGE so that existing records are updated instead of duplicated.

You must consider the following before you change the object selection:

• The Pipeline has to be in the Initialized or Disabled state. A Pipeline in a transitional state returns a 409. A Pipeline in the Failed state returns a 400, and a Pipeline that is being deleted returns a 404. An object name that the Pipeline does not have also returns a 404, and Hevo checks every name before it writes anything.

• The request is applied as a partial update. An object you do not specify keeps its current selection and load mode.

• The request must specify at least one object, and an object may be specified only once. The selection_mode value is optional, is not case-sensitive, and defaults to RETAIN_EXISTING. With INCLUDE_ALL or EXCLUDE_ALL you can leave the objects list out, or send it empty. Unless the Pipeline is disabled, a request that would leave the Pipeline with no object selected, or an object with no field selected, is rejected.

• Selecting an object starts replicating it, and its historical load runs on the next Pipeline run.

• Deselecting an object stops replication but leaves the Destination table and the data already loaded into it untouched.

• include_fields and exclude_fields can be set only on an object with selected set to true.

• Custom primary keys are not accepted. A request that sends primary_keys returns a 400. Primary keys come from your Source.

• A field name that the object does not have is rejected.

  A field named in both lists, or twice in one list, is also rejected, and so is a blank name. Names are matched exactly, so they are case-sensitive.

• You may not be able to exclude a primary-key or foreign-key field. If the List All Pipeline Objects endpoint does not return the selectable field for an object, you can exclude a primary-key field after you set its load_mode to APPEND.

• The load_mode value must be one of the object’s allowed_load_modes, and the object’s allow_load_mode_selection value must be true. OVERWRITE is never accepted here. An object with a selectable value of false cannot be selected.

Perform the following steps to change the object selection:

1. Navigate to the Manage Objects in a Pipeline (PATCH) endpoint.

2. In the Path Params section, specify the pipeline_id. This is the unique identifier of the Pipeline with the objects that you want to change.

   

3. In the Body Params section, specify selection_mode and the objects you want to change. Read Selection Modes and Object Settings for the values each one accepts.

   

4. The API request is simultaneously generated in the cURL Request block to the right. Click Try It! to make the API request and change the object selection.

   

For example, the following request selects one object in the Pipeline with the id 12345 and loads it in MERGE mode:

PATCH /api/v2/pipelines/12345/objects

Send the following request body with it:

{
  "selection_mode": "RETAIN_EXISTING",
  "objects": [
    {
      "name": "public.orders",
      "selected": true,
      "load_mode": "MERGE"
    }
  ]
}

Selection Modes

The selection_mode field sets the selection of every object you do not specify in the request:

Object Settings

Each object in the objects array accepts the following settings:

Once the change is accepted, you will get a response code 202. If the Pipeline is in the Disabled state, it remains Disabled.

Read the Pipeline at regular intervals until its state stops changing, because a replicating Pipeline is reported as Initializing while the change is applied. Refer to Reading one Pipeline for the steps.

Note:

• If your client or proxy cannot send a PATCH request, send the same request body as a POST request to the Manage Objects in a Pipeline (POST) endpoint.
• In version 1, call the Manage Objects in a Pipeline endpoint instead. Version 1 takes a different request body. It names the objects in include_objects and exclude_objects alongside selection_mode, rather than in a single objects array, and it does not accept a load mode or field selection for each object.

Load Modes

The load mode sets the method Hevo uses to write an object's records into its Destination table:

An object inherits the Pipeline's load mode unless you set one on the object.

Object Groups

Some Sources group related objects into a parent object and its child objects. The parent object decides when its child objects can be selected. An object returns its parent object in group, and the rule that its parent object applies in child_selection_mode:

Read group, child_selection_mode, and selectable on an object before you change its selection, so that you know the rule that applies to it. For example, an object that returns conversation_thread in group and AUTO_FOLLOW_HEAD in child_selection_mode is a child object of conversation_thread. To replicate it, you select the conversation_thread object instead, and Hevo includes that object along with every other child object of conversation_thread.

A request that changes a child object's selection on its own is rejected, and the response names the parent object to select instead. Objects are named by their fully qualified name.

If the List All Pipeline Objects endpoint does not return the selectable field for an object, you may not be able to name its child objects in the request. In that case, change the parent object instead, and the child objects take its selection.

Viewing the Field Mapping

To read one object with its field mapping, you need to call the Get Details of a Pipeline Object endpoint.

Perform the following steps to view the field mapping:

1. Navigate to the Get Details of a Pipeline Object endpoint.

2. In the Path Params section, specify the pipeline_id and the object_name, which is the object’s fully qualified name. Read Listing Objects for the steps to find the fully qualified name.

   

3. The API request is simultaneously generated in the cURL Request block to the right. Click Try It! to make the API request and read the object.

   

For example, the following request reads one object in the Pipeline with the id 12345:

GET /api/v2/pipelines/12345/objects/public.orders

Once the request is complete, you will get a response code 200 with every Source field, the Destination column it maps to, the data type in the Source and in the Destination, and whether the field is part of the primary key. Each field returns source_name, source_type, destination_name, destination_type, status, primary_key, foreign_key, selected and selectable.

Tracking Replication Progress

Hevo maintains a position for each object to identify the point up to which its data has been replicated from the Source. To read the position of every active object in a Pipeline, you need to call the Get Replication Position of a Pipeline endpoint.

Perform the following steps to read the replication position:

1. Navigate to the Get Replication Position of a Pipeline endpoint.

2. In the Path Params section, specify the pipeline_id. This is the unique identifier of the Pipeline with the objects that you want to read.

   

3. The API request is simultaneously generated in the cURL Request block to the right. Click Try It! to make the API request and read the replication position.

   

For example, the following request reads the replication position of the objects in the Pipeline with the id 12345:

GET /api/v2/pipelines/12345/position

Once the request is complete, you will get a response code 200 with the position of each active object. The format depends on the Source type. A PostgreSQL, MySQL or Oracle Source returns the UTC time of the last change that Hevo replicated, such as Aug 27, 2026 13:05 UTC for PostgreSQL and Aug 27, 2026, 13:05 UTC for MySQL and Oracle, and a SQL Server Source returns its commit and change log sequence numbers.

The position field is null for an object that has not been ingested yet, and for every object in the response when the position is temporarily unavailable. The request still succeeds, so read it again after a short interval.

Each object replicates independently, so a Pipeline has no single overall position.