Skip to main content

Managing Airbyte Cloud

This page will help you manage your Airbyte Cloud workspaces and understand Airbyte Cloud limitations.

Manage your Airbyte Cloud workspace

An Airbyte Workspace allows you to collaborate with other users and manage connections under a shared billing account.

info

Airbyte credits are assigned per workspace and cannot be transferred between workspaces.

Add users to your workspace

To add a user to your workspace:

  1. On the Airbyte Cloud dashboard, click Settings in the navigation bar.

  2. In the Workspace Settings sidebar, click Access Management.

  3. Click + New user.

  4. On the Add new users dialog, enter the email address of the user you want to invite to your workspace. Click Send invitation.

    info

    The user will have access to only the workspace you invited them to. Also note that they will be added as a workspace admin by default.

Switch between multiple workspaces

To switch between workspaces:

  1. On the Airbyte Cloud dashboard, click the current workspace name under the Airbyte logo in the navigation bar.

  2. Click View all workspaces.

  3. Click the name of the workspace you want to switch to.

Rename a workspace

To rename a workspace:

  1. On the Airbyte Cloud dashboard, click Settings in the navigation bar.

  2. In the Workspace Settings sidebar, click General Settings.

  3. In the Workspace name field, enter the new name for your workspace. Click Save.

Delete a workspace

To delete a workspace:

  1. On the Airbyte Cloud dashboard, click Settings in the navigation bar.

  2. In the Workspace Settings sidebar, click General Settings.

  3. Click Delete your workspace.

Single workspace vs. multiple workspaces

You can use one or multiple workspaces with Airbyte Cloud.

Access

Number of workspacesBenefitsConsiderations
SingleAll users in a workspace have access to the same data.If you add a user to a workspace, you cannot limit their access to specific data within that workspace.
MultipleYou can create multiple workspaces to allow certain users to access the data.Since you have to manage user access for each workspace individually, it can get complicated if you have many users in multiple workspaces.

Billing

Number of workspacesBenefitsConsiderations
SingleYou can use the same payment method for all purchases.Credits pay for the use of resources in a workspace when you run a sync. Resource usage cannot be divided and paid for separately (for example, you cannot bill different departments in your organization for the usage of some credits in one workspace).
MultipleWorkspaces are independent of each other, so you can use a different payment method card for each workspace (for example, different credit cards per department in your organization).You can use the same payment method for different workspaces, but each workspace is billed separately. Managing billing for each workspace can become complicated if you have many workspaces.

Manage Airbyte Cloud notifications

To set up Slack notifications:

  1. On the Airbyte Cloud dashboard, click Settings in the navigation bar.

  2. In the Workspace Settings sidebar, click Notifications.

  3. Create an Incoming Webhook for Slack.

  4. Navigate back to the Airbyte Cloud dashboard > Settings > Notifications and enter the Webhook URL. Click Save changes.

  5. Toggle the Send notifications when sync fails and Send notifications when sync succeeds buttons as required.

Understand Airbyte Cloud limits

Understanding the following limitations will help you better manage Airbyte Cloud:

  • Max number of workspaces per user: 100
  • Max number of sources in a workspace: 100
  • Max number of destinations in a workspace: 100
  • Max number of connection in a workspace: 100
  • Max number of streams that can be returned by a source in a discover call: 1K
  • Max number of streams that can be configured to sync in a single connection: 1K
  • Size of a single record: 100MB
  • Shortest sync schedule: Every 60 min
  • Schedule accuracy: +/- 30 min

View the sync log summary

The sync log summary shows if a sync succeeded, failed, cancelled, or is currently running.

To view the sync log summary:

  1. On the Airbyte Cloud dashboard, click Connections in the navigation bar.

    The Connections page displays.

  2. Click a connection in the list to view its sync history.

  3. Under Sync History, a list of syncs for that connection is displayed.

    The sync status is displayed (Succeeded, Cancelled, Failed, or Running) along with the summary of the sync log.

  4. For more information about a sync, click the sync log dropdown.

Sync log summary

Succeeded

The log summary for a successful sync shows the following data.

LogDescription
x GB (also measured in KB, MB)Amount of data moved during the sync. If basic normalization is on, the amount of data would not change since normalization occurs in the destination.
x emitted recordsNumber of records read from the source during the sync.
x committed recordsNumber of records the destination confirmed it received.
xh xm xsTotal time (hours, minutes, seconds) for the sync and basic normalization, if enabled, to complete.
Sync or ResetThe action that was performed (either a sync or a reset).
note

In a successful sync, the number of emitted records and committed records should be the same.

Cancelled

The log summary for a cancelled sync may show the following data.

LogDescription
NaN BytesSince the sync was cancelled, the log does not show how much data was moved before cancelling.
No recordsSince the sync was cancelled, the log does not show emitted or committed records.
xh xm xsTotal time (hours, minutes, seconds) between the beginning of sync and when it was cancelled.
Sync or ResetThe action that was performed (either a sync or a reset).
Failure OriginThe cause of the sync cancellation.
note

Airbyte will try to sync your data three times. After a third failure, it will stop attempting to sync.

Failed

The log summary for a failed sync may show the following data.

LogDescription
NaN BytesSince the sync failed, the amount of data is zero.
No recordsSince the sync failed, there are no emitted or committed records.
xh xm xsTotal time (hours, minutes, seconds) between the beginning of the sync and when it failed.
Sync or ResetThe action that was performed (either a sync or a reset).
Failure OriginThe cause of the sync failure.

Running

The sync log summary displays Running when the sync is actively running.

Edit stream configuration

  1. On the Airbyte Cloud dashboard, click Connections in the navigation bar and then click a connection in the list you want to change.

    The Connection page displays.

  2. Click Replication.

    The Replication tab displays.

The Transfer and Streams settings include the following parameters:

ParameterDescription
Replication FrequencyHow often the data syncs
Destination NamespaceWhere the replicated data is written
Destination Stream PrefixHelps you identify streams from different connectors
note

These parameters apply to all streams in the connection.

If you need to use cron scheduling:

  1. In the Replication Frequency dropdown, click Cron.
  2. Enter a cron expression and choose a time zone to create a sync schedule.
info

Only one sync per connection can run at a time. If cron schedules a sync to run before the last sync finishes, the sync will start after the last one completes.

In the Activate the streams you want to sync section, you can make changes to any stream you choose.

To search for a stream:

  1. Click the Search stream name search box.

  2. Type the name of the stream you want to find.

  3. Streams matching your search are displayed in the list.

To change individual stream configuration:

Single Edit Gif 7

  1. In the Sync column of the stream, toggle the sync on or off.

  2. Click the dropdown arrow in the Sync mode column and select the sync mode you want to apply.

note

Depending on the sync mode you select, you may need to choose a cursor or primary key.

  1. If there is a dropdown arrow in the Cursor or Primary key fields, click the dropdown arrow and choose the cursor or primary key.

To change multiple stream configurations:

Batch Edit gif 5

  1. Click the first checkbox in the table header to select all streams in the connection.

  2. Deselect the checkboxes of streams you do not want to apply these changes to.

  3. In the highlighted header of the table, toggle the sync on or off.

  4. Click the dropdown arrow in the Sync mode column and select the sync mode you want to apply to these streams.

  5. If there is a dropdown arrow in the Cursor or Primary key fields of the highlighted table header, click the dropdown arrow and choose the cursor or primary key.

  6. Click Apply to apply these changes to the streams you selected, or click Cancel to discard the changes.

To save the changes:

  1. Click Save changes, or click Cancel to discard the changes.

  2. The Stream configuration changed dialog displays. This gives you the option to reset streams when you save the changes.

caution

Airbyte recommends that you reset the streams. A reset will delete data in the destination of the affected streams and then re-sync that data. Skipping the reset is discouraged and might lead to unexpected behavior.

  1. Click Save connection, or click Cancel to close the dialog.

To refresh the source schema:

  1. Click Refresh source schema to fetch the schema of your data source.

  2. If there are changes to the schema, the Refreshed source schema dialog displays them.

Buy credits

This section guides you through purchasing credits on Airbyte Cloud. An Airbyte credit is a unit of measure used to pay for Airbyte resources when you run a sync.

To buy credits:

  1. On the Airbyte Cloud dashboard, click the coin icon in the navigation bar.

    The Credits page displays.

  2. If you are unsure of how many credits you need, click Talk to Sales to find the right amount for your team.

  3. Click Buy credits. The Stripe payment page displays.

  4. If you want to change the amount of credits, click Qty 200. The Update quantity dialog displays, and you can either type the amount or use minus (-) or plus (+) to change the quantity. Click Update.

    note

    Purchase limits:

    • Minimum: 100 credits
    • Maximum: 999 credits

    To buy more credits or a subscription plan, reach out to Sales.

  5. Fill out the payment information.

    After you enter your billing address, sales tax is calculated and added to the total.

  6. Click Pay.

    Your payment is processed, and the Credits page displays the updated quantity of credits, total credit usage, and the credit usage per connection.

    A receipt for your purchase is sent to your email. Email us for an invoice.

    note

    Credits expire after one year if they are not used.