Nexus Datasets
Nexus datasets is a solution to sync up with the current nexus data sets API. Data can be extracted from the nexus API after providing the required data in the config. Data set name should be provided aliong with the other details to sync the data. API Documentation
Prerequisites
- Nexus customer organization with Data Mesh rights
- Export data set created in the customer organization
- Data API Agent user configured in the customer organization
Airbyte OSS and Airbyte Cloud
- Name of the data set to be synced
- Data API Agent user name / Cliend access key ID / Secret key for the Data API Agent user
- Data Key (API Key) for the customer organization
Setup guide
Step 1: Nexus Configuration
Step 1.1: Create Nexus Customer Organization
- Create customer organization in the nexus platform
Step 1.2: Configure Data Mesh in the Customer Organization
- Log into the admin side of the nexus application
- Set up the Data Mesh rule for the organization
- Create users with the Data Mesh previledges
- Make sure this user can be logged in to the nexus platfrom
Step 1.3: Configure export data pipeline in the Nexus
- Log into the nexus platform using created user in the step 2
- Go to Analytics » Data Mesh Console
- Click on Export tab to go into the exportable data sets
- Click Create to create new export pipeline using the existing models
- Schedule the data set to run
Step 1.4: Summary
- Base URL for the nexus platform
- Nexus customer organization with relevant user with rights
- Export data set
- Data API Agent user (DAPI user)
- DAPI user id / access key id / secret key
- Data key (API key)
Step 2: Set up the source connector in Airbyte
For Airbyte Cloud:
- Log into your Airbyte Cloud account.
- In the left navigation bar, click Sources. In the top-right corner, click + new source.
- On the source setup page, select Nexus Datasets from the Source type dropdown and enter a name for this connector.
- Add Base URL
- Add Dataset Name
- Add Infor Streaming Mode (default Full)
- Add User ID
- Add Access Key ID
- Add Secret Key
- Add API Key
- Click
Set up source
For Airbyte OSS:
- Log into your Airbyte Cloud account.
- In the left navigation bar, click Sources. In the top-right corner, click + new source.
- On the source setup page, select Nexus Datasets from the Source type dropdown and enter a name for this connector.
- Add Base URL
- Add Dataset Name
- Add Infor Streaming Mode (default Full)
- Add User ID
- Add Access Key ID
- Add Secret Key
- Add API Key
- Click
Set up source.
Configuration
| Input | Type | Description | Default Value |
|---|---|---|---|
base_url | string | Base URL. Enter base url for your data set | |
dataset_name | string | Name of the dataset. Enter dataset name to be synced | |
infor_streaming_mode | string | Name of the dataset. Enter dataset name to be synced | |
user_id | string | Data API agent user ID. Enter DAPI agent user id configured in the nexus | |
access_key_id | string | Access key ID. Enter access key ID for the DAPI agent user | |
secret_key | string | Secret key. Enter secret key for the DAPI agent user | |
api_key | string | Data API key. Enter data API key for the organization |
Supported sync modes
The Nexus Datasets source connector supports the following sync modes:
- Full Refresh
- Incremental
Streams
| Stream Name | Primary Key | Pagination | Supports Full Sync | Supports Incremental |
|---|---|---|---|---|
| datasets | ❌ | DefaultPaginator | ✅ | ✅ |
Custom Components
NexusCustomAuthenticator is a primary component which handles the HMAC authentication for nexus API.
HMAC stands for Hash-based Message Authentication Code. In HMAC authentication, every request is independently established using a cryptographic hash function. For each API request, the client computes a hashed "signature" using a secret key and submits it in the Authorization header.
Please refer https://developer.infornexus.com/api/authentication-choices/hmac for more details to get the data to calculate the HMAC signature.
Error Codes
200 : Data set is ready to stream 202 : Data set is not ready but try again later. 304 : Data set is not ready, check the source again.