Contributing to Airbyte
Thank you for your interest in contributing! We love community contributions. Contribution guidelines are listed below. If you're unsure about how to start contributing or have any questions even after reading them, feel free to ask us on Slack in the #dev or #general channel.
However, for those who want a bit more guidance on the best way to contribute to Airbyte, read on. This document will cover what we're looking for. By addressing the points below, the chances that we can quickly merge or address your contributions will increase.
Code of conduct
Please follow our Code of conduct in the context of any contributions made to Airbyte.
Airbyte specification
Before you can start contributing, you need to understand Airbyte's data protocol specification.
First-time contributors, welcome!
We appreciate first time contributors and we are happy to assist you in getting started. In case of questions, just reach out to us via email or Slack!
Here is a list of easy good first issues to do.
Contributing to the codebase
We gladly welcome all improvements to the codebase.
Steps to contributing code
1. Open an issue, or find a similar one.
Before jumping into the code please first:
- Verify if an existing connector or platform GitHub issue matches your contribution project.
- If you don't find an existing issue, create a new connector or platform issue to explain what you want to achieve.
- Assign the issue to yourself and add a comment to tell that you want to work on this.
This will enable our team to make sure your contribution does not overlap with existing works and will comply with the design orientation we are currently heading the product toward. If you do not receive an update on the issue from our team, please ping us on Slack!
2. Code your contribution
- To contribute to a connector, fork the Connector repository. To contribute to the Airbyte platform, fork our Platform repository.
- If contributing a new connector, check out our new connectors guide.
- Open a branch for your work.
- Code, and please write tests.
- Ensure all tests pass. For connectors, this includes acceptance tests as well.
- For connectors, make sure to increment the connector's version according to our Semantic Versioning guidelines.
3. Open a pull request
- Rebase master with your branch before submitting a pull request.
- Open the pull request.
- Wait for a review from a community maintainer or our team.
4. Review process
When we review, we look at:
- Does the PR solve the issue?
- Is the proposed solution reasonable?
- Is it tested? (unit tests or integration tests)
- Is it introducing security risks? Once your PR passes, we will merge it 🎉.
New connectors
It's easy to add your own connector to Airbyte! Since Airbyte connectors are encapsulated within Docker containers, you can use any language you like. Here are some links on how to add sources and destinations. We haven't built the documentation for all languages yet, so don't hesitate to reach out to us if you'd like help developing connectors in other languages.
For sources, simply head over to our Python CDK.
The CDK currently does not support creating destinations, but it will very soon.
- See Building new connectors to get started.
- Since we frequently build connectors in Python, on top of Singer or in Java, we've created generator libraries to get you started quickly: Build Python Source Connectors and Build Java Destination Connectors
- Integration tests (tests that run a connector's image against an external resource) can be run one of three ways, as detailed here
Please note that, at no point in time, we will ask you to maintain your connector. The goal is that the Airbyte team and the community helps maintain the connector.
Semantic versioning for connectors
Changes to connector behavior should always be accompanied by a version bump and a changelog entry. We use semantic versioning to version changes to connectors. Since connectors are a bit different from APIs, we have our own take on semantic versioning, focusing on maintaining the best user experience of using a connector.
- Major: a version in which a change is made which requires manual intervention (update to config or configured catalog) for an existing connection to continue to succeed, or one in which data that was previously being synced will no longer be synced
- Minor: a version that introduces user-facing functionality in a backwards compatible manner
- Patch: a version that introduces backwards compatible bug fixes or performance improvements
Examples
Here are some examples of code changes and their respective version changes:
| Change | Impact | Version Change |
|---|---|---|
Adding a required parameter to a connector's spec | Users will have to add the new parameter to their config | Major |
Changing a format of a parameter in a connector's spec from a single parameter to a oneOf | Users will have to edit their config to define their old parameter value in the oneOf format | Major |
Removing a stream from a connector's catalog | Data that was being synced will no longer be synced | Major |
Renaming a stream in a connector's catalog | Users will have to update the name of the stream in their catalog | Major |
Removing a column from a stream in a connector's catalog | Users will have to remove that column from their catalog, data that was being synced will no longer be synced | Major |
Renaming a column from a stream in a connector's catalog | Users will have to update the name of the column in their catalog | Major |
Changing the datatype for a column of a stream in a connector's catalog | Users will have to update that data type in their catalog, data that was being synced will have changed format | Major |
Adding a non-required parameter to a connector's spec | Users will have the option to use the required parameter in the future | Minor |
Adding a stream in a connector's catalog | Additional data will be synced | Minor |
Adding a column to a stream's schema in a connector's catalog | Additional data will be synced | Minor |
Updating the format of the connector's STATE | Incremental streams will automatically run a full refresh only for the next sync | Patch |
| Optimizing a connector's performance | Syncs will be faster | Patch |
| Fixing a bug in a connector | Some syncs that would have failed will now succeed | Patch |
Trying to contribute, and don't see the change you want to make in this list? Call it out in your PR and your reviewer will help you pick the correct type of version change. Feel free to contribute the results back to this list!
Airbyte CI workflows
Contributing to documentation
Our goal is to keep our docs comprehensive and updated. If you would like to help us in doing so, we are grateful for any kind of contribution:
- Report missing content
- Fix errors in existing docs
- Help us in adding to the docs
The contributing guide for docs can be found here.
Contributing community content
We welcome contributions as new tutorials / showcases / articles, or to any of the existing guides on our tutorials page:
- Fix errors in existing tutorials
- Add new tutorials (please reach out to us if you have ideas to avoid duplicate work)
- Request tutorials
We have a repo dedicated to community content. Everything is documented there.
Feel free to submit a pull request in this repo, if you have something to add even if it's not related to anything mentioned above.
Other ways to contribute
Upvoting issues, feature and connector requests
You are welcome to add your own reactions to the existing issues. We will take them in consideration in our prioritization efforts, especially for connectors.
❤️ means that this task is CRITICAL to you.
👍 means it is important to you.
Requesting new features
To request new features, please create an issue on this project.
If you would like to suggest a new feature, we ask that you please use our issue template. It contains a few essential questions that help us understand the problem you are looking to solve and how you think your recommendation will address it. We also tag incoming issues from this template with the "community_new" label. This lets our teams quickly see what has been raised and better address the community recommendations.
To see what has already been proposed by the community, you can look here.
Watch out for duplicates! If you are creating a new platform issue, please check open, or recently closed.
Requesting new connectors
This is very similar to requesting new features. The template will change a bit and all connector requests will be tagged with the “community” and “area/connectors” labels.
To see what has already been proposed by the community, you can look here. Again, watch out for duplicates!
Reporting bugs
Bug reports help us make Airbyte better for everyone. We provide a preconfigured template for bugs to make it very clear what information we need.
Please search within our already reported bugs before raising a new one to make sure you're not raising a duplicate.
Reporting security issues
Please do not create a public GitHub issue. If you've found a security issue, please email us directly at security@airbyte.io instead of raising an issue.