Deploy Airbyte on AWS (Amazon EC2)
This page guides you through deploying Airbyte Open Source on an Amazon EC2 instance by setting up the deployment environment, installing and starting Airbyte, and connecting it to the Amazon EC2 instance.
The instructions have been tested on Amazon Linux 2 AMI (HVM).
- To test Airbyte, we recommend a
- To deploy Airbyte in a production environment, we recommend a
- Make sure your Docker Desktop app is running to ensure all services run smoothly
- Create and download an SSH key to connect to the instance
Set up the environment
- To connect to your instance, run the following command on your local terminal:
SSH_KEY=~/Downloads/dataline-key-airbyte.pem # the file path you downloaded the key
INSTANCE_IP=REPLACE_WITH_YOUR_INSTANCE_IP # find your IP address in the EC2 console under the Instances tab
chmod 400 $SSH_KEY # or ssh will complain that the key has the wrong permissions
ssh -i $SSH_KEY ec2-user@$INSTANCE_IP # connect to the aws ec2 instance AMI and the your private IP address
- To install Docker, run the following command in your SSH session on the instance terminal:
sudo yum update -y
sudo yum install -y docker
sudo service docker start
sudo usermod -a -G docker $USER
- To install
docker-compose, run the following command in your ssh session on the instance terminal:
sudo yum install -y docker-compose-plugin
docker compose version
- To close the SSH connection, run the following command in your SSH session on the instance terminal:
Install and start Airbyte
In your local terminal, run the following commands:
- Connect to your instance:
ssh -i $SSH_KEY ec2-user@$INSTANCE_IP
- Install Airbyte:
mkdir airbyte && cd airbyte
docker compose up -d # run the Docker container
Connect to Airbyte
For security reasons, we strongly recommend not exposing Airbyte on Internet available ports.
- Create an SSH tunnel for port 8000:
If you want to use different ports, modify
API_URL in your .env file and restart Airbyte.
Run the following commands in your workstation terminal from the downloaded key folder:
# In your workstation terminal
ssh -i $SSH_KEY -L 8000:localhost:8000 -N -f ec2-user@$INSTANCE_IP
http://localhost:8000to verify the deployment.
Get Airbyte logs in CloudWatch
Follow this guide to get your logs from your Airbyte Docker containers in CloudWatch.
If you encounter any issues, reach out to our community on Slack.