On Azure(VM)
The instructions have been tested on Azure VM Linux (ubuntu 18.04)

Launch Azure Cloud Shell

Launch cloud shell by going to https://shell.azure.com/bash

Create a new virtual machine

Create resource group

1
# Inside Azure cloud shell
2
rgName=airbyte-demo
3
rgLocation=eastus
4
az group create --name $rgName --location $rgLocation
Copied!

Create virtual machine

1
# Inside Azure cloud shell
2
userName=byteuser
3
vmName=airbyte
4
dnsName=$(head -3 /dev/urandom | tr -dc a-z | cut -c -16)
5
publicIp=$(az vm create --resource-group $rgName \
6
--name $vmName --image UbuntuLTS \
7
--admin-username $userName \
8
--public-ip-address-dns-name $dnsName \
9
--generate-ssh-keys --query 'publicIpAddress' -o json)
10
echo $publicIp
Copied!
This step will create a virtual machine and add a user account named byteuser. The --generate-ssh-keys option will generate a new ssh key and put it to the default key location (~/.ssh)
Note: Copy the publicIpAddress output, you will need this address later to connect from your workstation.

Download SSH key

1
# Inside Azure cloud shell
2
download ~/.ssh/id_rsa
Copied!
Make sure to update the permissions on the private key, or you'll get an error telling you that permissions for this file are too open.
1
chmod 600 ./$YOUR_PATH_TO_DOWNLOADS/id_rsa
Copied!
Above command will generate download link and give you pop-up on right bottom side, click on Click here to download your file. to download private key. Note: Save this file, you will need it to connect to your VM in Connect to Airbyte step.

Connect to virtual machine

If you get this error: Could not resolve hostname "XX.XXX.X.XXX": Name or service not known, just manually enter the publicIp host name when running the ssh command.
1
# Inside Azure cloud shell
2
ssh $userName@$publicIp
Copied!

Install environment

  • Install docker
1
# Inside Azure cloud shell
2
sudo apt-get update -y
3
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y
4
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
5
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
6
sudo apt-get update
7
sudo apt-get install docker-ce docker-ce-cli -y
8
sudo usermod -a -G docker $USER
Copied!
  • Install docker-compose
1
# Inside Azure cloud shell
2
sudo wget https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m) -O /usr/local/bin/docker-compose
3
sudo chmod +x /usr/local/bin/docker-compose
4
docker-compose --version
Copied!
  • Close the ssh connection to ensure the group modification is taken into account
1
# Inside Azure cloud shell
2
logout
Copied!
  • Reconnect to virtual machine
1
# Inside Azure cloud shell
2
ssh $userName@$publicIp
Copied!

Install and Start Airbyte

1
# Inside Azure cloud shell
2
mkdir airbyte && cd airbyte
3
wget https://raw.githubusercontent.com/airbytehq/airbyte/master/{.env,docker-compose.yaml}
4
sudo docker-compose up -d
Copied!

Connect to Airbyte

For security reasons, we strongly recommend to not expose Airbyte on Internet available ports. Future versions will add support for SSL & Authentication.
This part assumes that you have access to a terminal on your workstation
  • Create ssh tunnel for port 8000
    1
    # Inside your workstation terminal
    2
    # 1. Replace $SSH_KEY with private key path downloaded from earlier steps
    3
    # 2. Replace $INSTANCE_IP with publicIpAddress noted from earlier steps
    4
    ssh -N -L 8000:localhost:8000 -i $SSH_KEY [email protected]$INSTANCE_IP
    Copied!
  • Just visit http://localhost:8000 in your browser and start moving some data!

Troubleshooting

If you encounter any issues, just connect to our Slack. Our community will help! We also have a FAQ section in our docs for common problems.
Last modified 2mo ago