Skip to main content

Reddit

Availability
Available CoreAvailable StandardAvailable PlusAvailable ProAvailable Enterprise FlexAvailable PyAirbyte
Connector type (beta)
Available Data replicationNot available Agent(learn more)
Support Level
Marketplace
Connector Version
0.0.68 (last updated 2 days ago)
Data workers used
0.2 per sync (Learn more)
Definition ID
3ed344ac-4099-402c-bf83-1cfdc53295d9

Overview​

The Reddit source supports Full Refresh as well as Incremental syncs.

Full Refresh sync means every time a sync is run, Airbyte will copy all rows in the tables and columns you set up for replication into the destination in a new table. Incremental sync means only changed resources are copied from Reddit. For the first run, it will be a Full Refresh sync.

Setup guide​

Step 1: Create a Reddit app​

  1. Visit Reddit's app preferences page and select create another app.
  2. Choose the app type:
    • script if you want to authenticate with your own Reddit username and password (simplest; does not work for accounts with two-factor authentication).
    • web app if you want to authenticate with an OAuth 2.0 refresh token.
  3. Set a redirect URI (any URL you control, for example http://localhost:8080) and save the app.
  4. Copy the Client ID (shown below the app name) and the Client Secret (labeled secret).

Step 2: Choose an authentication method​

Reddit access tokens expire after one hour, so pick one of the first two options to let Airbyte refresh the token on every sync.

Option A: Reddit username and password (script app)​

Select Reddit username and password (script app) and enter the Client ID and Client Secret from Step 1 together with the Reddit username and password of the account that owns the app. Airbyte requests a fresh access token with the OAuth 2.0 password grant before each sync.

Option B: OAuth 2.0 refresh token​

Select OAuth 2.0 (refresh token) and enter the Client ID, Client Secret and a refresh token. To obtain a refresh token, complete the Reddit authorization code flow with duration=permanent and the scopes you need (identity, read, privatemessages, mysubreddits), then exchange the returned code at https://www.reddit.com/api/v1/access_token (HTTP Basic auth with the Client ID as the username and the Client Secret as the password, body grant_type=authorization_code&code=<code>&redirect_uri=<redirect_uri>). Copy the refresh_token from the response.

Option C: Access token (expires after one hour)​

Select Access token (expires after one hour) and paste an access token obtained manually, for example with Postman:

  • Request - POST https://www.reddit.com/api/v1/access_token
  • Authorization - Basic Auth - username: <CLIENT_ID>, password: <CLIENT_SECRET>
  • Body - x-www-form-urlencoded - grant_type: password, username: YOUR_REDDIT_USERNAME, password: YOUR_REDDIT_PASSWORD

The access_token in the response is the API Key. Because Airbyte cannot refresh it, this option only works for one-off syncs; existing connections configured with the legacy top-level api_key field are migrated to this option automatically.

Records and rate limiting​

  • The Reddit API has rate limiting of 100 queries per minute (QPM) per OAuth client ID. It is handled with an exponential backoff strategy, with maximum 3 retries.
  • Access tokens expire after one hour. With the username/password or refresh token options Airbyte obtains a new token automatically; with the access token option a new token must be generated manually.
  • Reddit requires a descriptive User-Agent; the connector sends airbyte:source-reddit:v1 (+https://docs.airbyte.com/integrations/sources/reddit) on every request.
  • The Reddit API has a hard limit of fetching 1000 records per single stream call with subsequent pagination.

Configuration​

InputTypeDescriptionDefault Value
credentialsobjectAuthentication. One of: OAuth 2.0 refresh token (client_id, client_secret, refresh_token), Reddit username and password (client_id, client_secret, username, password), or a short-lived access token (api_key).
querystringQuery. Specifies the query for searching in reddits and subredditsairbyte
include_over_18booleanInclude over 18 flag. Includes mature contentfalse
exactbooleanExact. Specifies exact keyword and reduces distractions
limitnumberLimit. Max records per page limit1000
subredditsarraySubreddits. Subreddits for exploration (funny, r/funny and /r/funny are all accepted)[r/funny, r/AskReddit]
start_datestringStart date.

Streams​

Stream NamePrimary KeyPaginationSupports Full SyncSupports Incremental
selfnameNo pagination✅❌
searchDefaultPaginator✅❌
subreddit_searchDefaultPaginator✅❌
message_inboxDefaultPaginator✅❌
subreddit_popularDefaultPaginator✅❌
subreddit_exploreDefaultPaginator✅✅

IP allow list​

If you use Airbyte Cloud and your organization restricts access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

Reference​

Config fields reference

Field
Type
Property name
object
credentials
string
start_date
boolean
exact
boolean
include_over_18
number
limit
string
query
array
subreddits

Changelog​

Expand to review
VersionDatePull RequestSubject
0.0.682026-10-1388766Update dependencies
0.0.672026-10-0788257Add refreshable OAuth 2.0 (refresh token / password grant) authentication, send a descriptive User-Agent, fix subreddit_explore to iterate over the configured subreddits and read its created cursor, and harden pagination against non-object data responses
0.0.662026-10-0688007Update dependencies
0.0.652026-09-2987320Update dependencies
0.0.642026-09-2286793Update dependencies
0.0.632026-09-1586199Update dependencies
0.0.622026-09-0885627Update dependencies
0.0.612026-08-1884754Update dependencies
0.0.602026-08-1184090Update dependencies
0.0.592026-08-0483607Update dependencies
0.0.582026-07-2883067Update dependencies
0.0.572026-07-2182583Update dependencies
0.0.562026-07-1481975Update dependencies
0.0.552026-06-3081234Update dependencies
0.0.542026-06-2380628Update dependencies
0.0.532026-06-1680049Update dependencies
0.0.522026-06-0979512Update dependencies
0.0.512026-06-0278919Update dependencies
0.0.502026-04-2877395Update dependencies
0.0.492026-04-2176717Update dependencies
0.0.482026-03-3175878Update dependencies
0.0.472026-03-1774911Update dependencies
0.0.462026-02-2473532Update dependencies
0.0.452026-02-0372651Update dependencies
0.0.442026-01-2072024Update dependencies
0.0.432026-01-1471479Update dependencies
0.0.422025-12-1870617Update dependencies
0.0.412025-11-2570060Update dependencies
0.0.402025-11-1869624Update dependencies
0.0.392025-10-2968912Update dependencies
0.0.382025-10-2168335Update dependencies
0.0.372025-10-1467872Update dependencies
0.0.362025-10-0867541Update dependencies
0.0.352025-09-3066443Update dependencies
0.0.342025-09-0965716Update dependencies
0.0.332025-08-2465446Update dependencies
0.0.322025-08-1064831Update dependencies
0.0.312025-08-0264457Update dependencies
0.0.302025-07-1963628Update dependencies
0.0.292025-07-1263054Update dependencies
0.0.282025-07-0562697Update dependencies
0.0.272025-06-2862217Update dependencies
0.0.262025-06-2161787Update dependencies
0.0.252025-06-1460557Update dependencies
0.0.242025-05-1060162Update dependencies
0.0.232025-05-0359473Update dependencies
0.0.222025-04-2759106Update dependencies
0.0.212025-04-1958458Update dependencies
0.0.202025-04-1257923Update dependencies
0.0.192025-04-0557298Update dependencies
0.0.182025-03-2956770Update dependencies
0.0.172025-03-2256163Update dependencies
0.0.162025-03-0855061Update dependencies
0.0.152025-02-2354586Update dependencies
0.0.142025-02-1553957Update dependencies
0.0.132025-02-0853455Update dependencies
0.0.122025-02-0153004Update dependencies
0.0.112025-01-2552494Update dependencies
0.0.102025-01-1851854Update dependencies
0.0.92025-01-1151376Update dependencies
0.0.82024-12-2850683Update dependencies
0.0.72024-12-2150232Update dependencies
0.0.62024-12-1449697Update dependencies
0.0.52024-12-1249368Update dependencies
0.0.42024-12-1149104Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
0.0.32024-10-2947827Update dependencies
0.0.22024-10-2847542Update dependencies
0.0.12024-08-2344579Initial release by btkcodedev via Connector Builder
Was this page helpful?