Orb Sync Engine

Continuously synchronizes an Orb account to a Postgres database.

Motivation

Sometimes you want to analyze your billing data using SQL. Even more importantly, you want to join your billing data to your product/business data.

This server synchronizes your Orb account to a Postgres database. It can be a new database, or an existing Postgres database.

How it works

• Creates a new schema orb in a Postgres database, with tables & columns matching Orb.
• Exposes a /webhooks endpoint that listens to any Orb webhooks.
• Inserts/updates/deletes changes into the tables whenever there is a change to Orb.

Not implemented

• This will not do an initial load of existing Orb data. You should use CSV loads for this. We might implement this in the future.
• Backfill of data
• Entities that are not supported through webhooks like plans
• orb-sync-lib is not yet published as standalone npm package (could be used in serverless functions or anywhere else)
• Database migration not handled automatically, need to run migrations from db/migrations manually for now

Supported Webhooks

• billable_metric.edited
• price.edited
• customer.balance_transaction_created
• customer.created
• customer.credit_balance_depleted
• customer.credit_balance_dropped
• customer.edited
• data_exports.transfer_success (ignored)
• data_exports.transfer_error (ignored)
• subscription.created
• subscription.started
• subscription.fixed_fee_quantity_updated
• subscription.edited
• subscription.ended
• subscription.plan_changed
• subscription.usage_exceeded
• subscription.cost_exceeded
• subscription.plan_version_change_scheduled
• subscription.plan_version_changed
• invoice.issue_failed
• invoice.issued
• invoice.payment_failed
• invoice.payment_processing
• invoice.payment_succeeded
• invoice.edited
• invoice.manually_marked_as_void
• invoice.manually_marked_as_paid
• invoice.undo_mark_as_paid
• invoice.sync_succeded
• invoice.sync_failed
• credit_note.issued
• credit_note.marked_as_void
• resource_event.test

Development

Develop

• Start a Postgres instance using cd docker && docker-compose up
• mv .env.sample .env and adjust values as necessary
• npm run dev:node-fastify to start the local server
• npm run test to run tests across the repo

Building Docker

cd apps/node-fastify
docker build -t orb-sync-engine .
# Ensure env vars are set
docker run -p 8080:8080 orb-sync-engine

Inspiration

Inspired by stripe-sync-engine