Deployment

Environments

EnvironmentURLBranchDatabaseStorage
Productiondatahub.iomaindatahub-cloud (Neon)datahub-cloud (R2)
Stagingstaging-cloud.datahub.iostagingdatahub-cloud-staging (Neon)datahub-cloud-staging (R2)
Preview*.vercel.appPR branchesstaging DBstaging storage

Deployment Pipeline

Deployments are managed by Vercel:

  1. PR Preview: Every PR gets an automatic Vercel preview deployment using the staging database/storage.
  2. Staging: Merging to staging deploys to the staging environment.
  3. Production: Rebasing staging onto main deploys to production.

The build command (pnpm build) runs Prisma migrations automatically:

prisma migrate deploy && prisma generate && next build

Infrastructure

Databases

PostgreSQL hosted on Neon (via Vercel integration):

  • Production: datahub-cloud
  • Staging: datahub-cloud-staging
  • Development: Local PostgreSQL via Docker (see DEVELOP.md)

Content Storage

Cloudflare R2 buckets:

  • Production: datahub-cloud
  • Staging: datahub-cloud-staging
  • Development: Local MinIO via Docker (see DEVELOP.md)

Authentication

GitHub OAuth applications (under Datopian account):

  • Production: DataHub Cloud
  • Staging: DataHub Cloud - Staging
  • Development: DataHub Cloud - Dev

Domain Configuration

Root Domain (datahub.io)

All routes are served by this single Next.js app. The middleware (middleware.ts) handles routing:

  • Landing pages (/, /pricing, /collections, /solutions, /publish) are rewritten to app/home/*
  • /@username paths are rewritten to app/user/*
  • /<publication>/<post> paths route to app/[publication]/[post]/*
  • /dashboard and /login have auth guards
  • Custom domains are rewritten to /_domain={hostname}/*

Subdomains

  • Production: datahub.io
  • Staging: staging-datahub.io (or staging-cloud.datahub.io)

Background Jobs (Inngest)

  • Production/Staging: Inngest Cloud (https://app.inngest.com, invite required)
  • Development: Local Inngest dev server via make dev

Cloudflare Worker (content processing)

A separate Cloudflare Worker processes markdown uploads:

  • Repo: datahub-cloudflare-workers
  • Deployed independently to Cloudflare
  • Consumes R2 queue events (prod/staging) or MinIO webhooks (dev)
  • See ARCHITECTURE.md for details

Environment Variables

Production and staging environment variables are configured in Vercel project settings.

Required variables (see .env.example for full list):

  • POSTGRES_PRISMA_URL / POSTGRES_URL_NON_POOLING - Database connections
  • NEXTAUTH_SECRET - Session encryption key
  • NEXT_PUBLIC_AUTH_GITHUB_ID / AUTH_GITHUB_SECRET - GitHub OAuth
  • S3_* variables - R2 storage configuration
  • GH_ACCESS_TOKEN / GH_WEBHOOK_SECRET - GitHub integration
  • INNGEST_APP_ID - Inngest configuration
  • NEXT_PUBLIC_ROOT_DOMAIN / NEXT_PUBLIC_CLOUD_DOMAIN - Domain configuration

CI/CD

GitHub Actions workflows (.github/workflows/):

WorkflowTriggerDescription
e2e.ymlPR to main/staging, push to stagingE2E tests against Vercel preview deployment
lint.ymlPR to main/staging, push to stagingESLint checks
unit.ymlPR to main/staging, push to mainJest unit tests
sync-ga-views.ymlScheduled (daily)Sync Google Analytics view counts