Setup bigquery

Configure service account key to authenticate to google bigquery

• Open a terminal in the d2e directory.
  • Run the following command to set an environment variable pointing to the json key file path in your local file system.

    GOOGLE_APPLICATION_CREDENTIALS=<path-to-the-json-file>

  • Then run the following commands

    GIT_BASE_DIR=$(pwd)
    PROJECT_NAME=$(grep -E '^PROJECT_NAME=' .env 2>/dev/null | awk -F'=' '{print $2}' | tr -d '"') 
    PROJECT_NAME=${PROJECT_NAME:-"d2e"}
    PREFECT_DOCKER_VOLUMES_CUSTOM="'[\"${PROJECT_NAME:-d2e}_trex:/app/duckdb_data\", \"${PROJECT_NAME:-d2e}_cdw-config-cachedb-data-1:/app/cdw-config/duckdb_data\", \"${PROJECT_NAME:-d2e}_r-libs:/home/docker/plugins/R/site-library\",\"$GIT_BASE_DIR/cache/vocab/transformed:/app/vocab\", \"$GIT_BASE_DIR/cache/synpuf1k:/app/synpuf1k\", \"$GOOGLE_APPLICATION_CREDENTIALS:/app/google-credentials.json\"]'"
    echo PREFECT_DOCKER_VOLUMES_CUSTOM=$PREFECT_DOCKER_VOLUMES_CUSTOM >> .env
    echo GOOGLE_APPLICATION_CREDENTIALS=$GOOGLE_APPLICATION_CREDENTIALS >> .env

Add database connection details and credentials

• Log in to the Researcher Portal using a Chrome web browser

  • https://localhost:443/portal - local workstation
  • https://<FQDN>/portal - remote server

• Login as the new admin user

• Switch to Admin Portal

• Select Setup on top right

• URL is now:

  • https://localhost:443/portal/systemadmin/setup - local workstation
  • https://<FQDN>/portal/systemadmin/setup - remote server

• Select Databases Configure button

• Select Add database

• Enter the database information.

• Click Save

name	value	note
Database ID	big_query	display name
Project		Your big query project name
Dataset		Your big query dataset name from the project

• Run the following commands to stop and start d2e

• d2e stop
  d2e start