Connect Datawise to BigQuery to enable Schema Radar detection and warehouse-level lineage.
Prerequisites
You need a Google Cloud service account with the appropriate roles granted. Datawise authenticates to BigQuery using a service account key file.
Create a dedicated service account for Datawise rather than reusing an existing one.
Create a Service Account
- In the Google Cloud Console, go to IAM & Admin > Service Accounts.
- Click Create Service Account. Give it a name (e.g.,
datawise-ingestion) and click Create and Continue.
- Grant the following roles:
BigQuery Data ViewerBigQuery Job UserBigQuery Metadata Viewer
- Click Done.
- Open the service account, go to Keys, click Add Key > Create New Key, and select JSON.
- Download the JSON key file. You’ll use values from this file in the connector form.
Required Permissions
Datawise only reads schema metadata and does not access the data in your tables. Grant the service account access only to the datasets you want Datawise to monitor. You can use the allow and deny pattern filters in the connector form to further control what gets ingested.
- In BigQuery, open the dataset you want to share.
- Click Sharing > Permissions > Add Principal.
- Enter the service account email (format:
name@project.iam.gserviceaccount.com).
- Grant the
BigQuery Data Viewer role.
- Repeat for each dataset you want Datawise to monitor.
For production environments, use dataset-level access. This limits Datawise’s footprint to exactly the datasets your team has decided to track. You can always add more datasets later by repeating the sharing step.
Connection Setup
Go to Connectors, click Add Connector, and select BigQuery.
Required Fields
These values come from the service account JSON key file you downloaded.
| Field | Where to find it |
|---|
| Project ID | Top-level project_id field in the JSON key file. |
| Private Key | The private_key field. Includes the -----BEGIN RSA PRIVATE KEY----- header and footer. |
| Private Key ID | The private_key_id field. |
| Client Email | The client_email field. Format: name@project.iam.gserviceaccount.com. |
| Client ID | The client_id field. |
| Name | A display name for this connector in Datawise. |
Filters
| Scope | Field | Description |
|---|
| Project | Allow Patterns | Restrict to matching project IDs. |
| Dataset | Allow Patterns | Only ingest matching datasets (e.g., prod_*). |
| Dataset | Deny Patterns | Exclude matching datasets (e.g., *_staging). |
| Table | Allow Patterns | Only ingest matching tables. |
| Table | Deny Patterns | Exclude matching tables. |
| View | Allow Patterns | Only ingest matching views. |
| View | Deny Patterns | Exclude matching views. |
Settings
| Setting | Default | Description |
|---|
| Include Tables | On | Ingest table metadata and schema. |
| Include Usage Statistics | Off | Collect query usage data. Requires additional BigQuery permissions. |
| Enable Table Profiling | Off | Collect row counts and column statistics. |
| Enable Stateful Ingestion | Off | Process only new or changed assets on subsequent runs. |
Schedule
| Field | Description |
|---|
| Run on a schedule | Recommended. |
| Schedule | Cron-style schedule. Default: every day at 00:00. |
| Timezone | The timezone used to evaluate the schedule. |
Testing the Connection
If the test fails, check:
- That the private key was copied in full, including the header and footer lines.
- That the service account has the required roles at the correct level (project or dataset).
- That the Project ID matches the project where the datasets live.
After Connecting
Once connected:
- BigQuery datasets, tables, and views appear in Data Assets.
- Schema Radar begins detecting schema changes on your configured schedule.
- Lineage from BigQuery populates when dbt Cloud is also connected.
