Before you start
- A Google service account key (download the JSON from the Google Cloud console).
- The service account needs three things on the project: read table data and metadata, create query jobs, and create Storage read sessions (Trino’s default table-read path uses the BigQuery Storage Read API). The simplest grant is BigQuery Data Viewer plus BigQuery User.
roles/bigquery.userbundles bothbigquery.jobs.createandbigquery.readsessions.create, so Job User alone is not sufficient for table scans. Query bytes are billed to the project running the job (see the billing project field below). - The project ID that holds the data, taken from the Google Cloud console. This is the project you want to query, which is not necessarily the service account’s own project.
Register the integration
From Integrations, choose Register integration and select Google BigQuery.
There is no auth method to choose: the connector supports exactly one durable credential shape (a service account key), so there is nothing to select between.
Capabilities available for this kind: driver (required), federation.
Test connection
Authenticates the service account, lists the project’s datasets, and runs a dry-run query job against the billing project. These three checks answer the three things that actually go wrong:- The key is rejected (the service account is revoked, deleted, or the machine clock is skewed).
- The project ID is wrong (reported as “not found”), or the service account has no BigQuery access to it (reported as “not accessible”, with the role to grant).
- The service account can read metadata but cannot create query jobs: it has Data Viewer but not the job-create permission on the billing project. This is the subtle one. Without the job-create check the integration would look healthy and then fail every query, because federation reads every row through a job. The dry run validates the permission without scanning a byte or billing anything.
Test connection validates token, dataset access and job creation. It does not separately probe
bigquery.readsessions.create (the BigQuery Storage Read API has no dry-run and needs a real table, so it cannot be checked on an empty project). Grant BigQuery User rather than only Job User, as Before you start describes, and both job creation and read sessions are covered.In Explore
Each BigQuery dataset appears as a schema, and each table under it with its BigQuery column types (STRING, INT64, TIMESTAMP, NUMERIC, and so on). Views appear only when Read views is enabled, matching the engine, which cannot expose a view without a materialization dataset. A dataset or table whose own metadata cannot be read is skipped rather than breaking the whole catalog.
Limitations
- Read-only: this kind connects Dippr to a BigQuery project’s data, not the other way around.
- No tunnel support. BigQuery is reached over the public Google API, not a private network, so there is nothing to route through a bastion.
- Reading views requires a materialization dataset; without one, views are not exposed.
- What you can query through the engine is governed by your own roles and grants, not by this record.
Equivalent dip command
Register a Google BigQuery source
--secret-stdin reads key=value, one per line. A downloaded service-account key is pretty-printed JSON spanning many lines, so it cannot be piped in directly; jq -c . compacts it onto the single line the service_account_key= assignment needs. (No jq? python3 -c 'import json,sys;print("service_account_key="+json.dumps(json.load(open("sa-key.json"))))' does the same.)Troubleshooting
Related
Integrations overview
Which sources you can connect, and how a source becomes a catalog.
Test connection and secrets
How Test connection works, and how your credentials are stored.