A pipeline owner is the service user a hosted pipeline runs as. Every run, scheduled or started with Run now, queries your engine with the owner’s roles, so the owner decides which tables a run can read and write. A pipeline with no owner does not run. On a pipeline that is not paused, Run now shows an error saying the pipeline has no owner, and a scheduled one shows Blocked in the pipelines list with the reason “no owner credential”. A paused pipeline reports that it is paused first; resume it and it reports the missing owner.

What you need

Service users and their API keys are created in the console only. dip has no command that creates either one. Creating one is a Members action, which the built-in data_engineer role does not include, so a data engineer usually asks an administrator to prepare the owner service user and its key, then attaches it. A custom role with the Members privileges is still limited to what it can reach itself: it can assign the service user a role only when its own grants cover everything that role grants, and mint a key only for a service user whose roles it covers, with capabilities it could use.

Set up an owner

1

Create the service user

Go to Settings → Service users and create one for the pipeline, for example svc-dbt-finance. Give each pipeline, or each group of pipelines that touch the same data, its own service user, so a key you revoke stops only the runs it belongs to. See Service users and API keys.
2

Give it the least access the project needs

Assign roles that reach the catalogs and schemas the dbt project reads, and the ones it builds tables in. What that is depends on your project: a project that only builds views in one schema needs much less than one that rebuilds a whole catalog. A custom role with data grants on exactly those schemas is the narrowest choice. The built-in data_engineer role can read and write every catalog, so use it only when the project really does.
3

Mint its API key

Select New API key on the service user’s row and tick query_engines:execute_sql. That is the one capability a run uses, so leave every other box empty. Copy the key: it is shown once.
4

Attach the owner to the pipeline

Pipe the key into dip pipeline set-owner. The key is read from standard input only, never from a flag, so it stays out of your shell history and process list:
dip prints dip: set owner of "finance-dbt" to service user "svc-dbt-finance", and the pipeline’s Overview tab names the owner.
owner(service_user="...") in pipeline.py does not attach an owner. It records who should own the pipeline, and dip pipeline push never assigns or changes one. When pipeline.py names an owner and the pipeline has none, push says so and prints the set-owner command to run, with the workspace you pushed to and the key piped in from an OWNER_KEY variable. When the pipeline already has a different owner, push says that owner stays and prints the command that would switch it. An external pipeline (register --external) is never run by Dippr, so it needs no owner.

Keep credentials out of the pipeline

A pipeline declaration (pipeline.py, and the pipeline.json it writes) must not carry credentials, in any dbt var, whatever the var is called. The declaration is baked into the pipeline image, so anyone who can pull that image can read it. Where the run’s credential comes from depends on who runs the pipeline:
  • A pipeline Dippr runs (dip pipeline push) runs as its owner. Attach the owner’s key with dip pipeline set-owner, which reads it from standard input only:
  • An external pipeline (dip pipeline register --external) is run by your own orchestrator and needs no owner. Give its runs their credentials in that orchestrator, for example as environment variables your dbt profile reads, never in the declaration. Reporting an external pipeline’s runs to Dippr is not available yet, so its runs do not appear in Dippr; see pipelines:report.
dip pipeline check, dip pipeline build and dip pipeline push (both before Docker runs), and registration, all refuse a declaration whose dbt vars, or the arguments of its run-operation macro, have a key that looks like a secret: one containing password, passwd, secret, token, credential, api_key, private_key or access_key, in any case. The last three also match written with - or with no separator (apikey). The check names the key:
dip pipeline build and dip pipeline push check the declaration pipeline.py produces on your machine. When pipeline.py cannot run there (no python3, or it imports a package only the image installs), they check the pipeline.json already on disk, and the image build checks the declaration itself. That works because dip pipeline check, dip pipeline build and a dip pipeline push that builds an image all require your project’s copy of the pipeline SDK (altimate_pipeline/ and altimate_app/) to be the one your dip ships. push --re-register builds nothing, so it does not check it. A project created with an older dip keeps its older copy, so they stop, name the files that differ, and tell you to run dip pipeline sdk-sync, which refreshes only those two directories. This check looks at key names only. A secret under a name it does not match, such as db_pwd, is not refused, so do not rely on it: keep credentials out of every var.
An image already in your registry keeps whatever its declaration carried. If a credential was ever in a dbt var of a pushed image, treat it as exposed and rotate it, then push without the var.

Your credential and the owner key are different

Two credentials are involved, and they do different jobs:
  • Your credential is what dip pipeline set-owner signs in with: your dip login, or in CI an API key with pipelines:manage. It needs permission to change the pipeline. It is never stored on the pipeline, and runs never use it.
  • The owner key is the service user’s alt_ key you pipe in. Dippr stores it encrypted for your default data plane and never shows it again. A run uses it to get a short-lived engine token; the dbt run itself only ever receives that token, never the key.
Do not use one key for both. A CI job that deploys pipelines needs pipelines:manage; the owner needs only query_engines:execute_sql.

Who can attach an owner

Anyone whose role allows changing pipelines can attach or replace an owner. That covers the built-in data_engineer, data_admin and account_admin roles, and any custom role granted the Pipelines update privilege. From CI, use an API key holding pipelines:manage whose service user has such a role. You can attach an owner only if you could assign every role that service user holds yourself. A run queries your engine with the owner’s roles, so attaching an owner hands those roles to the pipeline, and Dippr checks them the same way it checks a role assignment. If the owner holds a role you could not assign, dip pipeline set-owner is refused, names the roles, and changes nothing:
Ask someone who can assign those roles, such as an administrator, to attach the owner, or give the owner service user narrower roles. data_admin and account_admin can attach any owner. The check counts every role you hold, unless your session is acting as a single one of your roles. Then only that role counts, so an owner holding more than it is refused even when your other roles, an admin role included, would cover it. Runs act as the service user the key belongs to, so the key you pipe in must be one minted for the service user you name with --service-user. Dippr checks this when you attach the owner: a key that belongs to a different service user or to another tenant, that is unknown, revoked or expired, or that does not hold query_engines:execute_sql, is refused and nothing is stored. Keep owner keys where only the people who deploy that pipeline can read them, and give owner service users no roles beyond what their pipelines need.

Rotate the owner key

Rotate without a gap by attaching the new key before revoking the old one:
  1. On the service user’s row, select New API key and tick query_engines:execute_sql.
  2. Attach it: printf %s "$NEW_OWNER_KEY" | dip pipeline set-owner finance-dbt --service-user svc-dbt-finance --key-stdin. Runs that start from now on use the new key.
  3. Expand Show keys and select Revoke on the old key.
To hand the pipeline to a different service user, run the same command with that service user’s name and its key.

Revoke the owner key

Revoking the key under Show keys stops new runs straight away. The next run, scheduled or manual, fails before dbt starts, with the error owner_credential_refused. A run already under way keeps the short-lived engine token it started with until that token expires. To stop the schedule while you sort out a new owner, Pause the pipeline first.

When a run will not start

Service users and API keys

Create the service user and mint its key.

Roles and permissions

Build a custom role with grants on the schemas a pipeline touches.

dip pipeline commands

Every dip pipeline command, including set-owner.

A pipeline's runs and results

Read a run’s error when it cannot start.