Ingest Documentation via API

Upload a Sphinx-Needs documentation build to your ubTrace instance over HTTP, straight from a CI pipeline (GitHub Actions, GitLab CI, Azure Pipelines, Jenkins, or any tool that can run curl).

This is the recommended path when:

• Your CI runs in a different network than the ubTrace server and cannot mount a shared folder.

• You want every push to a branch or tag to publish a new version automatically.

• You prefer a single HTTP call over copying files into input_build/.

The round trip looks like this:

1. An admin creates an API token in the ubTrace admin panel.

2. The token is stored as a CI secret (for example a GitHub Actions secret).

3. The CI job builds the Sphinx-Needs documentation with ubt_sphinx.

4. The CI job packages the output into a tar.gz (or zip) archive.

5. The CI job uploads the archive to the /v1/ingest endpoint with curl.

6. The ubTrace worker picks up the archive and publishes the new version.

Prerequisites

Before you start, make sure you have:

• A running ubTrace instance that you can reach from your CI network. The API base URL is the same URL you use in the browser, with /api appended – for example https://ubtrace.example.com/api. In the bundled offline deployment this defaults to http://<host>:7150/api.

• Admin access to the ubTrace admin panel. Only users with the ubtrace-admin role can create or revoke API tokens. Regular editors and viewers cannot.

• A local Sphinx project that builds successfully with ubt_sphinx. The build must produce docs/ubtrace/needs.json – this is the file ubTrace uses to detect a valid artifact.

Step 1 – Create an API Token

An API token is a long, random secret that lets a machine call ubTrace without a username or password. You create one token per CI pipeline.

1. Open the ubTrace admin panel in your browser. In the default offline deployment this is http://<host>:7156.

2. Sign in with an account that has the ubtrace-admin role.

3. Go to Settings → API Tokens.

4. Click Create token.

5. Fill in the form:

   • Name – something you will recognize later, for example GitHub Actions CI or gitlab-docs-pipeline. This label is only for you; it is never sent as part of the token.

   • Scopes – shown for information; they are not selectable. Every token ubTrace issues carries both scopes that exist today:

     • Ingest – uploads via POST /v1/ingest/....

     • Delete – DELETE on a version or a project under /v1/artifacts/organizations/.... This is what lets a pipeline clean up after itself; see Removing a version.

     A token cannot read need data, read user data, or change settings – but because the two scopes are issued together, a leaked token can delete a version or a whole project. Treat it as a write credential: store it as a CI secret (Step 2), give it an expiry, and revoke it the moment it may have been exposed.

   • Expiry date (optional) – pick a date after which the token stops working. If you leave this empty, the token is valid until you revoke it. For CI pipelines a 6- or 12-month expiry is a good default – it forces a rotation without being disruptive.

6. Click Create. ubTrace shows the full token exactly once, in a dialog like ubt_abcd1234....

7. Copy the token immediately. Once you close the dialog you cannot see the full value again; you can only see the prefix (ubt_abcd...) in the list. If you lose the token, revoke it and create a new one.

Note

Where is the token stored on the server? ubTrace stores only a one-way hash of the token (argon2id) plus the short prefix. The server cannot reconstruct the original value, so even a database dump does not leak working tokens. This is the same pattern GitHub and GitLab use for personal access tokens.

Note

Token limits. Each admin can own up to 25 active tokens. If you hit the limit, revoke unused tokens from the list before creating a new one.

Step 2 – Store the Token as a CI Secret

Never paste the token into a workflow file or commit it to a repository. Store it as a CI secret and read it from an environment variable at runtime.

The examples below all use the name UBTRACE_INGEST_TOKEN. You can pick any name; just be consistent.

GitHub Actions

1. In your repository on GitHub, go to Settings → Secrets and variables → Actions.

2. Click New repository secret.

3. Set Name to UBTRACE_INGEST_TOKEN and Secret to the token value you copied in Step 1.

4. Save. The secret is now available to workflows in this repository as ${{ secrets.UBTRACE_INGEST_TOKEN }}.

Other CI systems

The same pattern works on every major CI platform – only the UI differs:

• GitLab CI: Settings → CI/CD → Variables. Add a masked, protected variable named UBTRACE_INGEST_TOKEN. Read it in .gitlab-ci.yml with $UBTRACE_INGEST_TOKEN.

• Azure Pipelines: Project Settings → Pipelines → Library. Add a secret variable to a variable group, then reference it with $(UBTRACE_INGEST_TOKEN).

• Jenkins: Manage Jenkins → Credentials. Add a “Secret text” credential, then bind it inside your pipeline with withCredentials([string(credentialsId: 'ubtrace-ingest-token', variable: 'UBTRACE_INGEST_TOKEN')]) { ... }.

• CircleCI / Bitbucket Pipelines / TeamCity / Drone: each ships a built-in secrets store. The contract is always the same – expose the token as an environment variable named UBTRACE_INGEST_TOKEN.

Step 3 – Build, Package, and Upload (GitHub Actions)

The workflow below does the full round trip. Copy it into .github/workflows/publish-docs.yml in your documentation repository and adjust the three env: values at the top.

name: Publish docs to ubTrace

on:
  push:
    branches: [main]
    tags: ['v*']

env:
  # The URL of your ubTrace API. Do NOT include a trailing slash.
  UBTRACE_URL: https://ubtrace.example.com/api
  # The organization and project identifiers you want to publish into.
  # Must already exist or be creatable on first upload (see "Identifiers" below).
  UBTRACE_ORG: mycompany
  UBTRACE_PROJECT: my-product

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repository
        uses: actions/checkout@v6

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install ubt_sphinx
        run: |
          pip install --upgrade pip
          # Pin to a known-good release so a future ubt_sphinx version cannot
          # silently break the pipeline. Bump deliberately after testing.
          pip install "ubt_sphinx==0.8.1" --extra-index-url https://pypi.useblocks.com

      - name: Determine the version identifier
        id: version
        run: |
          # Use the git tag when the workflow is triggered by a tag (e.g. "v1.2.3"),
          # otherwise use the short commit SHA. Pick whatever fits your release flow.
          if [[ "${GITHUB_REF_TYPE}" == "tag" ]]; then
            echo "version=${GITHUB_REF_NAME}" >> "$GITHUB_OUTPUT"
          else
            echo "version=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT"
          fi

      - name: Build the documentation with ubt_sphinx
        env:
          # Read by conf.py so the builder's output path contains the
          # version we want to publish (see note below the workflow).
          UBTRACE_VERSION: ${{ steps.version.outputs.version }}
        run: |
          # Adjust "docs/" if your Sphinx source lives somewhere else.
          # The ubtrace builder writes its output to:
          #   docs/_build/ubtrace/<ubtrace_organization>/<ubtrace_project>/<ubtrace_version>/
          # -- the values on the right come from conf.py, not from this step.
          sphinx-build -b ubtrace docs docs/_build/ubtrace

      - name: Package the build output as build.tar.gz
        env:
          UBTRACE_VERSION: ${{ steps.version.outputs.version }}
        run: |
          # The archive MUST contain "docs/ubtrace/needs.json" at its root.
          # Tar from inside the version directory so the three top-level
          # entries of the archive are "config/", "docs/", and (optionally) metadata.
          BUILD_DIR="docs/_build/ubtrace/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}"
          test -f "${BUILD_DIR}/docs/ubtrace/needs.json" \
            || { echo "needs.json missing -- check that conf.py values match UBTRACE_ORG/PROJECT/VERSION"; exit 1; }
          tar -czf "${GITHUB_WORKSPACE}/build.tar.gz" -C "${BUILD_DIR}" .

      - name: Upload to ubTrace
        env:
          UBTRACE_INGEST_TOKEN: ${{ secrets.UBTRACE_INGEST_TOKEN }}
          UBTRACE_VERSION: ${{ steps.version.outputs.version }}
        run: |
          curl --fail-with-body --show-error \
            -H "Authorization: Bearer ${UBTRACE_INGEST_TOKEN}" \
            -F "file=@build.tar.gz" \
            "${UBTRACE_URL}/v1/ingest/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}?overwrite=true"

--fail-with-body makes curl exit with a non-zero status on any HTTP error (4xx/5xx) and still print the response body, so your CI log shows exactly what went wrong.

Important

The three identifiers in the URL (${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}) must match the values the Sphinx build produces. The builder takes its output path from ubtrace_organization, ubtrace_project, and ubtrace_version in conf.py. The simplest way to keep them in sync is to read the version from an environment variable in conf.py:

# conf.py
import os
ubtrace_organization = "mycompany"
ubtrace_project = "my-product"
ubtrace_version = os.environ.get("UBTRACE_VERSION", "1")

If the archive also contains config/ubtrace_project.toml (which ubt_sphinx produces automatically), its organization, project_id, and version fields must equal the URL parameters – otherwise the upload is rejected with 400. See TOML identity check below.

Step 4 – Manual Upload with curl

You can run the same upload from your laptop whenever you need to publish a version by hand (for example during testing or a one-off migration):

# Replace the placeholders with your own values first.
export UBTRACE_URL=https://ubtrace.example.com/api
export UBTRACE_INGEST_TOKEN=ubt_your_token_here
export UBTRACE_ORG=mycompany
export UBTRACE_PROJECT=my-product
export UBTRACE_VERSION=v1

# Build the docs. ubt_sphinx writes its output under
#   docs/_build/ubtrace/<ubtrace_organization>/<ubtrace_project>/<ubtrace_version>/
# so make sure those values in conf.py match UBTRACE_ORG/PROJECT/VERSION above.
sphinx-build -b ubtrace docs docs/_build/ubtrace

# Package the version directory (the one that directly contains docs/ubtrace/needs.json):
BUILD_DIR="docs/_build/ubtrace/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}"
tar -czf /tmp/build.tar.gz -C "${BUILD_DIR}" .

# Upload it:
curl --fail-with-body --show-error \
  -H "Authorization: Bearer ${UBTRACE_INGEST_TOKEN}" \
  -F "file=@/tmp/build.tar.gz" \
  "${UBTRACE_URL}/v1/ingest/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}"

The overwrite Flag

By default, ubTrace rejects an upload for a version that already exists (HTTP 409 Conflict). This stops you from accidentally replacing a published release.

Append ?overwrite=true to the URL to allow replacement:

POST /v1/ingest/mycompany/my-product/v1?overwrite=true

When to use it:

• Recommended: pipelines that publish a moving pointer such as main or latest – you want every push to replace the previous build.

• Recommended: the ${GITHUB_SHA::7} pattern from the workflow above – the same commit should always produce the same artifact.

• Not recommended: signed releases (v1.2.3). Treat them as immutable – publish each release exactly once.

Recording a Release Date

A version’s release date – the date the release was released – cannot be derived from the upload. The date ubTrace ingested a bundle is not it, and neither is the date the bundle was built: a release cut in March and rebuilt in August was still released in March.

Declare it on the ingest call with ?releasedAt=, an ISO 8601 date or instant:

POST /v1/ingest/mycompany/my-product/v1.2.0?releasedAt=2026-03-15

The ingest call is the only place this value can be given, because a release date is a fact about one version. It is deliberately not read from the project configuration: a project-level setting would stamp every version of the project with a single date.

What happens on a re-ingest:

• You supply no releasedAt – the stored date is kept. Re-uploading a version to pick up a rebuilt bundle never moves its release date, and never erases one.

• You supply a releasedAt – it replaces the stored one. This is how a mistyped date is corrected: re-ingest the version with the right one.

• You supply an empty value (for example a CI variable that was never set) – it is treated as if you had supplied nothing. The date is never recorded as an empty value.

A version whose release date was never declared reports releasedAt: null on the version-read endpoints. That is not recorded, and ubTrace will not guess one for you.

Note

releasedAt is separate from the ingest and build timestamps, which ubTrace records on its own. Reports that order or filter versions by release date use this value; those that need to know when a bundle was last loaded keep using the load timestamp.

Archive Layout

ubTrace accepts two archive formats:

• tar.gz (recommended, produced by tar -czf)

• zip

The format is detected automatically from the file’s magic bytes. If you prefer to be explicit you can pass ?format=tar.gz or ?format=zip.

The archive must contain the ubt_sphinx output of one version at its root – that is, docs/ubtrace/needs.json must sit directly inside the archive, not nested in extra parent folders:

build.tar.gz
├── docs/
│   └── ubtrace/
│       ├── needs.json                 ← required
│       └── *.fjson                    ← page fragments
└── config/
    ├── ubtrace_project.toml           ← optional but recommended; may carry
    │                                     an [ubtrace.git] block (branch,
    │                                     commit, pull request)
    └── ubtrace_global_context.json    ← optional; names the Sphinx that built it

ubt_sphinx writes this exact structure under _build/ubtrace/<org>/<project>/<version>/. The simplest way to get the right archive layout is to tar from inside that version directory – see the BUILD_DIR line in the workflow above.

TOML identity check

If the archive contains config/ubtrace_project.toml (ubt_sphinx produces this file by default), ubTrace validates each field that is set in its [ubtrace.project] table:

[ubtrace.project]
organization = "mycompany"
project_id = "my-product"
version = "v1"

• organization must equal the organizationId in the URL.

• project_id must equal the projectId in the URL.

• version must equal the versionId in the URL.

A mismatch returns 400 Bad Request with a message listing which fields differ. Fields that are not set in the TOML file are not checked. If the TOML file itself is absent from the archive, no identity check is performed – but we recommend shipping it, because it catches “wrong identifier” mistakes before they corrupt the database.

Note

The table itself is not optional once the file exists: a ubtrace_project.toml with no [ubtrace.project] table is rejected with 400 and ingest.invalid_toml. Writing the three keys at the top level of the file instead of under the table is the usual cause.

Recorded toolchain versions

At ingest, ubTrace records which toolchain produced the bundle and keeps it against that version, so that two versions of the same project can be checked for toolchain drift exactly instead of by eye:

• the documentation-builder version – the Sphinx that ran the build, read from the build context ubt_sphinx dumps once per build into config/ubtrace_global_context.json. A bundle assembled with a stock sphinx-build -b json instead of ubt_sphinx is read too: that builder writes the same dump as globalcontext.json beside needs.json. The per-page .fjson files never carry this value;

• the needs-library version – the Sphinx-Needs release that produced needs.json, read from the creator block that library writes into the file itself, together with the program name it declares.

Nothing about your build has to change for this: both values already exist in a stock ubt_sphinx output, and they are re-read on every ingest, so re-uploading an existing archive is enough to populate them.

Read them back per version on either of:

• GET .../projects/{projectId}/versions-with-dimensions – a toolchain object on every entry, so two versions can be compared in one call;

• GET .../projects/{projectId}/versions/{versionId}/health – the same object for a single version, alongside its component diagnostics.

The toolchain key is always present. Each of its fields is null when the value was not recorded – a version ingested before this feature existed, or an archive whose producer emitted no such metadata. That is a normal response, never an error, and it does not count against the version’s health. Treat null as unknown: two versions that both report null are not known to share a toolchain, they are simply both unrecorded.

Recorded git provenance

A build can also state which source state produced it – the branch, the commit, and, when the build is for a pull request, that pull request’s number. ubTrace records the values against the version, so an administrator can confirm which source state an ingested build came from.

Unlike the toolchain above, this is not read out of the bundle automatically: nothing in a Sphinx build knows your branch. You state it, in either of two places.

In the bundle, as an optional [ubtrace.git] block in config/ubtrace_project.toml:

[ubtrace.git]
branch = "feature/new-brakes"
commit = "9f1c2ab7e4d3c5b1a0f8e7d6c5b4a3928170f6e5"
pull_request = "1274"

On the ingest call, as optional query parameters – usually the easier half in CI, since the pipeline already holds these values as variables:

curl -X POST \
  -H "Authorization: Bearer ${UBTRACE_TOKEN}" \
  -F "file=@build.tar.gz" \
  "${UBTRACE_URL}/v1/ingest/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}?gitBranch=feature/new-brakes&gitCommit=9f1c2ab&gitPullRequest=1274"

Every field of both sources is optional, and so is each source. An ingest that supplies neither behaves exactly as it always has, and records nothing.

Precedence

Where both sources state the same field, the query parameter wins. The TOML block is baked into the artifact when it is built; the parameter is what your pipeline states at the moment of the upload, and it is the only way to record provenance for a bundle whose builder emits no block at all.

Precedence is settled field by field, and only a parameter that actually carries a value overrides anything. An empty parameter means not supplied and falls through to the block – so a pipeline that unconditionally passes ?gitBranch=${CI_BRANCH} with the variable unset does not erase a branch the bundle itself declared.

Note

A value that is present but empty is stored as not recorded, never as an empty string. Two versions that both record nothing are unknown, not two versions that share a branch.

A malformed value is treated differently by source, on purpose. A query parameter is an explicit statement by the caller, so a gitPullRequest that is not a number is rejected with 400. The same value inside the TOML block is dropped with a warning instead and the rest of the block is still recorded – a generated configuration file must never fail a build.

Reading it back

Per version, on either of:

• GET .../projects/{projectId}/versions-with-dimensions – a git object on every entry;

• GET .../projects/{projectId}/versions/{versionId}/health – the same object for a single version.

The git key is always present, and each of its fields is null when the value was not recorded. That is a normal response, never an error, and it does not count against the version’s health. Re-ingesting a version with different values updates what is recorded; re-ingesting it with none clears it.

Maximum archive size

• Default limit: 500 MB per upload.

• Configurable per instance via the INGEST_MAX_FILE_SIZE environment variable (value in bytes). See Installation for where to set it.

• The absolute hard cap enforced by the upload library is 1 GB. Archives larger than that are rejected before they reach the application layer.

If your archive is close to the limit, first check whether _images or generated artifacts got included by accident. A typical ubt_sphinx output for a large multi-project documentation is well under 100 MB.

Maximum coverage input size

Once an archive is accepted, the ub-worker runs coverage preprocessing over the extracted needs.json (the ubt-coverage step). This is a separate limit from the upload cap above and applies to the uncompressed needs.json.

• Default limit: 1 GiB (1073741824 bytes) per needs.json.

• Configurable via the UBT_COVERAGE_MAX_INPUT_BYTES environment variable on the ub-worker service (value in bytes). An unset, empty, non-numeric, or zero value keeps the 1 GiB default. See Installation for where to set it.

• When to raise it: content-rich projects – many thousands of needs with large descriptions or dense link graphs – can produce a needs.json that exceeds 1 GiB. Raise the limit so the worker accepts them; e.g. set UBT_COVERAGE_MAX_INPUT_BYTES=4294967296 for a 4 GiB ceiling.

Identifiers (organizationId, projectId, versionId)

The three segments in the URL POST /v1/ingest/<organizationId>/<projectId>/<versionId> name the location of the artifact inside ubTrace and map 1-to-1 to folder names in input_build/.

Allowed characters

• organizationId and projectId: letters (a-z, A-Z), digits (0-9), underscore (_), and hyphen (-).

• versionId: the same set plus dot (.) – so v1.2.3 and 1.0 are valid version identifiers.

Length

• 1 to 255 characters per segment.

Consistency with the Sphinx build

• The three identifiers should match ubtrace_organization, ubtrace_project, and ubtrace_version in your conf.py so that the folder the builder writes to matches the URL you upload to.

• If the archive contains config/ubtrace_project.toml, mismatched fields cause a 400 Bad Request. See TOML identity check.

Choosing a ``versionId``

• Free-form within the character set above. Common patterns: v1, 1.2.3, main, latest, or the short commit SHA (${GITHUB_SHA::7}).

Error Responses

Every error response is JSON and includes a message field with the reason.

HTTP	Meaning	What to check
401 Unauthorized	The token is missing, malformed, or the Authorization header is wrong.	Confirm the header reads Authorization: Bearer ubt_... (note the Bearer prefix and the single space). Re-check the secret value.
403 Forbidden	The token is valid but does not have the ingest scope, has been revoked, or has expired.	Create a fresh token in the admin panel and update the CI secret.
400 Bad Request	The archive layout is wrong (docs/ubtrace/needs.json missing), the archive format cannot be detected, or an identifier contains illegal characters.	Unpack the archive locally with tar -tzf build.tar.gz | head – the first entries should include docs/ubtrace/needs.json. Also double-check your organization/project/version identifiers.
409 Conflict	The version already exists and overwrite=true was not set.	Add ?overwrite=true if replacement is intentional, or publish under a new version ID.
413 Payload Too Large	The archive is larger than INGEST_MAX_FILE_SIZE (default 500 MB). The same limit is enforced by both the backend API and the Next.js frontend’s middleware proxy, so it applies whether you call the API directly or through the frontend.	Trim unused assets from the Sphinx output or raise the limit in the ubTrace deployment.
429 Too Many Requests	The CI pipeline is uploading faster than the rate limiter allows.	Retry with exponential backoff. A single production pipeline should never hit this in practice.
5xx	ubTrace is unhealthy.	Check make status on the server and the ubtrace-api container logs.

Step 5 – Verify the Upload

After the curl call returns 201 Created, the worker scans the input_build/ folder (default: every 30 seconds) and imports the new version. Import usually finishes within a minute for a typical project.

Verify in the UI:

1. Open the ubTrace web app and sign in.

2. Navigate to your organization → project.

3. The new version should appear in the version switcher at the top of the page.

Verify via the API – useful for ad-hoc inspection from a developer machine:

# Lists all versions known for the project. Your new version should be in the list.
curl -H "Authorization: Bearer ${UBTRACE_USER_TOKEN}" \
  "${UBTRACE_URL}/v1/artifacts/organizations/${UBTRACE_ORG}/projects/${UBTRACE_PROJECT}/versions"

Note

The list-versions endpoint uses user authentication (Keycloak/OIDC JWT), not the ingest API token. UBTRACE_USER_TOKEN here is a JWT access token obtained by signing in as a normal user, not the ubt_... token from Step 1. The ingest scope only grants upload access – it cannot read artifacts. For CI pipelines, treat the 201 Created response from /v1/ingest as confirmation that the archive was accepted; use the UI path above for end-to-end verification.

If the version is accepted by /v1/ingest but never appears in the list, check the ub-worker container logs – it will report the reason (for example a malformed needs.json).

Per-pull-request ingestion

Added in version 1.12: Git provenance made this recipe practical – a per-pull-request version can now state the branch, commit and pull request it came from.

A reviewer looking at a pull request wants to see that pull request’s documentation, not the target branch’s. That means one ingested version per open pull request, published on every push and removed again when the pull request is merged or closed.

Both halves are your pipeline’s job. ubTrace does not poll your forge, so it never learns that a pull request closed, and it has no age-based cleanup of its own: a version exists until something deletes it. Ship the two workflows below together – the first one alone accumulates versions indefinitely.

Choosing the version identifier

Name the version after the pull request, not after the commit:

pr-1274

The identifier must be stable for the life of the pull request. Combined with ?overwrite=true this means every push to the branch replaces the same version in place, so a 40-commit pull request still costs one version and the cleanup step has exactly one identifier to delete.

Warning

Do not use ${GITHUB_SHA::7} as the version for pull-request builds. It mints a new version on every push, so a busy pull request leaves dozens behind and the cleanup step would have to enumerate them. The short-SHA pattern in Step 3 is for branch builds, where nothing later has to find the version again.

Pick a prefix that cannot collide with a release version. pr- in front of the number is enough: a release is v1.2.3, a pull request is pr-1274, and the two name spaces never meet.

As everywhere else, the identifier in the ingest URL must equal the ubtrace_version the Sphinx build was made with – see the note under Step 3 – Build, Package, and Upload (GitHub Actions).

Note

Provenance does not change the identifier. Unlike [ubtrace.dimensions], nothing in the [ubtrace.git] block or in the git* query parameters takes part in how a version is keyed. Re-ingesting pr-1274 with a new commit updates the recorded provenance of the same version; it does not create a second one.

A project that does declare [ubtrace.dimensions] is the one case where the identifier you delete is not the one you uploaded: the technical versionId is derived from the dimension set, so read it back from GET .../projects/{projectId}/versions-with-dimensions and delete that value. See ubtrace.dimensions.

Publishing the pull request’s build

This workflow is Step 3’s, with three changes: it triggers on pull requests, it derives the version from the pull request number, and it records the git provenance of the build.

name: Publish PR docs to ubTrace

on:
  pull_request:
    types: [opened, synchronize, reopened]

env:
  UBTRACE_URL: https://ubtrace.example.com/api
  UBTRACE_ORG: mycompany
  UBTRACE_PROJECT: my-product

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repository
        uses: actions/checkout@v6

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install ubt_sphinx
        run: |
          pip install --upgrade pip
          pip install "ubt_sphinx==0.8.1" --extra-index-url https://pypi.useblocks.com

      - name: Build the documentation with ubt_sphinx
        env:
          # One version per pull request, stable across pushes.
          UBTRACE_VERSION: pr-${{ github.event.pull_request.number }}
        run: |
          sphinx-build -b ubtrace docs docs/_build/ubtrace

      - name: Package the build output as build.tar.gz
        env:
          UBTRACE_VERSION: pr-${{ github.event.pull_request.number }}
        run: |
          BUILD_DIR="docs/_build/ubtrace/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}"
          test -f "${BUILD_DIR}/docs/ubtrace/needs.json" \
            || { echo "needs.json missing -- check conf.py values"; exit 1; }
          tar -czf "${GITHUB_WORKSPACE}/build.tar.gz" -C "${BUILD_DIR}" .

      - name: Upload to ubTrace with git provenance
        env:
          UBTRACE_INGEST_TOKEN: ${{ secrets.UBTRACE_INGEST_TOKEN }}
          UBTRACE_VERSION: pr-${{ github.event.pull_request.number }}
          # The head of the pull request -- the state a reviewer is looking
          # at. github.sha on a pull_request event is the merge commit, which
          # exists in no branch, so record the head SHA instead.
          GIT_BRANCH: ${{ github.event.pull_request.head.ref }}
          GIT_COMMIT: ${{ github.event.pull_request.head.sha }}
          GIT_PR: ${{ github.event.pull_request.number }}
        run: |
          curl --fail-with-body --show-error \
            -H "Authorization: Bearer ${UBTRACE_INGEST_TOKEN}" \
            -F "file=@build.tar.gz" \
            --url-query "gitBranch=${GIT_BRANCH}" \
            --url-query "gitCommit=${GIT_COMMIT}" \
            --url-query "gitPullRequest=${GIT_PR}" \
            --url-query "overwrite=true" \
            --url "${UBTRACE_URL}/v1/ingest/${UBTRACE_ORG}/${UBTRACE_PROJECT}/${UBTRACE_VERSION}"

Important

Encode gitBranch rather than pasting it into the URL. Branch names routinely contain / and may contain # or &, which truncate the parameter or corrupt the ones after it.

--url-query is the flag that does this on an upload: it appends an encoded parameter to the query string without touching the request method. Do not reach for --get --data-urlencode here – --get forces the request to GET, which contradicts the -F upload, and curl rejects the combination outright (exit code 2, and no output at all under -s).

--url-query needs curl 7.87 or newer (December 2022; the GitHub Actions Ubuntu runners are well past it). On an older curl, percent-encode the branch yourself and append it to the URL literally.

overwrite=true is required here, not optional: the second push to the pull request would otherwise be rejected with 409 Conflict.

Cleaning up when the pull request closes

Warning

Deleting a version currently fails on every shipped deployment. The API container mounts the output volume read-only in all three delivery channels – Docker Compose for development, Docker Compose for production, and the Helm chart – while the deletion removes that version’s directory on that volume. The call therefore ends in 500 Internal Server Error, and the cleanup job below goes red.

The failure lands after the database work has been committed, so the outcome is a partial deletion:

• the version’s PostgreSQL rows and Elasticsearch documents are removed, and the version disappears from the UI and the version list;

• its input_build/ directory is removed;

• its directory on the output volume is not reclaimed – the disk space stays used, and nothing later reclaims it;

• no VERSION_DELETED activity event is emitted, so the removal leaves no audit record and the project’s catalogue entry is not recomputed.

Tracked as issue #2121 and scheduled for v1.13.0. The recipe is documented in full here so the pipeline is ready the day the fix ships; until then, expect the cleanup step to fail and the build output of closed pull requests to accumulate. Add continue-on-error: true to the step if a red job on every closed pull request is not acceptable in the meantime – but do not read a green job as evidence that the disk was reclaimed.

Delete the version from a second workflow triggered by the pull request closing. GitHub fires closed for both a merge and an abandoned pull request, so one workflow covers both outcomes:

name: Remove PR docs from ubTrace

on:
  pull_request:
    types: [closed]

env:
  UBTRACE_URL: https://ubtrace.example.com/api
  UBTRACE_ORG: mycompany
  UBTRACE_PROJECT: my-product

jobs:
  cleanup:
    runs-on: ubuntu-latest
    steps:
      - name: Delete the pull request's version
        env:
          UBTRACE_INGEST_TOKEN: ${{ secrets.UBTRACE_INGEST_TOKEN }}
          UBTRACE_VERSION: pr-${{ github.event.pull_request.number }}
        run: |
          # ifExists=true keeps the job green when there is nothing to
          # delete -- a pull request closed before any docs build ran.
          curl --fail-with-body --show-error -X DELETE \
            -H "Authorization: Bearer ${UBTRACE_INGEST_TOKEN}" \
            "${UBTRACE_URL}/v1/artifacts/organizations/${UBTRACE_ORG}/projects/${UBTRACE_PROJECT}/versions/${UBTRACE_VERSION}?ifExists=true"

The endpoint is documented in full under Removing a version.

Fork pull requests

Neither workflow works for a pull request opened from a fork, and the gap covers the publishing half as much as the cleanup half.

GitHub does run pull_request workflows for a fork’s pull request. What it withholds is repository secrets: secrets.UBTRACE_INGEST_TOKEN is empty in such a run, so both jobs execute and both curl calls fail with an empty bearer token. For a fork pull request the effect is therefore not “cleanup occasionally misses one” – no version is published in the first place, and the recipe does not work at all for external contributors.

There is no way to hand a repository secret to a workflow triggered by a fork’s pull_request event. pull_request_target does receive secrets, but it runs the base branch’s workflow against the base checkout, so it cannot build the fork’s documentation. For a repository that takes external contributions, either accept that ubTrace previews are for same-repository pull requests only, or publish from a separate maintainer-approved workflow.

Reconcile periodically rather than trusting the cleanup hook alone: a scheduled job that lists the project’s versions, keeps the ones whose git.pullRequest is still open in your forge, and deletes the rest is a dozen lines and closes the gap – both for fork pull requests and for cleanup runs that failed. It is subject to the same deletion defect as the hook (see the warning above).

Note

Pace the reconcile. The DELETE endpoint has no rate-limit entry of its own, so the deployment’s default applies – in a production deployment 100 requests per minute per source address, and a scheduled job deletes from a single runner IP. A first reconcile against a repository with a long pull-request history is exactly the burst that trips it. Delete in batches with a short pause between them.

Storage implications

A pull-request version costs what a release version costs. Nothing about per-pull-request ingestion is cheaper per version. Each one stores:

• rows in PostgreSQL – needs, coverage, schema violations, ASPICE readiness, and the dimension projection;

• documents in the project’s Elasticsearch index;

• the expanded archive under input_build/{org}/{project}/{version}/ – larger on disk than the tar.gz you uploaded;

• the worker’s build output under the ubTrace output volume.

Nothing expires. ubTrace has no version-retention policy, no maximum version count, and no age-based garbage collection. A version stays until a DELETE removes it, so the steady-state cost of the recipe is:

one version's footprint  ×  number of open pull requests

with the release versions you already keep on top of that. Two levers keep that bounded, and both are in the recipe above:

• A stable per-pull-request identifier plus overwrite=true, so a pull request costs one version rather than one per push. This is the difference between a bounded and an unbounded footprint – get it wrong and a busy repository grows without limit.

• The cleanup workflow, so the count tracks open pull requests rather than every pull request the repository has ever had. Until issue #2121 ships, this lever is partial: deleting a version reclaims its PostgreSQL rows and its Elasticsearch documents, but not its directory on the output volume. Size that volume against every pull request the repository has had, not only the simultaneously open ones, until the fix lands.

Size the deployment’s storage against the peak number of simultaneously open pull requests, not the average. Plan for the documentation build to grow, too: INGEST_MAX_FILE_SIZE (see Installation) caps a single upload, not the accumulated total.

Removing a version

Warning

This endpoint currently fails on every shipped deployment, because the API mounts the output volume read-only. It returns 500 Internal Server Error after having already removed the version’s database rows and Elasticsearch documents, leaves the version’s directory on the output volume behind, and emits no VERSION_DELETED activity event. Tracked as issue #2121, scheduled for v1.13.0. The contract below is what the endpoint is specified to do and what it will do once the fix ships.

Deletes one version and everything derived from it:

DELETE /v1/artifacts/organizations/{org}/projects/{project}/versions/{versionId}

Responses:

Status	Meaning
204 No Content	Deleted. There is no response body.
404 Not Found	No such version – unless ?ifExists=true was supplied, which turns this case into 204.
403 Forbidden	The API token does not carry the delete scope. Not reachable with a user JWT, and not reachable with a token issued by the admin panel, which always carries the scope – see Authentication below.

Query parameters:

• ifExists – when true, a version that is not there is reported as 204 rather than 404. Use it in any pipeline that deletes unconditionally, so a cleanup job stays green whether or not a build was ever published.

Authentication. Either a user JWT (Keycloak/OIDC) or an API token carrying the delete scope.

Warning

Deletion is not restricted by project or by role. Any authenticated ubTrace user is accepted, whatever their role, and may delete any version of any project. Read access is filtered per project and per role; deletion is not. Treat the ability to sign in to ubTrace as the ability to delete, and restrict who can sign in accordingly.

The delete scope is not a fence either: every token the admin panel issues carries it, because the scopes are not selectable, so an ingest-only token cannot currently be minted (see Step 1).

What is removed. The version’s needs, coverage, schema violations and ASPICE readiness rows; its dimension projection; any materialized version-diff export that used it as either side; its Elasticsearch documents; and its directories under input_build/ and the output volume. The project’s catalogue entry is recomputed from the versions that remain, and the entry disappears when the deleted version was the last one.

Warning

Deletion is immediate and irreversible. There is no soft delete, no trash, and no undo – restoring a deleted version means ingesting it again. Note that the URL takes a version id; dropping that segment addresses the project, and DELETE on that path removes the project and every version in it.

Each successful deletion emits a VERSION_DELETED activity event recording the actor, so the removals a cleanup pipeline performs are auditable after the fact. The event is emitted last, after the filesystem cleanup, so on a deployment affected by issue #2121 it is not written – the rows are gone and no activity record names who removed them. Until that fix ships, do not rely on the activity log as the audit trail for deletions.

Revoking a Token

If a token is leaked, or a CI pipeline is retired:

1. Open Settings → API Tokens in the admin panel.

2. Find the token by its name or prefix.

3. Click Revoke.

Revocation is immediate. The next request with that token returns 403 Forbidden. Deleting and re-creating a token in your CI secret store rotates it without any server change.

Related Pages

• Version Health States – what each per-component health state means, including why a version with no schema-validation report is healthy.

• Sphinx ubTrace Builder – how to install and configure ubt_sphinx.

• Project Integration – the alternative folder-based import (input_build/ and input_src/) for environments that can mount a shared volume.

• Installation – environment variables, including INGEST_MAX_FILE_SIZE.