Skip to main content
How tickets are analysed

Pipeline Health

Check how ticket processing is going: runs that completed, failed or timed out, tickets being analysed right now, recent incoming notifications and pipeline errors.

Written By Chris Scaminaci

Last updated About 2 hours ago

Pipeline Health is a read-only diagnostics page for ticket processing in your organisation. It shows how many incoming notifications from HaloPSA completed, failed or timed out, which tickets are being analysed right now, and which errors the pipeline logged. Use it when a ticket is missing from Home, when analysis seems stuck, or when you suspect HaloPSA is not reaching QuantumOps.

Before you start: you need the Administrator role. The Pipeline Health link under System in the sidebar is shown only to Administrators; anyone else who types the address gets an access-denied page (see Access denied and sign-in errors). The page reads your organisation's data only and changes nothing.

Choose a time window

  1. Open Pipeline Health from System in the sidebar.
  2. Pick a window in the list at the top right: Last hour, Last 6 hours, Last 24 hours (the default) or Last 7 days. The page reloads.
  3. Select Refresh to read the latest data without changing the window.

Every time on this page is shown in UTC, not in your organisation's time zone.

The line under the page title shows the per-ticket processing budget, in seconds, or off. See The per-ticket processing budget.

Read the summary tiles

TileWhat it counts in the window
Webhook runsEvery incoming notification QuantumOps logged, from HaloPSA and from Slack. The tile shows 1000+ when there are more than 1,000.
CompletedRuns that finished normally.
ProcessingRuns that were logged and have not finished.
FailedRuns that ended with an error.
Timed outRuns that went past the per-ticket processing budget.
Tickets running QATickets that hold a processing lock, with the number over budget and abandoned in brackets.

Runs skipped because processing was paused or the ticket is excluded are counted in Webhook runs and shown in the run list, but have no tile of their own.

Tickets currently locked in QA processing

While QuantumOps analyses a ticket it holds a lock on that ticket, so two runs never analyse it at the same time. Tickets currently locked in QA processing lists the tickets whose lock is live.

ColumnMeaning
TicketThe HaloPSA ticket number.
Started (UTC)When the run took the lock.
ElapsedHow long the run has held it.
Lock (webhook)An identifier of the run that holds the lock.
StateRunning, or Over budget — stale lock when the run has held the lock longer than the budget.

When no ticket is being analysed the tile says "No ticket is holding a live QA lock right now."

Abandoned locks

A lock older than 15 minutes is treated as stale: the run that took it has died. Stale locks are not listed. They are counted as abandoned in the Tickets running QA tile, and the page notes that they are past the 15-minute window, so the next QA attempt on that ticket reclaims them. You do not need to clear them by hand.

At the default budget (900 seconds, the same 15 minutes), a run that passes the budget releases its lock and leaves this list. Over budget — stale lock therefore appears only when the budget is set shorter than 15 minutes. A lock that is still held after 15 minutes is counted as abandoned.

The per-ticket processing budget

The budget limits how long one ticket's analysis may run. The default is 900 seconds (15 minutes). A budget of 0 turns it off, and the line under the page title then reads off. The budget is an instance setting, so you cannot change it here. To change it, contact support as described in Getting help.

When a ticket's analysis passes the budget:

  • The run is marked TimedOut in Recent webhook runs, with an error that says QA processing exceeded the budget.
  • A PipelineTimeout entry is added to Pipeline errors and timeouts. Its message says the run may still be executing in the background, so the analysis can still finish after the page shows a timeout.
  • At the default budget, the ticket's lock is released so the next update to the ticket can start a new run. With a budget shorter than 15 minutes, the lock is not released at the timeout. It stays held until the run finishes or the 15-minute window has passed, and the ticket shows Over budget — stale lock meanwhile.

Recent webhook runs

Recent webhook runs lists the newest runs in the window. The heading shows how many are listed, for example showing 200 of 1000+, newest first. The page lists at most 200 rows. Choose a shorter window or a status chip to see the rest.

Use the chips All, Processing, Completed, Failed and TimedOut to filter by status. Skipped runs appear only under All.

ColumnMeaning
Time (UTC)When QuantumOps received the notification.
EndpointThe kind of notification, as in the next table.
TicketThe ticket number, or a dash when the notification has none.
StatusProcessing, Completed, Failed, TimedOut or Skipped.
ErrorThe reason for a failure, timeout or skip, cut off after 160 characters.
EndpointMeaning
NewTicketQANotificationA new ticket.
NewTriageAssistNotificationTriage assist for a new ticket (the QuantumOps Triage Assist webhook). A completed run shows the note "Triage preprocessing initiated (deprecated endpoint)" in Error.
TicketUpdateNotificationA ticket was updated.
TicketClosedNotificationA ticket was closed.
EndUserEmailA customer e-mail arrived on a ticket.

Other names appear for less common requests, such as a categorisation or research request. Slack events and Slack commands are logged in the same list under names that start with Slack.

Pipeline errors and timeouts

Pipeline errors and timeouts shows up to the 100 newest errors in the window that came from ticket processing, with the count in the heading. The error log is shared with other parts of QuantumOps, so the page keeps only entries that belong to the ticket pipeline, plus every timeout. Timeouts are highlighted.

ColumnMeaning
Time (UTC)When the error was logged.
SourceThe part of the pipeline that logged it.
TypeThe kind of error, such as PipelineTimeout.
MessageThe error text, cut off after 220 characters.

When there is nothing to show, the tile says "No pipeline errors logged in this window."

Diagnose a problem

A ticket is missing from Home. Choose a window that covers the time the ticket was created, then look for its number under Recent webhook runs.

  • No row at all: HaloPSA did not send a notification. Check the HaloPSA connection and its webhooks in Your HaloPSA connection.
  • Status Skipped, error "Processing paused": processing was paused. Resume it. A ticket that was already analysed can be reprocessed from Home. A new ticket is analysed on its next HaloPSA update, or from Process Ticket Now on its QuantumOps tab in HaloPSA (see The Ticket Dashboard in HaloPSA). See also Pausing and resuming AI processing.
  • Status Skipped, error "Ticket excluded by configuration": the ticket matched an exclusion rule, so it is deliberately not analysed.
  • Status Failed, error starting "Ticket could not be read from Halo": HaloPSA would not return the ticket, for example because the connection credentials stopped working or the ticket was deleted. The rest of the error repeats HaloPSA's message.
  • Status Completed: analysis ran. Look for the ticket in Home again; see Home: the QA Pipeline.

Processing looks stuck. Look at Tickets currently locked in QA processing.

  • A ticket listed as Running with a short Elapsed is still being analysed.
  • A ticket that is not listed, with a number in abandoned, is not stuck. The next attempt reclaims its lock. Update the ticket in HaloPSA, or reprocess it from Home.
  • Several TimedOut runs in a row point to a slow or failing AI provider. Read the Message column under Pipeline errors and timeouts.

Webhooks are failing. Select the Failed chip and read the Error column.

  • "Invalid signature" means QuantumOps rejected a notification because its signature did not match. Check the webhooks in Your HaloPSA connection.
  • Any other message is the error that stopped that run. Compare it with the entries under Pipeline errors and timeouts at the same time.