Argus — Data Platform Monitor

Timeline 00:00 → 24:00 of the selected day · one row per pipeline · click a run bar for upstream & downstream lineage

Timeline 00:00 → 24:00 of the selected day · one row per workflow · click a run bar for lineage

Active streaming alerts

Incoming messages per feed (gaps show as drops to zero)

Latency per feed (separate charts)

SLA breach minutes per feed

Event Hub feeds (fixed topics)

Three topics are monitored, configured in the ⚙ Config tab (namespace evhns-edpprod-uaenorth):

PNR → edp2.local.dih.pnr.prod.realtime
Inventory → edp2.local.dih.inventory.prod.realtime
Ticket → edp2.local.dih.ticket.prod.realtime

Latency, throughput, incoming/outgoing/captured message counts come from Azure Monitor metrics on these Event Hubs. Set the latency SLA and stall threshold above; alerts and the AI Agent use them.

Data Quality Status

Open this tab to load.
Reads the DQ dashboard from your Databricks SQL Warehouse (configured in the ⚙ Config tab).

No Issues Affected Feed Click a coloured DOW box to see its trend (served from the daily cache). Use “Refresh cache” once a day to recompute and store all boxes.

Dependency execution status Double-click a card to open its execution tree

Platform overview

Success rate: ADF vs Databricks

Selected day

Cumulative runs through the day

Running total by hour

Failures over the day

Failed runs per hour, both platforms

Busiest hours

Total runs started per hour

Reliability & duration

Outcome mix (all runs)

ADF + Databricks combined

Success rate by hour

Combined %, dips reveal bad windows

Run duration distribution

How long runs take (all runs)

Average duration by hour

Minutes, watch for slow-downs

Azure Data Factory

Runs by status

Selected day

Runs per hour

00:00 → 24:00, succeeded vs failed

Failures by pipeline

Where the errors concentrate

Databricks Workflows

Runs by status

Selected day

Runs per hour

00:00 → 24:00, succeeded vs failed

Failures by workflow

Where the errors concentrate

Succeeded Failed Running Pending / overdue Scheduled (not yet due) Blocked (parent failed)

Tick one or more dependencies above to see their execution status.

Loading…

My datasets

Group ADF pipelines and Databricks workflows under a dataset name (e.g. "PNR", "Inventory", "Sales"). Each page can then be filtered to one dataset or all. ADF Monitoring only lists datasets that contain pipelines; Databricks Workflows only lists datasets that contain workflows. Datasets are per-user — these are yours.

ADF pipelines

Set each pipeline's criticality and whether it pages on-call. This colours its border everywhere — ADF Monitoring, the dependency tree and Execution. Criticality is shared across all users.

Gold Silver Bronze PDPagerDuty

Click "Fetch from ADF" to load pipeline names from the configured factory.

Databricks workflows

Click "Fetch from Databricks" to load workflow names from the workspace.

Loading…

Backend API

The frontend talks to the Argus backend (app.py), which holds your credentials server-side and calls Azure / Databricks for you. When this page is served by the backend itself, leave the base URL empty.

not checked

Platform connections managed in Azure

ADF, Databricks, Event Hub and Databricks SQL credentials are no longer stored here.

They come from App Service → Configuration → Application settings (ideally Key Vault references), so secrets never sit in the database and are never sent to the browser.

The collector uses them every 5 minutes to refresh the data. This app only ever reads PostgreSQL, which is why every tab loads instantly and shows the same numbers for everyone.

To change a credential, update the Application Setting in Azure and restart the app.

Settings used: ADF_TENANT_ID ADF_CLIENT_ID ADF_CLIENT_SECRET ADF_SUBSCRIPTION_ID ADF_RESOURCE_GROUP ADF_FACTORY_NAME DBX_HOST DBX_TOKEN EH_TENANT_ID EH_CLIENT_ID EH_CLIENT_SECRET EH_SUBSCRIPTION_ID EH_RESOURCE_GROUP EH_NAMESPACE DBSQL_HOST DBSQL_HTTP_PATH DBSQL_TOKEN DBSQL_CATALOG

Data Quality collection window

The collector refreshes the DQ matrix only inside this window (GST). Outside it, the last snapshot is kept. Set both to 0 and 24 to collect all day.

START HOUR (GST, 0–23)

END HOUR (GST, 1–24)

Streaming latency (Databricks)

Real ingestion delay is read from each feed's landing table: enqueuedTime − the source timestamp inside json_body, averaged per time window. Give the delta path (or catalog.schema.table), the JSON path holding the origin timestamp, and the table's own timestamp column (feeds differ: most use enqueuedTime, ticket uses ingestion_timestamp). Runs on the Databricks SQL Warehouse configured below.

PNR TABLE / DELTA PATH

PNR JSON PATH

PNR TIMESTAMP COLUMN

INVENTORY TABLE / DELTA PATH

INVENTORY JSON PATH

INVENTORY TIMESTAMP COLUMN

WINDOW (MIN)

Event Hub feeds

Which Event Hubs to chart on the Streaming tab. The feed name is the label shown on the cards and charts; the Event Hub name is the entity inside the namespace. Saved to the database.

Dashboard dependencies

Pick which dependency flows appear in the “Dependency execution status” strip on the Dashboards tab and in the Manager view. Tick none to show them all.

⬢ HandoverIQ

ADF · DATABRICKS · AOAI

DEMO MODE
MTD spend: —

Argus2.0

Success rate: ADF vs Databricks

Cumulative runs through the day

Failures over the day

Busiest hours

Outcome mix (all runs)

Success rate by hour

Run duration distribution

Average duration by hour

Runs by status

Runs per hour

Failures by pipeline

Runs by status

Runs per hour

Failures by workflow

My datasets

Backend API

Platform connections managed in Azure

Data Quality collection window

Streaming latency (Databricks)

Event Hub feeds

Dashboard dependencies

Built for Modern Data Support.

Sign in