JOBS¶

Purpose: The Jobs page helps a data engineer or admin: - Track and monitor the status of data pipeline runs - See at a glance which pipeline executions succeeded or failed - Investigate failed jobs (especially recurring issues, like MySQL → ClickHouse failures) - Monitor job durations and how each run was triggered - Drill into individual job details for deeper troubleshooting

Accessing Jobs¶

Navigation: ETL Management → Jobs

Jobs List Page¶

Summary Cards¶

Four (five) stat cards give a quick health check of all job executions:

Metric	Count
Total Jobs	15
Running	0
Completed	5
Failed	10
Cancelled	0

Toolbar¶

• A search bar to look up jobs by name/keyword
• A status filter dropdown (currently "All Statuses")
• A date filter dropdown (currently "Created Date")
• A sort/filter icon
• A Refresh button to reload job data

Jobs Table¶

The main table lists individual job executions with these columns:

• Status — badge showing COMPLETED (green) or FAILED (red)
• Pipeline — the name of the pipeline that was run (e.g., test Pipeline, Clickhouse, Mysql to ClickhouseTest, Mysqltest)
• Job ID — a unique UUID identifying the specific run
• Created — date and time the job was created/executed
• Duration — how long the job took to run (in seconds)
• Records — number of records processed (shown as — if none were recorded)
• Trigger — how the job was started (e.g., Manual)
• Actions — an eye icon to view more details about that job

Notes: Status filter can be used to narrow the table to just Failed, Completed, Running, or Cancelled jobs. The Records column will show — for jobs that failed before processing began.

Job Execution Details Page¶

Click on the eye icon under Actions opens the Job Execution Details page for that run.

Header¶

• Job name with a status badge (e.g., COMPLETED, FAILED)
• Started timestamp
• Job ID
• A back arrow to return to the jobs list
• Auto Refresh toggle and a manual Refresh button

Tabs¶

Three tabs organize the details:

• Overview (default view)
• Execution Logs
• Metrics & Performance

Summary Cards¶

Metric	Value
Duration	e.g., 0.091s
Records Extracted	e.g., 0
Records Transformed	e.g., 0
Records Loaded	e.g., 0

Job Information¶

A detailed key-value breakdown of the run:

Field	Description
Job ID	Unique identifier for this run
Tenant ID	Identifier for the tenant/organization
Pipeline ID	Identifier for the pipeline definition
Pipeline Name	Name of the pipeline
Status	Current run status
Trigger	How the job was started (e.g., Manual)
Triggered By	User/API key ID that triggered the run
Created At	Timestamp the job was created
Started At	Timestamp the job began execution
Completed At	Timestamp the job finished

Performance Metrics¶

• Success Rate — percentage of records successfully processed

Metadata¶

• TriggeredFrom — indicates the interface/source used to trigger the job (e.g., pipeline-builder)

Notes:

• Use the Execution Logs tab to see step-by-step log output for the run.
• Use the Metrics & Performance tab for more granular performance data beyond the Overview summary cards.
• Success Rate is calculated from records processed, not from job status — a job can show COMPLETED with a 0% success rate if no records were processed.
• Auto Refresh is useful for monitoring a job that's still in progress.

Frequently Asked Questions (FAQ)¶

Q: What's the difference between the Jobs page and the Pipelines page?

A: The Pipelines page is where you build and configure ETL pipelines. The Jobs page is a monitoring/history view showing individual execution runs of those pipelines — each time a pipeline runs, it generates a job record here.

Q: What does each job status mean?

A: - COMPLETED — the job finished running - FAILED — the job encountered an error and did not complete successfully - RUNNING — the job is currently in progress - CANCELLED — the job was stopped before completion

Q: Why does a job show COMPLETED but with a low or 0% success rate?

A: Success Rate is calculated based on records actually processed, not the overall job status. A job can technically finish running (COMPLETED) while processing zero or very few records successfully — for example, if the source had no matching data or most records failed validation.

Q: Why does the Records column show "—" for some jobs?

A: This typically means the job failed before it reached the record-processing stage, so there's no record count to display.

Q: How do I investigate a failed job?

A: Click the eye icon in the Actions column to open the Job Execution Details page, then check the Execution Logs tab for step-by-step log output showing where and why the failure occurred.

Q: What's the difference between the Overview, Execution Logs, and Metrics & Performance tabs?

A: - Overview — high-level summary (duration, records extracted/transformed/loaded, job information) - Execution Logs — detailed step-by-step log output for troubleshooting - Metrics & Performance — more granular performance data beyond what's shown in the Overview

Q: What does "Triggered By" and "TriggeredFrom" tell me?

A: Triggered By identifies the specific user or API key that started the run. TriggeredFrom identifies the interface or source the trigger came from (e.g., pipeline-builder). Together they answer "who" and "from where" for a given run.

Q: When should I use Auto Refresh?

A: Use it when monitoring a job that's still RUNNING, so the details page updates automatically without needing to manually click Refresh.

Q: I'm seeing recurring failures for a specific pipeline (e.g., MySQL → ClickHouse) — where should I start?

A: Filter the Jobs table by that pipeline name or by Failed status, then open the Execution Logs for a few recent failed runs to look for a common error pattern (e.g., connection issues, schema mismatches, or data type errors).

Q: Can I filter jobs by a specific date range?

A: Yes — use the date filter dropdown in the toolbar (currently defaulting to "Created Date") alongside the status filter to narrow down results.

Q: What does the Duration field measure?

A: The total time (in seconds) the job took to run, from start to completion or failure.