EM Automation and the Job System
Oracle observability post #8— the last post covered Compliance Standards and continuous security assurance. This one is about automation: how to use OEM's job system to run routine DBA operations on schedule across your entire estate, build chained workflows that replace shell scripts, and make failure visible before it becomes a problem.
Every Oracle environment has a list of tasks that run on a schedule. Statistics gathering. Log archiving. RMAN backup verification. Tablespace usage reports. Health checks before maintenance windows. On a small estate, cron jobs and DBMS_SCHEDULER handle this well enough. On a larger estate — 50 databases, 200 hosts, multiple data centers — that approach creates a coordination problem. The jobs run, but the output is scattered across servers, nobody is watching for failures, and a missed execution goes unnoticed until something breaks downstream.
OEM 24ai's job system solves this by centralizing job scheduling, execution, and monitoring across your entire estate. One place to define what runs, where, and when. One place to see what succeeded and what failed. One place to get paged when something doesn't run. No logging into individual servers, no reconciling output from disparate cron logs.
Job Types in OEM 24ai
OEM supports several job types, each designed for a specific class of work:
OS Command — runs a shell script or command on the agent host. Good for host-level maintenance: log rotation, file cleanup, pre/post-patching steps, space management. The script runs in the OS context of the Management Agent user.
SQL Script — executes SQL against an Oracle database target. Script runs in the database context, output is captured in OEM. Good for statistics operations, data purges, custom health checks, report generation.
RMAN Script — executes an RMAN script against a database. The job handles connection and output capture. Good for backup validation, image copies, crosschecks, restore testing.
SQL*Plus Script — runs SQLPlus with a script you supply. More flexible than SQL Script for operations that use SQLPlus-specific commands or complex formatting. Use when you need control over output formatting or SQL*Plus settings.
Multi-Task Job — a sequence of steps where each step is its own job type. Steps can run serially or in parallel, with pass/fail conditions controlling flow. This is where OEM's job system becomes genuinely powerful for operational automation.
Creating and Scheduling Jobs (EM CLI)
The console job wizard works for one-off setup. For repeatable, scriptable job management, EM CLI is the right tool.> Note: EM CLI syntax below is representative. Exact parameter names vary by OEM version and verb type. Always validate against your environment: emcli help create_job and emcli help get_jobs.
Create a SQL script job targeting multiple databases
emcli create_job \
-name="PROD_STATS_GATHER_WEEKLY" \
-type="SQL Script" \
-target_list="PRODDB01:oracle_database,PRODDB02:oracle_database,PRODDB03:oracle_database" \
-input_file="sql_script:/opt/oracle/scripts/gather_stats.sql" \
-schedule_type="weekly" \
-start_time="2026-10-05 02:00:00" \
-repeat_hours=168 \
-tz="America/Chicago"
Create an OS command job targeting a host group
emcli create_job \
-name="PROD_HOST_LOG_CLEANUP" \
-type="OSCommand" \
-target_list="Production_Hosts:group" \
-os_command="/opt/oracle/scripts/cleanup_alert_logs.sh" \
-schedule_type="daily" \
-start_time="2026-10-01 01:00:00" \
-tz="America/Chicago"
Submit a job immediately (outside its schedule)
emcli submit_job \
-name="PROD_STATS_GATHER_WEEKLY"
Get job execution history
emcli get_jobs \
-name="PROD_STATS_GATHER_WEEKLY" \
-status="Succeeded,Failed,Running"
Output includes execution time, duration, target, and exit status. For failed jobs, use the console to drill into full output and error messages.
Job Libraries: Reusable Templates
If the same job runs against multiple target sets with minor variations, the Job Library saves you from redefining it each time.
A Job Library entry is a saved job definition without a schedule or specific target list attached. When you need to run it, you create a job from the library entry, add the target list and schedule, and submit.
Console path: Enterprise → Job → Library
This is particularly useful for:
Standard maintenance scripts that run against different groups on different cadences (production on Sunday, development on Tuesday)
Jobs that different DBAs submit against their own target subsets
Patching and upgrade pre-checks that get rerun at each patching cycle
Build the library as you build the jobs. By the time you have 10–15 standardized operations in the library, onboarding a new DBA to your team's operational practices becomes significantly easier.
Multi-Task Jobs: Chained Workflows
A Multi-Task Job is a sequence of steps where each step is its own job type. Steps can be:
Serial — step 2 starts only after step 1 completes
Parallel — steps run concurrently; a final step waits for all to complete
Conditional — step 3 runs only if step 2 succeeded; step 4 runs only if step 2 failed
Example: pre-maintenance health check workflow
Run tablespace usage query (SQL Script) — verify no tablespace is above 85%
If step 1 succeeds → run invalid objects check (SQL Script)
If step 1 or 2 fails → generate alert incident (via notification rule)
Run RMAN crosscheck (RMAN Script) — confirm backup catalog is clean
Run listener status check (OS Command) — confirm listeners are up
This kind of workflow would previously live in a shell script that called multiple utilities, parsed exit codes, and had its own error handling. OEM's Multi-Task Job does it natively: per-step status, per-step output capture, conditional branching, and full visibility in the console — no custom orchestration code.
Console path: Enterprise → Job → Activity → Create Job → Multi-Task
Job Output and Failure Visibility
Every job execution captures:
Start and end time
Target name and host
Exit status (Succeeded, Failed, Stopped)
Full output log — stdout/stderr for OS commands, SQL output for database scripts
Console path: Enterprise → Job → Activity — filterable by status, target, time range, and job name.
For long-running jobs, you can monitor live output in the console rather than waiting for completion or tailing a log on the host.
Connecting to incident management: Create incident rules that fire on job failure. A failed RMAN crosscheck at 3am becomes an OEM incident, which triggers your on-call notification via the same infrastructure that handles your performance alerts. One notification framework for both real-time monitoring and scheduled job failures.
Console path for notification rules: Enterprise → Monitoring → Incident Rules → Create Rule → select "Job Execution" as the event type → filter on status = Failed and severity = your threshold.
What to Automate First
If you're starting from scratch with OEM job automation, three categories give the most return for the setup investment:
Statistics gathering. Most environments still run this ad-hoc or through DBMS_SCHEDULER jobs with no centralized visibility. Moving it to OEM gives you a single place to confirm it ran on every production database, see how long it took, and get paged if it fails. The group targeting approach means one job definition covers your entire production database fleet.
Log and trace file cleanup. Alert logs, trace files, and audit files accumulate on every Oracle host. An OS Command job targeting your host group, running weekly, keeps disk pressure under control without requiring manual intervention. Schedule it to run before your disk utilization monitoring alerts fire — not after.
RMAN backup verification. Your backups run. But is someone checking that they succeeded? An RMAN crosscheck job with a failure notification rule turns "I assume the backups are good" into "I know the backups are good, and here's when I last verified." This one change has caught real backup failures in multiple customer environments before the database team realized there was a problem.
The Anti-Patterns
Using OEM jobs to replace DBMS_SCHEDULER. For database-internal scheduling — maintenance window management, stats gathering that runs inside a maintenance window, application-layer batch jobs — DBMS_SCHEDULER is the right tool. OEM jobs are for cross-estate coordination and operations that need centralized visibility and failure alerting. Don't duplicate the same job in both systems.
No output retention policy. OEM stores job output in the Management Repository. Long-running jobs with verbose SQL output accumulate fast. Set a retention period in the job settings (Enterprise → Job → Settings) rather than letting the repository grow unchecked.
Submitting jobs manually when automation is available. If a DBA is running the same operation by hand every week, it should be scheduled. The manual step adds risk — forgotten runs, inconsistent timing, no output retention — without adding value. If you're doing it more than twice, build the job.
No failure notification on scheduled jobs. A job that fails silently is worse than no job at all. It gives you false confidence that the task ran. Every scheduled job that matters should have a notification rule on failure. This is a one-time setup cost per job type.
One monolithic OS command job. Breaking complex operations into Multi-Task Job steps gives you per-step status, per-step output, and conditional branching. A shell script that does everything in sequence tells you "it failed" — a Multi-Task Job tells you which step failed, what its output was, and which subsequent steps were skipped or redirected.
Ignoring duration trends. Statistics gathering that used to take 45 minutes and now takes 3 hours is a signal. OEM captures job duration for every execution. Review it periodically. An unexplained duration increase is often your first indicator of a growing data volume problem, degraded I/O, or a locking issue.
What Good Looks Like
In a well-configured estate, the OEM job system handles the operational repetition: statistics, cleanup, verification, health checks, pre-maintenance checks. All of it runs on schedule, against the right target groups, with output captured and failures surfaced as incidents. DBAs aren't logging into servers to run manual checks; they're responding to alerts when automation finds a problem.
The Job Library has reusable templates for the 10–15 standard operations your team runs regularly. Multi-Task Jobs handle workflows that used to live in shell scripts nobody maintained. The job activity view gives you a single answer to "did everything run last night?"
When something fails — and eventually something will — you know immediately, you have the full output, and you can correlate the failure with other events on the same target.
The Bottom Line
OEM 24ai's job system turns routine DBA operations from manual tasks into managed, monitored automation. OS commands, SQL scripts, RMAN scripts, and chained Multi-Task workflows cover the operational workloads that most teams are still running manually or through undocumented cron jobs.
The investment is upfront: defining the jobs, building the library, wiring up the notification rules. The return is every subsequent execution running automatically, with output retained and failures paged — not discovered days later when someone notices a side effect.
Coming soon in this series: OEM 24ai Incident Management and Notification Rules — how to build a notification framework that gets the right alert to the right person at the right time, and how to avoid the alert fatigue that makes monitoring systems useless.