Background Jobs

Question 1

What are the main types of background jobs ?

Answer 1

Background jobs typically include one or more of the following types of jobs:

• CPU-intensive jobs, such as mathematical calculations or structural model analysis.
• I/O-intensive jobs, such as executing a series of storage transactions or indexing files.
• Batch jobs, such as nightly data updates or scheduled processing.
• Long-running workflows, such as order fulfillment, or provisioning services and systems.
• Sensitive-data processing where the task is handed off to a more secure location for processing. For example, you might not want to process sensitive data within a web app. Instead, you might use a pattern such as the Gatekeeper pattern to transfer the data to an isolated background process that has access to protected storage.

Question 2

What are the main types of triggers for a background job ?

Answer 2

Background jobs can be initiated in several different ways. They fall into one of the following categories:

• Event-driven triggers. The task is started in response to an event, typically an action taken by a user or a step in a workflow.
• Schedule-driven triggers. The task is invoked on a schedule based on a timer. This might be a recurring schedule or a one-off invocation that is specified for a later time.

Question 3

Describe some Event-driven invocation patterns

Answer 3

Event-driven invocation uses a trigger to start the background task. Examples of using event-driven triggers include:

• The UI or another job places a message in a queue. The message contains data about an action that has taken place, such as the user placing an order. The background task listens on this queue and detects the arrival of a new message. It reads the message and uses the data in it as the input to the background job. This pattern is known as asynchronous message-based communication.
• The UI or another job saves or updates a value in storage. The background task monitors the storage and detects changes. It reads the data and uses it as the input to the background job.
• The UI or another job makes a request to an endpoint, such as an HTTPS URI, or an API that is exposed as a web service. It passes the data that is required to complete the background task as part of the request. The endpoint or web service invokes the background task, which uses the data as its input.

Question 4

Describe some Schedule-driven invocation patterns

Answer 4

Schedule-driven invocation uses a timer to start the background task. Examples of using schedule-driven triggers include:

• A timer that is running locally within the application or as part of the application’s operating system invokes a background task on a regular basis.
• A timer that is running in a different application, such as Azure Logic Apps, sends a request to an API or web service on a regular basis. The API or web service invokes the background task.
• A separate process or application starts a timer that causes the background task to be invoked once after a specified time delay, or at a specific time.

Typical examples of tasks that are suited to schedule-driven invocation include batch-processing routines (such as updating related-products lists for users based on their recent behavior), routine data processing tasks (such as updating indexes or generating accumulated results), data analysis for daily reports, data retention cleanup, and data consistency checks.

Question 5

What is the main downside of background jobs regarding the return of results?

Answer 5

Background jobs execute asynchronously in a separate process, or even in a separate location, from the UI or the process that invoked the background task. Ideally, background tasks are “fire and forget” operations, and their execution progress has no impact on the UI or the calling process. This means that the calling process does not wait for completion of the tasks.
Therefore, it cannot automatically detect when the task ends.

Question 6

As background jobs don’t return results in a common way, how could these results be provided to the caller ?

Answer 6

If you require a background task to communicate with the calling task to indicate progress or completion, you must implement a mechanism for this. Some examples are:

• Write a status indicator value to storage that is accessible to the UI or caller task, which can monitor or check this value when required. Other data that the background task must return to the caller can be placed into the same storage.
• Establish a reply queue that the UI or caller listens on. The background task can send messages to the queue that indicate status and completion. Data that the background task must return to the caller can be placed into the messages.
• Expose an API or endpoint from the background task that the UI or caller can access to obtain status information. Data that the background task must return to the caller can be included in the response.
• Have the background task call back to the UI or caller through an API to indicate status at predefined points or on completion. This might be through events raised locally or through a publish-and-subscribe mechanism. Data that the background task must return to the caller can be included in the request or event payload.