Batch Service Overview

Overview¶

Batch service is used when the data is transferred from a file or DB system to another file or DB system. Even if the data is frequently processed like online transactions, the batch service can be used depending on the way of transfer. The batch service is initiated by requesting ISM the required data(a file or table data) not by sending data directly. If ISM receives the request of data transfer, ISM inquires(or fetches) the data and send it to the target system as it has defined on rule. If there is too large data to be processed, the batch service divides the data into segmentations to process the transactions. When an error occurs on processing the segmentation process, it is processed again from the data of the error occurrence.

On batch service, the work that should be processed before or after the job can be defined. We call it "Task", and the work before batch service execution is "Pre Task", the work after the service is "Post Task". The kinds of task are,

• File Task
• Table Task
• Tellnet Task
• Service Task
• Stored Procedure Task

The task is executed as a separated transaction from the batch service.

Batch Service Runtime Configuration¶

Batch service comprises of batch agent which requests batch process, batch manager, and the interface process which conducts interface between each source and target system. Batch service can be configured in either way, master-backup mode or active-active mode. Even the batch service is configured to active-active mode, the master batch manager has to be discriminated by each server or group.

The service flow is as follows,

1. The batch agent sends the execution request to the batch manager which is registered to the agent.
2. After the batch manager receive the job request from the agent, it sends the internal source system interface process the data verification(fetching file, verification of table data) request.
3. Since the source system interface process complete the process, the batch manager send the data refection(reflecting file transfer and table data) request to the target system interface process.
4. After the target system interface process finish the process, the batch manager save the result of the process and exit the process.
5. After the batch agent send the process request, it also send the verification request of the processed result in certain period(Default: 100 times per 3 seconds).
6. The batch manager verify whether the process is completed or not and return the status to the agent.
7. Since the process is completed, the batch manager return the result to the agent, and the agent receive and print it to the screen, terminate it by setting the exit code to the result code.

When any failure occurs on the transmission process process occurs, the batch manager of the master server rout the request and execute the service. The batch manager of the master server determines the location of the interface process according to the location of the source interface process. However, if there's a shared disk between master and backup server, and it is accessible from both sides, execute the interface with the transmission process regardless of the location of the source process. For example, the organization of the source interface process executing on master server and the target interface executing on backup server is available.

Batch Agent

Batch agent plays a the trigger of the batch service. It sends the data related information which has to be processed on batch service - filename, process object data verification condition - to the batch manager to start the process. It also periodically check the execution result, finish the process by setting the result as the exit code of the process.

Batch Manager

Batch Manager controls the whole processes of batch service. It delivers directions of data inquiry/applying to each interface process, and goes on the next stage when the result is successful. Batch manager conduct pre-requisite operation – pre task – and post production - post task. Each Pre task, post task, and batch service conduct the next job only after the previous job is completed normally. Batch manager communicates with batch agent and notify the result of the process to the batch agent.

Interface Process

Interface process plays a role of conducting the interface between the source and target which fetching the data or transferring it. The division into the source and output is divided by the roles in batch service, each interface process do both of the roles of the source and the target.

Common Process

ISM requires a transmission interface process for each related system. Thus, the more file or DB systems ISM have, the more interface processes are followed. However, in the condition of the amount of service executions aren't many or the time required is short, it must be a waste of time to assign a process for each of them. If these interface processes can be integrated into a single or a few processes, the system resources can be managed more efficiently. For this reason, the batch interface process of the systems which integrates the processes can be executed by the "Common Process".

"Common process" is that defining the "alternative process" among the systems registered to "common system" by the batch interface type. If the alternative system is defined for each registered system of ISM, the batch manager finds the alternative process and sends the service execution request not to the very system but to the alternative system process.

Alternative system can be defined for each type of batch source and batch target. For file system, both of source interface and target interface are processable through the alternative system. For DB system, however, only the source interface is processable through the alternative system.

Batch Service Forwarding

If the batch manager determines that it cannot process more job requests, another batch manager run in another server can handle the jobs. Once the agent send the request of batch, the batch manager compare current processing jobs to the maximum number of jobs. If the current processing request is greater than the number, order direction "forward" to the agent, and the agent send the service processing request to backup batch manager.

The maximum amount of simultaneously processable is defined on ism.xml.

<batch>
<job threshold="30"/> <!-- Maximum number of simultaneous processing = 30 -->
</batch>

System Interface Pattern

The patterns usable in batch interface as follows. Interface type is synchronous way, but it doesn't conduct response conversion using process results nor manage the response data.

Sync classification	Request/Response classification	Note
ACK transfer	Inquire data in synchronous way and apply the data	Do not process the process of response data.

• The patterns defined here aren't the patterns in general meaning but the patterns for ISM to register the batch service.