The standard to define the types of the service is the target system. File batch service means when the target system is sending file data. File batch service comprised of File to File, File to DB, File to other(file standard)
The transferable data as a file is as follows. These are using FTP supported by ISM excepts the custom method.
| Category | Description | Note |
|---|---|---|
| Number of Files | Transferable files are more than one. | If the file are multiple, process it using wildcard. |
| File Location | Connect to FTP and it should be on the relocatable location. | Relocatable location include absolute path. If not, limit it on the cases of the absolute path's available. |
| File Type | Ascii data or binary data | Define whether ascii/binary for each system. It cannot be used as file based. |
| File Size | FTP Transferable file size | Transferable in the coverage of FTP. If the file is too big, longer timeout than the time required to sending file should is needed. |
It supports two different way below.
| Way | Content |
|---|---|
| Passive | The way the server doesn't work when the FTP Client is exchanging the data |
| Active | The way the server is running when FTP Client is exchanging the data |
ISM exclusively supports Unicode type. No supports guaranteed for fixed length of data process using query handler of other Korean encodings.
When the file transfer is required by the source system, the service send the file registered by rule or request parameter to the target system.
When the batch service is requested by the batch agent, defined and call the source and target files. If both are not assigned, follow the transmission protocol registered on rule. Refer the service request protocol on batch agent installation guide.
Both of the source and target system connects to the rule registered server by FTP. If there's a lot of server is registered, try to reconnect to the another server after five attempts. If the connections to the all server fails, stop trying.
Define the unit as file application. When the application is created, the default value is active. If there's no value on set on runtime, use the passive method as default.
Define the unit as integration service. If there's no value on set on runtime, use ascii as default file transmission mode.
Compare the size of file of the object file to send and the sent file. To check the size of file, current FTP account should have the permission of Dir. The information of the size of file is extracted from the results of list command of FTP by FTP parser. If there's no permission of Dir, cannot inquire the list of files. If the size of file cannot be verified, The following value must be set on ism.xml.
<batch> <file checkSize=”false”/> </batch>
Define the unit as application. Data Output Form is used to extract the file information from FTP parser. FTP parser analyze the returned string in units of string, the delimiter of the analysis of parser is space. The number of spaces are changed to the data output form so that the form must be set by numbers.
Direct connection to the source/target system. Move to the root directory defined on rule. Execute Dir command on the target directory.
$>ftp deai Connected to deai. 220 deai FTP server (Version 4.2 Thu Jul 12 13:39:06 CDT 2007) ready. User (deai:(none)): bwadmin 331 Password required for bwadmin. Password: 230-Last unsuccessful login: Tue Jul 22 10:32:54 KORST 2008 on /dev/pts/57 from 192.7.106.176 230-Last login: Tue Jul 22 13:30:04 KORST 2008 on ftp from ::ffff:192.7.105.13 230 User bwadmin logged in. ftp> cd ISM1.1 250 CWD command successful. ftp> dir 200 PORT command successful. 150 Opening data connection for /bin/ls. total 265384 -rw-r----- 1 bwadmin bwadmin 8552836 Jul 02 17:21 BWMonitorPatch.jar -rw-r--r-- 1 bwadmin bwadmin 35 May 12 2007 Licence.txt drwxr-xr-x 2 bwadmin bwadmin 256 Feb 25 19:40 Uninstaller drwxr-xr-x 6 bwadmin bwadmin 256 Apr 22 10:02 ba drwxr-xr-x 6 bwadmin bwadmin 256 Apr 22 10:02 batch_agent drwxr-xr-x 4 bwadmin bwadmin 4096 Jun 17 20:47 bin drwxr-xr-x 3 bwadmin bwadmin 256 Jul 02 16:41 bwmonitor drwxr-xr-x 3 bwadmin bwadmin 256 Jul 02 16:41 bwmonitor.backip drwxr-xr-x 5 bwadmin bwadmin 4096 Jul 15 13:30 data ==> The bolded underlined part is the date output form. The form expressed above is the default FTP date output form.
ftp data output form default form is MM DD HH:mm.
data form e.g)
drwxr-xr-x 5 bwadmin bwadmin 4096 7M 2D 13:30 data
output form format : MM DD HH:mm
Define the custom handler for file data transmission. Define it when other methods are used besides FTP on file transfer. Custom handler is defined by the system.
<batch> <file system_id handler=””/> </batch>
Absolute Path can refer to the root directory of the system according to the authority of the FTP account. Because it could effect to the system security, determines whether to allow the root directory or not following the policy.
<batch> <path absolute=”true”/> </batch>
If true, allow absolute path. If false, do not allow the absolute path. If it is supposed that the path of the registered source or target file is ‘/snd/input’ and the input source filename is ‘/root/input.dat', the processing mode is as follows.
| Allow the absolute path | Do not allow the absolute path |
|---|---|
| /root/input.dat | /snd/input/root/input.dat |
Since the absolute path is allowed, ignore the registered path and use the one already registered. If not, process the file attaching the input path to already registered path.
| Regulation Category | Description | Processing Status | Processing Mode |
|---|---|---|---|
| Delete the source file after the transmission completed | Delete the file of the source system after the transmission completed | Permission | Delete the file of the source system after the transmission completed. If the source file has a wildcard, delete all the files corresponding to wildcard. |
| Rule Category | Description | Processing Status | Processing Mode |
|---|---|---|---|
| Transfer the file in source filename | Transfer the source filename as the target system's filename. | Permission | The filename generated on the target file is identical to the filename of the source file, but the directory information of the source file isn't used. Since the source filename is used, do not use the input file name on request or rule registered target filename. |
| Define the target filename and transfer | Define the target filename and transfer on request. | Permission | Use the filename defined by request and transfer Use the directory information of the target defined by the request If no directory is defined, use the target directory registered on rule. |
| Duplicate the target file | If the target file already exist, duplicate the If the target file is already existed, overwrite on it. | Permission | |
| Adding target file | If the target file is already existed, add it to previous file. | Permission | |
| Define the privilege(mask) of the target file | Define the privilege of the target file. The privilege information is defined by numbers. Ex) rw-rw-rw : 666 | Permission | After the transfer the target file, change the authority information to the defined value. |
The rule defining the source file as follows. The rule applied equally to File - DB Batch.
To explain the defining rules below, fix the directory and file name as follows.
| Input | rule | Permission status | Processing type | |
|---|---|---|---|---|
| File name | Directory | File name | ||
| /snd/biz/in.dat | /snd | in_r.txt | Permission | Process by input file Connect to FTP and get the file by input file name get /snd/biz/in.dat |
| snd/biz/in.dat | /snd | in_r.txt | Not admitted | Even though the same directory is repeated, process it to be attached to the the path registered on rule because there's no absolute path. Since there's no pertinent directory, the process fails. get /snd/snd/biz/in.dat |
| biz/in.dat | /snd | inpu_r.txt | Permission | Because there's no absolute path on input file, process it to be attached to the the path registered on rule. get /snd/biz/in.dat |
| in.dat | /snd | in_r.txt | Because there's no absolute path on input file, process it to be attached to the the path registered on rule. Since there's no pertinent directory, the process fails. get /snd/in.dat | |
| /snd/biz/in.dat | /snd/biz | in_r.txt | Permitted | In permission of the absolute path get /snd/biz/in.dat When the absolute path isn't admitted get /snd/biz/snd/biz/in.dat |
| No input | /snd/biz | in_r.txt | Not admitted | Combine the absolute path and the filename registered on rule and process it Since the file isn't exist, the process fails. get /snd/biz/in_r.txt |
| *.dat | /snd/biz | in_r.txt | Permission | Combine the absolute path and the filename registered on rule and process it get /snd/biz/*.dat |
| No input | /snd/biz | *.dat | Permission | Combine the absolute path and the filename registered on rule and process it get /snd/biz/*.dat |
The defining mode below only applies on the transfer rule which isn't using the source file name.
To describe the defining rule below, fix the directory and filename as follow.
| Transfer filename | Rule information | Permission status | Processing mode | |
|---|---|---|---|---|
| Filename | Directory | |||
| /rcv/biz/in.dat | /rcv | in_r.txt | Permission | Process it according to the input file. Connect to FTP, and transfer file in the input filename. put /rcv/biz/in.dat |
| rcv/biz/in.dat | / rcv | in_r.txt | Not admitted | Even though the same directory is repeated, process it to be attached to the the path registered on rule because there's no absolute path. Since there's no pertinent directory, the process fails. put /rcv/rcv/biz/in.dat |
| biz/in.dat | / rcv | inpu_r.txt | Permission | Because there's no absolute path on input file, process it to be attached to the the path registered on rule. put /rcv/biz/in.dat |
| in.dat | / rcv | in_r.txt | Permission | Since there's no pertinent directory, the process fails. put /rcv/in.dat |
| /rcv/biz/in.dat | /rcv/biz | in_r.txt | Permission | Allow the absolute path put /rcv/biz/in.dat Do not allow the absolute path put /rcv/biz/rcv/biz/in.dat |
| No input | /rcv/biz | in_r.txt | Permission | Combine the absolute path and the filename registered on rule and process it put /rcv/biz/in_r.txt |
| *.dat | /rcv/biz | in_r.txt | Not admitted | If the input file is wildcard, only the source filename is available. |
| No input | /rcv/biz | *.dat | Not admitted | If the input file is wildcard, only the source filename is available. |
Define the directory to fetch the source file. Directory is indispensable information, if there's none, define the current directory(./). Define the source filename. If there's no filename defined, it should be inputted on batch service request. Define the status whether to delete the source file or not after the transfer is completed.
Define the directory to save the transfer object file. Directory is indispensable information, if there's none, define the current directory(./). Define the target filename. If there's no filename defined, it should be inputted on batch service request, or the source file usage status should be on. Define whether to use the source filename or not. Since the source filename is used, even if the target filename is defined, it is sent by the source filename. Define the processing mode when the target file existed in the target defined directory.
Processing modes are the three below.
If there's a conversion required for the input file data, register the information for data parsing. Input file data structure can have multiple field groups.
Define the information below when the parsing of the input file data is required.
| Input data type | Field delimiter | Record delimiter |
|---|---|---|
| Fixed length data | X | O |
| Delimiter data | O | O |
Fix the delimiter as follows for convenience.
| Field delimiter | Record delimiter | Field length definition | Permission status | Processing mode |
|---|---|---|---|---|
| No input | No input | 0 and fixed length mixed | Not admitted | Since there's no field delimiter, the try of data parsing fails because the field length value isn't accurate. |
| No input | No input | Fixed length | Permission | Data parsing according to the field length. Since there's no record delimiter Data number * Field group length(fixed length) is the actual size of the file. |
| No input | \n | Fixed length | Permission | Data parsing according to the field length Since there's record delimiter Data number * (Field group length + Record delimiter) is the actual size of the file. |
| | | No input | Fixed length | Permission | Data parsing according to the field delimiter Since there's no record delimiter Field delimiter number / Field number within the field group is the actual size of the file. |
| | | \n | Fixed length | Permission | Data parsing according to the field delimiter Since there's record delimiter The number of the record delimiter is the actual processing object data number. |
| | | \n | 0 and fixed length mixed | Permission | Data parsing according to the field delimiter Since there's record delimiter The number of the record delimiter is the actual processing object data number. |
In sending file, the target filename can be generated in a specific rule. On definition of the file service, the filename class can be defined so that can be changed in every the batch service execution. File name Management Class inherit the TargetFileNameGenerator and be implemented. The contents which has to be implemented in filename management class is the code which replaces the parameters included in the filename. Attach '#' to the both sides of the value to be declared as a parameter.
For this example, the file is generated by taking DATE and COUNT as the parameters. Filename Management Class is implemented to return the proper value for each parameter. The convert() method of the Filename Management Class is called by each parameter unit. In the example, because the parameter is two, call the convert() method twice.
convert(“DATE”); convert(“COUNT”);
Filename Management Class is implemented as follows.
import com.ism.common.file.TargetFilenameGenerator;
public class FileNameGenerator extends TargetFilenameGenerator {
public String convert(String exp) {
return “XXX”;
}
}
Implemented class is included to the class path so that execution process can refer it, or included in ism_custom.jar.
Form the interface services as the source and the target, and define the routing rule and mapping rule in the way the request is sent from the source to the target. Define the service flow from the source service to the target service by the service model. Determines whether to sequential process or parallel process in the definition of the service flow. The parallel process isn't available on batch service.
| Properties | Description |
|---|---|
| Pattern | System Interface Pattern Set the ACK transmission. |
| Properties | Description | Accept Value |
|---|---|---|
| Routing Determining Method | Find the way to search the target service. | All |
| Routing Input Field | When the routing determining method isn't All, define the input field. | |
| Routing Pattern Value | Define the assigned input field value, pattern, or the result of the calculation. Send the request to the target service which has the same value set. | |
| Service Mapping | Define the conversion rule between the source service and the target service | Mapping Rule |
| Error Mapping | Defines the conversion rule when the error response returned from the target service. | |
| Target Service | Define the target service to send the request according to the routing pattern value. | Target Service |
| Properties | Description |
|---|---|
| Pattern | System Interface Pattern Set the ACK transmission. |
Batch service does not support the parallel processing.
Integration Service defines the source and the target system to the service model result, and other rules on batch service execution.
Other information besides the processing flow as the batch integration service is as follows.
Define the object system to execute the source and target interface services defined on the service model, and the timeout value to be used on the interface with the pertinent systems. The timeout value is only available on file interface.
In the case sending data form file to DB, there should be the transaction unit defined for transaction process. The transaction unit is the same as the data inquiry unit number(fetch number).
If the size of the file data(number) is bigger than the transaction unit, divide the file data to the transaction unit.
Because it isn't possible to know whether the size of the file data which will be inputted on DB is bigger than the transaction unit or not, fetch the file and separate it. On file separation job, divide the file according to its record size(Row) registered on rule.
| Field Delimiter | Record Delimiter | Support Status | Note |
|---|---|---|---|
| In use | In use | Supported | No limits for the length of field delimiter. No limits for the record delimiter. |
| In use | None | Supported | |
| None(length) | In use | Supported | |
| None(length) | None | Supported | The file which the contents is written in a single line viewed on editor because it has no record delimiter. |
In the case the data structure registered on rule and the actual data structure is different, use Custom File Separator to separate the file. Custom seperation class is defined as an integration service unit.
Transaction is only supported by transaction unit. No transaction process per item is supported.
SQL query to input data to the source DB is generated on ISM. The input data for query generation of ISM are three as follow.
ISM generated or supported SQL query's forms as follow.
In the case below, there should be a date form defined to process the date type. The form designed by format is the date type supported by Java. On description, the date form is fixed below for convenience. Date form - yyyyMMdd
| Source Data | Target Data | Processing Mode | ||
|---|---|---|---|---|
| Type | Format | Type | Format | |
| Character(Including Char, Varchar) | Date | yyyyMMdd | On inquiry, suppose that the data fits on yyyyMMdd form and convert it to Date and process it. | |
| Date | yyyyMMdd | Date | yyyyMMdd | >The result of the data is processed as String, as the result, String should be fit on yyyyMMdd form. On the input on the target, it is converted to yyyyMMdd Date type. |
| Date | yyyMMdd | Chracter | The result of the data is processed as String, as the result, String should be fit on yyyyMMdd form. | |
Follow the source service definition of File-File batch service.
Target Service defines the DB calculation to be executed when the data is applied Because the data reflecting SQL query is generated automatically, only define the reflecting target table name. In the case the data reflecting table should be changed according to the executing time of the process, define the table name management class. If the data reflecting Query cannot be generated by the information defined by ISM, define Custom Query Handler.
Follow the file data structure definition of File-File batch service.
Target DB transfer data structure cannot have multiple field groups. Target DB transfer data must be the delimiter length data. There's no need to input both of field delimiter and the record delimiter. The structure definition of date type follow the processing rule of File-DB batch's Date type processing rule. Define whether to use SQL Function on the field group when the SQL Function is required on the generation of target SQL query If there's update query generation is required, define the key field on the definition of the field group.
Form the interface services as the source and the target, and define the routing rule and mapping rule in the way the request is sent from the source to the target. Define the service flow from the source service to the target service by the service model. Determines whether to sequential process or parallel process in the definition of the service flow. The parallel process is not used on batch service.
| Properties | Description |
|---|---|
| Pattern |
System Interface Pattern Set ACK transmission. |
| Properties | Description | Value |
|---|---|---|
| Routing Determining Methods | Determines the way to find the target service. | All |
| Routing Input Field | If the routing determining method isn't All, define the input field. | |
| Routing Pattern Value | Define the value of the designated input field, pattern, or the result value of the calculation. The same value is sent to request the target service. | |
| Service Mapping | Define the conversion rule between the source and the target service. | Mapping Rule. |
| Error Mapping | Defines the conversion rule when the error response returns from the target service. | |
| Target Service | Define the target service to send the request according to the routing pattern value. | Target Service |
| Properties | Description |
|---|---|
| Pattern | System Interface Pattern Set the ACK transmission. |
Integration Service defines the source and target system on service model result, and the rules of other Batch Service on run Other information besides processing flow for Batch Integration Service is as follows.
Define the object system to execute the source and target interface services defined on the service model, and the timeout value to be used on the interface with the pertinent systems. The timeout value is only available on file interface.