Spring Batch Hello World Tutorial

21 Flares Twitter 0 Facebook 1 Google+ 20 Pin It Share 0 LinkedIn 0 Filament.io 21 Flares ×

Spring Batch is a framework from the Spring family that makes processing of batch jobs easier, modular and scalable.  Typical Batch processing programs to processes billions of transactions every day for large enterprises. These can be consolidation of month end sales, correction of errors, price and loyalty corrections, rate adjustments, insurance benefit determinations and so on. Spring batch is a robust framework for such batch processing needs. It provides developers with design patterns and framework code that solves common batch problems and allows developers to focus more on the business requirements and less on complex batch infrastructure. Spring Batch is a  POJO-based development framework utilizing and providing all the goodness of Spring framework. There is a misconception among many developers that it is a scheduling framework. It is NOT. It works in collaboration with many popular open source and commercial scheduling systems like Quartz, Control – M etc.

It provides APIs to read/write resources, transaction management, restart failed jobs, monitor jobs and techniques like multi threading and partitioning. In short, it provides solutions to almost all problems faced in the processing of complex batch operations. This tutorial will show you how to create a batch application to read data from a flat file and write it to a database table after processing.

Before we proceed with the tutorial let us look at some of the elements of a spring batch application. In Spring batch, a Job consists of a series of steps and each step consists of either a [ read-process-write ] task or a tasklet ( single operation).

Read – process – write Task

A step can contain a read-process-write task. Reader reads data from a file or a database or any other data source. The process part processes each record and finally the write phase saves the results of  the processing. For example, a step may read data from an excel file, process it and write it into the database.

Tasklet

If your step consists of only a single operation like calling a stored procedure or making a web service call, it can be implemented as a tasklet. The Tasklet is a simple interface that has one method, execute.

A Job  consists of  1 or more steps.
A Step = 1 READ-PROCESS-WRITE or 1 Tasklet.
Job = Chained Steps ->  { Step A -> Step B -> Step 3 } 

 Spring Batch Hello World Application

This hello world application demonstrates all that is required to start development with Spring batch. This app does the following

  1. Reads an Input CSV File – ( ItemReader )
  2. Processes the file -( ItemProcessor )
  3. Writes the data to MYSQL Database – ( ItemWriter )

Project Structure

Using Eclipse IDE, create a simple maven project. You can use the values com.javahash.batch as groupId and SpringBatch as artifactId while creating the maven project.

The complete project structure will be

SpringBatch-Project-Structure

 

 

Job Definition

The batch job definition is defined in orderprocess-job-config.xml

In the above XML, we import 2 other xml files namely, datasource-config.xml and job-context.xml. These XML files holds the configuration related to datasource ( MySQL in this case) and spring batch framework configurations respectively. We shall look at those files later.

Line 18: We have defined a job with id “hellojob” that consists of a single step and single tasklet.For read/write scenarios spring batch uses chunk oriented processing. Items are read one at a time by an item reader and are aggregated into a  ‘chunk’ of  specified size. When the number of read items in the chunk reaches the specified limit, the contents of the chunk are sent to the item writer and written to the  data source. The size of the chunk is configured as a commit limit.

 Line 30: Holds the definition of the reader. Spring batch provides many out of the box reader implementations to read the data from different sources. Since we are reading, csv files, we are using the FlatFileItemReader provided by Spring batch. The resource attribute refers to the input file to be processed. The value attribute holds the path where the input files are residing. In this example we use a classpath resource. classpath:input/orders.csv indicates look for orders.csv inside the input folder starting from the root directory in the classpath. We have src/main/resources in the classpath. Hence the input/orders.csv is put there. The linesToSkip attribute indicates the number of lines that should be ignored by the reader before actual processing begins. Here we have a blank row as the first line, so we are skipping it. The lineMapper attribute defines the configuration of the component that will perform the line reads. In this instance we use a Spring Batch implementation called DefaultLineMapper which requires a line tokenizer component to split the line contents into individual fields.

The fieldSetMapper attribute refers to the custom class com.javahash.batch.OrderDataMapper that takes a Field Set and maps the fields to instance variables on the OrderDataMapper Java Object.

Line 48:  The writer component is responsible for writing data items, in this case OrderDataMapper POJOs to the database.

OrderDataMapper

Maps the fields from the CSV file to Java Object’s instance variables

OrderItemWriter

Let us now take a look at the other 2 xmls we imported into orderprocess-job-config.xml

job-context.xml

The above xml stores the configuration information required for the spring batch framework to work. We have configured a datasource for the job repository and the datasource maps to mysql.

datasource-config.xml

This file contains the datasource configuration. We have also configured scripts to auto create the spring batch related meta tables.

Running the Job

Running the spring batch job involves

  • Initializing and starting the spring context
  • Get reference to JobLauncher class
  • Execute the run method of JobLaucher

Download Source Code

Source Code – SpringBatch Tutorial

  • Ashokkumar Shanmugam

    Nice