网站建设如何设定关键字,it外包收费,网上购物商城介绍,公司做公司网站广告Spring batch 系列文章
Spring Batch教程#xff08;一#xff09; 简单的介绍以及通过springbatch将xml文件转成txt文件 Spring Batch教程#xff08;二#xff09;示例#xff1a;将txt文件转成xml文件以及读取xml文件内容存储到数据库mysql Spring Batch教程#xff…Spring batch 系列文章
Spring Batch教程一 简单的介绍以及通过springbatch将xml文件转成txt文件 Spring Batch教程二示例将txt文件转成xml文件以及读取xml文件内容存储到数据库mysql Spring Batch教程三示例从mysql中读取数据写入文本和从多个文本中读取内容写入mysql Spring Batch教程四tasklet使用示例spring batch的定时任务使用 Spring Batch教程五spring boot实现batch功能注解示例读写文本文件 Spring Batch教程六spring boot实现batch功能注解示例读文件写入mysql 文章目录 Spring batch 系列文章一、示例1从mysql中读取数据写入txt文件1、maven依赖2、创建mysql 表并插入数据3、PersonInfo bean4、建立RowMapper5、创建ItemProcessor实现类6、添加Job listener(JobExecutionListener)7、进行job的配置1、job配置2、数据源配置 8、创建一个运行job的main类9、验证1、控制台输出2、程序结果输出 二、示例2从多数据源文件读取写入mysql1、maven依赖2、创建表3、PersonInfo bean4、建立FieldSetMapper5、创建ItemProcessor实现类6、添加Job listener(JobExecutionListener)7、进行job的配置1、数据源配置2、hibernate配置3、job配置 8、创建一个运行job的main类9、准备测试数据10、验证1、控制台输出2、程序结果输出 本文介绍了2个示例即从mysql中读取数据写入文本和从多个文本中读取内容写入mysql。 本文使用的是jdk版本最新版本的spring core和springb batch用不了。
一、示例1从mysql中读取数据写入txt文件
1、maven依赖 propertiesproject.build.sourceEncodingUTF-8/project.build.sourceEncodingspringframework.version5.2.25.RELEASE/springframework.versionjoda-time.version2.12.5/joda-time.versionmysql.connector.version5.1.31/mysql.connector.versionspringbatch.version4.2.8.RELEASE/springbatch.version/propertiesdependenciesdependencygroupIdorg.springframework.batch/groupIdartifactIdspring-batch-core/artifactIdversion${springbatch.version}/version/dependencydependencygroupIdorg.springframework.batch/groupIdartifactIdspring-batch-infrastructure/artifactIdversion${springbatch.version}/version/dependencydependencygroupIdorg.springframework/groupIdartifactIdspring-core/artifactIdversion${springframework.version}/version/dependencydependencygroupIdorg.springframework/groupIdartifactIdspring-oxm/artifactIdversion${springframework.version}/version/dependencydependencygroupIdorg.springframework/groupIdartifactIdspring-jdbc/artifactIdversion${springframework.version}/version/dependencydependencygroupIdjoda-time/groupIdartifactIdjoda-time/artifactIdversion${joda-time.version}/version/dependencydependencygroupIdmysql/groupIdartifactIdmysql-connector-java/artifactIdversion${mysql.connector.version}/version/dependency/dependencies
2、创建mysql 表并插入数据
DROP TABLE IF EXISTS personinfo;
CREATE TABLE personinfo (name varchar(30) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,birthday varchar(15) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,salary double NOT NULL
) ENGINE InnoDB CHARACTER SET utf8 COLLATE utf8_general_ci ROW_FORMAT Dynamic;-- ----------------------------
-- Records of personinfo
-- ----------------------------
INSERT INTO personinfo VALUES (alanchanchn, 1985-02-01, 76);
INSERT INTO personinfo VALUES (alan, 1979-09-01, 91.5);
INSERT INTO personinfo VALUES (chan, 1993-03-01, 92);
INSERT INTO personinfo VALUES (alanchan, 1995-08-01, 83);SET FOREIGN_KEY_CHECKS 1;3、PersonInfo bean
import lombok.Data;/*** author alanchan**/
Data
public class PersonInfo {private String name;private String birthday;private double salary;
}
4、建立RowMapper
import java.sql.ResultSet;
import java.sql.SQLException;import org.springframework.jdbc.core.RowMapper;import com.win.mysql2xml.bean.PersonInfo;/*** * author alanchan**/
public class PersonInfoRowMapper implements RowMapperPersonInfo {public PersonInfo mapRow(ResultSet rs, int rowNum) throws SQLException {PersonInfo personInfo new PersonInfo();personInfo.setName(rs.getString(name));personInfo.setBirthday(rs.getString(birthday));personInfo.setSalary(rs.getDouble(salary));return personInfo;}}
5、创建ItemProcessor实现类
本示例仅仅是过滤一下salary小于77的设置为salary*1.3。
import org.springframework.batch.item.ItemProcessor;import com.win.mysql2xml.bean.PersonInfo;/*** * author alanchan**/
public class PersonInfoItemProcessor implements ItemProcessorPersonInfo, PersonInfo {public PersonInfo process(PersonInfo personInfo) throws Exception {System.out.println(Processing result : personInfo);if (personInfo.getSalary() 77) {PersonInfo tempPersonInfo new PersonInfo();tempPersonInfo.setName(personInfo.getName());tempPersonInfo.setBirthday(personInfo.getBirthday());tempPersonInfo.setSalary(personInfo.getSalary() * 1.3);personInfo tempPersonInfo;}return personInfo;}}
6、添加Job listener(JobExecutionListener)
import java.util.List;import org.joda.time.DateTime;
//import org.joda.time.DateTime;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;/*** * author alanchan**/
public class PersonInfoJobListener implements JobExecutionListener {private DateTime startTime, stopTime;public void beforeJob(JobExecution jobExecution) {startTime new DateTime();System.out.println(job开始 at : startTime);}public void afterJob(JobExecution jobExecution) {stopTime new DateTime();System.out.println(job结束 at : stopTime);System.out.println(任务耗时毫秒 : getTimeInMillis(startTime, stopTime));if (jobExecution.getStatus() BatchStatus.COMPLETED) {System.out.println(job任务完成);// Here you can perform some other business logic like cleanup} else if (jobExecution.getStatus() BatchStatus.FAILED) {System.out.println(job任务异常如下 );ListThrowable exceptionList jobExecution.getAllFailureExceptions();for (Throwable th : exceptionList) {System.err.println(异常 : th.getLocalizedMessage());}}}private long getTimeInMillis(DateTime start, DateTime stop) {return stop.getMillis() - start.getMillis();}}
7、进行job的配置
1、job配置
文件位置/sping-batch/src/main/resources/spring-batch-context4.xml
beans xmlnshttp://www.springframework.org/schema/beansxmlns:batchhttp://www.springframework.org/schema/batchxmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexsi:schemaLocationhttp://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsdhttp://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsdimport resourceclasspath:context-datasource.xml/!-- JobRepository and JobLauncher are configuration/setup classes --bean idjobRepository classorg.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean /bean idjobLauncher classorg.springframework.batch.core.launch.support.SimpleJobLauncherproperty namejobRepository refjobRepository //bean!-- ItemReader which reads from database and returns the row mapped by rowMapper --bean iddatabaseItemReader classorg.springframework.batch.item.database.JdbcCursorItemReaderproperty namedataSource refdataSource /property namesql valueSELECT name,birthday,salary FROM personinfo /property namerowMapperbean classcom.win.mysql2xml.PersonInfoRowMapper //property/bean!-- ItemWriter writes a line into output flat file --bean idflatFileItemWriter classorg.springframework.batch.item.file.FlatFileItemWriter scopestepproperty nameresource valuefile:d:/personInfo.txt /property namelineAggregator!-- An Aggregator which converts an object into delimited list of strings --bean classorg.springframework.batch.item.file.transform.DelimitedLineAggregatorproperty namedelimiter value| /property namefieldExtractor!-- Extractor which returns the value of beans property through reflection --bean classorg.springframework.batch.item.file.transform.BeanWrapperFieldExtractorproperty namenames valuename,birthday,salary //bean/property/bean/property/bean!-- Optional JobExecutionListener to perform business logic before and after the job --bean idjobListener classcom.win.mysql2xml.PersonInfoJobListener /!-- Optional ItemProcessor to perform business logic/filtering on the input records --bean iditemProcessor classcom.win.mysql2xml.PersonInfoItemProcessor /!-- Step will need a transaction manager --bean idtransactionManager classorg.springframework.batch.support.transaction.ResourcelessTransactionManager /!-- Actual Job --batch:job idpersonInfoJobbatch:step idstep1batch:tasklet transaction-managertransactionManagerbatch:chunk readerdatabaseItemReader writerflatFileItemWriter processoritemProcessor commit-interval10 //batch:tasklet/batch:stepbatch:listenersbatch:listener refjobListener //batch:listeners/batch:job/beans2、数据源配置
文件位置/sping-batch/src/main/resources/context-datasource.xml
beans xmlnshttp://www.springframework.org/schema/beansxmlns:batchhttp://www.springframework.org/schema/batch xmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexsi:schemaLocationhttp://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans-4.0.xsdbean iddataSource classorg.springframework.jdbc.datasource.DriverManagerDataSourceproperty namedriverClassName valuecom.mysql.jdbc.Driver /property nameurl valuejdbc:mysql://192.168.10.44:3306/test /property nameusername valueroot /property namepassword value1234 //bean/beans8、创建一个运行job的main类 import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;/*** * author alanchan**/
public class App {SuppressWarnings(resource)public static void main(String areg[]) {ApplicationContext context new ClassPathXmlApplicationContext(spring-batch-context4.xml);JobLauncher jobLauncher (JobLauncher) context.getBean(jobLauncher);Job job (Job) context.getBean(personInfoJob);try {JobExecution execution jobLauncher.run(job, new JobParameters());System.out.println(Job执行状态 : execution.getStatus());} catch (JobExecutionException e) {System.out.println(Job 执行失败);e.printStackTrace();}}
}
9、验证
运行程序 查看输出文件内以及控制台内容
1、控制台输出
job开始 at :2023-07-21T10:49:44.68308:00
Processing result :PersonInfo(namealanchanchn, birthday1985-02-01, salary76.0)
Processing result :PersonInfo(namealan, birthday1979-09-01, salary91.5)
Processing result :PersonInfo(namechan, birthday1993-03-01, salary92.0)
Processing result :PersonInfo(namealanchan, birthday1995-08-01, salary83.0)
job结束 at :2023-07-21T10:49:44.91808:00
任务耗时毫秒 :235
job任务完成
Job执行状态 : COMPLETED2、程序结果输出 二、示例2从多数据源文件读取写入mysql
1、maven依赖
在这里插入代码片2、创建表
DROP TABLE IF EXISTS personinfo;
CREATE TABLE personinfo (id int(11) NOT NULL AUTO_INCREMENT,name varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,salary double(10, 2) NOT NULL,birthday varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,PRIMARY KEY (id) USING BTREE
) ENGINE InnoDB AUTO_INCREMENT 1 CHARACTER SET utf8 COLLATE utf8_general_ci ROW_FORMAT Dynamic;3、PersonInfo bean
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;import lombok.Data;/*** author alanchan**/
Data
Entity
Table(name personinfo)
public class PersonInfo {IdGeneratedValue(strategy GenerationType.IDENTITY)private int id;Column(name name, nullable false)private String name;Column(name birthday, nullable false)private String birthday;Column(name salary, nullable false)private double salary;
}4、建立FieldSetMapper
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;import com.win.multireaderhibernatewriter.bean.PersonInfo;/*** * author alanchan**/
public class PersonInfoFieldSetMapper implements FieldSetMapperPersonInfo {public PersonInfo mapFieldSet(FieldSet fieldSet) throws BindException {PersonInfo personInfo new PersonInfo();personInfo.setName(fieldSet.readString(0));personInfo.setBirthday(fieldSet.readString(1));personInfo.setSalary(fieldSet.readDouble(2));return personInfo;}}
5、创建ItemProcessor实现类
import org.springframework.batch.item.ItemProcessor;import com.win.multireaderhibernatewriter.bean.PersonInfo;/*** * author alanchan**/
public class PersonInfoItemProcessor implements ItemProcessorPersonInfo, PersonInfo {public PersonInfo process(PersonInfo personInfo) throws Exception {System.out.println(Processing result : personInfo);if (personInfo.getSalary() 60) {PersonInfo tempPersonInfo new PersonInfo();tempPersonInfo.setName(personInfo.getName());tempPersonInfo.setBirthday(personInfo.getBirthday());tempPersonInfo.setSalary(personInfo.getSalary() * 1.5);personInfo tempPersonInfo;}return personInfo;}
}
6、添加Job listener(JobExecutionListener)
import java.util.List;import org.joda.time.DateTime;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;/*** * author chenw**/
public class PersonInfoJobListener implements JobExecutionListener {private DateTime startTime, stopTime;public void beforeJob(JobExecution jobExecution) {startTime new DateTime();System.out.println(job开始 at : startTime);}public void afterJob(JobExecution jobExecution) {stopTime new DateTime();System.out.println(job结束 at : stopTime);System.out.println(任务耗时毫秒 : getTimeInMillis(startTime, stopTime));if (jobExecution.getStatus() BatchStatus.COMPLETED) {System.out.println(job任务完成);// Here you can perform some other business logic like cleanup} else if (jobExecution.getStatus() BatchStatus.FAILED) {System.out.println(job任务异常如下 );ListThrowable exceptionList jobExecution.getAllFailureExceptions();for (Throwable th : exceptionList) {System.err.println(异常 : th.getLocalizedMessage());}}}private long getTimeInMillis(DateTime start, DateTime stop) {return stop.getMillis() - start.getMillis();}}7、进行job的配置
1、数据源配置
文件位置/sping-batch/src/main/resources/context-datasource2.xml
beans xmlnshttp://www.springframework.org/schema/beansxmlns:batchhttp://www.springframework.org/schema/batchxmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexsi:schemaLocationhttp://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans-4.0.xsdbean iddataSource classcom.mchange.v2.c3p0.ComboPooledDataSource destroy-methodcloseproperty namedriverClass valuecom.mysql.jdbc.Driver /property namejdbcUrl valuejdbc:mysql://192.168.10.44:3306/test /property nameuser valueroot /property namepassword value1234 //bean/beans2、hibernate配置
文件位置/sping-batch/src/main/resources/context-model.xml
?xml version1.0 encodingUTF-8?
beans xmlnshttp://www.springframework.org/schema/beansxmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexmlns:contexthttp://www.springframework.org/schema/contextxmlns:txhttp://www.springframework.org/schema/txxmlns:aophttp://www.springframework.org/schema/aopxsi:schemaLocationhttp://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsdhttp://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsdhttp://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsdhttp://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.0.xsddefault-autowirebyName default-init-methodinitimport resourceclasspath:context-datasource2.xml/bean idsessionFactory classorg.springframework.orm.hibernate5.LocalSessionFactoryBean property namedataSource refdataSource/property namepackagesToScanlistvaluecom.win.multireaderhibernatewriter.bean/value/list/propertyproperty namehibernatePropertiespropsprop keyhibernate.dialectorg.hibernate.dialect.MySQLDialect/propprop keyhibernate.show_sqltrue/prop !-- prop keyhibernate.format_sqltrue/prop --/props/property/beanbean idtransactionManager classorg.springframework.orm.hibernate5.HibernateTransactionManager /tx:annotation-driven transaction-managertransactionManager//beans3、job配置
文件位置/sping-batch/src/main/resources/spring-batch-context5.xml
beans xmlnshttp://www.springframework.org/schema/beansxmlns:batchhttp://www.springframework.org/schema/batchxmlns:xsihttp://www.w3.org/2001/XMLSchema-instancexsi:schemaLocationhttp://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsdhttp://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsdimport resourceclasspath:context-model.xml/!-- JobRepository and JobLauncher are configuration/setup classes --bean idjobRepository classorg.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean /bean idjobLauncher classorg.springframework.batch.core.launch.support.SimpleJobLauncherproperty namejobRepository refjobRepository //beanbean idmultiResourceItemReader classorg.springframework.batch.item.file.MultiResourceItemReaderproperty nameresources valueclasspath:testmultireader/personinfo*.txt /property namedelegate refflatFileItemReader //bean!-- ItemReader reads a complete line one by one from input file --bean idflatFileItemReader classorg.springframework.batch.item.file.FlatFileItemReader scopestepproperty namelineMapperbean classorg.springframework.batch.item.file.mapping.DefaultLineMapperproperty namefieldSetMapper!-- Mapper which maps each individual items in a record to properties in POJO --bean classcom.win.multireaderhibernatewriter.PersonInfoFieldSetMapper //propertyproperty namelineTokenizer!-- A tokenizer class to be used when items in input record are separated by specific characters --bean classorg.springframework.batch.item.file.transform.DelimitedLineTokenizerproperty namedelimiter value| //bean/property/bean/property/bean!-- ItemWriter which writes data to database --bean iddatabaseItemWriter classorg.springframework.batch.item.database.HibernateItemWriterproperty namesessionFactory refsessionFactory //bean!-- Optional ItemProcessor to perform business logic/filtering on the input records --bean iditemProcessor classcom.win.multireaderhibernatewriter.PersonInfoItemProcessor /!-- Optional JobExecutionListener to perform business logic before and after the job --bean idjobListener classcom.win.multireaderhibernatewriter.PersonInfoJobListener /!-- Actual Job --batch:job idpersonInfoJobbatch:step idstep1batch:tasklet transaction-managertransactionManagerbatch:chunk readermultiResourceItemReader writerdatabaseItemWriter processoritemProcessor commit-interval10 //batch:tasklet/batch:stepbatch:listenersbatch:listener refjobListener //batch:listeners/batch:job/beans8、创建一个运行job的main类 import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionException;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;/*** * author alanchan**/
public class App {SuppressWarnings(resource)public static void main(String areg[]) {ApplicationContext context new ClassPathXmlApplicationContext(spring-batch-context5.xml);JobLauncher jobLauncher (JobLauncher) context.getBean(jobLauncher);Job job (Job) context.getBean(personInfoJob);try {JobExecution execution jobLauncher.run(job, new JobParameters());System.out.println(Job 执行状态 : execution.getStatus());} catch (JobExecutionException e) {System.out.println(Job 失败);e.printStackTrace();}}
}
9、准备测试数据
文件目录位置/sping-batch/src/main/resources/testmultireader 数据源文件都在该部目录下
/sping-batch/src/main/resources/testmultireader/personinfo-1.txt
alanchanchn|1985-02-01|98.8
alan|1979-09-01|91.5
/sping-batch/src/main/resources/testmultireader/personinfo-2.txt
zhangsan|1998-03-01|92.0
lisi|1995-08-01|60/sping-batch/src/main/resources/testmultireader/personinfo-3.txt
wangking|1989-04-01|18.0
tony|1995-08-01|86.0/sping-batch/src/main/resources/testmultireader/personinfo-4.txt
zhaoqin|1997-03-01|36.0
sunmonkey|1999-08-01|23.010、验证
1、控制台输出
job开始 at :2023-07-21T13:00:47.78008:00
Processing result :PersonInfo(id0, namealanchanchn, birthday1985-02-01, salary98.8)
Processing result :PersonInfo(id0, namealan, birthday1979-09-01, salary91.5)
Processing result :PersonInfo(id0, namezhangsan, birthday1998-03-01, salary92.0)
Processing result :PersonInfo(id0, namelisi, birthday1995-08-01, salary60.0)
Processing result :PersonInfo(id0, namewangking, birthday1989-04-01, salary18.0)
Processing result :PersonInfo(id0, nametony, birthday1995-08-01, salary86.0)
Processing result :PersonInfo(id0, namezhaoqin, birthday1997-03-01, salary36.0)
Processing result :PersonInfo(id0, namesunmonkey, birthday1999-08-01, salary23.0)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
Hibernate: insert into personinfo (birthday, name, salary) values (?, ?, ?)
job结束 at :2023-07-21T13:00:48.04408:00
任务耗时毫秒 :264
job任务完成
Job 执行状态 : COMPLETED2、程序结果输出 以上介绍了2个示例即从mysql中读取数据写入文本和从多个文本中读取内容写入mysql。
