Nightly build for the 4.3.x branch

Build: #18 was successful Scheduled with changes by Mahmoud Ben Hassine <mbenhassine@vmware.com>

Code commits

Git

  • Mahmoud Ben Hassine <mbenhassine@vmware.com>

    Mahmoud Ben Hassine <mbenhassine@vmware.com> b96ac3cb30d33bd2c25a3a2f496367fb9085c8fd

    Fix PartitionParserTests
    Some tests in PartitionParserTests were
    failing intermittently due to the usage
    of non-synchronized shared state between
    concurrent threads.

    This commit updates the test code to use
    `AtomicInteger` instead of `int` for the
    state shared between concurrent threads.

    (cherry picked from commit 98fba4a03bb13ef407a52c4fd01b9566eaf61085)

    • spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/PartitionParserTests.java (version b96ac3cb30d33bd2c25a3a2f496367fb9085c8fd)
  • Mahmoud Ben Hassine <mbenhassine@vmware.com>

    Mahmoud Ben Hassine <mbenhassine@vmware.com> af3eff3474abcad5fb5ad1636ca60d609b3dd61a

    Fix KafkaItemReaderTests
    Tests in this class fail intermittently because
    they send messages to Kafka in an asynchronous
    way and assert on the results immediately without
    waiting for the send operation to complete.

    This commit updates the tests to wait for
    send results before asserting on them
    (similar to a140a9f5).

    (cherry picked from commit 5826111b853d2ae4a2e6f061ae1a079ad524da89)

    • spring-batch-infrastructure/src/test/java/org/springframework/batch/item/kafka/KafkaItemReaderTests.java (version af3eff3474abcad5fb5ad1636ca60d609b3dd61a)
  • Mahmoud Ben Hassine <mbenhassine@vmware.com>

    Mahmoud Ben Hassine <mbenhassine@vmware.com> c1fbed0007586aa0303520825c3d88b2a699aaa8

    Fix OptimisticLockingFailureTests#testAsyncStopOfStartingJob
    This test was failing intermittently due to an incorrect
    way of waiting for a job execution to finish, which is:

    ```
    while(jobExecution.isRunning()) {
       // wait for async launched job to complete execution
    }
    ```

    `JobExecution#isRunning()` is based on the status
    of the job execution in memory which might not be
    persisted yet. Here is an excerpt from the Javadoc:

    ```
    Test if this JobExecution indicates that it is
    running. It should be noted that this does not
    necessarily mean that it has been persisted as
    such yet.
    ```

    That's why in the case where `isRunning` returns
    false and the JobExecution is not persisted yet
    (which is still in a running status in db),
    the second attempt of re-running the job fails
    with a `JobExecutionAlreadyRunningException`.

    This commit fixes the loop by continuously
    checking the status of the Job execution
    in the job repository until it reaches one
    of the end statuses.

    Issue #1121

    (cherry picked from commit cf2642223d6736514d8d9c6574e72d486ee590b5)

    • spring-batch-core/src/test/java/org/springframework/batch/core/repository/dao/OptimisticLockingFailureTests.java (version c1fbed0007586aa0303520825c3d88b2a699aaa8)
  • Mahmoud Ben Hassine <mbenhassine@vmware.com>

    Mahmoud Ben Hassine <mbenhassine@vmware.com> 4fefe4c101f174a1b8397285db3b751d68fdd1eb

    Fix flaky tests
    JsrSplitParsingTests#test and DecisionStepTests#testDecisionAfterSplit
    were failing intermittently with:

    ```
    org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
        at org.springframework.batch.core.jsr.job.flow.JsrFlowJob.doExecute(JsrFlowJob.java:88) ~[main/:?]
        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) [main/:?]
        at org.springframework.batch.core.jsr.launch.JsrJobOperator$2.run(JsrJobOperator.java:674) [main/:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
    Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=flow1.step1 at state=flow1.step1.flow1.step1 with exception
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:178) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
        ... 1 more
    Caused by: org.springframework.dao.ConcurrencyFailureException: PreparedStatementCallback; SQL [INSERT INTO BATCH_STEP_EXECUTION_CONTEXT (SHORT_CONTEXT, SERIALIZED_CONTEXT, STEP_EXECUTION_ID) VALUES(?, ?, ?)]; transaction rollback: serialization failure; nested exception is java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
        at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:73) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:79) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1541) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:667) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
        at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
        at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
        at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
        at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
        at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
        ... 1 more
    Caused by: java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
        at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
        at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
        at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
        at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
        at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
        at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
        at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
        at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
        ... 1 more
    Caused by: org.hsqldb.HsqlException: transaction rollback: serialization failure
        at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.Session.handleAbortTransaction(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.Session.executeCompiledStatement(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.Session.execute(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
        at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
        at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
        at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
        at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
        at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
        at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
        at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
        at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
        at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
        at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
        ... 1 more
    ```

    This failure is due to a concurrency issue when executing
    the split flow with the default SimpleAsyncTaskExecutor.

    Several attempts have been made to fix this issue with no success:

    * use `@DirtiesContext` annotation
    * use a separate db for each test
    * use `READ_COMMITTED` isolation level in the job repository
    * downgrade hsqldb from v2.5.1 to v2.4.1 (since v2.5.1 has
    introduced several changes in the MVCC mode)

    The issue seems to be related to how the in-memory database
    is shared between tests *and* how the test context is cached
    (ie the combination of both).

    This commit moves these tests to a separate test class which
    seem to fix the issue.

    (cherry picked from commit 109073130216b950f3b1201ec6d4764a799d071c)

    • spring-batch-core/src/test/java/org/springframework/batch/core/jsr/configuration/xml/JsrSplitParsingTests.java (version 4fefe4c101f174a1b8397285db3b751d68fdd1eb)
    • spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/DecisionStepTests.java (version 4fefe4c101f174a1b8397285db3b751d68fdd1eb)
    • spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/SplitTests.java (version 4fefe4c101f174a1b8397285db3b751d68fdd1eb)
    • spring-batch-core/src/test/resources/META-INF/batch-jobs/SplitTests-testDecisionAfterSplit-context.xml (version 4fefe4c101f174a1b8397285db3b751d68fdd1eb)
    • spring-batch-core/src/test/resources/META-INF/batch-jobs/SplitTests-testSplit-context.xml (version 4fefe4c101f174a1b8397285db3b751d68fdd1eb)
  • Mahmoud Ben Hassine <mbenhassine@vmware.com>

    Mahmoud Ben Hassine <mbenhassine@vmware.com> 8f110d6d4f5114ccd83d22cdd5bf371faeeee595

    Remove unused code in DecisionStepTests
    (cherry picked from commit 287eace69fd03fc24d101f03bcbc9944b4cb717f)

    • spring-batch-core/src/test/java/org/springframework/batch/core/jsr/step/DecisionStepTests.java (version 8f110d6d4f5114ccd83d22cdd5bf371faeeee595)