Gradle build plan for 4.3.x branch

Build: #5 was successful Changes by Mahmoud Ben Hassine <mbenhassine@vmware.com>

Build result summary

Details

Completed
Duration
6 minutes
Labels
None
Agent
bamboo-15.af.pivotal.io
Revision
b96ac3cb30d33bd2c25a3a2f496367fb9085c8fd b96ac3cb30d33bd2c25a3a2f496367fb9085c8fd
Total tests
3911
Successful since
#3 ()

Tests

Code commits

Author Commit Message Commit date
Mahmoud Ben Hassine <mbenhassine@vmware.com> Mahmoud Ben Hassine <mbenhassine@vmware.com> b96ac3cb30d33bd2c25a3a2f496367fb9085c8fd b96ac3cb30d33bd2c25a3a2f496367fb9085c8fd Fix PartitionParserTests
Some tests in PartitionParserTests were
failing intermittently due to the usage
of non-synchronized shared state between
concurrent threads.

This commit updates the test code to use
`AtomicInteger` instead of `int` for the
state shared between concurrent threads.

(cherry picked from commit 98fba4a03bb13ef407a52c4fd01b9566eaf61085)
Mahmoud Ben Hassine <mbenhassine@vmware.com> Mahmoud Ben Hassine <mbenhassine@vmware.com> af3eff3474abcad5fb5ad1636ca60d609b3dd61a af3eff3474abcad5fb5ad1636ca60d609b3dd61a Fix KafkaItemReaderTests
Tests in this class fail intermittently because
they send messages to Kafka in an asynchronous
way and assert on the results immediately without
waiting for the send operation to complete.

This commit updates the tests to wait for
send results before asserting on them
(similar to a140a9f5).

(cherry picked from commit 5826111b853d2ae4a2e6f061ae1a079ad524da89)
Mahmoud Ben Hassine <mbenhassine@vmware.com> Mahmoud Ben Hassine <mbenhassine@vmware.com> c1fbed0007586aa0303520825c3d88b2a699aaa8 c1fbed0007586aa0303520825c3d88b2a699aaa8 Fix OptimisticLockingFailureTests#testAsyncStopOfStartingJob
This test was failing intermittently due to an incorrect
way of waiting for a job execution to finish, which is:

```
while(jobExecution.isRunning()) {
   // wait for async launched job to complete execution
}
```

`JobExecution#isRunning()` is based on the status
of the job execution in memory which might not be
persisted yet. Here is an excerpt from the Javadoc:

```
Test if this JobExecution indicates that it is
running. It should be noted that this does not
necessarily mean that it has been persisted as
such yet.
```

That's why in the case where `isRunning` returns
false and the JobExecution is not persisted yet
(which is still in a running status in db),
the second attempt of re-running the job fails
with a `JobExecutionAlreadyRunningException`.

This commit fixes the loop by continuously
checking the status of the Job execution
in the job repository until it reaches one
of the end statuses.

Issue #1121

(cherry picked from commit cf2642223d6736514d8d9c6574e72d486ee590b5)
Mahmoud Ben Hassine <mbenhassine@vmware.com> Mahmoud Ben Hassine <mbenhassine@vmware.com> 4fefe4c101f174a1b8397285db3b751d68fdd1eb 4fefe4c101f174a1b8397285db3b751d68fdd1eb Fix flaky tests
JsrSplitParsingTests#test and DecisionStepTests#testDecisionAfterSplit
were failing intermittently with:

```
org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
    at org.springframework.batch.core.jsr.job.flow.JsrFlowJob.doExecute(JsrFlowJob.java:88) ~[main/:?]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) [main/:?]
    at org.springframework.batch.core.jsr.launch.JsrJobOperator$2.run(JsrJobOperator.java:674) [main/:?]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=flow1.step1 at state=flow1.step1.flow1.step1 with exception
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:178) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
    ... 1 more
Caused by: org.springframework.dao.ConcurrencyFailureException: PreparedStatementCallback; SQL [INSERT INTO BATCH_STEP_EXECUTION_CONTEXT (SHORT_CONTEXT, SERIALIZED_CONTEXT, STEP_EXECUTION_ID) VALUES(?, ?, ?)]; transaction rollback: serialization failure; nested exception is java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
    at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:73) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:79) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1541) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:667) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
    at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
    at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
    at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
    ... 1 more
Caused by: java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
    at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
    at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
    at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
    at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
    at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
    ... 1 more
Caused by: org.hsqldb.HsqlException: transaction rollback: serialization failure
    at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.Session.handleAbortTransaction(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.Session.executeCompiledStatement(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.Session.execute(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
    at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
    at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
    at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
    at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
    ... 1 more
```

This failure is due to a concurrency issue when executing
the split flow with the default SimpleAsyncTaskExecutor.

Several attempts have been made to fix this issue with no success:

* use `@DirtiesContext` annotation
* use a separate db for each test
* use `READ_COMMITTED` isolation level in the job repository
* downgrade hsqldb from v2.5.1 to v2.4.1 (since v2.5.1 has
introduced several changes in the MVCC mode)

The issue seems to be related to how the in-memory database
is shared between tests *and* how the test context is cached
(ie the combination of both).

This commit moves these tests to a separate test class which
seem to fix the issue.

(cherry picked from commit 109073130216b950f3b1201ec6d4764a799d071c)
Mahmoud Ben Hassine <mbenhassine@vmware.com> Mahmoud Ben Hassine <mbenhassine@vmware.com> 8f110d6d4f5114ccd83d22cdd5bf371faeeee595 8f110d6d4f5114ccd83d22cdd5bf371faeeee595 Remove unused code in DecisionStepTests
(cherry picked from commit 287eace69fd03fc24d101f03bcbc9944b4cb717f)