Infinispan cache is in status "TERMINATED" during Integrationtests

Migration
1

Hi!
I am currently migrating to the A12 version 2026.06-ext0. This involves the change from hazelcast to infinispan. With Infinispan we now get the problem that Integrationtests are getting errors that the cache is in status “TERMINATED” with a stacktrace looking like this:

org.infinispan.commons.IllegalLifecycleStateException: ISPN000323: Cache 'validationCache' is in 'TERMINATED' state and so it does not accept new invocations. Either restart it or recreate the cache container.
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:540)
at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:567)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:670)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at com.mgmtp.a12.dataservices.server.util.internal.KernelCachesPreloader.listenOnServicesInitializationFinished(KernelCachesPreloader.java:104)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:158)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:112)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.infinispan.commons.IllegalLifecycleStateException: ISPN000323: Cache 'validationCache' is in 'TERMINATED' state and so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.impl.InvocationContextInterceptor.ignoreCommand(InvocationContextInterceptor.java:101)
at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:90)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:238)
at org.infinispan.cache.impl.InvocationHelper.doInvoke(InvocationHelper.java:329)
at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:112)
at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:554)
at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:548)
at org.infinispan.cache.impl.EncoderCache.get(EncoderCache.java:675)

We have the DirtiesContext annotation on every integrationtests and I am suspecting that the caches are not properly reset after the first integration test.

One of our integrationtests relies on the caches working and fails now because of that.

Any idea how to solve that?

2

Hi, looks like KernelCachesPreloader warms caches asynchronously (@Async, no custom executor configured, so it falls back to Spring’s SimpleAsyncTaskExecutor) and dispatches per-model work onto the JVM-wide common ForkJoinPool. Neither is synchronized with the Spring ApplicationContext lifecycle. When @DirtiesContext tears down the context, the EmbeddedCacheManager bean is stopped immediately (destroyMethod = “stop”), but a still-running warm-up thread from that context can then hit the cache, causing ISPN000323. Hazelcast tolerated this; Infinispan enforces lifecycle state strictly.

Here’s LLMs suggestion:

Workaround: force async execution to run synchronously in your test config, so cache preloading completes before context startup finishes — same approach dataservices-server-init-app already uses for clean shutdown:

  @Configuration
  static class SyncAsyncTestConfig implements AsyncConfigurer {
      @Override
      public Executor getAsyncExecutor() {
          return new SyncTaskExecutor();
      }
  }

Another workaround could be mgmtp.a12.dataservices.initialization.pre-compile.enabled-for-models: [] in your tests.

3

Thank you for the kind response!

Unfortunately neither of those helped. We already have an AutoConfigurer, but replacing that one by the SyncTaskExecutor did not result in any changed behavior. Furthermore, we already disabled pre-compiling for all models using that configuration to save time in the integration tests.

The integration test that does not pass calls DocumentService#update, which results in the before mentioned exception on the cache “com.mgmtp.a12.kernel.md.model.api.IDocumentModel”. This happens during a DocumentBeforeUpdateEvent Listener, which calls UAA to check for permissions and eventually tries to call DefaultDocumentModelResolver#getUserModel, which fails to access that cache. Maybe that context helps?

4

I have an update on that problem:
I changed the cache manager for the integrationtests to the ConcurrentMapCacheManager. This resolved the initial exception, introduces however a new exception when trying to create the dsLiquibaseProperties bean in UAA:
Caused by: java.lang.IllegalStateException: org.springframework.web.context.support.GenericWebApplicationContext@7364eed1 has been closed already

This further indicates that for some reason UAA (and maybe other components) use a closed spring application context.

5

It seems I have found the bug in UAA. Spring does not reset static fields when the context is dirtied as far as I know. PropertyChangesChecker in UAA has a static field called JAVERS with beans that are apparently not reset after the context is closed and thus contain old beans.

The following workaround
@AfterAll
static void afterAll() throws NoSuchFieldException, IllegalAccessException {
Field field = PropertyChangesChecker.class.getDeclaredField("JAVERS");
field.setAccessible(true);
field.set(null, null);
}
worked for me.

This does not fix the other terminated caches during KernelCachesPreloader, but since our integration tests do not rely on those to work, this works for now.