Failed to instantiate [com.mgmtp.a12.dataservices.search.SearchService] with DS 36.1.0

tim-steep-bit · August 28, 2023, 8:00am

Because of important fixes we need to update to DataServices 36.1.0.
But when starting the backend or run our tests we experience the following problem:

Error creating bean with name 'documentQueryService': Unsatisfied dependency expressed through field 'searchService': Error creating bean with name 'searchService' defined in class path resource [com/mgmtp/a12/dataservices/search/internal/SearchServiceConfiguration.class]: Failed to instantiate [com.mgmtp.a12.dataservices.search.SearchService]: Factory method 'searchService' threw exception with message: java.lang.reflect.InvocationTargetException

stack trace

Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'documentQueryService': Unsatisfied dependency expressed through field 'searchService': Error creating bean with name 'searchService' defined in class path resource [com/mgmtp/a12/dataservices/search/internal/SearchServiceConfiguration.class]: Failed to instantiate [com.mgmtp.a12.dataservices.search.SearchService]: Factory method 'searchService' threw exception with message: java.lang.reflect.InvocationTargetException
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:716) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:696) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:145) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:483) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1416) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:597) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1417) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1337) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:888) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-6.0.11.jar:6.0.11]
	... 33 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'searchService' defined in class path resource [com/mgmtp/a12/dataservices/search/internal/SearchServiceConfiguration.class]: Failed to instantiate [com.mgmtp.a12.dataservices.search.SearchService]: Factory method 'searchService' threw exception with message: java.lang.reflect.InvocationTargetException
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:659) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:647) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1332) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1162) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:560) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1417) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1337) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:713) ~[spring-beans-6.0.11.jar:6.0.11]
	... 48 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.mgmtp.a12.dataservices.search.SearchService]: Factory method 'searchService' threw exception with message: java.lang.reflect.InvocationTargetException
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:171) ~[spring-beans-6.0.11.jar:6.0.11]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:655) ~[spring-beans-6.0.11.jar:6.0.11]
	... 61 common frames omitted
Caused by: java.lang.IllegalStateException: java.lang.reflect.InvocationTargetException
	at com.mgmtp.a12.dataservices.search.internal.lucene.LuceneSearchServiceFactory.newSearchService(LuceneSearchServiceFactory.java:58) ~[dataservices-core-36.1.0.jar:na]
	at com.mgmtp.a12.dataservices.search.internal.SearchServiceFactory.newSearchService(SearchServiceFactory.java:56) ~[dataservices-core-36.1.0.jar:na]
	at com.mgmtp.a12.dataservices.search.internal.SearchServiceConfiguration.searchService(SearchServiceConfiguration.java:78) ~[dataservices-core-spring-boot-autoconfigure-36.1.0.jar:na]
	at com.mgmtp.a12.dataservices.search.internal.SearchServiceConfiguration$$SpringCGLIB$$0.CGLIB$searchService$3(<generated>) ~[dataservices-core-spring-boot-autoconfigure-36.1.0.jar:na]
	at com.mgmtp.a12.dataservices.search.internal.SearchServiceConfiguration$$SpringCGLIB$$2.invoke(<generated>) ~[dataservices-core-spring-boot-autoconfigure-36.1.0.jar:na]
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258) ~[spring-core-6.0.11.jar:6.0.11]
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-6.0.11.jar:6.0.11]
	at com.mgmtp.a12.dataservices.search.internal.SearchServiceConfiguration$$SpringCGLIB$$0.searchService(<generated>) ~[dataservices-core-spring-boot-autoconfigure-36.1.0.jar:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:139) ~[spring-beans-6.0.11.jar:6.0.11]
	... 62 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[na:na]
	at com.mgmtp.a12.dataservices.search.internal.lucene.LuceneSearchServiceFactory.newSearchService(LuceneSearchServiceFactory.java:55) ~[dataservices-core-36.1.0.jar:na]
	... 74 common frames omitted
Caused by: java.util.ServiceConfigurationError: org.apache.lucene.codecs.Codec: Provider org.apache.lucene.codecs.simpletext.SimpleTextCodec could not be instantiated
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) ~[na:na]
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813) ~[na:na]
	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729) ~[na:na]
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403) ~[na:na]
	at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:68) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.util.NamedSPILoader.<init>(NamedSPILoader.java:52) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.util.NamedSPILoader.<init>(NamedSPILoader.java:38) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:45) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.index.LiveIndexWriterConfig.<init>(LiveIndexWriterConfig.java:128) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.index.IndexWriterConfig.<init>(IndexWriterConfig.java:145) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.createIndexWriterConfig(DirectoryTaxonomyWriter.java:267) ~[lucene-facet-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:153) ~[lucene-facet-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:292) ~[lucene-facet-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39]
	at com.mgmtp.a12.dataservices.search.internal.lucene.LuceneFacetSupport.<init>(LuceneFacetSupport.java:61) ~[dataservices-core-36.1.0.jar:na]
	at com.mgmtp.a12.dataservices.search.internal.lucene.AbstractLuceneSearchService.<init>(AbstractLuceneSearchService.java:95) ~[dataservices-core-36.1.0.jar:na]
	at com.mgmtp.a12.dataservices.search.internal.lucene.LuceneSearchService.<init>(LuceneSearchService.java:33) ~[dataservices-core-36.1.0.jar:na]
	... 80 common frames omitted
Caused by: java.lang.NoClassDefFoundError: org/apache/lucene/codecs/BufferingKnnVectorsWriter
	at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[na:na]
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012) ~[na:na]
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150) ~[na:na]
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862) ~[na:na]
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760) ~[na:na]
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681) ~[na:na]
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639) ~[na:na]
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
	at org.apache.lucene.codecs.simpletext.SimpleTextCodec.<init>(SimpleTextCodec.java:50) ~[lucene-codecs-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[na:na]
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789) ~[na:na]
	... 95 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.codecs.BufferingKnnVectorsWriter
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
	... 111 common frames omitted

Besides using the new artifact we did not change anything about our configuration and code as there are no relevant changes for us listed on getA12.

Does anybody know why this is broken now? And how this can be fixed?

tim-steep-bit · August 28, 2023, 10:27am

Thanks to some help we were able to solve it quickly.
The lucene versions where not properly aligned resulting in some problems. Configuring the build.gradle like this solved the problem:

    implementation(a12Libs.dataservices.server.app) {
        [...]
        exclude group: "org.apache.lucene"
    }

    implementation "org.apache.lucene:lucene-analysis-common:9.4.2"
    implementation "org.apache.lucene:lucene-core:9.4.2"
    implementation "org.apache.lucene:lucene-facet:9.4.2"

anon67142020 · June 3, 2024, 10:07pm

This behaviour is still there in dataservices 36.3.7.

I wonder if this caused by conflicting versions of lucene specified in dataservices-parent.

there are top-level dependency entries for lucene-core, lucene-facet and lucene-analysis-common. All are set to 9.4.2.
then there is solr-core in version 9.3.0, which in turn provides / requires plenty of lucene-dependencies, all in version 9.7.0 (e.g. lucene-codecs).

In our project we import dataservices-parent as bom and just have one compile-scope dependency to dataservices-server-app. All should work well, but we end up with a mixed bag of lucene dependencies (and OPs error):

I am hestistant to apply OPs solutions, as it strips all the additional lucene-dependencies (solr might need at runtime).

I would rather go forward and fix lucene-core, lucene-facet and lucene-analysis-common at the version required by solr (i.e. 9.7.0), hence (excuse my maven):

        <dependency>
            <groupId>org.apache.lucene</groupId>
            <artifactId>lucene-core</artifactId>
            <version>9.7.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.lucene</groupId>
            <artifactId>lucene-facet</artifactId>
            <version>9.7.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.lucene</groupId>
            <artifactId>lucene-analysis-common</artifactId>
            <version>9.7.0</version>
        </dependency>

What is the recommended approach here? Thanks.

denise-narrow-token · June 5, 2024, 8:47am

Hi @anon67142020,

I have forwarded your question to the Data Services team. They have let me know that they are working on this in A12-15920 and the next Data Services version 36.3.8 will have lucene core fixed at 9.7.0.

I wil update this thread once this Data Services version has been released. Hopefully this will solve your problem.

Best regards,
Denise from the Discourse team

anon67142020 · June 5, 2024, 11:07am

Thank you, this anwers my question.