Hello everyone,
I am currently looking for an implementation option for the following requirement. We need to import larger datasets consisting of 2000+ documents of a specific model. The import is initiated online by a user via the A12 frontend and not, for example, by an administrator. The process is therefore time-critical. Our problem is as follows. If we use the usual abstraction level and use the DocumentPersister / DocumentPersisterService, which is also used by the standard CSV importer, an update of the SOLR index is triggered when each individual document is created, which massively extends the runtime. Importing 6000 documents takes over an hour on a powerful system. An update request against SOLR always takes more than 100 ms. Ideally, the import should run synchronously and the SOLR index should only be updated with the new documents at the end.
Do you have any advice on how this requirement can be met in terms of performance? Possibly also with a view to the planned change to 2023.06.