When using KernelDocumentService.validateDocument() in a bulk import scenario (e.g., importing 1000 records from Excel), the method DocumentModelCodeGenerator.calculateModelHash() is called 1000 times , causing performance degradation. Each call serializes the entire IDocumentModel to JSON, which is expensive and unnecessary when the model hasn’t changed.
Use Case Context
We have an Excel import feature that:
- Reads data rows from an Excel file (up to 5000 records)
- Converts each row to an
IDocument - Validates each document using
KernelDocumentService.validateDocument() - Saves valid documents to the database
for (int index = 0; index < dataRows.size(); index++) {
IDocument documentFromRow = dataConvertor.createDocumentFromDataRow(documentModel, propertyMapper, row);
Optional<IDocumentValidationResult> result = kernelDocumentService.validateDocument(documentFromRow, locale);
// process result...
}
Validation Call Chain
KernelDocumentService.validateDocument(IDocument document, Locale locale)
-> KernelDocumentService.validatePartially(IDocument document, Locale locale)
-> DocumentAbstractRtService.validatePart(IDocument document, Set<IEntityInstance> relevantEntityInstances, Locale locale)
-> DocumentAbstractRtService.callValidator(ValidationMethod validationMethod, IDocument document, Set<IEntityInstance> relevantEntityInstances, Locale locale)
->
- Calls modelProvider.getMetaModel(documentModel, variant)
- Calls modelProvider.getValidator(documentModel, variant)
-> DynamicModelProvider.getValidator(IDocumentModel documentModel, String variant)
-> DynamicModelProvider.getMetaModel(IDocumentModel documentModel, String variant)
-> DynamicModelProvider.getModelCode(IDocumentModel documentModel, String variant)
-> DocumentModelCodeGenerator.generateCode(IDocumentModel documentModel, String variant)
-> calculateModelHash(documentModel)
-> DocumentModelCodeGenerator.getSerializedModel(IDocumentModel documentModel)
For 1000 documents of the same model, validateDocument() is called 1000 times.
The problem
In DocumentModelCodeGenerator.generateCode() :
public IModelCode generateCode(final IDocumentModel documentModel, final String variant) {
String modelCodeIdentifier = modelIdProvider.getDocumentModelId(documentModel, variant);
IModelCode modelCode = cache.getModelCode(modelCodeIdentifier);
// ⚠️ THIS IS CALLED EVERY TIME - even when cache already has the code
final String modelHash = calculateModelHash(documentModel);
if (modelCode == null || !StringUtils.equals(modelHash, modelCode.getModelHash())) {
// Code generation only happens on cache miss (good!)
synchronized (lock) {
// ... generate code and cache it
}
}
return modelCode;
}
The calculateModelHash() method serializes the entire IDocumentModel to JSON on every call just to compare hashes, even when the cache already contains valid code.
My question:
- Is it necessary to call
calculateModelHash()on every validation request? How can we reduce this in the bulk import use case? - Is there any existing support for batch validation? if not, how can we implement a short term solution in our side, so we can replace when there is a support.
Note
A12 2024.06 ext9