Hi guys,
My version of A12 is 2023.06-ext4 with DS 36.3.1.
I’m trying to implement a custom document repository (IDocumentRepository).
Since I only need custom logic for the create method, I am passing the calls to all the other methods to the default document repository. In the A12 Tutorial: Data Services Introduction this is also done this way GetA12 Login.
The problem is that in the overview no documents are shown anymore, and an exception is thrown:
java.lang.IllegalStateException: Duplicate key AnonHint-document/12 (attempted merging values ... and ...)
at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at com.mgmtp.a12.dataservices.document.persistence.internal.DefaultDocumentService.load(DefaultDocumentService.java:225)
I found out that the cause is due to the method findDocumentsByDocRefs in the document repository.
Because now with my custom repository there are two repositories - the custom one and the default one. The DefaultDocumentService is calling findDocumentsByDocRefs twice - once on the custom repository and once on the default.
This leads to duplicated references and explains the resulting exception.
I could work around this issue by just returning an empty list for findDocumentsByDocRefs in my custom repository. But that feels unsafe to me, since I don’t know if there are other places where this gets called.
So my qustion is if that is a bug in the DefaultDocumentService or do I need to handle it in some other way in my custom repository and thus is the tutorial, I mentioned earlier, outdated regarding this part?
Hello,
please implement your repository so it returns unique documents. That’s the rule for repositories.
You can not store the same document in several repositories.
For customizations, please check customization events like DocumentBeforeCreate and similar ones.
Hi @petr-high-peak, thank you already for the reply!
Unfortunately, I don’t really understand what you mean with that, could you please elaborate more on it?
Please consider that in my custom repository, all methods that return something are just using the default document repository, as I don’t need to customize them.
The only custom code is inside the create method where I need to set the created_by field, and then I’m again calling the default document repository’s create to save it. So I’m not storing the same document multiple times.
I have also checked on the provided Event Listeners of DS GetA12 Login. But I didn’t find any suiting one, because I need the DataServicesDocument to be able to set created_by and only need it when creating new documents.
Moin @raffael-broad-linden,
could you please share your custom implementation of the IDocumentRepository?
Kind regards,
Jan
Repositories are not intended for customizing the current default repository. If you need some customizations, it’s better to use customization events.
If there is no other way around and you must reimplement the repository, and all documents are stored using your customized repository, then you must disable the default repository and use your customized one as the default one.
Also, it is not a good idea to rely on internal code which our DefaultDocumentRepository is. Then you can easily get into trouble if we change implementation, which is possible for internal code in the non-breaking release and even in the patch version.
So I’d discourage implementing your custom repo this way.
Best regards,
Petr
For events documentation see GetA12 Login
Hi @petr-high-peak, thank you for the answer!
I think there is no other way around implementing an IDocumentRepository, I could not find a suiting event from the documentation.
I thought the approved which was used in the Tutorial would be valid, so I followed it.
Hi @jan-static-daemon,
here is my custom implementation, it is similar to the one from the Backend Tutorial. I only need to customize the create method:
import com.mgmtp.a12.dataservices.document.DataServicesDocument;
import com.mgmtp.a12.dataservices.document.DocumentReference;
import com.mgmtp.a12.dataservices.document.persistence.IDocumentRepository;
import com.mgmtp.a12.kernel.md.document.api.IDocument;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import lombok.NonNull;
@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class HintDocumentRepository implements IDocumentRepository {
private final IDocumentRepository defaultDocumentRepository;
public HintDocumentRepository(@Qualifier("defaultDocumentRepository") IDocumentRepository defaultDocumentRepository) {
this.defaultDocumentRepository = defaultDocumentRepository;
}
@Override
public boolean supports(IDocument document) {
return HINT_MODEL_NAMES.contains(document.getDocumentModelId());
}
@Override
public void create(DataServicesDocument dataServicesDocument) {
HintDocumentWrapper.fromIDocument(dataServicesDocument.getKernelDocument()).ifPresent(hintDocument -> {
boolean isAnonymous = hintDocument.isAnonymous();
String creator =
(isAnonymous ? hintDocument.getName() : hintDocument.getEmail().orElseThrow()).toLowerCase();
dataServicesDocument.setCreatedBy(creator);
dataServicesDocument.setModifiedBy(creator);
});
defaultDocumentRepository.create(dataServicesDocument);
}
@Override
public void update(DataServicesDocument dataServicesDocument) {
defaultDocumentRepository.update(dataServicesDocument);
}
@Override
public void delete(DocumentReference documentReference) {
defaultDocumentRepository.delete(documentReference);
}
@Override
public Optional<DataServicesDocument> getByDocumentReference(DocumentReference documentReference) {
return defaultDocumentRepository.getByDocumentReference(documentReference);
}
@Override
public @NonNull List<DocumentReference> findAllDocRefsForModel(String modelId) {
return defaultDocumentRepository.findAllDocRefsForModel(modelId);
}
@Override
public @NonNull List<DataServicesDocument> findDocumentsByDocRefs(List<DocumentReference> docRefs) {
// This is the workaround I was mentioning
// In the tutorial this would also just call the defaultDocumentRepository, but then I get the exception
// In the tutorial the DS version is different, so that is prob. the reason why it is still working there
return Collections.emptyList();
}
@Override
public Optional<String> findModelForDocument(String documentId) {
return defaultDocumentRepository.findModelForDocument(documentId);
}
}
And I am using @Order instead of @Primary, because otherwise it is not used to create.
If I understand it right, you want to set different users in some cases, right? So, you’re right that it’s not possible to handle it using events for now, but in the near future, as soon as the new metadata concept is introduced it will be feasible.
But for now, repositories must be isolated from each other. You can not provide the same document through multiple repos. So in your case, you should unregister the default DS repo and use just your own.
Hi @petr-high-peak thanks for the response!
Yes, that’s right, I only want to set the user on create.
I think I understand that my custom repository should not provide the documents, since they are already provided by the default repo. But unregistering the default repo would mean I had to implement the whole interface myself, and I don’t think that is realistic, since I only need that small custom code part.
What about returning empty lists and optionals in the methods which provide documents in my custom repo? Then only the default one would provide them, is that a valid solution as well?
Then it would look like this:
@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class HintDocumentRepository implements IDocumentRepository {
private final IDocumentRepository defaultDocumentRepository;
public HintDocumentRepository(@Qualifier("defaultDocumentRepository") IDocumentRepository defaultDocumentRepository) {
this.defaultDocumentRepository = defaultDocumentRepository;
}
@Override
public boolean supports(IDocument document) {
return HINT_MODEL_NAMES.contains(document.getDocumentModelId());
}
@Override
public void create(DataServicesDocument dataServicesDocument) {
// Custom things...
defaultDocumentRepository.create(dataServicesDocument);
}
@Override
public void update(DataServicesDocument dataServicesDocument) {
defaultDocumentRepository.update(dataServicesDocument);
}
@Override
public void delete(DocumentReference documentReference) {
defaultDocumentRepository.delete(documentReference);
}
@Override
public Optional<DataServicesDocument> getByDocumentReference(DocumentReference documentReference) {
return Optional.empty();
}
@Override
public @NonNull List<DocumentReference> findAllDocRefsForModel(String modelId) {
return Collections.emptyList();
}
@Override
public @NonNull List<DataServicesDocument> findDocumentsByDocRefs(List<DocumentReference> docRefs) {
return Collections.emptyList();
}
@Override
public Optional<String> findModelForDocument(String documentId) {
return defaultDocumentRepository.findModelForDocument(documentId);
}
}
The methods getByDocumentReference, findAllDocRefsForModel and findDocumentsByDocRefs would just return empty lists/optional, so my repo wouldn’t provide documents, right?
Moin @raffael-broad-linden,
thanks for pointing out the issue that you had with the tutorial. We are discussing it internally at the moment, but we have a quality gap in the code and the way on how to customize the persistence.
The code in the chapter “Document Manipulation” regarding the customization of the IDocumentRepository doesn’t resemble best-practices on how to use A12 Data Services.
We will remove this step from the written tutorial and rework it to display the best and recommended way to use custom persistence in A12.
Kind regards,
Jan
Hi @jan-static-daemon, thank you for the clarification!
It would be nice if updates regarding this could be posted here, so me and others are provided with guidance to this topic.
Hi @petr-high-peak,
could you please share your thoughts about my proposed solution from the comment Custom document repository duplicate key exception - #11 by rengelbart, to let me know if that approach is valid?
Hi @raffael-broad-linden , I think your solution could work as a hotfix until the new metadata concept is released. So you will handle your specific models by your new repo when mutating data, but you’ll blind the output of your repo so that all documents will be returned just by the default repo.
It’s not a nice hack, but as there is no other solution to this, you could survive on this until the proper solution is available.
Have a nice day.
Petr
Hi @petr-high-peak,
thank you very much for the help with this!
Best regards,
Raffael