We are in the process of upgrading from 2022.02 to 2022.06
We use a VisibilityDataChecker to implement an attribute based authorization through an external service. This is used in the WF task list (as that uses DS since 2022.02) as well as the “regular” document overview (LIST_DOCUMENTS).
However, I was told that the VisibilityDataChecker is deprecated in 2022.06, but I couldn’t find anything in the migation guide except for:
LIST_DOCUMENTS operation no longer checks document read permission for every document.
To re-enable this behavior use configuration key mgmtp.a12.dataservices.jsonRpc.listDocuments.additionalReadScopeCheck.enabled
Does this relate to the usage of the VisibilityDataChecker? Or does this affect the use of the UAA scopes?
If that is really deprecated, then what is the recommended replacement to filter documents through an external service?
We are investigating if the use of CDDs can help us improve the performance of the task list. If we switch to using CDDs, would that change the approach for filtering based on document attributes?
Hi Thomas,
The migration instructions indeed seem to have a gap or are at least not very explicit about this.
We will follow up…
Hi @thomas-soft-grove ,
We have removed visiblity filters completely because it was not compatible with UAA repository access. We have thought about your use-case and therefore we have introduced a key mgmtp.a12.dataservices.jsonRpc.listDocuments.additionalReadScopeCheck.enabled which will enable you to have the same behavior in your application as in 22.02 release. If this key is set to true the documents are loaded using loaders and the DocumentAfterLoadEventis published. You need to register EventListener for this event and contact KROSS to check the permissions. If the user is not allowed to access the document, please throw an exception. The value true must be also set for mgmtp.a12.dataservices.jsonRpc.listDocuments.skipNonLoadableDocuments as it was in 22.02 to skip the documents that user is not allowed to see and continue loading other documents.
This should address your case. I apologize for the inadequate migration notes. I have created a ticket <INTERNAL_LINK> to improve it.
OK, thanks.
We are currently evaluating if using CDDs will help with our performance problems. In case we implement CDDs, will that have any impact on how to use that EventListener? (My understanding is that we would use LIST_CDDS instead of LIST_DOCUMENTS)
Currently we are serving the complete CDD from the index, therefore there is no document loading (for documents in the CDD) issued therefore there are no DocumentAfterLoadEvent fired. So it would not be possible for you to call KROSS for the permission checks of those documents. We could publish a new event when the CDD is being put into the LIST_CDD result set so you can make permission checks, but this will not help you with performance issues. If you have 10 CDDs per page and each CDD consists of 10 documents, you will have to issue 100 document permission checks just for this single page.
Furthermore, I think you will have a problem with the security audit since the content of the CDDs is stored in the Solr index entirely.
The DocumentAfterLoadEvent doesn’t give me the current user, so how do I know for which user this event is fired?
Well, currently we have to load 100 complete documents (including large attachments!) in order to do the permission check. Only loading “documents” consisting of 5 small strings should massively improve performance.
This line String currentUser = SecurityContextHolder.getContext().getAuthentication().getName(); will get you the current user but I have asked @tuan-stable-gale to help you with setting up UAA repository access with KROSS. I think it will be a cleaner solution because all the security related concerns can be in one place and in one A12 product
Attachments can be moved out of documents in 34.1.0 because we have introduced Database Attachment Persister which has extension points for encryption. Furthermore, it is possible to limit the number of documents returned from the query by injecting filters into all queries. This should dramatically reduce the number of requests to KROSS and also the number of rejected documents. The only remaining question is how to set up these filters. Is it possible to get some pre-filtering documents based on user properties?
We need one call per document to KROSS to determine if a document is visible, there is no way avoid that.
Those checks are actually quite fast (typically < 10ms per document). The bottleneck is the fact, that we have to fully load each document to get access to the 5 attributes we need. This takes somewhere between 50ms to 500ms - so substantially slower than the authorization check through KROSS.
Given the nature of the authorization checks in the OZG platform, there is no way to specify “static” filters for the Solr query to limit the number of documents. The only way to get an acceptable performance here, is to only load those attributes that are needed for the check, rather than the full document.
Even without attachments, we are looking at documents that might be several MB in size.
We could extend our search service so that it would return more data from the search index (not just docRef), These properties would then end up in the query resultset. Therefore, there will not be a need to further load documents just for permission checks, you can use properties directly from search results.Currently we do not have this enabled but underlying search index supports this feature.
The broader the filters you will have in your application the more likely it will be that the result set will contain documents that the user is not allowed to see. My suggestion here would be to limit the size of the results sets if possible. There might be an option to have multiple overviews with different use-cases in mind. Clerks in the buro should have different documents displayed then the citizen posting the taxes in app. The documents are from the same model but the properties of the document can be used to limit the results. Other limitations can be made based on mandatory timeframes i.e., always display the results in 3 month batches or other delimiters if they can be used.
Believe me: we have investigated all possible ways to limit the number of documents in the overview, but in the end there is no way to define a static filter. We have to pass the attributes from the document to the backend for evaluation.
Throwing an exception doesn’t seem to work - at least not in the context of the Workflows module.
When I do that, the whole LIST_DOCUMENTS call is terminated and the frontend just shows “an error occurred”.
This is the stacktrace (cleaned up to only show the A12 classes.
2022-08-08 14:23:12.001 WARN 3532 --- [nio-8088-exec-9] c.g.jsonrpc4j.JsonRpcBasicServer : Error in JSON-RPC Service
java.lang.RuntimeException: Access to document 3148 denied
at com.mgmtp.<PROJECT_NAME>.workflows.camunda.authorization.OzgVisibilityChecker.checkAccess(OzgVisibilityChecker.java:103)
...
at com.mgmtp.a12.dataservices.document.persistence.AbstractDocumentLoader.loadDocument(AbstractDocumentLoader.java:50)
at com.mgmtp.a12.dataservices.document.persistence.AbstractDocumentLoader$$FastClassBySpringCGLIB$$fade8c64.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
...
at com.mgmtp.a12.dataservices.document.persistence.internal.DefaultDocumentLoader$$EnhancerBySpringCGLIB$$c65dca17.loadDocument(<generated>)
at com.mgmtp.a12.dataservices.document.operation.ListDocumentsOperation.lambda$secureAccess$0(ListDocumentsOperation.java:123)
at com.mgmtp.a12.dataservices.document.operation.internal.ListDocumentsSupport.secureAccess(ListDocumentsSupport.java:21)
at com.mgmtp.a12.dataservices.document.operation.ListDocumentsOperation.secureAccess(ListDocumentsOperation.java:122)
...
at com.mgmtp.a12.dataservices.document.operation.ListDocumentsOperation.rpc(ListDocumentsOperation.java:100)
at com.mgmtp.a12.dataservices.document.operation.ListDocumentsOperation$$FastClassBySpringCGLIB$$94b0d2d9.invoke(<generated>)
...
at com.mgmtp.a12.dataservices.rpc.internal.JsonRpcOperationDispatcher.handleRequest(JsonRpcOperationDispatcher.java:128)
at com.mgmtp.a12.dataservices.rpc.internal.JsonRpcOperationDispatcher$$FastClassBySpringCGLIB$$da48344b.invoke(<generated>)
...
at com.mgmtp.a12.dataservices.rpc.internal.JsonRpcOperationDispatcher$$EnhancerBySpringCGLIB$$227307e2.handleRequest(<generated>)
at com.mgmtp.a12.workflows.config.DataServicesRequestMapping.forward(DataServicesRequestMapping.kt:31)
...
Our application.properties contains these two properties:
mgmtp.a12.dataservices.jsonRpc.listDocuments.skipNonLoadableDocuments.enabled=true
mgmtp.a12.dataservices.jsonRpc.listDocuments.additionalReadScopeCheck.enabled=true
Did I do something wrong in my EventListener? Or is this maybe a DS or WF bug? Or do I need another property to ignore the exception that is used for the control flow in DS?
Hi @thomas-soft-grove,
The exception you must throw is exactly org.springframework.security.access.AccessDeniedException. No other exceptions are handled and are rethrown. If you use this AccessDeniedException, then it is handled in list documents together with skipNonLoadableDocuments and the document is just filtered out without throwing an exception.
Thanks, that seems to work.
We are happy that this works for you, we are planning some further improvements to security that would allow you to call KROSS from authorizationDefinition file in A12S-2826. We should be able to release it this week.