Inconsistent paging even with sorting defined

Data Services Query API
1

In the a12 docs there is this hint about problems with paging when no sorting is defined: QUERY JSON-RPC → Request → sort

It is strongly recommended to include a sorting specification in the query. Paging is mandatory, and paging without sorting may lead to inconsistent results.

But we now came across a problem where even with sorting (on a date field with many documents sharing the same date), the pagination result is inconsisten: when trying to fetch all document entries by going through the pages some documents appear double, others are missing.

Is this a known problem? I didn’t find any mention about this in the docs or in discourse.

We already found out that we can circumvent this by adding an additional sort for the /__meta/docRef field. Did somebody else maybe find a better solution? Or a nice way to automatically add this sorting config to all query calls automatically?

2

For anyone interested, we solved this with aspects to inject an additional filter into the query:

@Aspect
@Component
class QuerySortTiebreakerAspect {

    @Around("execution(* com.mgmtp.a12.dataservices.query.QueryService.query(..))")
    fun appendDocRefSort(pjp: ProceedingJoinPoint): Any? {
        val queryRoot = pjp.args[0] as? QueryRoot ?: return pjp.proceed()
        val current = queryRoot.sort.orEmpty()
        val modified = if (current.any { it.field == DOC_REF_FIELD }) {
            queryRoot
        } else {
            queryRoot.toBuilder()
                .sort(current + Order(DOC_REF_FIELD, Order.Direction.ASC))
                .build()
        }
        return pjp.proceed(arrayOf(modified, pjp.args[1]))
    }

    companion object {
        private const val DOC_REF_FIELD = "/__meta/docRef"
    }
}