Entity limit in a repeatable group

Question on behalf of @stefan-sparse-array:

After loading about 18k entities into a single document, the whole database connection seems to be crashing. (I actually do not even want to display the repeated group, just some meta data that has a single repeatability.) So I have the following questions:

  1. Is there any limit of entities in a repeatable group?

  2. Do you have any experience with a large number of entities in a running application? What is the best way to handle such large amounts of data? Would using relationships help or will they not solve the problem of huge data instances?

Answer on behalf of @erik-radiant-gorge:

  1. The modelling limit for entities in a repeatable group is 999999.

  2. Using relationships will most likely help since you will no longer have all 6k (or however many) entities in a single document, that will then be passed to the validation kernel for computations/validations.
    However, it also means that you cannot enforce (complex) consistency rules crossing this relationship using the A12 validation capabilities. If that is what you need, you would have to implement it with a custom solution.

Answer on behalf of @ansgar-solar-lake:

On top of the modelling limit, please keep the end to end handling of documents in mind: Documents are treated as storage and transport unit. Therefore, if you have a very large number of entries in a repetable group, then they are part of that unit and stored atomically with the document and sent around as such.