Performance issue in form with many repeat entries

Engines Form Engine
1

We are experiencing performance issues in a form with a lot of repeat entries.

We have no idea how to drill down on the problem, identify possible - modeling? - culprits and make the form performing better.

Any help is greatly appreciated. Thanks.

Issues
Given a document with around 600 repeat entries, we experience the following wait / response times in our application (already reproducible with the Form Model Preview).

  • New entry: 5 s
  • Detail screen > First selection: 9 s
  • Detail screen > Second selection: 13 s
  • Detail screen > Fill any field: at least 5 s

I attach a screen cast to show this:
performance-issue-repeat-smaller-repeated

Of course this is not the greatest experience for our users. Also, there are plans to allow even more repeat entries in this model (up to 10.000).

Models
The underlying document model basically consists of one central repeat group (/Belegliste/Beleg), which currently can hold up to 1000 entries. We have a couple of fields and quite a number of rules and computations, some of which perform aggregations. We use some enumerations with categories.

In the form model, we make some use of dependencies, often based upon said enumerations.

Questions
How can I find out what consumes the most time?

Is it computations / validations or some form concepts, such as dependencies (in general)?

Can I somehow profile the processes following a user interaction? Can I turn on helpful logging in the kernel / form engine?

Are there any best and worst practices when it comes to modelling with respect to performance?

Could someone please have a look at the attached models and maybe point out critical sections?

Maybe using composed data might ease the situation, though I am not clear on how exactly. However, as we are still on an older A12 version, we don’t think that this is a short-term solution for us.

Reproduction
I attached the models (A12 2023.06.2.0) and a document.

Models:
BeleglisteAusgaben.json (187.8 KB)
DomainBedingungen.json (43.6 KB)
DomainBeleglisteAusgaben.json (378.7 KB)
DomainKalkulationshilfen.json (44.1 KB)
Kalkulationshilfen.json (15.0 KB)

Document:
beleglisteGross.json (995.9 KB)

Steps

  • Start SME and load models
  • Start Form Model Preview for BeleglisteAusgaben
  • Import From File > beleglisteGross.json
  • Navigate to “Neuer Beleg” and open repeat detail screen
  • Fill fields in repeat detail screen
2

Hi @anon67142020,

“How can I find out what consumes the most time?”

I profiled your form&data in our local dev setup (using chrome profiler) and I think you can do the same in yours.

“Is it computations / validations or some form concepts, such as dependencies (in general)?”

It is definitely computations, probably in combination with dependencies. We are currently working on an optimization for 24.06 (A12-15052) - but it will probably only improve the slowest cases (9s/13s) and not the 5s case.

“Are there any best and worst practices when it comes to modelling with respect to performance?”

At the moment, forms in A12 are handled completely on the client-side (also in the CDM / composed data case). This includes all validations, computations and dependency logic. Therefore I think the only thing that you can do is to load less data (=entries) at a time. Unfortunately, we don’t have an out of the box solution for that.

“Maybe using composed data might ease the situation”

This would help but only if you use CDM on the server-side, e.g. to do cross-entry validation / computations. As soon as you open the CDM with the same form model, it will behave like now.

I will also ask the kernel team to have a look at your models, maybe they have more ideas.

3

I’m wondering if this shouldn’t be replaced by a different top level modeling approach. The currently single document is divided into 3 different documents. One “Belegliste” that only contains the overall data. Then there are lots of “Beleg” documents with a relation to “Belegliste”. And last there is a CDM containing all the aggregations over the different “Beleg” documents for one Belegliste. I haven’t looked into the details of the document models to say for sure.

The UI would look very different though because the repeat would need to be replaced by an overview and the Belegliste form would be separated from the CDM if you would go with plain A12.

But I would like to hear opinions from someone from the A12 modeling team if this is the approach intended by A12.

4

This is an interesting idea. So what you are suggesting is:

Belegliste --relation-- Beleg* --relation-- AggregatedBeleglisteCdm

This would actually be a charming design. However, the single “Beleg” needs some overall data from “Belegliste” and I wouldn’t know how to get these data in a relationship. I could copy the data from Belegliste into Beleg but I am afraid of the resulting synchronisation hell. Is there a nice way to achieve this?

Your suggestion also gives me another idea: assuming aggregations are expensive, maybe we could divide by “data flow” and achieve some improvements that way. What I mean is

  • Beleg needs data from Belegliste: this should be inexpensive, as we get single data values
  • AggregatedBeleglisteCdm aggregates over Beleg data: this is expensive, but I don’t need it when adding a new Beleg

We could end up with something like this:

AggregatedBeleglisteCdm --relation-- Belegliste --repeat-- Beleg*

Belegliste-Form is used for adding new Beleg entries. As there are no aggregations there, we possible reduce computation time. AggregatedBeleglisteCdm-Form is only used for displaying (aggregated) data. Does this sound feasible?

5

I had more of your last suggestion in mind than the first. But anyway I would really appreciate some feedback from the A12 modeling team.

6

However, the single “Beleg” needs some overall data from “Belegliste” and I wouldn’t know how to get these data in a relationship. I could copy the data from Belegliste into Beleg but I am afraid of the resulting synchronisation hell. Is there a nice way to achieve this?

If I understand it correctly, you could use a CDM approach only for the Beleg form that also integrates data from the Belegliste document. In this direction it’s a to-1 relationship (I assume) and thus it wouldn’t load a lot of data.

7

From a modeling perspective, the way suggested by @andreas-fresh-mesa / @erik-radiant-gorge sounds quite valid. Our overall suggestion would be quite similar, so modeling the case based on relationships (and a CDM) between these entities to reduce the amount of processed data within the FE and subsequent loading times. But at the end it’s pretty hard to estimate (or even guess) what performance gains you’ll get from these changes - I think you have to try out what works best for you.
I hope you at least have a starting point for possible options for action @anon67142020 - please get back to us in case of further issues or if you were (hopefully) successful to share your solution! :slight_smile:

8

I’ve taken a look at the models and believe that you could remove some of the computations.

I started a new thread, so the way the the computation rules can be modeled may be discussed separatly to the structural / data modeling topics

9

Thanks for all your helpful replies and suggestions! We will try these and report back once we get it working.