Performance issues and Computations

Modeling
,
1

Continuing the discussion from Performance issue in form with many repeat entries:

I’ve started a separate thread as the original discussion was moving towards a discussion of the data graph.

As @baschir-loud-bluff mentioned in the original thread, Computation Rules can lead to performance issues as they need to be checked on loading and recalculated with every value change.

There are a number of Computation Rules modeled in the Groups “Summen” and “SummenZN” which calculate as helper fields. For example, Gesamt_ESF, Gesamt_EFRE, Gesamt_LM and Gesamt_EMFAF are calculate for each line of the repeat “Summen” so that the result of one of these fields can be shown in the Field “Gesamt”.

The calculations carried out in these Computation Rules are, with a few exceptions very similar. As a result, you may be able to remove a number of these Computation Rules.

I tried to re-model the Computation Rules “Computation_Gesamt” so that the rules and fields “Gesmat_ESF”, “Gesamt_EFRE”, “Gesamt_LM”, “Gesamt_EMFAF” and “Strukturfond” could be removed from “Summen”.

Common Precondition

GroupFilled(RuleGroup)
And
AllFieldsFilled(Kostenart, /Bedingungen/Strukturfond)
and
AtLeastOneFieldFilled(../Beleg*/Betraege/GezahlterBetrag)

Precondition 1 for the “normal situation”. If you need to explicitly state the accepted values of the Kostenart, you can easily change the operator here.

([/Bedingungen/Strukturfond] == "ESF"
    And NoFieldValueIncludedInValueList(Kostenart In "5668", "5788"))
Or ([/Bedingungen/Strukturfond] == "EFRE"
    And NoFieldValueIncludedInValueList(Kostenart In "7017", "4523"))
Or ([/Bedingungen/Strukturfond] == "LM"
    And NoFieldValueIncludedInValueList(Kostenart In "7023"))
Or ([/Bedingungen/Strukturfond] == "EMFAF")

Calculation 1

Sum(../Beleg*/Betraege/GezahlterBetrag Having [../Beleg/Kostenart] == [$Kostenart] )

This uses the $-Operator to iterate through the Konstenart selected for each line in “Summe”. You can find more information in the documentation or in the Kernel Language Tutorial (Step 3).

The exceptions to this calculation can then be added individually. I only noticed 5 exceptions, for example:
Precondition 2

[/Bedingungen/Strukturfond] == "ESF"
And [Kostenart] == "5668"
And [/Bedingungen/RestkostenpauschaleNichtRelevant] == False

Calculation 2

RoundDown({Sum(/Belegliste/Beleg*/Betraege/GezahlterBetrag Having [/Belegliste/Beleg/Kostenart->RTKP_PAUSCHALE]
        == "JA") * [/Bedingungen/Restkostenpauschale] / 100}, 2)

As this Computation Rule has 6 entries in the Computation Table, I also feel that it is easier to understand than the current situation.

I believe this approach can be applied to most of the calculations in “Summe” and “SummeNZ” so that you reduce the number of computed fields by about 20 fields per line in the repetition.

It’s very difficult to reverse engineer all these computations in your document model for you, but I strongly believe that you can reduce the number of helper fields. If you can go through them and move calculations from helper fields to a single field for the end result, this should lead to better performance.

2

@malcolm-silver-ice Thank you for taking the time to delve into our model and your suggestions.

Regarding the $-operator: from what A12-Version on is this operator available? (we are lagging behind and are still on 2021.06)

3

Hi @anon67142020,

The $-operator was introduced with Kernel version 27.2.0 and as a result you can use this feature with the 2022.06-ext4 installer or later.

I’ve had the time to do some further investigations and based on the data that you provided, the above changes make very little difference. There’s only a single line in the repeat “Summe” where these changes are proposed.

Within the group “Beleg”, there are some smaller changes you can make

  • Remove “DifferenzZumStichtag” and add this calculation to “Computation_RelevantZN”
  • Change data type of RelevantZN to Boolean (previously a restricted number field)
  • Adapt the calculation that counts the RelevantZN fields that are true

The biggest change that you can make is relating to your dependencies. For example, the field “Eigereicht” is used as the trigger for 16 Dependent Fields. By restructuring the Document Model, you can reduce this to a 1 Dependent Group and 3 Dependent Fields. This change meant that the sample data needed to be restructured but thanks to the helper fields that we removed, the file didn’t get any bigger.

I did the performance analysis recommended by @baschir-loud-bluff and got the following results.
Please note:

  1. You need a web based Simple Model Editor to run these tests as developer tools are not accessible in locally installed Simple Model Editors.
  2. These results are based on a single test
Action Original Post Original Models Updated Models % Improvement
New entry 5 s 1.2 s 1 s 17 %
Detail screen > First selection 9 s 1.7 s 1.5 s 12 %
Detail screen > Second selection 13 s 2.4 s 0.9 s 63 %
Detail screen > Fill any field 5 s 0.8 s 0.8 s 0 %
Sum of Times 32 s 6.1 s 4.2 s 31 %

I’ve attached the models so you can look at this yourself.

Regarding these results, please remember:

  • The relative durations between your tests and my tests are similar. I conclude that the changes to the model will have a positive effect regardless of the version you are using.
  • There have been significant performance improvements in the 2023.06 release line and there’s more coming.
  • To achieve these results I made large changes to your models which may not completely reflect the original state. These models represent a suggestion about how you can improve performance through modeling

Recommendations

  1. Reduce the number of helper fields where possible → you will save less data and recalculate less fields
  2. Reduce the number of dependencies. Organize fields which the same dependency logic in groups so that you can use dependent groups.
  3. Reduce the number of redundant elements in the models. Analyse the dependent controls that you need so that you can group these sensibly too (see Vorabfrage section)

Models (2023.06-ext5 Version as I needed to migrate them to use a web-based Simple Model Editor)
BeleglisteAusgaben_Update.json (169.5 KB)
beleglisteGross_update.json (934.2 KB)
DomainBedingungen_Update.json (42.5 KB)
DomainBeleglisteAusgaben_Update.json (221.4 KB)
DomainKalkulationshilfen_Update.json (42.8 KB)

4

@malcolm-silver-ice thanks again for the additional analysis.

Just a quick check regarding

the above changes make very little difference

Does it mean, that in your adjustments, combining dependencies / grouping dependent fields had a larger effect than eliminating sum-fields and combining computations?

5

The data that you provided only had a single sum row in it. The models you provided allowed for up to 100 sum rows. As the computations would be carried out for each individual row, this could lead to poor performance. Only a single row was present, so you could say that this was a hidden problem which would show itself as more sum rows were added.

So, yes, in the case of your single test data set, I saw more improvements in the performance when I changed the way the dependencies worked.

That being said, the recommendations above are still true and both dependencies and calculations can lead to reduced performance. If you wanted to look more closely at the dependencies, dependencies that set or clear a value are more “expensive” as they can lead to recalculation of other fields, causing a double negative affect.

6

Thanks for the clarification.
I’d like to mark one of your answers as “solution”, but I cannot access the available action (probably because you created the topic).