Document Migration

I wonder what is the best way to keep our persisted documents up-to-date when document models have been changed. This surely is a common requirement, right?

There is in fact a dedicated section in the documentation here GetA12 (Development→Components→Data Services→Data Migration Support), but it is both assertive and vague at the same time.

It starts by saying “Data Services (DS) provides predefined migration steps for common use cases.”
So yay, that’s great! Common use cases surely are “renaming a field”, “moving a field to a group”, “creating a field” and so on. But when reading further this is clearly not the case. I still have no idea what “common use cases” are referred to.

The subsection “Custom migration” refers to an example (which is a bit hard to locate, because it only mentions an artifact that contains the actual example code without providing any link).

Furthermore, that example only shows the boiler plate needed to set up a migration. The actual “migration” here is only a log statement :frowning:

There have been similar question here on discourse before, but the best answer I have found is this https://discourse.geta12.com/t/recommended-approach-for-data-migration/1531/2?u=sfriedrich and it refers to example code that is only available internally on an old git branch. It also still uses old “Document” class and XML document structures (that are now JSON) and needs to rebuild the search index for all document of a specific model.

It seems to me the main problems with the migration approach are

  • the migration steps are only started after a12 is done initializing, so on the one hand DS Java APIs could be used (like DocumentService), but on the other hand the actual persisted document structure are incompatible with the current models, so most DS Java API can actually not be used
  • after a document has been migrated the search index has to be updated accordingly

An accessible full example for real common use cases (like creating or renaming a field) would be great.

Also, I wonder if it is feasible to create a migration module that

  • kicks in before A12 is fully initialized
  • offers actual convenience function for common use cases (like move/rename a field with this path to that path)
  • works on JSON
  • directly accesses the DB
  • selectively updates the search index
  • … so when a12 ds is initialized everything is already migrated and ready to be used

Opinions?

Hi @stephen-warm-graph ,

DS currently provides a framework for writing migrations; however, it does not include functionality for executing document migrations. Our recommended approach to using the migration framework is outlined in the referenced Discourse post.

I have raised ticket A12-16640 in anticipation that customer projects would require more comprehensive migration support. Contrary to expectations, such demand has not materialized. Please utilize this ticket to document the migration scenarios you would like to see addressed. We can then use your project requirements to further refine and prioritize the development of this feature.

Hello @stephen-warm-graph, I am a member of the Discourse team. I can see that there was a new response to your question on Document Migration. Do you find it helpful, or should the question remain open?

Thanks ahead for your feedback and have a nice rest of the day!

Moin @stephen-warm-graph,

in the Project Template we have a small example on how the Data Migration API from Data Services can be used to remove a specific field in a Document Model.
You can find the example in server/init/src/main/java/com/mgmtp/a12/template/server/init/migration.