A way to migrate existing documents during startup

Peace,

In one of our current example, we changed the given role for new registered user and give this role access to specific documents. Now we already have some registered users, which have an old role, so we want to migrate them. Of course, this is possible with a SQL-Migration, but is there a framework which will migrate existing documents during startup ?

A framework

SQL-Migration have some negative aspects like, missing historization or making deployments more harder, because we have to coordinate the SQL-Migration with the deployment explicitly.

FYI: We are using A12 version 2020.02 (Yeah we have to update, but the version shouldn’t be a problem here :smiley:)

Peace out,
Tjorben

Maybe you could have a look at this: <INTERNAL_LINK> (Should be the same with your 2020.02 version I would expect)
And there is a key: mgm.services.core.migrate-on-startup that by default set to true to run this migration data.
Be aware also if you have your server clustered, make sure to make it cluster-safe.
:slight_smile:

Hey,

fortunately we’re not running clustered (yet), so we don’t have to take care there :slight_smile:

But that brings me to the question, if services is cluster-aware or has some kind of cluster support included? Like leader election, and only the leader will execute migrations and update the index afterwards?

Also, will the spring application report readiness before or after the migrations have been executed? The documentation says it will run once the application is completely bootstrapped. So does this mean the application is ready to use and we’d need to block access until migrations are done? Or is this part of the startup still?

Thanks in advance!

Hello Max,

I think Services has provided a tool call service-cli for the awareness of cluster setup in the customer project. Maybe you could have a look at this link for more detail: <INTERNAL_LINK> Probably it’s not available in your version of A12 but you could get the idea out of that to be ready for the future. :diamond_shape_with_a_dot_inside:
However, you have to do the configuration of “the leader” on your own.

For the second point, maybe you can have a closer look in the services-cores package from Service where you can quickly see the file com.mgmtp.a12.services.startup.ServicesCoreStartupListener#onApplicationEvent describe how it runs during start up phase. When the event ContextRefreshEvent is published, it will try to run the migration first, then publish another event for reindexing of the data and so on until the start up phase is done then your application is ready to use (another event ServicesStartupFinishedEvent is fired at that time).

Cheers :beers:

Not sure if this is what I mean. Having an external tool to interact with services does not seem feasible when you have a setup with replicated services instances for the same application. I think the application itself needs to be aware of clustering, so even in case of an unexpected restart the migrations and re-indexation is not executed again.

On the readiness: I basically meant the Spring Boot actuator endpoint /health/readiness. In case this reports readiness before the migrations were executed and the index is built, traffic might be routed to the container before that leading to unexpected responses or even failures.

Best,
Max