For our Demo content upload I need to ensure that a certain ID of a document is retained - so that in consecutive runs it gets updated and not created again (and of course to retain soft links to from other documents).
Is there a way to achieve this?
The API I found in documentation <INTERNAL_LINK> which you can reference to your ID to update it.
Currently, it is impossible to create a document with ID set. IDs for documents come from the database sequence. It is not possible to achieve this even with import documents feature because database ID is metadata information which is kept outside of document itself.
We had some discussions about a new version of docs endpoint which would enable batch operations but we have not discussed this option. Personally, I would like to avoid it because it would create a problem with the database sequences. You could specify a document id that is a next_val for the sequence and the next insert would fail because of the primary key (document id) already exists. There are ways how to solve this but so far we were not told this is an issue. I have faced a similar problem when I was preparing a load test environment. I have created a database dump from existing DB and set next_val for sequences to the max document_id.
@marcus-stellar-pool could it be a way to create document using LiquidBase approach to have new document during startup (before the indexing happens)? so that we can use it for further process?
The services version with data migration will be released this Wednesday only. In the migration step, you could do whatever you want but the broken sequences would have to be taken care off. And I think that <PROJECT_NAME> have their own migration in mind that we currently do not support
It is an extremely interesting question what we really need.
First of all we have one point where we have direct references to documents via it’s ID as a soft reference. If the content is inserted into the system we need to get sure that it stored as id 42.
On the other hand we have a need to add documents to the system and if they are already there we may need to update them.
Perhaps asking for documents with a certain id is plain wrong.
- If you reference documents use only project specific identifiers
- If you want to know if a document is there better search for the project specific id rather than the document id of the platform server
The point i really like sbout our current solution is that the content loader is not part of the platform server but a client which can simply be restartet to maintin the documetns in the platform server. Feels much cleaner than an integrated solution.
This is not the first time that I heard problems like this occurred in client projects. It all boils down to the identity problems of our solution. Currently we are exposing database ids which is not the way to go. Especially if we want to enable different store options for subsets of documents. We are discussing now different ways how we could refer to the documents and models in A12. We would like to move away from database ids or simple ids and move towards references which would identify entities in the store regardless of the implementation. Therefore there will be no database ids anymore and it would be able for client project to implement their own persistence driver which would handle save (insert, update) operations as the client projects would require it.
hmm, the idea to hide the persistenc ID is a good thing. But your proposal to implement an project specific persistence driver sounds to me to throw out the baby with the bath water.
It is nice to have the platform server as an serchable high performace ready to use persistence backend - in structure quite similar to a mongoDB or so - but in A12 (description may include wishes)
Adding the need to add a custom persistence driver to this would directly destroy that and may make your life a tad harder since you have to provide a consistent set of features and quality levels independend of the persistence backend.
Perhaps relying completely on custom IDs in the documents would be a simpler solution. They most problaby are not the real IDs (which are hidden). But thea are for example the ‘natural’ target of internal references …
If given in a document they will be respected as ID and a respecitve create or update would be preformed.
What do you think?
I would really be sorry if the install and run approach of the platform server would be abandoned.
I have made too broad statements 
We are do not plan to abandon current approach of services. We will still provide all the functionality as we are having now. We would however have different inner structures which would be extensible for client projects to use. I.e.: Default implementation of this persistence driver would still persist documents in the database in the single table as it is done now.
But since there would be possible to have different persistence drivers implemented, we cannot rely on the database ids as we do now. Or you can have your persistence driver which would store documents of certain model in the dedicated table. We need this right now because currently we have special documents (user,roles, access rights) in the same table which is causing us a lot of troubles. Furthermore, we are working on concepts which would result in different kinds special documents. And from our current experience we found out that we need to have more customizable solution which also we could use. Because of that reasons, we need to come up with the document identity and references that would work with this concept as well as with the current implementation.
Yep.
And hence I want to stress that there might be a huge difference in perceived ID and internal ID. if the need to have certain IDs for documents would need to tinker with the persistence structure - and even forcing me to customize well working stuff I would think this is a pitty.
An ‘external ID’ which is independent of all this would solve this or a statement ‘there is no such thing as an ID’ would solve this as well. Both concept would have consequences for soft or A12 internal references (when they come)