Update Vocabulary via Dropbox

Data Management
,
1

My idea was to allow users to expand a very vibrant vocabulary by themselves. When they have new items, they drop a CSV file into a Dropbox, adding the lines to the vocabulary. The system will then remove the CSV file.
Some may view the loss of sovereignty over parts of the master data as problematic - nonetheless, I scripted the following Dropbox procedure:

import csv

def process(transaction):
    voc = transaction.getVocabularyForUpdate("VOCABULARY_CODE")
    new_terms = {}
    with open(transaction.getIncoming().getAbsolutePath()) as f:
        csv_file = csv.reader(f, delimiter=";")
        for row in csv_file:
            new_terms[row[0]] = row[1]
    for key, value in new_terms.items():
        term = transaction.createNewVocabularyTerm()
        term.setCode(key)
        term.setLabel(value)
        voc.addTerm(term)
        transaction.getLogger().info(
            "%s (%s) was added to 'VOCABULARY_CODE'." % (value, key)
        )

The code seems to work, but the process doesn’t end since it waits forever to register data on openBIS. It stops when I delete the file manually, but the data never leaves the pre-commit state and is not registered in the vocabulary.

2023-10-21 14:41:34 Compile python script
2023-10-21 14:41:34 Start processing
2023-10-21 14:41:34 Start registration
2023-10-21 14:41:34 Label (CODE) was added to 'VOCABULARY_CODE'.
2023-10-21 14:41:34 Prepared registration of 0 data sets:

2023-10-21 14:41:34 Preparation ready
2023-10-21 14:41:34 In pre-commit state; no data needed to be moved.
2023-10-21 14:41:34 About to register metadata with AS: registrationId(40837)
2023-10-21 14:41:34 Will try to register data in openbis.
2023-10-21 14:41:34 Error in registrating data in application server
...
2023-10-21 15:01:35 The incoming file was deleted before registration. Nothing was registered in openBIS.
2023-10-21 15:01:35 Responding to error [OPENBIS_REGISTRATION_FAILURE] by performing action LEAVE_UNTOUCHED on .......csv

Does a dataset have to be created every time?
What would be the move to terminate the process but still have the content of the csv file registered in openBIS?
Thanks for any input helping me to understand the problem.
Cheers,
Volker

2

Hi,

are you using the dropbox only to register a vocabulary? Dropboxes are meant for datasets regsitration.
I would recommend to use pyBIS in your case.