I was wondering if it would be possible to customise and/or add new plug-ins for exporting and archving data. Our university develops and hosts a platform called CoScine that is also intended to share and collaboarate with data. One use-case I could foresee would be that researchers distributed across multiple institutions can use this to share data (and then the final export can be made to zenodo).
Sharing data within openBIS is possible as well if I understood things correctly - but then the data would have to be downloaded for each further use. The Coscine facilities are linked to an S3 object store that is also accessible from the cluster environment, etc - essentially, the researchers could, if possible, export the data they want to share there and it can then be used by others withouth having to copy data across.
Similarly, would it be possible to add an export option that uses tar instead of zip (or an option to choose)? We’re also trying to find ways to circument the “small file problem” for machine learning, WebDataset (https://webdataset.github.io) seems a promising approach. We could, of course, download the data from openBIS, create the archives, upload this to a different store, register the meta-data in openBis - but I was wondering if this process could be made easier for the user and some steps integrated in openBIS.
I understand, of course, that such a functionality would have to be developed - if we had (our could find) ressources for someone to help, how feasible would it be to do so?
all the best