Interact with openBIS via python from scripts

Dear @MichalO

I was wondering if you were able to find out more in the meantime why there is only one linked dataset shown in the ELN UI?

many thanks again
Ulrich

Dear Ulrich,

I have pinged the development team, drawing the attention to the issue again as a likely bug report. I think they have a good description of the problem done by you in this thread. Hope someone from it will contact you directly, if they need explanations.

all the best,
Michal

1 Like

I wanted to bring some light regarding the unusual behaviour you experienced when using Link datasets.

When there are several link datasets belonging to the same repositoryId, only the latest one is shown, because it contains the most up to date information of the repository.

The same behaviour can be obtained when there are several datasets sharing the same historyId.

This is a feature used to group datasets belonging to the same history of the same data.

I hope things make more sense now.

Dear @juanf

many thanks - although I have to admit that I’m still slightly confused…
I had, perhaps naively, assummed that the main difference between datasets stored in openBIS and linked datasets is that they are, well, linked, ie not present in the instance - but otherwise would behave the same.

If I have a number of linked and “normal” datasets in a collection in an invetory, the UI looks like this:
(as above)

so here only the latest linked dataset shows up, however, if I query that space in pyBIS, I get them all:

my_datasets = oBis.get_datasets(sample='/MATERIALS/EBSDSIM/EBSDSIMMASTER22')
print(my_datasets)
   permId                 type        experiment                          sample                              registrationDate     modificationDate     location                                                             status     presentInArchive    size
--  ---------------------  ----------  ----------------------------------  ----------------------------------  -------------------  -------------------  -------------------------------------------------------------------  ---------  ------------------  ------
 0  20230509161844189-258  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-05-09 16:18:44  2023-05-09 16:18:44
 1  20230322100240630-160  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-03-22 10:02:41  2023-03-22 10:02:41
 2  20230509162755872-261  OTHER_DATA  /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-05-09 16:27:56  2023-05-09 16:27:56
 3  20230322094931096-158  OTHER_DATA  /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-05-09 16:44:47  2023-05-09 16:44:47
 4  20230322104042558-161  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-03-22 10:40:43  2023-03-22 10:40:43
 5  20230509165417032-263  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-05-09 16:54:18  2023-05-09 16:54:18  683D22F7-39C0-4E5C-B664-70E221D4CD86/68/cc/30/20230509165417032-263  AVAILABLE  False               450301
 6  20230324104947554-162  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-03-24 10:49:48  2023-03-24 10:49:48
 7  20230509165558983-264  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-05-09 16:55:59  2023-05-09 16:55:59  683D22F7-39C0-4E5C-B664-70E221D4CD86/36/7d/4a/20230509165558983-264  AVAILABLE  False               450301
 8  20230324131502079-170  RAW_DATA    /MATERIALS/EBSDSIM/EBSD_SIM_MASTER  /MATERIALS/EBSDSIM/EBSDSIMMASTER22  2023-03-24 13:15:02  2023-03-24 13:15:02

However, if I put them in a Lab-space, it looks slightly differently:
The upper part shows all datasets, whether they are linked or not, whereas the lower part only shows the most recent linked dataset, but all others.

and if I query this from pyBIS, I also get all of them:

my_datasets = oBis.get_datasets(sample='/ULRICHKERZEL/TEST_0001/UKSAMPLE_001')
print(my_datasets)

   permId                 type      experiment                               sample                                registrationDate     modificationDate     location                                                             status     presentInArchive    size
--  ---------------------  --------  ---------------------------------------  ------------------------------------  -------------------  -------------------  -------------------------------------------------------------------  ---------  ------------------  ------
 0  20230324135639443-179  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-03-24 13:56:39  2023-03-24 13:56:39
 1  20230324135841606-180  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-03-24 13:58:42  2023-03-24 13:58:42
 2  20230324141521987-184  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-03-24 14:15:22  2023-03-24 14:15:22
 3  20230324142532210-187  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-03-24 14:25:32  2023-03-24 14:25:32
 4  20230214163941939-82   RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-02-14 16:39:42  2023-02-14 16:39:42  683D22F7-39C0-4E5C-B664-70E221D4CD86/25/d1/17/20230214163941939-82   AVAILABLE  False               450301
 5  20230630075613506-275  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-06-30 07:56:14  2023-06-30 07:56:14  683D22F7-39C0-4E5C-B664-70E221D4CD86/56/89/a9/20230630075613506-275  AVAILABLE  False               204800
 6  20230324132741144-174  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-03-24 13:27:41  2023-03-24 13:27:41
 7  20230630075641824-276  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-06-30 07:56:42  2023-06-30 07:56:42  683D22F7-39C0-4E5C-B664-70E221D4CD86/7b/fa/a3/20230630075641824-276  AVAILABLE  False               204800
 8  20230630075705734-277  RAW_DATA  /ULRICHKERZEL/TEST_0001/TEST_0001_EXP_1  /ULRICHKERZEL/TEST_0001/UKSAMPLE_001  2023-06-30 07:57:06  2023-06-30 07:57:06  683D22F7-39C0-4E5C-B664-70E221D4CD86/7d/5f/e1/20230630075705734-277  AVAILABLE  False               614400

Many thanks
Ulrich

What I described is ELN-LIMS only feature, APIs will always return everything.

Why?

  • ELN-LIMS is a particular application that targets particular use cases of how link datasets and datasets with history id are used.
  • APIs are for programmers so they can do any application they want. openBIS is a platform.

Why you where able to see all datasets in other place of the UI? We can consider this a bug on the ELN-LIMS.