Left-hand main menu stuck loading ("permanent loading") for some users — multi-group setup

Hi all,

For about two days now, the left-hand navigation menu in the ELN-LIMS UI fails to build for some of our users, it just sits in a permanent loading state. Login itself works fine; the problem only appears after the user is logged in, when the left-side menu (Lab Notebook, Inventory, Utilities, …) should load. For the affected users, currently nothing in that menu comes up, not even Utilities.

The affected users seem to be distributed more or less at random, and we can’t find an obvious commonality between them. We run a multi-user-group (server) setup on version 20.10.11.1.

It appeared quite suddenly and I can’t correlate it with any specific change, though it’s possible that someone modified user or group settings. My initial assumption was that the menu is assembled from the user/group settings (i.e. the visible spaces and objects depend on how those are configured), so I first suspected a settings change, but our log analysis now points elsewhere (see below).

What we have found so far:

Digging into the AS log (bis.log), the stuck menu correlates with repeated failures of getSamples calls that are passed malformed sample identifiers. Example log lines:

ACCESS.ApplicationServerApiLogger - [USER:'…' SPACE:'…']: get-samples  SAMPLE_IDS('[/16/97/5, /82/57/7]')ERROR OPERATION.ApplicationServerApi - exception while processing 'IApplicationServerApi.getSamples'Caused by: java.lang.IllegalArgumentException: Sample identifier can not contain more than three '/': /86/71/47/8671476at …helper.sample.FullSampleIdentifier.extractParts(FullSampleIdentifier.java:140)at …helper.sample.FullSampleIdentifier.(FullSampleIdentifier.java:47)at …helper.sample.ListSampleTechIdByIdentifier.groupIdentifiers(ListSampleTechIdByIdentifier.java:89)at …asapi.v3.ApplicationServerApi.getSamples(ApplicationServerApi.java:972)

We traced where these identifiers come from: they are substrings of embedded image URLs in the rich-text $DOCUMENT / description property of ELN objects. The ELN file-service stores images under a path that is sharded by the leading characters of the file UUID, e.g.:

<img src="/openbis/openbis/file-service/eln-lims/16/97/5b/16975b84-68dc-…/….png">
<img src="/openbis/openbis/file-service/eln-lims/7b/82/57/7b825750-2082-…/….png">

Something on the rendering path takes the sharded prefix (/16/97/5b/…) and feeds /16/97/5 into getSamples as if it were a sample identifier. Prefixes that are short or alphanumeric simply resolve to “not found”, but when the prefix is purely numeric and longer it produces a 4-segment identifier (e.g. /47/75/71/477571, /86/71/47/8671476), which throws the IllegalArgumentException shown above. That exception appears to abort the batch the UI uses to build the left navigation/preview, so the menu never finishes loading. It is reproducible for users who open image-heavy lab-notebook entries, and we see the same failing call both from interactive sessions and via Personal Access Tokens (i.e. from a routine that parses document content).

A few questions for the community:

  1. Is this getSamples-on-image-path behaviour present in a stock 20.10.11.X ELN-LIMS, or does it indicate a custom plugin / customization on our side?
  2. Which component resolves object references inside rich-text $DOCUMENT content? We’d like to fix the parser so it excludes file-service / image URLs and only resolves real object links.
  3. More generally, is there a recommended way to compare the effective settings (and resolved spaces/objects) of an affected vs. a working user, in case settings still play a role?

Thanks in advance for any pointers!