Data upload failing in dockerized openBIS 20.10.12 - download-url configuration issue

ELN, LIMS, Data management functionalities
1

Hi openBIS community,

I’m experiencing issues with file uploads in my openBIS instance and would appreciate any guidance.

Setup:

  • openBIS version: 20.10.12

  • Deployment: Docker (using the official openbis/openbis-app:20.10.12 image)

  • Configuration: Using the standard docker-compose.yml and .env files provided by openBIS

  • Host OS: Ubuntu Server (latest version)

  • Network: Container exposed on ports 8080 and 8081

Problem: When attempting to upload files through the web interface, the upload fails without a clear error message (Error message: “Upload failed”). There are no error logs in the openBIS application server or datastore server logs.

Browser Console Error: The browser shows a failed HTTPS request to the datastore server:

Request URL: https://<server-ip>/datastore_server/session_workspace_file_upload?filename=...
Status: Failed to load response data

Additionally, there’s a 404 error:

http://<server-ip>:8080/openbis/resources/api/v3/dss/dto/service/id/CustomDssServiceCode.js
Status Code: 404 Not Found

Troubleshooting Already Done:

  1. Fixed FQDN configuration: Changed OPENBIS_FQDN in openbis-app.env from local.openbis.ch to the actual server IP address.

  2. Verified service.properties entries:

    • host-address = http://<server-ip> (in DSS config)

    • server-url = http://openbis-app:8080 (internal communication between AS and DSS)

    • download-url = http://<server-ip> (originally was https://)

  3. Port accessibility: Port 8081 is accessible from external clients:

   curl http://<server-ip>:8081/datastore_server/
   # Returns: Error: Request URI '/datastore_server' expected to start with '/datastore_server/'
  1. Internal AS↔DSS communication: Both curl tests work inside the container:
   curl http://openbis-app:8080/openbis/  # Works
   curl http://localhost:8080/openbis/    # Works
  1. Port binding verification: Port 8080 is listening on all interfaces:
   tcp6  0  0  :::8080  :::*  LISTEN
  1. Modified startup script: Changed the start-openbis.sh script to use http:// instead of https:// for download-url, but the browser still attempts HTTPS connections.

Current Configuration Files:

docker-compose.yml:

version: "3.7"
services:
  db:
    container_name: openbis-db
    image: postgres:15
    environment:
      - POSTGRES_PASSWORD=
      - PGDATA=/var/lib/postgresql/data
      - POSTGRES_HOST_AUTH_METHOD=trust
    volumes:
      - openbis-db-data:/var/lib/postgresql/data
    networks:
      - openbis-network

  app:
    container_name: openbis-app
    image: openbis/openbis-app:20.10.12
    hostname: openbis-app
    depends_on:
      - db
    env_file:
      - openbis-app.env
    volumes:
      - openbis-app-data:/data
      - openbis-app-etc:/etc/openbis
      - openbis-app-logs:/var/log/openbis
    ports:
      - 8081:8081
      - 8080:8080
    networks:
      - openbis-network

volumes:
  openbis-db-data:
  openbis-app-data:
  openbis-app-etc:
  openbis-app-logs:

networks:
  openbis-network:
```

openbis-app.env:
```
OPENBIS_ADMIN_PASS=<password>
OPENBIS_DB_ADMIN_PASS=<password>
OPENBIS_DB_ADMIN_USER=postgres
OPENBIS_DB_APP_PASS=<password>
OPENBIS_DB_APP_USER=openbis
OPENBIS_DB_HOST=openbis-db
OPENBIS_FQDN=<server-ip>

Questions:

  1. Why is the browser attempting HTTPS connections when download-url is configured to use HTTP?

  2. Is there additional configuration needed in the web application itself to force HTTP for datastore uploads?

  3. Is the 404 error for CustomDssServiceCode.js related to the upload failure, or is it a separate issue?

  4. Should I be using a reverse proxy (nginx/traefik) with SSL termination for production deployments, or is there a way to make HTTP-only uploads work properly?

Any help would be greatly appreciated!

2

Hi Olli,
Seems like your configurations are correct in general.
I think you need a webserver with reverse-proxy facing the client to re-route the traffic to as (port 8080) and dss (port 8081).
My test setup uses an apache2 server deployed directly on the host with following config:
apache2/sites-available/vhosts.conf:

<VirtualHost *:443>

ServerName your.server.name
DocumentRoot “/var/www/html”

SSLEngine on
SSLCipherSuite AES256+EECDH:AES256+EDH
SSLProtocol All -SSLv2 -SSLv3
SSLHonorCipherOrder On
SSLCompression off
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
SSLCertificateChainFile /etc/ssl/certs/ssl-cert-snakeoil.pem

<Directory “/var/www/html”>
AllowOverride All
Options -Indexes +FollowSymLinks
Require all granted

AllowEncodedSlashes on
RedirectMatch ^/$ /openbis/webapp/eln-lims/
RewriteRule ^/openbis$ /openbis/ [R,L]
RewriteRule ^/datastore_server$ /datastore_server/ [R,L]
RewriteRule ^/eln$ /eln/ [R,L]
ProxyPass /openbis/ http://openbis.app.container.ip:8080/openbis/ timeout=600 Keepalive=Off retry=0
ProxyPassReverse /openbis/ http://openbis.app.container.ip:8080/openbis/
ProxyPass /datastore_server/ http://openbis.app.container.ip:8081/datastore_server/ nocanon timeout=600 Keepalive=Off retry=0
ProxyPassReverse /datastore_server/ http://openbis.app.container.ip:8081/datastore_server/

</VirtualHost>

In this way, the traffic from client will route from port 443 to either AS or DSS by the webserver and you don’t have to worry about the internal addresses in the openbis-app container.

Hope this helps a little.

Best,
Filip

3

Hi Filip,

Thank you so much for your help! Your suggestion to use a reverse proxy was exactly what was needed to solve the problem.

Solution Summary:

I implemented an nginx reverse proxy as an additional Docker container (following the ingress pattern from the official documentation). This resolved all upload issues.

What was causing the problem:

The browser was attempting to make HTTPS requests directly to the datastore server on port 8081, but since openBIS was configured with download-url = https://<server-ip> without an actual HTTPS endpoint on that port, the connection failed silently. The internal communication between AS and DSS was also problematic because the server-url was pointing to external IPs instead of internal Docker hostnames.

Final working configuration:

  1. Added nginx ingress container to docker-compose.yml:

    • Exposes only ports 80 (HTTP) and 443 (HTTPS) to the outside

    • Routes /openbis/ → Application Server (port 8080)

    • Routes /datastore_server/ → Datastore Server (port 8081)

    • Handles SSL termination at the proxy level

  2. Removed direct port exposure for ports 8080 and 8081 from the openbis-app container (only exposed internally within the Docker network)

  3. Set download-url = https://<server-ip> in both AS and DSS configurations

  4. Set server-url = http://openbis-app:8080 in DSS configuration (for internal AS↔DSS communication using Docker service names)

Now all client requests go through the reverse proxy on port 443, and the proxy forwards them to the appropriate backend service. File uploads work perfectly!

As a workaround while debugging, I also successfully implemented the drop-box feature for server-side file ingestion, which bypasses the web upload mechanism entirely. This proved to be a useful alternative for batch uploads and automated workflows.

Thank you again for pointing me in the right direction with the reverse proxy suggestion. For anyone else experiencing similar issues: the key insight is that browser-based uploads require proper SSL termination and unified routing through a single HTTPS endpoint.

Best regards,
Olli

1 Like
4

Dear @olli.schermer and @farman ,
thanks for this structured conversation.

That is indeed the correct approach and I am aware openBIS is still not very clear to setup correctly for first time users. The configuration variables and descriptions are still in some places misleading but we are working to clarify more and more and conversation like this is very helpful to be aware where are obstacles.

I will just add few comments here as perhaps this thread might be useful for future.

  • we intentionally build everything with https and we will continue more into this direction with all obstacles about certificates, performance and extra layer. Encryption by default is our goal to improve security overall.
  • there should be never a case to modify start-openbis.sh script, we try to maintain this script very clearly written, obvious and simple. All configuration of openBIS should be finally supported by Docker Environment variables.
  • we will try to maintain very basic examples of configuration needed for deployment in openBIS Continous Delivery repository, we use it to continous deploy Docker containers of openBIS.
  • we are working on to provide also a docker container for reverse proxy and/or as an ingress for openBIS Application to make examples even simpler. It will be published in Docker Hub
    openbis/openbis-ingress - Docker Image
1 Like