Dear @eikemq,
we do not provide detailed guide of migration because standalone setup is flexible and individual setups are inconsistent. We are trying to standardize that within our work to have openBIS in container.
Strategically a migration is nothing else like backup and restore procedures. That is why we focus to provide as many details how to backup openBIS installation and how to restore. It is generally more universal with room of individual setups.
Recently, I began restoring standalone setups into containerized ones so I briefly share few points.
-
We do not intent to provide more detailed guides from you have already seen.
Backup
Backup — Python documentation
and restore
Restore — Python documentation
fulfill what should be included.
-
I always setup clean and fresh container with the same openBIS release, I verify if the environment is working correctly before I begin restoration of standalone system.
-
For database restoration I drop existing databases…
docker exec -i -u postgres openbis-db dropdb -U postgres multi_dataset_archive_prod;
docker exec -i -u postgres openbis-db dropdb -U postgres openbis_prod;
docker exec -i -u postgres openbis-db dropdb -U postgres pathinfo_prod;
and I create them again
docker exec -i -u postgres openbis-db psql -c "create database multi_dataset_archive_prod with owner \"openbis\" encoding 'UTF8' template=template0 tablespace=pg_default;";
docker exec -i -u postgres openbis-db psql -c "alter database multi_dataset_archive_prod set default_with_oids = off;";
docker exec -i -u postgres openbis-db psql -c "alter database multi_dataset_archive_prod set join_collapse_limit = '32';";
docker exec -i -u postgres openbis-db psql -c "alter database multi_dataset_archive_prod set from_collapse_limit = '32';";
docker exec -i -u postgres openbis-db psql -c "create database openbis_prod with owner \"openbis\" encoding 'UTF8' template=template0 tablespace=pg_default;";
docker exec -i -u postgres openbis-db psql -c "alter database openbis_prod set default_with_oids = off;";
docker exec -i -u postgres openbis-db psql -c "alter database openbis_prod set join_collapse_limit = '32';";
docker exec -i -u postgres openbis-db psql -c "alter database openbis_prod set from_collapse_limit = '32';";
docker exec -i -u postgres openbis-db psql -c "create database pathinfo_prod with owner \"openbis\" encoding 'UTF8' template=template0 tablespace=pg_default;";
docker exec -i -u postgres openbis-db psql -c "alter database pathinfo_prod set default_with_oids = off;";
docker exec -i -u postgres openbis-db psql -c "alter database pathinfo_prod set join_collapse_limit = '32';";
docker exec -i -u postgres openbis-db psql -c "alter database pathinfo_prod set from_collapse_limit = '32';";
I restore the Postgres dump files in following command…
cat multi_dataset_archive_prod-db.dmp | docker exec -i -u postgres openbis-db pg_r
estore -d multi_dataset_archive_prod -U openbis --role=openbis --no-owner --exit-on-error;
cat openbis_prod-db.dmp | docker exec -i -u postgres openbis-db pg_restore -d open
bis_prod -U openbis --role=openbis --no-owner --exit-on-error;
cat pathinfo_prod-db.dmp | docker exec -i -u postgres openbis-db pg_restore -d pat
hinfo_prod -U openbis --role=openbis --no-owner --exit-on-error;
The openbis-app container should not be running during database restoration.
- File based user’s password I restore from passwd.backup file directly to the docker volume as openbis-app container should still be not running during this execution.
cat passwd.backup | sort >> /volumes/openbis-app-etc/_data/as/passwd;
- The structure of volume openbis-app-data is precreated. There is many directories indented to be permanently stored and generally it is “parent directory of the store directory”. We have approach to store there not only data files and content files
storeroot-dir = /data/openbis/store
file-server.repository-path = /data/openbis/raw-store
but all cashing, commands status files, task control files, session tokens, etc. Everything what expected to survive container reloads and restarts.
api-workspace
cache-workspace
commandqueue
doc
dropboxes
eln-lims-dropbox
eln-lims-dropbox-marker
examples
incoming-default
incoming-simple
log
post-registration
raw-store
samples
session-workspace
store
transactions
The content of “storeroot-dir” and “file-server.repository-path” have to be always restored, the rest is depending of configuration.
-
The most complex part is openBIS configuration which we ourselves not standardize yet either. The initial openBIS container is prepared to have minimal functional configuration items preset and the rest I keep as it is default in the openBIS installer. There is no other way to migrate complete configuration from standalone setup to container. My approach is to review each part of configuration and apply to new system as complete fresh configuration in docker compose environment file. It is to review each file in “openbis-app-etc” and compare with standalone system.
-
The difficult part is “core-plugins” as the design of openBIS did not separate functional code from configuration and we are working to improve that and simplify. However results are not easy as each plugin should be treated separately, and configured from beginning. Currently I use configuration management tool (Puppet and Ansible) to set and keep desired setting in each “.properties”, “.js” or “.json” file. Unfortunately it is far from simple and attitude should be decide by each openBIS operator alone.