Build, configure, and deploy¶

These instructions assume the reader is familiar with the process of deploying a KBase module, including the runtime and dev_container, and has access to a system with the KBase runtime installed. These instructions are based on the kbase-image-v26 runtime image.

Unlike many modules the WSS can be built and tested outside the dev_container, but the dev_container is required to build and test the scripts. These instructions are for deploying the server and so do not address the scripts. Building outside the dev_container means the Makefile uses several default values for deployment - if you wish to use other values deploy from the dev_container as usual.

Build the workspace service¶

First checkout the dev_container:

/kb$ sudo git clone https://github.com/kbase/dev_container
Cloning into 'dev_container'...
remote: Counting objects: 1097, done.
remote: Total 1097 (delta 0), reused 0 (delta 0), pack-reused 1097
Receiving objects: 100% (1097/1097), 138.81 KiB, done.
Resolving deltas: 100% (661/661), done.

Note

In the v26 image, /kb is owned by root. As an alternative to repetitive sudo s, chown /kb to the user.

Bootstrap and source the user environment file, which sets up Java and Perl paths which the WSS build needs:

/kb$ cd dev_container/
/kb/dev_container$ sudo ./bootstrap /kb/runtime/
/kb/dev_container$ source user-env.sh

Now the WSS may be built. If building inside the dev_container all the dependencies from the DEPENDENCIES file are required, but to build outside the dev_container, only the jars and workspace_deluxe repos are necessary:

~$ mkdir kb
~$ cd kb
~/kb$ git clone https://github.com/kbase/workspace_deluxe
Cloning into 'workspace_deluxe'...
remote: Counting objects: 21961, done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 21961 (delta 20), reused 0 (delta 0), pack-reused 21921
Receiving objects: 100% (21961/21961), 21.42 MiB | 16.27 MiB/s, done.
Resolving deltas: 100% (13979/13979), done.

~/kb$ git clone https://github.com/kbase/jars
Cloning into 'jars'...
remote: Counting objects: 1466, done.
remote: Total 1466 (delta 0), reused 0 (delta 0), pack-reused 1466
Receiving objects: 100% (1466/1466), 59.43 MiB | 21.49 MiB/s, done.
Resolving deltas: 100% (626/626), done.

~/kb$ cd workspace_deluxe/
~/kb/workspace_deluxe$ make
*snip*

make will build:

A workspace client jar in /dist/client
A workspace server jar in /dist
This documentation in /docs

Note

If the build fails due to a sphinx error, sphinx may require an upgrade to >= 1.3:

$ sudo pip install sphinx --upgrade

Service dependencies¶

The WSS requires MongoDB 3.6+ to run. The WSS may optionally use:

Any AWS S3 compatible storage system as a file storage backend.
Shock for linking WSS objects to Shock nodes.
The Handle Service to mediate linking workspace objects to Shock nodes (see Shock integration with the workspace service).
The Sample Service version 0.1.1 and above to allow linking workspace objects to samples (see Sample service integration with the workspace service).
Apache Kafka as a notification service.

The WSS is tested against:

The auth2 branch of the KBase fork of Shock version 0.9.6 (e9f0e1618e265042bf5cb96429995b5e6ec0a06a)
MongoDB version 3.6.23 with and without WiredTiger
Minio version 2019-05-23T00-29-34Z
Handle service commit aae2f70120e75d2ccccab1b1c01dbb9e8327eee8 with log.py commit b549c557e3c519e0a55eadf7863a93db25cd6806
Sample service commit b549c557e3c519e0a55eadf7863a93db25cd6806

Please see the respective service documentation to set up and run the services required.

Note

The WSS is only compatible with versions of Shock that have been patched to work with KBase authentication. As of this writing, that is only the version of Shock linked above.

Note

The alternative to S3 as a file storage backend is MongoDB GridFS. GridFS is simpler to set up, but locks the entire database when writing files. Since the workspace can consume very large files, this can cause a significant impact on other database operations.

Configuration¶

The workspace configuration is contained in the deploy.cfg file in the repository root (see Configuration parameters). Copy the provided deploy.cfg.example file to deploy.cfg to create the file.

Note

See Configure an event listener for configuration parameters for event listeners.

Warning

deploy.cfg contains several sets of credentials, and thus should be protected like any other file containing unencryted passwords or tokens. It is especially important to protect the credentials that the WSS uses to talk to S3 (backend-token) as they can be used to delete or corrupt the workspace data. At minimum, only the user that runs the WSS (which should not be root) should have read access to deploy.cfg. Also be aware that the deploy.cfg contents are copied to, by default, /kb/deployment/deployment.cfg when the workspace is deployed from the dev_container.

Configuration parameters¶

mongodb-host¶

Required: Yes

Description: Host and port of the MongoDB server, eg. localhost:27017

mongodb-database¶

Required: Yes

Description: Name of the workspace MongoDB database

mongodb-type-database¶

Required: If not delegating type operations

Description: Name of the workspace MongoDB types database. This database name must not be the same as mongodb-database.

Warning

Once any data has been saved by the workspace, changing the type database will result in unspecified behavior, including data corruption.

type-delegation-target¶

Required: If delegating type operations

Description: URL of the workspace service to which type operations should be delegated. If this parameter is set mongodb-type-database is ignored.

Warning

Read Workspace scaling carefully before delegating types.

Warning

Once any data has been saved by the workspace, changing the type database will result in unspecified behavior, including data corruption.

mongodb-retrywrites¶

Required: No

Description: Setting for the MongoDB retryWrites connection parameter. true is true, anything else is false. Defaults to false.

mongodb-user¶

Required: If the MongoDB instance requires authorization

Description: Username for an account with readWrite access to the MongoDB database

mongodb-pwd¶

Required: If the MongoDB instance requires authorization

Description: Password for an account with readWrite access to the MongoDB database

auth2-service-url¶

Required: Yes

Description: URL of the KBase authentication service MKII

auth2-ws-admin-read-only-roles¶

Required: No

Description: KBase authentication server custom roles that designate that the user possessing the role has authority to run administration methods requiring only read access. If a role is entered in this field, workspace administrator management is delegated to the KBase authentication server, and administrators specified in the configuration or added to the workspace database are ignored. Multiple roles may be specified as a comma separated list.

auth2-ws-admin-full-roles¶

Required: No

Description: KBase authentication server custom roles that designate that the user possessing the role has authority to run all administration methods. If a role is entered in this field, workspace administrator management is delegated to the KBase authentication server, and administrators specified in the configuration or added to the workspace database are ignored. Multiple roles may be specified as a comma separated list.

ignore-handle-service¶

Required: If not using handles

Description: Set to anything (true is good) to not use handles. In this case attempting to save an object with a handle will fail. Delete or leave blank to use handles (the default).

handle-service-url¶

Required: If using handles

Description: The URL of the Handle Service

handle-service-token¶

Required: If using handles

Description: Credentials for the account approved to assign/modify shock node ACLs.

ws-admin¶

Required: No

Description: the user name for a workspace administrator. This name, unlike names added via the administer API call, is not permanently stored in the database and thus the administrator will change if this name is changed and the server restarted. This administrator cannot be removed by the administer API call. If either auth2-ws-admin-read-only-roles or auth2-ws-admin-full-roles contain text, this parameter is ignored and workspace administrator management is delegated to the KBase authentication server.

backend-type¶

Required: Yes

Description: Determines which backend will be used to store the workspace object data. Either GridFS or S3. Note all data other than the object data is stored in MongoDB.

Warning

Once any data has been saved by the workspace, changing the backend type will result in unspecified behavior, including data corruption.