Working with AWS¶

Introduction¶

In addition to installing the server application on a physical cluster machine, you can deploy a fully configured cluster in the AWS cloud. This lets you use LigandScout's remote execution capabilities without access to an on-site cluster. AWS allows you to deploy arbitrarily large clusters that are billed per hour of up-time.

Everything described on this page runs entirely in your own AWS account. You do not need to contact Inte:Ligand first, and Inte:Ligand does not need access to your account. All you need is your LigandScout serial number.

Reference

AWS Landing Page: https://aws.amazon.com/

Clusters in the cloud have an important advantage over traditional ones: they are elastic, which means they scale up and down with the current workload. iserver exploits this by splitting large jobs into multiple smaller sub-jobs.

Consider a small cluster and a large screening job that is split into several sub-jobs, each planned to run on all CPU cores of a single node. The figure below illustrates this with five sub-jobs and an initial cluster size of two compute nodes. Two of the sub-jobs can start immediately while additional nodes are added to the cluster. These are then used to execute the three jobs waiting in the scheduler's queue. Once the number of sub-jobs drops below the number of nodes, the cluster scales back down and reduces running costs.

Overview¶

Assembling and setting up a physical HPC cluster requires weeks of effort from IT specialists, plus ongoing maintenance. Setting up a cloud cluster needs only basic command-line knowledge. This page guides you through five steps:

Create an AWS account
Create an AWS access key for programmatic access
Create a key pair for accessing the cluster nodes
Install AWS ParallelCluster
Write a cluster configuration and create your cluster

Tip

We recommend skimming the whole page before working through the steps in detail. It helps to understand how they connect.

Setting Up Your AWS Account¶

Tip

The screenshots in this section can be enlarged by clicking on them. Use the middle mouse button to open a screenshot in a new tab.

To deploy a cluster for virtual screening and conformer generation in the AWS cloud, you first need an AWS account.

Creating Your AWS Account¶

Go to https://aws.amazon.com and click on Create an AWS Account.

AWS also provides detailed instructions on account creation.

Creating an AWS Access Key¶

AWS ParallelCluster deploys clusters on your behalf. For it to create resources in your account, you need to provide a programmatic access key, which you can create and download from the web interface.

Go to the Security Credentials section of your AWS account, in the top right of the AWS Management Console:

There you can click Create New Access Key:

Warning

The secret access key can only be downloaded immediately after it is created. Store it somewhere safe — you will need it in a moment. If you did not save it, revoke the key and create a new one.

Creating a Key Pair¶

Next, create a key pair (public and private key) for accessing the cluster nodes. You need it to connect from LigandScout or KNIME to the LigandScout Remote server application running on the cluster's head node, and it also gives you manual SSH access to the nodes.

Navigate to Services -> EC2 -> Key Pairs -> Create Key Pair.

Choose the pem file format. The ppk format is not supported by LigandScout or KNIME.

Deploying a Cluster with AWS ParallelCluster¶

Deployment and configuration are handled by Amazon's ParallelCluster tool. You install it on your own desktop computer or notebook, and it sends the necessary instructions to AWS. In other words, you deploy clusters in the AWS cloud directly from your local computer.

Reference

AWS ParallelCluster on GitHub https://github.com/aws/aws-parallelcluster

The entry point for the official documentation is https://docs.aws.amazon.com/parallelcluster/latest/ug/what-is-aws-parallelcluster.html.

Installing AWS ParallelCluster¶

Follow the official installation instructions: https://docs.aws.amazon.com/parallelcluster/latest/ug/install-v3-parallelcluster.html

It does not matter which machine you install it on. You can also install it on several machines and use them in parallel.

Afterwards, run aws configure and supply:

Default region name: The AWS region your cluster should run in. The current region is shown in the top right of the AWS Management Console. Examples are eu-central-1, us-east-2, and sa-east-1.
AWS Access Key ID: The programmatic access key that lets AWS ParallelCluster create resources in your account. See above.
AWS Secret Access Key: The secret half of that key, which you downloaded earlier.

The Cluster Configuration File¶

A cluster is described by a YAML file. If you do not already know which VPC and subnet to use, running pcluster configure --config scratch-config.yaml walks you through the network settings interactively and is a good way to discover a usable subnet ID.

Warning

pcluster configure overwrites the file you point it at with a freshly generated configuration. Always give it a scratch filename, never the reference configuration below — otherwise it silently discards the CustomActions and SharedStorage blocks that install and run iserver, and your cluster will come up without them.

Reference

Interactive configuration: https://docs.aws.amazon.com/parallelcluster/latest/ug/install-v3-configuring.html

Every available setting: https://docs.aws.amazon.com/parallelcluster/latest/ug/cluster-configuration-file-v3.html

Download the reference configuration — cluster-config.yaml — and replace every REPLACE_ME value:

# Reference AWS ParallelCluster v3 configuration for LigandScout Remote.
#
# Replace every REPLACE_ME value, then create the cluster with:
#
#   pcluster create-cluster --cluster-name my-cluster \
#       --cluster-configuration cluster-config.yaml
#
# Documentation: https://docs.inteligand.com/ls-remote/working-with-aws/

# The region this cluster is built in. This setting overrides the default region
# you set with `aws configure`, so it must match the region your key pair and
# subnet live in. Examples: eu-central-1, us-east-2, sa-east-1.
Region: REPLACE_ME

Image:
  # Ubuntu 22.04 is the operating system Inte:Ligand tests against.
  Os: ubuntu2204

HeadNode:
  # The head node runs iserver and the Slurm controller. It does no screening
  # itself, so a small instance is enough.
  InstanceType: t3.large

  Networking:
    # A public subnet. Placing compute nodes in a private subnet instead would
    # require a NAT gateway, which bills by the hour whether or not you run jobs.
    SubnetId: subnet-REPLACE_ME
    ElasticIp: true

  Ssh:
    # The EC2 key pair you created in the AWS console. You need the matching
    # .pem file both for `pcluster ssh` and for connecting from LigandScout-XT.
    KeyName: REPLACE_ME

  LocalStorage:
    RootVolume:
      Size: 50

  CustomActions:
    OnNodeConfigured:
      # Installs LigandScout-XT and iserver onto the shared volume and starts
      # iserver as a systemd service.
      Script: https://docs.inteligand.com/ls-remote/aws/bootstrap-iserver.sh
      Args:
        # Your LigandScout serial number. LigandScout-XT activates itself with
        # it on first use.
        - "REPLACE_ME"
        # Optional: pin different LigandScout-XT and iserver archives by adding
        # their URLs as a second and third argument.

Scheduling:
  Scheduler: slurm

  SlurmSettings:
    # Minutes an idle compute node stays up before it is terminated.
    ScaledownIdletime: 5

  SlurmQueues:
    - Name: compute
      # Switch to SPOT for up to 90% lower compute cost. Screening and conformer
      # generation sub-jobs can be restarted individually, so an interrupted
      # node costs you one sub-job rather than the whole run.
      CapacityType: ONDEMAND

      Networking:
        # Note the plural: compute queues take a list, unlike the head node.
        SubnetIds:
          - subnet-REPLACE_ME
        # Compute nodes need outbound internet access to fetch their own
        # bootstrap data and to activate LigandScout. Subnets do not always
        # auto-assign public IPs, so request one explicitly.
        AssignPublicIp: true

      ComputeResources:
        - Name: screening
          InstanceType: c6i.2xlarge
          # MinCount 0 means you pay for compute only while jobs are running.
          MinCount: 0
          MaxCount: 10

SharedStorage:
  # Mounted on every node as /shared. The bootstrap script installs
  # LigandScout-XT and iserver here, and screening databases and job results
  # live here too. Size it for the databases you intend to use.
  - Name: shared
    MountDir: /shared
    StorageType: Ebs
    EbsSettings:
      Size: 200
      VolumeType: gp3
      # This volume and everything on it is destroyed together with the cluster.
      # Set to Retain if you would rather keep it.
      DeletionPolicy: Delete

The settings worth understanding before you create a cluster:

Region: The region the cluster is built in. This value overrides the default region you set with aws configure, so it has to match the region in which you created your key pair and subnet. Key pairs and subnets are region-scoped: one created in us-east-2 does not exist in eu-central-1.
Ssh.KeyName: The EC2 key pair you created above. You need the matching .pem file to connect from LigandScout.
HeadNode.Networking.SubnetId and SlurmQueues[].Networking.SubnetIds: Both normally hold the same subnet. Note that the head node takes a single value while the compute queue takes a list — that asymmetry is ParallelCluster's, not a typo.
CustomActions.OnNodeConfigured.Args: The first argument is your LigandScout serial number — the same serial you use to activate LigandScout on the desktop, issued by Inte:Ligand with your licence. It is mandatory, and the cluster build fails early and explicitly if it is wrong. You may optionally add a LigandScout-XT archive URL as the second argument and an iserver archive URL as the third, if you need to pin versions other than the tested defaults; to override only the third, pass the default LigandScout-XT URL as the second.
SlurmQueues[].ComputeResources[].InstanceType: The type of compute instance to use. Together with MaxCount this is the main cost factor. AWS offers everything from a single virtual CPU up to more than a hundred cores per machine, and iserver works with compute instances of all sizes.
SharedStorage.EbsSettings.Size: The size of the /shared volume in gigabytes. It holds the LigandScout-XT and iserver installations, your screening databases, and all job results. Size it for the databases you intend to use — it can be increased later.

Warning

Your serial number is stored in plain text in this file and in the resulting CloudFormation stack. Treat the configuration file like any other credential: do not commit it to a public repository.

Info

There is no fixed limit on the number of instances you can run. AWS applies a quota to the total number of CPU cores, which depends on a number of factors but is usually at least a few hundred. The AWS console shows your current quotas under Service Quotas; the relevant one is Running On-Demand All Standard (A, C, D, H, I, M, R, T, Z) instances. Quota increases can be requested.

What the Bootstrap Script Does¶

CustomActions.OnNodeConfigured points at a script hosted alongside this documentation. AWS ParallelCluster downloads and runs it on the head node once the base setup — networking, shared storage, and the Slurm scheduler — is complete.

The script installs LigandScout-XT and iserver onto the shared volume, writes your serial number so LigandScout-XT activates itself, activates it once immediately so that a wrong serial fails the build instead of surfacing later as a failed screening job, generates a matching application.properties, and registers iserver as a systemd service so it starts on boot and restarts if it stops. Finally it waits for iserver to answer on /status, so a cluster that reaches CREATE_COMPLETE is one where the server is actually running.

Compute nodes need no setup of their own. They reach the same installation over the shared volume, and because LigandScout-XT ships with its own Java runtime, no packages have to be installed on any node.

You can read the script here, download it as bootstrap-iserver.sh, or host your own modified copy and point Script: at that instead.

bootstrap-iserver.sh

#!/bin/bash
#
# Sets up the LigandScout Server (iserver) on an AWS ParallelCluster v3 head node.
#
# Run it from your cluster configuration as an OnNodeConfigured custom action:
#
#   HeadNode:
#     CustomActions:
#       OnNodeConfigured:
#         Script: https://docs.inteligand.com/ls-remote/aws/bootstrap-iserver.sh
#         Args:
#           - "<your LigandScout serial number>"
#
# Arguments:
#   $1  LigandScout serial number                        (required)
#   $2  URL of the LigandScout-XT Linux archive          (optional)
#   $3  URL of the iserver Linux archive                 (optional)
#
# The script installs both applications onto the shared volume, activates
# LigandScout-XT with your serial number, and registers iserver as a systemd
# service. Compute nodes need no setup of their own: they reach the same
# installation over the shared volume.
#
# Progress and errors are written to /var/log/iserver-bootstrap.log.

set -euo pipefail

exec > >(tee -a /var/log/iserver-bootstrap.log) 2>&1

# Pinned to a tested combination. Override via Args if you need a different one.
DEFAULT_XT_URL="https://www.inteligand.com/ligandscout-xt/downloads/LigandScout-XT_114_linux64_20260709.tar.gz"
DEFAULT_ISERVER_URL="https://www.inteligand.com/download/iserver/iserver_1.6.6_linux64.tar.gz"

SERIAL="${1:-}"
XT_URL="${2:-$DEFAULT_XT_URL}"
ISERVER_URL="${3:-$DEFAULT_ISERVER_URL}"

# The default login user of an Ubuntu ParallelCluster. Jobs run as this user.
CLUSTER_USER="ubuntu"

SHARED_DIR="/shared"
APPS_DIR="${SHARED_DIR}/apps"
XT_DIR="${APPS_DIR}/LigandScout-XT"
ISERVER_DIR="${APPS_DIR}/ligandscout_server"

echo "=== iserver bootstrap started $(date -u +'%Y-%m-%dT%H:%M:%SZ') ==="

if [ -z "${SERIAL}" ]; then
    echo "ERROR: no LigandScout serial number was given." >&2
    echo "       Add it as the first entry of CustomActions.OnNodeConfigured.Args" >&2
    echo "       in your cluster configuration file." >&2
    exit 1
fi

# Only the head node runs iserver. This guard makes the script harmless if it
# is ever attached to a compute queue as well.
if [ -r /opt/parallelcluster/cfnconfig ]; then
    # shellcheck disable=SC1091
    . /opt/parallelcluster/cfnconfig
    if [ "${cfn_node_type:-}" != "HeadNode" ]; then
        echo "Node type is '${cfn_node_type:-unknown}', not HeadNode. Nothing to do."
        exit 0
    fi
fi

# The host that LigandScout-XT contacts to activate. Checked up front so that a
# head node without outbound internet access fails immediately, with a message
# naming the cause, rather than ten minutes later at the activation step.
LICENSE_HOST_URL="http://license1.inteligand.com"

# ---------------------------------------------------------------------------
# Pre-flight: confirm everything this script depends on is reachable before
# spending several minutes downloading a third of a gigabyte.
# ---------------------------------------------------------------------------
check_reachable()
{
    local label="$1"
    local url="$2"

    printf '    %-18s ' "${label}"
    if curl -fsSL --head --connect-timeout 10 --max-time 20 --retry 1 --retry-delay 5 \
            "${url}" > /dev/null 2>&1; then
        echo "reachable"
        return 0
    fi
    echo "UNREACHABLE  (${url})"
    return 1
}

echo "--- Pre-flight checks"
preflight_failures=0
check_reachable "LigandScout-XT" "${XT_URL}" || preflight_failures=$((preflight_failures + 1))
check_reachable "iserver" "${ISERVER_URL}" || preflight_failures=$((preflight_failures + 1))
check_reachable "license server" "${LICENSE_HOST_URL}" || preflight_failures=$((preflight_failures + 1))

if [ "${preflight_failures}" -ne 0 ]; then
    echo "ERROR: ${preflight_failures} of 3 required hosts could not be reached from this" >&2
    echo "       head node, so the installation cannot proceed." >&2
    echo "" >&2
    echo "       The usual cause is that the head node has no outbound internet access." >&2
    echo "       Check that its subnet routes to an internet gateway and that the node" >&2
    echo "       has a public IP (HeadNode.Networking.ElasticIp: true), or that a NAT" >&2
    echo "       gateway is present if you placed it in a private subnet." >&2
    echo "" >&2
    echo "       If instead an archive URL is wrong or has been withdrawn, correct it in" >&2
    echo "       CustomActions.OnNodeConfigured.Args." >&2
    exit 1
fi

# ---------------------------------------------------------------------------
# Download and unpack an archive into a target directory.
#
# Both archives contain a single top-level directory whose name carries the
# version, so we unpack into a temporary location and rename. That keeps the
# installation paths stable across releases.
# ---------------------------------------------------------------------------
install_archive()
{
    local url="$1"
    local dest="$2"
    local tmp

    case "${dest}" in
        "${APPS_DIR}"/*) ;;
        *) echo "ERROR: refusing to install outside ${APPS_DIR}: ${dest}" >&2; exit 1 ;;
    esac

    tmp="$(mktemp -d)"

    echo "Downloading ${url}"
    curl -fSL --retry 5 --retry-delay 10 --retry-connrefused \
        "${url}" -o "${tmp}/archive.tar.gz"

    echo "Unpacking into ${dest}"
    mkdir -p "${tmp}/unpacked"
    tar -xzf "${tmp}/archive.tar.gz" -C "${tmp}/unpacked"

    rm -rf "${dest}"
    mkdir -p "$(dirname "${dest}")"

    local top_level entry_count
    entry_count="$(find "${tmp}/unpacked" -mindepth 1 -maxdepth 1 | wc -l)"
    top_level="$(find "${tmp}/unpacked" -mindepth 1 -maxdepth 1)"
    if [ "${entry_count}" -eq 1 ] && [ -d "${top_level}" ]; then
        mv "${top_level}" "${dest}"
    else
        mv "${tmp}/unpacked" "${dest}"
    fi

    rm -rf "${tmp}"
}

echo "--- Creating directories on the shared volume"
mkdir -p \
    "${APPS_DIR}" \
    "${SHARED_DIR}/jobs/screening" \
    "${SHARED_DIR}/jobs/confgen" \
    "${SHARED_DIR}/data/compound-databases/re-confgen" \
    "${SHARED_DIR}/data/compound-databases/upload" \
    "${SHARED_DIR}/data/input"

echo "--- Installing LigandScout-XT"
install_archive "${XT_URL}" "${XT_DIR}"

if [ ! -x "${XT_DIR}/iscreen" ]; then
    echo "ERROR: ${XT_DIR}/iscreen is missing. The archive layout is not what" >&2
    echo "       this script expects. Check the URL in Args." >&2
    exit 1
fi

# LigandScout-XT activates itself the first time a licensed tool runs and finds
# a file named 'serial' in its installation directory. Every cluster node reads
# the same file from the shared volume, so this single write covers the whole
# cluster, including nodes added later by autoscaling.
echo "--- Writing serial number for automatic activation"
printf '%s\n' "${SERIAL}" > "${XT_DIR}/serial"
chmod 600 "${XT_DIR}/serial"

echo "--- Installing iserver"
install_archive "${ISERVER_URL}" "${ISERVER_DIR}"

if [ ! -x "${ISERVER_DIR}/iserver" ]; then
    echo "ERROR: ${ISERVER_DIR}/iserver is missing. The archive layout is not" >&2
    echo "       what this script expects. Check the URL in Args." >&2
    exit 1
fi

# The launcher runs 'java -jar ilib-server-current.jar', but the release archive
# only contains the versioned jar. Normally update.sh creates this symlink; on a
# fresh install nothing has, so we do it here.
ISERVER_JAR="$(find "${ISERVER_DIR}" -maxdepth 1 -name 'ilib-server-*.jar' \
    ! -name 'ilib-server-current.jar' -exec basename {} \; | sort | tail -n 1)"

if [ -z "${ISERVER_JAR}" ]; then
    echo "ERROR: no ilib-server-*.jar found in ${ISERVER_DIR}." >&2
    echo "       The archive layout is not what this script expects." >&2
    exit 1
fi

ln -sfn "${ISERVER_JAR}" "${ISERVER_DIR}/ilib-server-current.jar"
echo "Linked ilib-server-current.jar -> ${ISERVER_JAR}"

# Release archives are packed with world-writable modes. chown alone would not
# correct that, so tighten the tree before handing it over.
chmod -R go-w "${XT_DIR}" "${ISERVER_DIR}"

echo "--- Writing iserver configuration"
cat > "${ISERVER_DIR}/application.properties" <<PROPERTIES
######################################################################
# Written by bootstrap-iserver.sh. Edit and restart with
#   sudo systemctl restart iserver
######################################################################

job.screening.directory=${SHARED_DIR}/jobs/screening
job.confgen.directory=${SHARED_DIR}/jobs/confgen

ligandscout.path=${XT_DIR}

ldb.directories=${SHARED_DIR}/data/compound-databases
ldb.directories.monitoring.recursive=true
ldb.directories.upload=${SHARED_DIR}/data/compound-databases/upload
job.confgen.databases.directory=${SHARED_DIR}/data/compound-databases/re-confgen

job.confgen.input.directories=${SHARED_DIR}/data/input
job.confgen.input.recursive=true

######################################################################
# Scheduler
######################################################################
scheduler=slurm
scheduler.slurm.partition=
scheduler.slurm.option.string=
scheduler.priority=0

scheduler.number.processors.screening=4
scheduler.number.processors.confgen=2

# Match these to the compute instance type in your cluster configuration.
iscreen.amount.cores=6
iscreen.memory=8
idbgen.memory=4
idbgen.memory.slaves=3

job.splitting.max.chunk.size=2000000
job.splitting.max.chunk.size.confgen=10000

merge.confgen.databases=false
move_finished_jobs=false
finished_jobs_directory=/home/<user>/jobs

######################################################################
# Logging
######################################################################
logging.level.root=INFO
logging.level.org.springframework=WARN
logging.level.org.springframework.web=WARN
logging.level.org.hibernate=WARN
logging.level.com.tupilabs.pbs=WARN
logging.level.ilib.server.grid.slurm=WARN

logging.directory=./logs
logging.allow.download=true

######################################################################
# Internal database
######################################################################
spring.datasource.url=jdbc:h2:./database/ilib-server;MVCC=true
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=admin
spring.datasource.password=password
spring.jpa.generate-ddl=true
spring.jpa.show-sql=false
spring.jpa.hibernate.ddl-auto=update
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.hibernate.use-new-id-generator-mappings=true
spring.h2.console.enabled=true
spring.h2.console.path=/console/

######################################################################
# Web server
######################################################################
server.tomcat.basedir=./tomcat-logs
server.tomcat.accesslog.enabled=true
server.tomcat.accesslog.pattern=%t %a "%r" %s (%D ms)

spring.servlet.multipart.max-file-size=32768MB
spring.servlet.multipart.max-request-size=32768MB

server.port=8080
# Bind to the loopback interface only. The REST API has no authentication, so it
# must not be exposed: LigandScout-XT and KNIME reach it through an SSH tunnel.
# Removing this line makes iserver listen on every interface, including the
# public one, and anyone who can reach port 8080 gains full control of the server.
server.address=127.0.0.1

######################################################################
# Startup self-tests
######################################################################
testing.skip=false
testing.paths=true
testing.scheduler=true
testing.quit-on-error=true
PROPERTIES

echo "--- Handing the shared volume to ${CLUSTER_USER}"
chown -R "${CLUSTER_USER}:${CLUSTER_USER}" "${SHARED_DIR}"

# Activate now rather than waiting for the first screening job. Activation is
# otherwise lazy, so an incorrect serial would produce a cluster that builds
# cleanly and then fails every job with an opaque licensing error.
echo "--- Activating LigandScout-XT"
if ! su - "${CLUSTER_USER}" -c "'${XT_DIR}/activation' -s '${SERIAL}'"; then
    echo "ERROR: LigandScout-XT could not be activated with the serial number" >&2
    echo "       supplied in CustomActions.OnNodeConfigured.Args." >&2
    echo "       Check that the serial is correct and that the head node can reach" >&2
    echo "       license1.inteligand.com. Screening jobs cannot run until this" >&2
    echo "       succeeds, so the cluster build is being failed deliberately." >&2
    exit 1
fi

# iserver runs on the Java runtime that ships inside LigandScout-XT, so the
# cluster needs no separate JDK installation. The launcher calls a bare 'java',
# which the PATH below resolves to the bundled runtime.
#
# /opt/slurm/bin must also be on that PATH: iserver invokes sbatch, sinfo,
# squeue and scancel by bare name, and its startup self-test quits the server
# if it cannot query the scheduler.
#
# Restart=always rather than on-failure: when the self-test fails, iserver exits
# cleanly with status 0, which on-failure would not catch.
echo "--- Registering the iserver systemd service"
cat > /etc/systemd/system/iserver.service <<UNIT
[Unit]
Description=LigandScout Server (iserver)
Documentation=https://docs.inteligand.com/ls-remote/working-with-aws/
After=network-online.target slurmctld.service
Wants=network-online.target

[Service]
Type=simple
User=${CLUSTER_USER}
Group=${CLUSTER_USER}
WorkingDirectory=${ISERVER_DIR}
Environment=JAVA_HOME=${XT_DIR}/jre
Environment=PATH=${XT_DIR}/jre/bin:/opt/slurm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ExecStart=${ISERVER_DIR}/iserver
Restart=always
RestartSec=15

[Install]
WantedBy=multi-user.target
UNIT

systemctl daemon-reload
systemctl enable --now iserver

echo "--- Waiting for iserver to answer on http://localhost:8080/status"
for attempt in $(seq 1 60); do
    if curl -fsS --max-time 5 http://localhost:8080/status > /dev/null 2>&1; then
        echo "iserver is up after ${attempt} attempt(s)."
        echo "=== iserver bootstrap finished $(date -u +'%Y-%m-%dT%H:%M:%SZ') ==="
        exit 0
    fi
    sleep 10
done

echo "ERROR: iserver did not answer within 10 minutes." >&2
echo "       Diagnose with: journalctl -u iserver" >&2
exit 1

Creating the Cluster¶

With the configuration file in place, create the cluster:

pcluster create-cluster --cluster-name my-cluster --cluster-configuration cluster-config.yaml

Creation takes roughly fifteen minutes, most of which is AWS building the network and node images. Watch progress with:

pcluster describe-cluster --cluster-name my-cluster

Once clusterStatus reads CREATE_COMPLETE, your cluster is ready for virtual screening and conformer generation through LigandScout and KNIME.

Info

If cluster creation fails, add --rollback-on-failure false so the cluster survives for inspection. Then log into the head node as described below and read /var/log/iserver-bootstrap.log for problems with the LigandScout installation, or /var/log/cfn-init.log and /var/log/cloud-init-output.log for problems with the base setup.

Connecting from LigandScout¶

The bootstrap script configures iserver to listen on port 8080 of the head node's loopback interface only (server.address=127.0.0.1), so it is not reachable from the network. LigandScout-XT reaches it through a built-in SSH tunnel; no inbound port needs to be opened.

Warning

The iserver REST API has no authentication. Anyone who can reach port 8080 has full control of the server, including your data. The loopback binding is what protects it — do not remove server.address from application.properties in order to connect without a tunnel, and do not open port 8080 in the head node's security group.

Get the head node's address with:

pcluster describe-cluster --cluster-name my-cluster --query headNode.publicIpAddress

Then, in LigandScout-XT, open the iServer Settings dialog and enter:

IP or Hostname: The head node's public IP address.
Port: 8080.
User: ubuntu — the default login user of an Ubuntu ParallelCluster.
Identity File: The .pem file of the key pair you created earlier.

Leave Use SSH Tunnel enabled. If your organisation reaches AWS through a bastion host, enable Use intermediate Host and fill in its details as well.

Reference

These settings are also available under Preferences → Program and Network Settings and are described in the LigandScout user manual, in the LigandScout Remote and Preferences chapters.

If LigandScout reports that it "established SSH Tunnel(s), but could not connect to the remote server application", the tunnel is fine and iserver is not running. Log into the head node and check:

sudo systemctl status iserver
sudo journalctl -u iserver

Working with AWS Clusters¶

Command-Line Interface¶

AWS ParallelCluster provides a command-line interface for interacting with your clusters.

Reference

All commands are documented at https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster-v3.html.

To log into the head node — for inspecting the cluster or making manual changes:

pcluster ssh --cluster-name my-cluster -i <path-to-key-file/key-name.pem>

To save on running costs when the cluster is not needed, stop the compute fleet:

pcluster update-compute-fleet --cluster-name my-cluster --status STOP_REQUESTED

Warning

This terminates every compute node, including nodes with jobs still running. Running jobs are killed, not drained. Wait until the queue is empty — check with squeue on the head node — before stopping the fleet, or restart the affected sub-jobs afterwards from the job monitoring dialog.

Start it again with:

pcluster update-compute-fleet --cluster-name my-cluster --status START_REQUESTED

Warning

Stopping the compute fleet does not stop the head node or the shared storage volume. The head node is usually a small, inexpensive instance, but it continues to bill, as does the volume. A cluster with its fleet stopped therefore has a small but non-zero running cost.

Once you no longer need a cluster, delete it:

pcluster delete-cluster --cluster-name my-cluster

Deleting a cluster terminates all its resources, after which no further costs are incurred.

Warning

Deletion also destroys the shared storage volume and everything on it, unless you set DeletionPolicy: Retain in your configuration. Save screening results locally before deleting, or create a snapshot of the volume first.

Updating to Newer `iserver` Versions¶

New clusters install the version pinned in the bootstrap script — currently iserver 1.6.6. You do not need to recreate a cluster to move to a newer one. Log into the head node as described above, then:

sudo systemctl stop iserver
cd /shared/apps/ligandscout_server/
./update.sh -v <version>
sudo systemctl start iserver

update.sh is described in more detail in the physical cluster installation instructions.

Warning

Pass an iserver version newer than the one you have. The Version Dependencies page records changelogs and compatibility notes, but its newest entry predates the version installed on a new cluster — do not treat the top of that page as the latest release. Contact support@inteligand.com if you are unsure which version to move to.

To pin the new version for future clusters as well, add its archive URL as the third entry of CustomActions.OnNodeConfigured.Args in your cluster configuration.

Screening Databases¶

LigandScout Remote creates new screening databases from local molecule libraries via conformer generation. If you already have databases you want to use, upload them to the shared volume. Any file transfer method works; since the head node is reachable over SSH, scp is the simplest:

scp -i <path-to-key-file/key-name.pem> -r /path/to/database.ldb2 ubuntu@<head-node-ip>:/shared/data/compound-databases

On Windows, use an SCP client such as WinSCP.

Once the upload finishes, refresh the server's database list using the Load Remote Database dialog in LigandScout. You can also upload databases directly from the LigandScout user interface.

Warning

Make sure the shared volume has enough free space for the databases you upload. If it does not, increase its size.

Creating Snapshots¶

To save the state of your cluster, create an EBS snapshot of the shared volume. Snapshots can be used as the basis for new volumes, so you do not have to re-upload databases or re-run conformer generation for a future cluster.

Go to Elastic Block Store -> Volumes in the AWS console and select the shared volume of your running cluster — the one whose size matches SharedStorage.EbsSettings.Size. The node root volumes are smaller. Choose Actions -> Create Snapshot and give it a recognisable name.

Under Elastic Block Store -> Snapshots you can then manage the snapshot and see its Snapshot ID. To base a future cluster's shared volume on it, add the ID to your configuration and keep Size at least as large as the snapshot:

SharedStorage:
  - Name: shared
    MountDir: /shared
    StorageType: Ebs
    EbsSettings:
      SnapshotId: snap-REPLACE_ME
      Size: 200
      VolumeType: gp3

Warning

Size defaults to 40 GiB if you omit it. A volume cannot be smaller than the snapshot it is restored from, so leaving it out makes cluster creation fail whenever the snapshot is larger than that.

Info

A restored snapshot brings back your screening databases and job results, but not a working installation: the bootstrap script replaces /shared/apps/LigandScout-XT and /shared/apps/ligandscout_server on every new cluster. That also resets iserver's internal job database, so the job history from the old cluster does not carry over — the result files under /shared/jobs do.

Databases in the older .ldb format are not readable by current iserver versions, which expect .ldb2. If you are carrying a volume forward from a cluster built before the move to LigandScout-XT, convert them on the head node with the idbupgrader tool in /shared/apps/LigandScout-XT.

Warning

Snapshots consume EBS storage and are billed accordingly, though the rate per gigabyte is well below that of a live volume. See https://aws.amazon.com/ebs/pricing/.

Increasing the Shared Volume Size¶

If you upload large screening databases or run extensive conformer generation jobs, you will eventually need more space on the shared volume. This is a two-step process.

In the AWS console, go to Services -> EC2 -> Elastic Block Store -> Volumes. You will see one root volume per node plus the shared volume, which is the largest. Select it, then click Actions -> Modify Volume:

Set the new size. Depending on how large the increase is, the volume stays in optimizing state for up to a few hours and performs below par during that time. The cluster remains usable throughout.
Extend the filesystem so the change is recognised. Log into the head node as described above and run df -h to find the device mounted on /shared, then:
```
sudo resize2fs /dev/<device>
```
Run df -h again to confirm.

Info

To reuse the larger volume for future clusters, create a snapshot and reference it from your configuration.

References

Modifying EBS volumes: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-modify-volume.html
Requesting a volume modification from the console: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/requesting-ebs-volume-modifications.html#modify-ebs-volume
Extending a Linux filesystem after a volume modification: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/recognize-expanded-volume-linux.html

Cost Management¶

Running a cloud cluster is inexpensive compared to owning hardware, but it is worth monitoring charges and consuming only the resources you actually need.

A good starting point is your AWS Billing Dashboard. To estimate costs before you create anything, use the AWS Pricing Calculator.

References

AWS Cost Management: https://aws.amazon.com/aws-cost-management/
EC2 On-Demand pricing: https://aws.amazon.com/ec2/pricing/on-demand/
EC2 Spot pricing: https://aws.amazon.com/ec2/spot/pricing/
EBS volume and snapshot pricing: https://aws.amazon.com/ebs/pricing/

Compute instances dominate the bill, so the levers that matter most are:

Let the cluster scale to zero: With MinCount: 0, you pay for compute only while jobs are running. ScaledownIdletime controls how many minutes an idle node waits before being terminated — lower values save money, higher values avoid paying the startup time of a node that is about to be needed again.
Keep nodes in a public subnet: A private-subnet layout requires a NAT gateway, which bills by the hour whether or not you run any jobs. The reference configuration therefore places both the head node and the compute nodes in a public subnet.
Delete clusters you have finished with: A cluster whose compute fleet is stopped still bills for the head node and the shared volume. Deleting the cluster stops all charges.
Right-size the shared volume: Storage bills continuously, regardless of whether the cluster is running.

Reducing Cost with Spot Instances¶

Besides on-demand instances, which cost a fixed amount at all times, AWS offers spot instances. The spot price follows supply and demand and can be up to 90% below the on-demand price. To use them, change the capacity type of your queue:

Scheduling:
  SlurmQueues:
    - Name: compute
      CapacityType: SPOT

AWS then charges the current spot market price, capped at the on-demand price, so the cost is guaranteed to be no higher than on-demand. You can also cap it yourself per compute resource:

      ComputeResources:
        - Name: screening
          InstanceType: c6i.2xlarge
          SpotPrice: 0.15

If you set a SpotPrice, AWS reclaims your instances once the market price exceeds it. Even without one, instances can be reclaimed when spot capacity runs short.

This is not a problem for LigandScout Remote. Screening and conformer generation jobs are fault tolerant: a failed sub-job can be restarted on its own without repeating the whole job. Right-click a failed sub-job in the job monitoring dialog of LigandScout or KNIME to restart it.

Reference

Spot instances overview: https://aws.amazon.com/ec2/spot/

Official documentation: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html

Spot settings in AWS ParallelCluster: https://docs.aws.amazon.com/parallelcluster/latest/ug/Scheduling-v3.html

Working with AWS¶

Introduction¶

Overview¶

Setting Up Your AWS Account¶

Creating Your AWS Account¶

Creating an AWS Access Key¶

Creating a Key Pair¶

Deploying a Cluster with AWS ParallelCluster¶

Installing AWS ParallelCluster¶

The Cluster Configuration File¶

What the Bootstrap Script Does¶

Creating the Cluster¶

Connecting from LigandScout¶

Working with AWS Clusters¶

Command-Line Interface¶

Updating to Newer iserver Versions¶

Screening Databases¶

Creating Snapshots¶

Increasing the Shared Volume Size¶

Cost Management¶

Reducing Cost with Spot Instances¶

Updating to Newer `iserver` Versions¶