A Viash component can be translated into one or more engines: Native, Docker; and one or more runners: Executable, Nextflow. Each of these engines and runners result in a different artifact:
Native engine: a single executable is generated which runs natively on the host system. This assumes all dependencies are already installed by the user and is therefore not reproducible. Requirements: Bash.
Docker engine: a single executable is generated but it runs inside a Docker container instead. The Docker engine specification can help you install custom dependencies and will take care of auto-mounting input/output files. Requirements: Bash, Docker.
Executable runner: a bash script is generated and can be run directly. The executable runner supports both the native and docker engine.
Nextflow runner: a Nextflow module which can be used as a standalone workflow or as a module in a larger workflow. Requirements: Nextflow and a containerization engine (e.g. Docker, Singularity, Podman).
Below we will create our first Viash component using any of the languages natively supported by Viash.
Create a script
When creating a new Viash component, you can write a new script or use a pre-existing script. Below is a script that simply copies an input file to an output destination.
The par variable(s) appear to be hard coded, but they’re not! When running this script with Viash, Viash will strip away the section between VIASH START and VIASH END, and replace it with parameter values at runtime. The values included in this script are thus entirely for development and debugging purposes. More information on how this works will be given in Variables and meta-variables.
Create a config
A Viash config file is a YAML file that describes the functionality of a component as well as the engine(s) and runner(s) it targets.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_bash
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_bash:latest'
[warning] Could not pull from 'example_bash:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_bash:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/bash/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/bash/foo.txt'.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_csharp
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_csharp:latest'
[warning] Could not pull from 'example_csharp:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_csharp:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/csharp/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/csharp/foo.txt'.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_js
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_js:latest'
[warning] Could not pull from 'example_js:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_js:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/js/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/js/foo.txt'
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_python
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_python:latest'
[warning] Could not pull from 'example_python:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_python:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/python/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/python/foo.txt'.
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_r
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_r:latest'
[warning] Could not pull from 'example_r:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_r:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/r/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/r/foo.txt'.
[1] TRUE
You can call use the component’s --help functionality to get an overview its parameters and descriptions.
viash run config.vsh.yaml ----help
example_scala
A minimal example component.
Arguments:
--input
type: file, required parameter, file must exist
example: file.txt
--output
type: file, required parameter, output, file must exist
example: output.txt
As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.
viash run config.vsh.yaml ----input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_scala:latest'
[warning] Could not pull from 'example_scala:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_scala:latest' with Dockerfile
warning: 1 deprecation; re-run with -deprecation for details
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/scala/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/scala/foo.txt'.
Note
The double dash (--) between the viash command and the arguments is used to signify the end of the arguments passed to Viash and the start of those passed to the script. If you forgot to add these, you’ll get an error similar to this:
viash run config.vsh.yaml \--input foo.txt \--output bar.txt
[scallop] Error: Unknown option 'input'
Build an executable
We will now turn the Viash component into an executable.
Use the viash build command to generate an executable.
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/bash/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/bash/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 411 Dec 4 09:10 config.vsh.yaml
-rw-r--r-- 1 runner docker 411 Dec 4 09:10 foo.txt
-rw-r--r-- 1 runner docker 411 Dec 4 09:13 output.txt
-rwxr-xr-x 1 runner docker 181 Dec 4 09:10 script.sh
drwxr-xr-x 2 runner docker 4096 Dec 4 09:12 target
You can run the executable by providing a value for --input and --output:
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/csharp/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/csharp/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 450 Dec 4 09:10 config.vsh.yaml
-rw-r--r-- 1 runner docker 450 Dec 4 09:10 foo.txt
-rw-r--r-- 1 runner docker 450 Dec 4 09:10 output.txt
-rw-r--r-- 1 runner docker 237 Dec 4 09:10 script.csx
drwxr-xr-x 2 runner docker 4096 Dec 4 09:12 target
You can run the executable by providing a value for --input and --output:
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/js/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/js/output.txt'
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 428 Dec 4 09:10 config.vsh.yaml
-rw-r--r-- 1 runner docker 428 Dec 4 09:11 foo.txt
-rw-r--r-- 1 runner docker 428 Dec 4 09:13 output.txt
-rwxr-xr-x 1 runner docker 282 Dec 4 09:10 script.js
drwxr-xr-x 2 runner docker 4096 Dec 4 09:12 target
You can run the executable by providing a value for --input and --output:
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/python/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/python/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 423 Dec 4 09:10 config.vsh.yaml
-rw-r--r-- 1 runner docker 423 Dec 4 09:11 foo.txt
-rw-r--r-- 1 runner docker 423 Dec 4 09:13 output.txt
-rwxr-xr-x 1 runner docker 216 Dec 4 09:10 script.py
drwxr-xr-x 2 runner docker 4096 Dec 4 09:12 target
You can run the executable by providing a value for --input and --output:
warning: 1 deprecation; re-run with -deprecation for details
Copying '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/scala/config.vsh.yaml' to '/viash_automount/tmp/Rtmpd48hmu/create_new_component603135e51fc6/scala/output.txt'.
This results in the following output:
ls-l
total 20
-rw-r--r-- 1 runner docker 462 Dec 4 09:10 config.vsh.yaml
-rw-r--r-- 1 runner docker 462 Dec 4 09:12 foo.txt
-rw-r--r-- 1 runner docker 462 Dec 4 09:13 output.txt
-rw-r--r-- 1 runner docker 435 Dec 4 09:10 script.scala
drwxr-xr-x 2 runner docker 4096 Dec 4 09:12 target