Create a new component

Building a reproducible Viash component.

A Viash component can be translated into one or more platforms: Native, Docker, Nextflow. Each of these platforms result in a different artifact:

Below we will create our first Viash component using any of the languages natively supported by Viash.

Create a script

When creating a new Viash component, you can write a new script or use a pre-existing script. Below is a script that simply copies an input file to an output destination.

Create a new file named script.sh and copy the following content inside of it. This script will copy an input file to an output destination.

#!/bin/bash

## VIASH START
par_input=path/to/file.txt
par_output=output.txt
## VIASH END

# copy file
echo "Copying '$par_input' to '$par_output'."
cp -r "$par_input" "$par_output"

Create a new file named script.csx and copy the following content inside of it. This script will copy an input file to an output destination.

using System.IO;

// VIASH START
var par = new {
  input = "path/to/file.txt",
  output = "output.txt"
};
// VIASH END

// copy file
Console.WriteLine($"Copying '{par.input}' to '{par.output}'.");
File.Copy(par.input, par.output, true);

Create a new file named script.js and copy the following content inside of it. This script will copy an input file to an output destination.

const fs = require('fs');

// VIASH START
let par = {
  'input': 'path/to/file.txt',
  'output': 'output.txt'
};
// VIASH END

// copy file
console.log(`Copying '${par['input']}' to '${par['output']}'`)
fs.copyFile(par['input'], par['output'], (err) => {
  if (err) throw err;
});

Create a new file named script.py and copy the following content inside of it. This script will copy an input file to an output destination.

import shutil

## VIASH START
par = {
  'input': 'file.txt',
  'output': 'output.txt'
}
## VIASH END

# copy file
print(f"Copying '{par['input']}' to '{par['output']}'.")
shutil.copyfile(par['input'], par['output'])

Create a new file named script.R and copy the following content inside of it. This script will copy an input file to an output destination.

## VIASH START
par <- list(
  "input" = 'file.txt',
  "output" = 'output.txt'
)
## VIASH END

# copy file
cat("Copying '", par$input, "' to '", par$output, "'.\n", sep = "")
file.copy(par$input, par$output)

Create a new file named script.scala and copy the following content inside of it. This script will copy an input file to an output destination.

import java.nio.file.StandardCopyOption.REPLACE_EXISTING
import java.nio.file.Files
import java.nio.file.Paths

// VIASH START
case class ViashPar(input: String, output: String)
val par = ViashPar(
  "path/to/file.txt",
  "output.txt"
)
// VIASH END

// copy file
println(s"Copying '${par.input}' to '${par.output}'.")
val fileIn = Paths.get(par.input)
val fileOut = Paths.get(par.output)
Files.copy(fileIn, fileOut, REPLACE_EXISTING)
Note

The par variable(s) appear to be hard coded, but they’re not! When running this script with Viash, Viash will strip away the section between VIASH START and VIASH END, and replace it with parameter values at runtime. The values included in this script are thus entirely for development and debugging purposes. More information on how this works will be given in Variables and meta-variables.

Create a config

A Viash config file is a YAML file that describes the functionality of a component and the platform(s) it targets.

Create a file named config.vsh.yaml and add the contents below based on your chosen scripting language.

functionality:
  name: example_bash
  description: A minimal example component.
  arguments:
    - type: file
      name: --input
      example: file.txt
      required: true
    - type: file
      name: --output
      direction: output
      example: output.txt
      required: true
  resources:
    - type: bash_script
      path: script.sh
platforms:
  - type: docker
    image: "bash:4.0"
  - type: native
  - type: nextflow

Create a file named config.vsh.yaml and add the contents below based on your chosen scripting language.

functionality:
  name: example_csharp
  description: A minimal example component.
  arguments:
    - type: file
      name: --input
      example: file.txt
      required: true
    - type: file
      name: --output
      direction: output
      example: output.txt
      required: true
  resources:
    - type: csharp_script
      path: script.csx
platforms:
  - type: docker
    image: "ghcr.io/data-intuitive/dotnet-script:1.3.1"
  - type: native
  - type: nextflow

Create a file named config.vsh.yaml and add the contents below based on your chosen scripting language.

functionality:
  name: example_js
  description: A minimal example component.
  arguments:
    - type: file
      name: --input
      example: file.txt
      required: true
    - type: file
      name: --output
      direction: output
      example: output.txt
      required: true
  resources:
    - type: javascript_script
      path: script.js
platforms:
  - type: docker
    image: "node:19-bullseye-slim"
  - type: native
  - type: nextflow

Create a file named config.vsh.yaml and add the contents below based on your chosen scripting language.

functionality:
  name: example_python
  description: A minimal example component.
  arguments:
    - type: file
      name: --input
      example: file.txt
      required: true
    - type: file
      name: --output
      direction: output
      example: output.txt
      required: true
  resources:
    - type: python_script
      path: script.py
platforms:
  - type: docker
    image: "python:3.10-slim"
  - type: native
  - type: nextflow

Create a file named config.vsh.yaml and add the contents below based on your chosen scripting language.

functionality:
  name: example_r
  description: A minimal example component.
  arguments:
    - type: file
      name: --input
      example: file.txt
      required: true
    - type: file
      name: --output
      direction: output
      example: output.txt
      required: true
  resources:
    - type: r_script
      path: script.R
platforms:
  - type: docker
    image: "eddelbuettel/r2u:22.04"
  - type: native
  - type: nextflow

Create a file named config.vsh.yaml and add the contents below based on your chosen scripting language.

functionality:
  name: example_scala
  description: A minimal example component.
  arguments:
    - type: file
      name: --input
      example: file.txt
      required: true
    - type: file
      name: --output
      direction: output
      example: output.txt
      required: true
  resources:
    - type: scala_script
      path: script.scala
platforms:
  - type: docker
    image: "sbtscala/scala-sbt:eclipse-temurin-19_36_1.7.2_2.13.10"
  - type: native
  - type: nextflow

Here’s a breakdown of the different sections:

  • functionality: A description of what the component does.
    • name: The name of the component.
    • arguments: The input and output parameters of the script.
    • resources: References to all necessary files and folders to make the component work.
  • platforms: Lists which platforms a component can target (i.e. Native, Docker or Nextflow).

Run the component

That’s it! With these two steps, you created your first component. Next, you can use the viash run command to test whether it actually works as intended.

You can call use the component’s --help functionality to get an overview its parameters and descriptions.

viash run config.vsh.yaml -- --help
example_bash

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.

viash run config.vsh.yaml -- --input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_bash:latest'
[warning] Could not pull from 'example_bash:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_bash:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/bash/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/bash/foo.txt'.

You can call use the component’s --help functionality to get an overview its parameters and descriptions.

viash run config.vsh.yaml -- --help
example_csharp

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.

viash run config.vsh.yaml -- --input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_csharp:latest'
[warning] Could not pull from 'example_csharp:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_csharp:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/csharp/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/csharp/foo.txt'.

You can call use the component’s --help functionality to get an overview its parameters and descriptions.

viash run config.vsh.yaml -- --help
example_js

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.

viash run config.vsh.yaml -- --input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_js:latest'
[warning] Could not pull from 'example_js:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_js:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/js/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/js/foo.txt'

You can call use the component’s --help functionality to get an overview its parameters and descriptions.

viash run config.vsh.yaml -- --help
example_python

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.

viash run config.vsh.yaml -- --input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_python:latest'
[warning] Could not pull from 'example_python:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_python:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/python/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/python/foo.txt'.

You can call use the component’s --help functionality to get an overview its parameters and descriptions.

viash run config.vsh.yaml -- --help
example_r

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.

viash run config.vsh.yaml -- --input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_r:latest'
[warning] Could not pull from 'example_r:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_r:latest' with Dockerfile
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/r/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/r/foo.txt'.
[1] TRUE

You can call use the component’s --help functionality to get an overview its parameters and descriptions.

viash run config.vsh.yaml -- --help
example_scala

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

As expected, this component has an --input and --output parameter. You can execute the component by providing values for these parameters.

viash run config.vsh.yaml -- --input config.vsh.yaml --output foo.txt
[notice] Checking if Docker image is available at 'example_scala:latest'
[warning] Could not pull from 'example_scala:latest'. Docker image doesn't exist or is not accessible.
[notice] Building container 'example_scala:latest' with Dockerfile
warning: 1 deprecation; re-run with -deprecation for details
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/scala/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/scala/foo.txt'.
Note

The double dash (--) between the viash command and the arguments is used to signify the end of the arguments passed to Viash and the start of those passed to the script. If you forgot to add these, you’ll get an error similar to this:

viash run config.vsh.yaml \
  --input foo.txt \
  --output bar.txt
[scallop] Error: Unknown option 'input'

Build an executable

We will now turn the Viash component into an executable.

Use the viash build command to generate an executable:

viash build config.vsh.yaml --output target

This will generate an executable in the target/ directory:

tree
.
├── config.vsh.yaml
├── foo.txt
├── script.sh
└── target
    └── example_bash

1 directory, 4 files

Use the viash build command to generate an executable:

viash build config.vsh.yaml --output target

This will generate an executable in the target/ directory:

tree
.
├── config.vsh.yaml
├── foo.txt
├── script.csx
└── target
    └── example_csharp

1 directory, 4 files

Use the viash build command to generate an executable:

viash build config.vsh.yaml --output target

This will generate an executable in the target/ directory:

tree
.
├── config.vsh.yaml
├── foo.txt
├── script.js
└── target
    └── example_js

1 directory, 4 files

Use the viash build command to generate an executable:

viash build config.vsh.yaml --output target

This will generate an executable in the target/ directory:

tree
.
├── config.vsh.yaml
├── foo.txt
├── script.py
└── target
    └── example_python

1 directory, 4 files

Use the viash build command to generate an executable:

viash build config.vsh.yaml --output target

This will generate an executable in the target/ directory:

tree
.
├── config.vsh.yaml
├── foo.txt
├── script.R
└── target
    └── example_r

1 directory, 4 files

Use the viash build command to generate an executable:

viash build config.vsh.yaml --output target

This will generate an executable in the target/ directory:

tree
.
├── config.vsh.yaml
├── foo.txt
├── script.scala
└── target
    └── example_scala

1 directory, 4 files

Displaying the help text

It’s often useful to know what arguments an executable expects before trying to run it.

To display its documentation, run the executable with just the --help argument:

target/example_bash --help
example_bash

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

To display its documentation, run the executable with just the --help argument:

target/example_csharp --help
example_csharp

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

To display its documentation, run the executable with just the --help argument:

target/example_js --help
example_js

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

To display its documentation, run the executable with just the --help argument:

target/example_python --help
example_python

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

To display its documentation, run the executable with just the --help argument:

target/example_r --help
example_r

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

To display its documentation, run the executable with just the --help argument:

target/example_scala --help
example_scala

A minimal example component.

Arguments:
    --input
        type: file, required parameter, file must exist
        example: file.txt

    --output
        type: file, required parameter, output, file must exist
        example: output.txt

This executable takes a file as input and will create an output file.

Running the executable

Running an executable is the same as any other executable on your system.

You can run the executable by providing a value for --input and --output:

target/example_bash --input config.vsh.yaml --output output.txt
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/bash/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/bash/output.txt'.

This results in the following output:

ls -l
total 20
-rw-r--r-- 1 runner docker  428 Feb 21 14:33 config.vsh.yaml
-rw-r--r-- 1 runner docker  428 Feb 21 14:33 foo.txt
-rw-r--r-- 1 runner docker  428 Feb 21 14:35 output.txt
-rwxr-xr-x 1 runner docker  181 Feb 21 14:33 script.sh
drwxr-xr-x 2 runner docker 4096 Feb 21 14:34 target

You can run the executable by providing a value for --input and --output:

target/example_csharp --input config.vsh.yaml --output output.txt
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/csharp/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/csharp/output.txt'.

This results in the following output:

ls -l
total 20
-rw-r--r-- 1 runner docker  467 Feb 21 14:33 config.vsh.yaml
-rw-r--r-- 1 runner docker  467 Feb 21 14:33 foo.txt
-rw-r--r-- 1 runner docker  467 Feb 21 14:33 output.txt
-rw-r--r-- 1 runner docker  237 Feb 21 14:33 script.csx
drwxr-xr-x 2 runner docker 4096 Feb 21 14:34 target

You can run the executable by providing a value for --input and --output:

target/example_js --input config.vsh.yaml --output output.txt
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/js/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/js/output.txt'

This results in the following output:

ls -l
total 20
-rw-r--r-- 1 runner docker  445 Feb 21 14:33 config.vsh.yaml
-rw-r--r-- 1 runner docker  445 Feb 21 14:33 foo.txt
-rw-r--r-- 1 runner docker  445 Feb 21 14:35 output.txt
-rwxr-xr-x 1 runner docker  282 Feb 21 14:33 script.js
drwxr-xr-x 2 runner docker 4096 Feb 21 14:34 target

You can run the executable by providing a value for --input and --output:

target/example_python --input config.vsh.yaml --output output.txt
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/python/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/python/output.txt'.

This results in the following output:

ls -l
total 20
-rw-r--r-- 1 runner docker  440 Feb 21 14:33 config.vsh.yaml
-rw-r--r-- 1 runner docker  440 Feb 21 14:33 foo.txt
-rw-r--r-- 1 runner docker  440 Feb 21 14:35 output.txt
-rwxr-xr-x 1 runner docker  216 Feb 21 14:33 script.py
drwxr-xr-x 2 runner docker 4096 Feb 21 14:34 target

You can run the executable by providing a value for --input and --output:

target/example_r --input config.vsh.yaml --output output.txt
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/r/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/r/output.txt'.
[1] TRUE

This results in the following output:

ls -l
total 20
-rw-r--r-- 1 runner docker  435 Feb 21 14:33 config.vsh.yaml
-rw-r--r-- 1 runner docker  435 Feb 21 14:34 foo.txt
-rw-r--r-- 1 runner docker  435 Feb 21 14:35 output.txt
-rwxr-xr-x 1 runner docker  207 Feb 21 14:33 script.R
drwxr-xr-x 2 runner docker 4096 Feb 21 14:34 target

You can run the executable by providing a value for --input and --output:

target/example_scala --input config.vsh.yaml --output output.txt
warning: 1 deprecation; re-run with -deprecation for details
Copying '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/scala/config.vsh.yaml' to '/viash_automount/tmp/Rtmp1bwLsE/create_new_component46185a94777f/scala/output.txt'.

This results in the following output:

ls -l
total 20
-rw-r--r-- 1 runner docker  479 Feb 21 14:33 config.vsh.yaml
-rw-r--r-- 1 runner docker  479 Feb 21 14:34 foo.txt
-rw-r--r-- 1 runner docker  479 Feb 21 14:35 output.txt
-rw-r--r-- 1 runner docker  435 Feb 21 14:33 script.scala
drwxr-xr-x 2 runner docker 4096 Feb 21 14:34 target