Unit testing

How to ensure high quality components.

We recommend adding unit tests to all of your components in a project as soon as possible. Implementing unit tests not only ensures that your component works, but it also makes the project more maintainable in the long run as you will be notified when a component breaks.

This page describes how to add a unit test to your component.

Create unit test

Below is an example of how to add a unit test to a Viash component.

#!/bin/bash

## VIASH START
meta_executable="target/example_bash"
## VIASH END

echo ">>> Create input test file"
echo "foo" > foo.txt

echo ">>> Run executable"
$meta_executable --input foo.txt --output bar.txt

echo ">>> Check whether output file exists"
[[ -f bar.txt ]] (echo "Output file could not be found!" && exit 1)

echo ">>> Check whether input and output file are the same"
cmp foo.txt bar.tx (echo "Input and output files are different!" && exit 1)

echo ">>> Test finished successfully"
using System;
using System.IO;
using System.Diagnostics;

// VIASH START
var meta = new {
  executable = "target/example_csharp"
};
// VIASH END

string inputPath = "foo.txt";
string outputPath = "bar.txt";
string content = "hello\nthere\n";

Console.WriteLine(">>> Create input test file");
File.WriteAllText(inputPath, content);

Console.WriteLine(">>> Run executable");

var startInfo = new ProcessStartInfo(meta.executable)
{
    Arguments = $"--input {inputPath} --output {outputPath}",
    UseShellExecute = false,
    RedirectStandardOutput = true,
    CreateNoWindow = true
};
using(var cmd = Process.Start(startInfo))
{
    cmd.WaitForExit();
}


Console.WriteLine(">>> Check whether output file exists");
if (!File.Exists(outputPath)) {
    Console.WriteLine("Output file was not found");
    Environment.Exit(1);
}

Console.WriteLine(">>> Check whether input and output file are the same");
var outputLines = File.ReadAllText(outputPath);
if (content != outputLines) {
    Console.WriteLine(
        "Input and output should be the same\n" +
        $"expected content: {content}\n" +
        $"found: {outputLines}\n"
    );
    Environment.Exit(1);
}

Console.WriteLine(">>> Test finished successfully");
const fs = require('fs');
const child_process = require('child_process');

// VIASH START
let meta = {
  'executable': 'target/example_js'
};
// VIASH END

const inputPath = 'foo.txt';
const outputPath = 'bar.txt';
const content = 'hello\nthere\n';

console.log('>>> Create input test file');
fs.writeFileSync(inputPath, content, 'utf8');

console.log('>>> Run executable');
const cmdArgs = [
  '--input', inputPath,
  '--output', outputPath
];
const child = child_process.spawnSync(meta["executable"], cmdArgs);
if (child.error) {
  console.error(`Error: ${child.error}`);
  process.exit(1);
}

console.log('>>> Check whether output file exists');
if (!fs.existsSync(outputPath)) {
  console.error('Output file was not found');
  process.exit(1);
}

console.log('>>> Check whether input and output file are the same');
const outputLines = fs.readFileSync(outputPath, 'utf8');
if (content !== outputLines) {
  console.error(
    `Input and output should be the same\n` +
    `expected content: ${content}\n` +
    `found: ${outputLines}\n`
  );
  process.exit(1);
}

console.log('>>> Test finished successfully');
import subprocess
import os

input_path = "foo.txt"
output_path = "bar.txt"
content = "hello\nthere\n"

## VIASH START
meta = {
  "executable": "target/example_python"
}
## VIASH END

print(">>> Create input test file")
with open(input_path, "w") as file:
  file.write(content)

print(">>> Run executable")
cmd_args = [
  meta["executable"],
  "--input", input_path,
  "--output", output_path
]
subprocess.run(cmd_args, check=True)

print(">>> Check whether output file exists")
assert os.path.exists(output_path), "Output file was not found"

print(">>> Check whether input and output file are the same")
with open(output_path, "r", encoding="utf8") as file:
  output_lines = file.read()

assert content == output_lines, \
  "Input and output should be the same" \
  f"expected content: {content}" \
  f"found: {output_lines}"

print(">>> Test finished successfully")
## VIASH START
meta <- list(
  "target/example_r"
)
## VIASH END

input_path <- "foo.txt"
output_path <- "bar.txt"
content <- c("hello", "there")

cat(">>> Create input test file\n")
writeLines(content, input_path)

cat(">>> Run executable\n")
system2(
  meta$executable,
  c(
    "--input", input_path,
    "--output", output_path
  )
)

cat(">>> Check whether output file exists\n")
if (!file.exists(output_path)) {
  stop("Output file was not found")
}

cat(">>> Check whether input and output file are the same\n")
output_lines <- readLines(output_path)

if (!identical(content, output_lines)) {
  stop(paste0(
    "Input and output should be the same\n",
    "expected content: ", content, "\n",
    "found: ", output_lines, "\n"
  ))
}

cat(">>> Test finished successfully\n")
import scala.sys.process._
import java.nio.file.{Files, Paths}
import scala.io.Source

// VIASH START
case class ViashMeta(executable: String)
val meta = ViashMeta(
  "target/example_scala"
)
// VIASH END
val inputPath = Paths.get("foo.txt")
val outputPath = Paths.get("bar.txt")
val content = "hello\nthere"

println(">>> Create input test file")
Files.write(inputPath, content.getBytes("UTF-8"))

println(">>> Run executable")
s"${meta.executable} --input $inputPath --output $outputPath".!

println(">>> Check whether output file exists")
assert(Files.exists(outputPath), "Output file not found")

println(">>> Check whether input and output file are the same")
val outputLines = Source.fromFile(outputPath.toFile()).getLines.mkString("\n")
assert(
  content == outputLines, 
  s"""Output not the same
expected: '$content'
found: '$outputLines'
""".stripMargin
)

println(">>> Test finished successfully")
  1. Create an input file “foo.txt” with some sample content.
  2. Run an executable file specified in the “meta” dictionary with the arguments “–input foo.txt” and “–output bar.txt”.
  3. Check whether the output file “bar.txt” exists.
  4. Check whether the content of the output file is the same as the content of the input file.
  5. If both checks pass, the script prints “Test finished successfully”.
Important

A test script doesn’t need to be written in the same scripting language as the main script, as long as all of the required dependencies are available. This means the main script could be written in R, but the unit test could be written in Bash.

Add test to config

Next, we need to add the unit test to the test_resources section in the Viash config.

name: example_bash
description: A minimal example component.
arguments:
  - type: file
    name: --input
    example: file.txt
    required: true
  - type: file
    name: --output
    direction: output
    example: output.txt
    required: true
resources:
  - type: bash_script
    path: script.sh
engines:
  - type: docker
    image: bash:4.0
  - type: native
runners:
  - type: executable
  - type: nextflow
test_resources:
  - type: bash_script
    path: test.sh
name: example_csharp
description: A minimal example component.
arguments:
  - type: file
    name: --input
    example: file.txt
    required: true
  - type: file
    name: --output
    direction: output
    example: output.txt
    required: true
resources:
  - type: csharp_script
    path: script.csx
engines:
  - type: docker
    image: ghcr.io/data-intuitive/dotnet-script:1.3.1
  - type: native
runners:
  - type: executable
  - type: nextflow
test_resources:
  - type: csharp_script
    path: test.csx
name: example_js
description: A minimal example component.
arguments:
  - type: file
    name: --input
    example: file.txt
    required: true
  - type: file
    name: --output
    direction: output
    example: output.txt
    required: true
resources:
  - type: javascript_script
    path: script.js
engines:
  - type: docker
    image: node:19-bullseye-slim
  - type: native
runners:
  - type: executable
  - type: nextflow
test_resources:
  - type: javascript_script
    path: test.js
name: example_python
description: A minimal example component.
arguments:
  - type: file
    name: --input
    example: file.txt
    required: true
  - type: file
    name: --output
    direction: output
    example: output.txt
    required: true
resources:
  - type: python_script
    path: script.py
engines:
  - type: docker
    image: python:3.10-slim
  - type: native
runners:
  - type: executable
  - type: nextflow
test_resources:
  - type: python_script
    path: test.py
name: example_r
description: A minimal example component.
arguments:
  - type: file
    name: --input
    example: file.txt
    required: true
  - type: file
    name: --output
    direction: output
    example: output.txt
    required: true
resources:
  - type: r_script
    path: script.R
engines:
  - type: docker
    image: eddelbuettel/r2u:22.04
  - type: native
runners:
  - type: executable
  - type: nextflow
test_resources:
  - type: r_script
    path: test.R
name: example_scala
description: A minimal example component.
arguments:
  - type: file
    name: --input
    example: file.txt
    required: true
  - type: file
    name: --output
    direction: output
    example: output.txt
    required: true
resources:
  - type: scala_script
    path: script.scala
engines:
  - type: docker
    image: sbtscala/scala-sbt:eclipse-temurin-19_36_1.7.2_2.13.10
  - type: native
runners:
  - type: executable
  - type: nextflow
test_resources:
  - type: scala_script
    path: test.scala

Note that if you can add multiple unit tests to the test_resources section, they will all be evaluated. You can also add other resources, similar to what is described in the page on adding resources.

Run the unit test

You can run the unit test as follows:

viash test config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_example_bash_1470395744433269633'
====================================================================
+/tmp/viash_test_example_bash_1470395744433269633/build_engine_environment/example_bash ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'example_bash:test' with Dockerfile
[info] docker build -t 'example_bash:test'  '/tmp/viash_test_example_bash_1470395744433269633/build_engine_environment' -f '/tmp/viash_test_example_bash_1470395744433269633/build_engine_environment/tmp/dockerbuild-example_bash-luXTQ9/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 276B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/bash:4.0
#2 DONE 0.5s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/1] FROM docker.io/library/bash:4.0@sha256:8c3543551fde9ee9e5598b641e615f3e78468f10e9219f83b136c41e649d8377
#4 resolve docker.io/library/bash:4.0@sha256:8c3543551fde9ee9e5598b641e615f3e78468f10e9219f83b136c41e649d8377 done
#4 extracting sha256:f7ee36c9aa34bbb665f975c76e5c0d1607f0674b94c84cfb0061f87006ea5d10
#4 sha256:f7ee36c9aa34bbb665f975c76e5c0d1607f0674b94c84cfb0061f87006ea5d10 3.79MB / 3.79MB 0.0s done
#4 sha256:642896822c35bdebe202256b2a1dd5f513d341d2b0b82297847b03063e328188 1.59kB / 1.59kB 0.0s done
#4 sha256:374a69975c29a767c60f9b064d012c6ea383bd8bbdb37779712e4e17a29de347 693.60kB / 693.60kB 0.0s done
#4 sha256:e9ab7edf3fc61a46c9d00084418f104282d7bedd0a2802d5398864bfbafd9179 337B / 337B 0.0s done
#4 sha256:8c3543551fde9ee9e5598b641e615f3e78468f10e9219f83b136c41e649d8377 10.06kB / 10.06kB done
#4 extracting sha256:f7ee36c9aa34bbb665f975c76e5c0d1607f0674b94c84cfb0061f87006ea5d10 0.1s done
#4 sha256:e092a2a0a266410f2236fce184a9b2fd9b5ad7f851f6cb01916b2f0c6417e4eb 1.70kB / 1.70kB done
#4 sha256:8cd1e2177265b4834dc31b304bc2e534d496df63840e4dc93593dd7c0d745e01 5.58kB / 5.58kB done
#4 extracting sha256:642896822c35bdebe202256b2a1dd5f513d341d2b0b82297847b03063e328188
#4 extracting sha256:642896822c35bdebe202256b2a1dd5f513d341d2b0b82297847b03063e328188 done
#4 extracting sha256:374a69975c29a767c60f9b064d012c6ea383bd8bbdb37779712e4e17a29de347 0.0s done
#4 extracting sha256:e9ab7edf3fc61a46c9d00084418f104282d7bedd0a2802d5398864bfbafd9179 done
#4 DONE 0.4s

#5 exporting to image
#5 exporting layers done
#5 writing image sha256:e360cd7738f72ffbd1852f2ea43682b5c007e8bb4a62e4a391778d1e462834b0 done
#5 naming to docker.io/library/example_bash:test done
#5 DONE 0.0s
====================================================================
+/tmp/viash_test_example_bash_1470395744433269633/test_test/test_executable
>>> Create input test file
>>> Run executable
Copying 'foo.txt' to 'bar.txt'.
>>> Check whether output file exists
>>> Check whether input and output file are the same
>>> Test finished successfully
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory
viash test config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_example_csharp_13949381452347629464'
====================================================================
+/tmp/viash_test_example_csharp_13949381452347629464/build_engine_environment/example_csharp ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'example_csharp:test' with Dockerfile
[info] docker build -t 'example_csharp:test'  '/tmp/viash_test_example_csharp_13949381452347629464/build_engine_environment' -f '/tmp/viash_test_example_csharp_13949381452347629464/build_engine_environment/tmp/dockerbuild-example_csharp-gamiqP/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 312B done
#1 DONE 0.0s

#2 [internal] load metadata for ghcr.io/data-intuitive/dotnet-script:1.3.1
#2 DONE 0.8s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/1] FROM ghcr.io/data-intuitive/dotnet-script:1.3.1@sha256:d95bf2a2b3e095e5b02960ab4ddd4674d798a1493ed0dc5cf427c77a5ed90ed3
#4 resolve ghcr.io/data-intuitive/dotnet-script:1.3.1@sha256:d95bf2a2b3e095e5b02960ab4ddd4674d798a1493ed0dc5cf427c77a5ed90ed3 done
#4 extracting sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139
#4 sha256:1102243946aeba437ebb7c7ab5559d465d660ed57a1f13888c89692401cd5bb6 7.71kB / 7.71kB done
#4 sha256:d772c6b09cca24a6ec7cb06facff8b7be509b608330bd035f91b70388a94d3b4 28.31MB / 31.26MB 0.1s
#4 sha256:2c56128c63bdf4a1d571ed930304f4eb77e8e2fde7ce03ddf6073e3df9abf8ad 2.10MB / 8.65MB 0.1s
#4 sha256:6c2be99410893469a9e0d8f0ff537b22f651479bf64c5dcf269d2e5267e85749 0B / 20.91MB 0.1s
#4 sha256:d95bf2a2b3e095e5b02960ab4ddd4674d798a1493ed0dc5cf427c77a5ed90ed3 856B / 856B done
#4 sha256:ae4d45b87ee640aec6f031a1695c6d814e0026ea6fb3220e3daaad90b0f09bda 2.21kB / 2.21kB done
#4 sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139 2.81MB / 2.81MB 0.1s done
#4 sha256:3ae3a38ed752842e75d1609c931b29e565b53c24229c87bf1ba64251b5f1f094 1.74MB / 1.74MB 0.1s done
#4 extracting sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139 0.1s done
#4 sha256:d772c6b09cca24a6ec7cb06facff8b7be509b608330bd035f91b70388a94d3b4 31.26MB / 31.26MB 0.3s
#4 sha256:2c56128c63bdf4a1d571ed930304f4eb77e8e2fde7ce03ddf6073e3df9abf8ad 8.65MB / 8.65MB 0.3s
#4 sha256:6c2be99410893469a9e0d8f0ff537b22f651479bf64c5dcf269d2e5267e85749 20.91MB / 20.91MB 0.3s
#4 sha256:d772c6b09cca24a6ec7cb06facff8b7be509b608330bd035f91b70388a94d3b4 31.26MB / 31.26MB 0.5s done
#4 extracting sha256:3ae3a38ed752842e75d1609c931b29e565b53c24229c87bf1ba64251b5f1f094 0.1s done
#4 sha256:2c56128c63bdf4a1d571ed930304f4eb77e8e2fde7ce03ddf6073e3df9abf8ad 8.65MB / 8.65MB 0.5s done
#4 sha256:6c2be99410893469a9e0d8f0ff537b22f651479bf64c5dcf269d2e5267e85749 20.91MB / 20.91MB 0.5s done
#4 sha256:5a0c86bcd85d690fae1d36f59c4041c4142835310a431e3f1b00446763d7db73 105.91MB / 105.91MB 0.8s
#4 sha256:4937da28ea60b32af53a8d507ea830228a05f22a7be2ae1ef0134bcd0e3bafba 68.90MB / 68.90MB 0.8s
#4 sha256:5e733ff9c30825005ec05ee10b784276e8034171be7eeaa10d6f876fc7bb5383 12.80MB / 12.80MB 0.6s done
#4 sha256:16def9f17d8d9988a074a1d3ae42cb041b261068b3a455c99141bb8df243e9a4 829.24kB / 829.24kB 0.7s done
#4 sha256:03ac08f8f1d3c646d8f456ad0e2307b56511e692bcef7a4d67ec194ba81f59e7 0B / 103B 1.1s
#4 sha256:03ac08f8f1d3c646d8f456ad0e2307b56511e692bcef7a4d67ec194ba81f59e7 103B / 103B 1.3s
#4 sha256:5a0c86bcd85d690fae1d36f59c4041c4142835310a431e3f1b00446763d7db73 105.91MB / 105.91MB 1.5s done
#4 sha256:4937da28ea60b32af53a8d507ea830228a05f22a7be2ae1ef0134bcd0e3bafba 68.90MB / 68.90MB 1.5s done
#4 sha256:03ac08f8f1d3c646d8f456ad0e2307b56511e692bcef7a4d67ec194ba81f59e7 103B / 103B 1.5s done
#4 extracting sha256:d772c6b09cca24a6ec7cb06facff8b7be509b608330bd035f91b70388a94d3b4 0.1s
#4 extracting sha256:d772c6b09cca24a6ec7cb06facff8b7be509b608330bd035f91b70388a94d3b4 0.3s done
#4 extracting sha256:2c56128c63bdf4a1d571ed930304f4eb77e8e2fde7ce03ddf6073e3df9abf8ad
#4 extracting sha256:2c56128c63bdf4a1d571ed930304f4eb77e8e2fde7ce03ddf6073e3df9abf8ad 0.1s done
#4 extracting sha256:6c2be99410893469a9e0d8f0ff537b22f651479bf64c5dcf269d2e5267e85749
#4 extracting sha256:6c2be99410893469a9e0d8f0ff537b22f651479bf64c5dcf269d2e5267e85749 0.2s done
#4 extracting sha256:5a0c86bcd85d690fae1d36f59c4041c4142835310a431e3f1b00446763d7db73
#4 extracting sha256:5a0c86bcd85d690fae1d36f59c4041c4142835310a431e3f1b00446763d7db73 1.5s done
#4 extracting sha256:5e733ff9c30825005ec05ee10b784276e8034171be7eeaa10d6f876fc7bb5383
#4 extracting sha256:5e733ff9c30825005ec05ee10b784276e8034171be7eeaa10d6f876fc7bb5383 0.2s done
#4 extracting sha256:4937da28ea60b32af53a8d507ea830228a05f22a7be2ae1ef0134bcd0e3bafba
#4 extracting sha256:4937da28ea60b32af53a8d507ea830228a05f22a7be2ae1ef0134bcd0e3bafba 0.4s done
#4 extracting sha256:16def9f17d8d9988a074a1d3ae42cb041b261068b3a455c99141bb8df243e9a4
#4 extracting sha256:16def9f17d8d9988a074a1d3ae42cb041b261068b3a455c99141bb8df243e9a4 0.0s done
#4 extracting sha256:03ac08f8f1d3c646d8f456ad0e2307b56511e692bcef7a4d67ec194ba81f59e7 done
#4 DONE 8.3s

#5 exporting to image
#5 exporting layers done
#5 writing image sha256:86b4f2dd2468dea49d39e786ffc5e447b14ee13ecb836e1b9a0c7906a4eac749 done
#5 naming to docker.io/library/example_csharp:test done
#5 DONE 0.0s
====================================================================
+/tmp/viash_test_example_csharp_13949381452347629464/test_test/test_executable
>>> Create input test file
>>> Run executable
>>> Check whether output file exists
>>> Check whether input and output file are the same
>>> Test finished successfully
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory
viash test config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_example_js_12120548170414026125'
====================================================================
+/tmp/viash_test_example_js_12120548170414026125/build_engine_environment/example_js ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'example_js:test' with Dockerfile
[info] docker build -t 'example_js:test'  '/tmp/viash_test_example_js_12120548170414026125/build_engine_environment' -f '/tmp/viash_test_example_js_12120548170414026125/build_engine_environment/tmp/dockerbuild-example_js-8mc4v6/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 287B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/node:19-bullseye-slim
#2 DONE 0.4s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/1] FROM docker.io/library/node:19-bullseye-slim@sha256:f58f1fcf5c9ff9e3752993edb4ed6dbd35697124c85a43f3b97aa054500b0534
#4 resolve docker.io/library/node:19-bullseye-slim@sha256:f58f1fcf5c9ff9e3752993edb4ed6dbd35697124c85a43f3b97aa054500b0534 done
#4 sha256:f58f1fcf5c9ff9e3752993edb4ed6dbd35697124c85a43f3b97aa054500b0534 1.21kB / 1.21kB done
#4 sha256:63b5ca0f90dbaf64465a83d46276e133eab30410f642808d158bf98928811999 1.37kB / 1.37kB done
#4 sha256:adbc02574b521bb652d771c8ac0282c724b9fe90c001f3bab0825c32092c9036 6.84kB / 6.84kB done
#4 sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1 1.05MB / 31.40MB 0.2s
#4 sha256:0bfad7312c25e414fc3d0e3640a77bd4b5c74d2118595670999afef9758cb49b 4.19kB / 4.19kB 0.0s done
#4 sha256:56db04c463e993ac037103947cd590d52fcddcb17ad04285a0ca4c3510c0738c 0B / 46.62MB 0.2s
#4 sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1 31.40MB / 31.40MB 0.3s
#4 sha256:56db04c463e993ac037103947cd590d52fcddcb17ad04285a0ca4c3510c0738c 45.09MB / 46.62MB 0.3s
#4 sha256:477c7ebfb8500396b86c69952acb17b909a0a63da41e03dd4046e2c809722c6e 2.76MB / 2.76MB 0.3s
#4 sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1 31.40MB / 31.40MB 0.5s done
#4 sha256:56db04c463e993ac037103947cd590d52fcddcb17ad04285a0ca4c3510c0738c 46.62MB / 46.62MB 0.8s done
#4 sha256:477c7ebfb8500396b86c69952acb17b909a0a63da41e03dd4046e2c809722c6e 2.76MB / 2.76MB 0.8s done
#4 extracting sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1 0.1s
#4 sha256:8bb4aa3e51be63d4a2e5bd30ab9cd478a916ed3c5c0212a44b96fd743e0e0f2f 450B / 450B 0.8s done
#4 extracting sha256:f03b40093957615593f2ed142961afb6b540507e0b47e3f7626ba5e02efbbbf1 0.9s done
#4 extracting sha256:0bfad7312c25e414fc3d0e3640a77bd4b5c74d2118595670999afef9758cb49b
#4 extracting sha256:0bfad7312c25e414fc3d0e3640a77bd4b5c74d2118595670999afef9758cb49b done
#4 extracting sha256:56db04c463e993ac037103947cd590d52fcddcb17ad04285a0ca4c3510c0738c
#4 extracting sha256:56db04c463e993ac037103947cd590d52fcddcb17ad04285a0ca4c3510c0738c 1.5s done
#4 extracting sha256:477c7ebfb8500396b86c69952acb17b909a0a63da41e03dd4046e2c809722c6e
#4 extracting sha256:477c7ebfb8500396b86c69952acb17b909a0a63da41e03dd4046e2c809722c6e 0.4s done
#4 extracting sha256:8bb4aa3e51be63d4a2e5bd30ab9cd478a916ed3c5c0212a44b96fd743e0e0f2f
#4 extracting sha256:8bb4aa3e51be63d4a2e5bd30ab9cd478a916ed3c5c0212a44b96fd743e0e0f2f done
#4 DONE 5.1s

#5 exporting to image
#5 exporting layers done
#5 writing image sha256:9f286a64588bc58f12a060d6fc2872f5baf9c7686a47e675cec3e8d006741dfc done
#5 naming to docker.io/library/example_js:test done
#5 DONE 0.0s
====================================================================
+/tmp/viash_test_example_js_12120548170414026125/test_test/test_executable
>>> Create input test file
>>> Run executable
>>> Check whether output file exists
>>> Check whether input and output file are the same
>>> Test finished successfully
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory
viash test config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_example_python_8032911729364741382'
====================================================================
+/tmp/viash_test_example_python_8032911729364741382/build_engine_environment/example_python ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'example_python:test' with Dockerfile
[info] docker build -t 'example_python:test'  '/tmp/viash_test_example_python_8032911729364741382/build_engine_environment' -f '/tmp/viash_test_example_python_8032911729364741382/build_engine_environment/tmp/dockerbuild-example_python-x1d94g/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 286B done
#1 DONE 0.0s

#2 [auth] library/python:pull token for registry-1.docker.io
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/python:3.10-slim
#3 DONE 1.0s

#4 [internal] load .dockerignore
#4 transferring context:
#4 transferring context: 2B done
#4 DONE 0.4s

#5 [1/1] FROM docker.io/library/python:3.10-slim@sha256:fd76ade0c607f27677bc04be3c60749f400eedc941d9e72967e19a4cedff80c2
#5 resolve docker.io/library/python:3.10-slim@sha256:fd76ade0c607f27677bc04be3c60749f400eedc941d9e72967e19a4cedff80c2 done
#5 sha256:8c97ebedc32fd60935cdf5992e935753e2a0f98231830028050e1e04bd3c13c2 1.75kB / 1.75kB done
#5 sha256:db825c749364f7f06c8fb165b360d9de94ba5c14727e2e446c1b4202a30224e7 5.49kB / 5.49kB done
#5 sha256:6310eb16bf4251731feab01e8f633bf5e2d75a657ccad97f420b1f83cce457be 22.02MB / 29.79MB 0.1s
#5 sha256:4bfad29f0e4c54033a1172e7cd01a051cef0fd435105fde568297cccc948e3cc 3.15MB / 4.27MB 0.1s
#5 sha256:33fd700e5760ffc8c4f8306b7e77aeb159d838d54c855d673df9e81a8ac3f620 3.15MB / 13.91MB 0.1s
#5 sha256:fd76ade0c607f27677bc04be3c60749f400eedc941d9e72967e19a4cedff80c2 10.37kB / 10.37kB done
#5 sha256:6310eb16bf4251731feab01e8f633bf5e2d75a657ccad97f420b1f83cce457be 29.79MB / 29.79MB 0.2s
#5 sha256:4bfad29f0e4c54033a1172e7cd01a051cef0fd435105fde568297cccc948e3cc 4.27MB / 4.27MB 0.2s
#5 sha256:33fd700e5760ffc8c4f8306b7e77aeb159d838d54c855d673df9e81a8ac3f620 13.91MB / 13.91MB 0.2s
#5 sha256:6310eb16bf4251731feab01e8f633bf5e2d75a657ccad97f420b1f83cce457be 29.79MB / 29.79MB 0.3s done
#5 sha256:4bfad29f0e4c54033a1172e7cd01a051cef0fd435105fde568297cccc948e3cc 4.27MB / 4.27MB 0.3s done
#5 sha256:33fd700e5760ffc8c4f8306b7e77aeb159d838d54c855d673df9e81a8ac3f620 13.91MB / 13.91MB 0.3s done
#5 extracting sha256:6310eb16bf4251731feab01e8f633bf5e2d75a657ccad97f420b1f83cce457be 0.1s
#5 sha256:9feb9947eea2d4a7b84504a6ef5ba25f1f86ec5626260c269297140f777beaa0 250B / 250B 0.3s done
#5 extracting sha256:6310eb16bf4251731feab01e8f633bf5e2d75a657ccad97f420b1f83cce457be 0.7s done
#5 extracting sha256:4bfad29f0e4c54033a1172e7cd01a051cef0fd435105fde568297cccc948e3cc 0.1s
#5 extracting sha256:4bfad29f0e4c54033a1172e7cd01a051cef0fd435105fde568297cccc948e3cc 0.1s done
#5 extracting sha256:33fd700e5760ffc8c4f8306b7e77aeb159d838d54c855d673df9e81a8ac3f620
#5 extracting sha256:33fd700e5760ffc8c4f8306b7e77aeb159d838d54c855d673df9e81a8ac3f620 0.5s done
#5 extracting sha256:9feb9947eea2d4a7b84504a6ef5ba25f1f86ec5626260c269297140f777beaa0
#5 extracting sha256:9feb9947eea2d4a7b84504a6ef5ba25f1f86ec5626260c269297140f777beaa0 done
#5 DONE 2.3s

#6 exporting to image
#6 exporting layers done
#6 writing image sha256:820923277ff697f5882541544f55311d768dc432dff23af16a9d2856f19373e3 done
#6 naming to docker.io/library/example_python:test done
#6 DONE 0.0s
====================================================================
+/tmp/viash_test_example_python_8032911729364741382/test_test/test_executable
Copying 'foo.txt' to 'bar.txt'.
>>> Create input test file
>>> Run executable
>>> Check whether output file exists
>>> Check whether input and output file are the same
>>> Test finished successfully
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory
viash test config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_example_r_5971090497479475884'
====================================================================
+/tmp/viash_test_example_r_5971090497479475884/build_engine_environment/example_r ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'example_r:test' with Dockerfile
[info] docker build -t 'example_r:test'  '/tmp/viash_test_example_r_5971090497479475884/build_engine_environment' -f '/tmp/viash_test_example_r_5971090497479475884/build_engine_environment/tmp/dockerbuild-example_r-mPs4br/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 287B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/eddelbuettel/r2u:22.04
#2 DONE 0.3s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/1] FROM docker.io/eddelbuettel/r2u:22.04@sha256:e71d4bd06713b2048f3725a5a06e987ca709fa1b33e730cda653e3f8db49a0d1
#4 resolve docker.io/eddelbuettel/r2u:22.04@sha256:e71d4bd06713b2048f3725a5a06e987ca709fa1b33e730cda653e3f8db49a0d1 done
#4 sha256:e71d4bd06713b2048f3725a5a06e987ca709fa1b33e730cda653e3f8db49a0d1 954B / 954B done
#4 sha256:3330f05f29b99d58110fa89411ef71a68de97574d65d829f392131e6892c9f10 5.87kB / 5.87kB done
#4 sha256:e96e057aae67380a4ddb16c337c5c3669d97fdff69ec537f02aa2cc30d814281 30.43MB / 30.43MB 0.1s
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 9.44MB / 235.01MB 0.1s
#4 sha256:e96e057aae67380a4ddb16c337c5c3669d97fdff69ec537f02aa2cc30d814281 30.43MB / 30.43MB 0.2s done
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 52.43MB / 235.01MB 0.2s
#4 sha256:af18caf2c1a90d5f23609a5d79925507c9adcd58c4e867391d943a7157e01a54 3.15MB / 72.25MB 0.2s
#4 extracting sha256:e96e057aae67380a4ddb16c337c5c3669d97fdff69ec537f02aa2cc30d814281
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 91.23MB / 235.01MB 0.3s
#4 sha256:af18caf2c1a90d5f23609a5d79925507c9adcd58c4e867391d943a7157e01a54 44.04MB / 72.25MB 0.3s
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 132.12MB / 235.01MB 0.4s
#4 sha256:af18caf2c1a90d5f23609a5d79925507c9adcd58c4e867391d943a7157e01a54 72.25MB / 72.25MB 0.4s
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 187.70MB / 235.01MB 0.5s
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 235.01MB / 235.01MB 0.6s
#4 extracting sha256:e96e057aae67380a4ddb16c337c5c3669d97fdff69ec537f02aa2cc30d814281 0.8s done
#4 sha256:af18caf2c1a90d5f23609a5d79925507c9adcd58c4e867391d943a7157e01a54 72.25MB / 72.25MB 1.1s done
#4 sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 235.01MB / 235.01MB 1.8s done
#4 extracting sha256:af18caf2c1a90d5f23609a5d79925507c9adcd58c4e867391d943a7157e01a54 0.1s
#4 extracting sha256:af18caf2c1a90d5f23609a5d79925507c9adcd58c4e867391d943a7157e01a54 2.1s done
#4 extracting sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6
#4 extracting sha256:aa48af1f9f911d7fdf489e254463aa5b8041f3ded7add5f65c86127c0d4acab6 4.0s done
#4 DONE 11.7s

#5 exporting to image
#5 exporting layers done
#5 writing image sha256:b847079fe95369e35353c38667cc842c04b7dc2408251466612522a4d9921f17 done
#5 naming to docker.io/library/example_r:test done
#5 DONE 0.0s
====================================================================
+/tmp/viash_test_example_r_5971090497479475884/test_test/test_executable
>>> Create input test file
>>> Run executable
Copying 'foo.txt' to 'bar.txt'.
[1] TRUE
>>> Check whether output file exists
>>> Check whether input and output file are the same
>>> Test finished successfully
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory
viash test config.vsh.yaml
Running tests in temporary directory: '/tmp/viash_test_example_scala_15438672778403597665'
====================================================================
+/tmp/viash_test_example_scala_15438672778403597665/build_engine_environment/example_scala ---verbosity 6 ---setup cachedbuild ---engine docker
[notice] Building container 'example_scala:test' with Dockerfile
[info] docker build -t 'example_scala:test'  '/tmp/viash_test_example_scala_15438672778403597665/build_engine_environment' -f '/tmp/viash_test_example_scala_15438672778403597665/build_engine_environment/tmp/dockerbuild-example_scala-gwByDq/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 323B done
#1 DONE 0.0s

#2 [auth] sbtscala/scala-sbt:pull token for registry-1.docker.io
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/sbtscala/scala-sbt:eclipse-temurin-19_36_1.7.2_2.13.10
#3 DONE 0.8s

#4 [internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [1/1] FROM docker.io/sbtscala/scala-sbt:eclipse-temurin-19_36_1.7.2_2.13.10@sha256:440234e283754375d5ab6bcb35e2d470f9dabeaaeb639d8c0708fac2d3707e0c
#5 resolve docker.io/sbtscala/scala-sbt:eclipse-temurin-19_36_1.7.2_2.13.10@sha256:440234e283754375d5ab6bcb35e2d470f9dabeaaeb639d8c0708fac2d3707e0c done
#5 sha256:76808e2ea8df2d9cac1d275426e51a2190369f706076c55ac2da3823bef0a019 10.89kB / 10.89kB done
#5 sha256:c5c735a83dbd58e1c83f8d52abdc51068e27c00405f53d59802d4954a6b5398a 8.39MB / 16.99MB 0.1s
#5 sha256:440234e283754375d5ab6bcb35e2d470f9dabeaaeb639d8c0708fac2d3707e0c 743B / 743B done
#5 sha256:301a8b74f71f85f3a31e9c7e7fedd5b001ead5bcf895bc2911c1d260e06bd987 30.43MB / 30.43MB 0.1s
#5 sha256:06478611ba16efc6663ccf5f22a29c990c53aed6848bc58c54c67dc32d8aef13 3.04kB / 3.04kB done
#5 sha256:c5c735a83dbd58e1c83f8d52abdc51068e27c00405f53d59802d4954a6b5398a 16.99MB / 16.99MB 0.2s
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 1.05MB / 200.85MB 0.3s
#5 extracting sha256:301a8b74f71f85f3a31e9c7e7fedd5b001ead5bcf895bc2911c1d260e06bd987
#5 sha256:c5c735a83dbd58e1c83f8d52abdc51068e27c00405f53d59802d4954a6b5398a 16.99MB / 16.99MB 0.3s done
#5 sha256:301a8b74f71f85f3a31e9c7e7fedd5b001ead5bcf895bc2911c1d260e06bd987 30.43MB / 30.43MB 0.3s done
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 44.04MB / 200.85MB 0.4s
#5 sha256:14d68137d870fdc4bc849fbe5c5571ca3e6d9a7b2c5bf1c7e928ccf716dd3b4a 3.15MB / 29.75MB 0.4s
#5 sha256:2af8e2a543d81b57575a2e457bd1e98ffc061b15639dac3be5ff6c2702b9ea1a 175B / 175B 0.4s done
#5 sha256:320659737461e071e3d809231861a7487daaa79805c57f97015ff020f34a939a 0B / 23.71MB 0.4s
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 95.42MB / 200.85MB 0.5s
#5 sha256:14d68137d870fdc4bc849fbe5c5571ca3e6d9a7b2c5bf1c7e928ccf716dd3b4a 28.31MB / 29.75MB 0.5s
#5 sha256:320659737461e071e3d809231861a7487daaa79805c57f97015ff020f34a939a 19.92MB / 23.71MB 0.5s
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 136.31MB / 200.85MB 0.7s
#5 sha256:320659737461e071e3d809231861a7487daaa79805c57f97015ff020f34a939a 23.71MB / 23.71MB 0.7s
#5 sha256:14d68137d870fdc4bc849fbe5c5571ca3e6d9a7b2c5bf1c7e928ccf716dd3b4a 29.75MB / 29.75MB 0.8s done
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 174.06MB / 200.85MB 0.9s
#5 sha256:320659737461e071e3d809231861a7487daaa79805c57f97015ff020f34a939a 23.71MB / 23.71MB 0.8s done
#5 sha256:c9480a4b593fb5abc117a5e01c80d8495c9fd89e743757730c712545f425c9f3 183B / 183B 0.9s done
#5 sha256:812216efaf878f5ff8086b6efdccec1cb090228107efd62db47d048c927c4f6a 0B / 4.33kB 0.9s
#5 sha256:1dea7716181a505cea41832dc84691b858a8f64471cb352d8c29cec06e428aeb 18.87MB / 40.48MB 0.9s
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 200.85MB / 200.85MB 1.1s
#5 sha256:812216efaf878f5ff8086b6efdccec1cb090228107efd62db47d048c927c4f6a 4.33kB / 4.33kB 0.9s done
#5 sha256:1dea7716181a505cea41832dc84691b858a8f64471cb352d8c29cec06e428aeb 40.48MB / 40.48MB 1.1s
#5 sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 42.99MB / 214.15MB 1.1s
#5 sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B 0.9s done
#5 sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 74.45MB / 214.15MB 1.2s
#5 sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 108.00MB / 214.15MB 1.3s
#5 extracting sha256:301a8b74f71f85f3a31e9c7e7fedd5b001ead5bcf895bc2911c1d260e06bd987 1.1s done
#5 sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 173.02MB / 214.15MB 1.5s
#5 sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 214.15MB / 214.15MB 1.6s
#5 sha256:1dea7716181a505cea41832dc84691b858a8f64471cb352d8c29cec06e428aeb 40.48MB / 40.48MB 1.9s done
#5 extracting sha256:c5c735a83dbd58e1c83f8d52abdc51068e27c00405f53d59802d4954a6b5398a
#5 sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 200.85MB / 200.85MB 3.2s done
#5 sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 214.15MB / 214.15MB 3.2s done
#5 sha256:2c0021742fe2b9491f62c3d89da10492c175081bd17d77efd0f22a298c37ad64 222B / 222B 3.2s done
#5 extracting sha256:c5c735a83dbd58e1c83f8d52abdc51068e27c00405f53d59802d4954a6b5398a 0.7s done
#5 extracting sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b
#5 extracting sha256:5fb9fa7513e6a72bf7daf0a75e97c441f1ed7ff69c8609d1c429fdf3a6dc846b 1.2s done
#5 extracting sha256:2af8e2a543d81b57575a2e457bd1e98ffc061b15639dac3be5ff6c2702b9ea1a
#5 extracting sha256:2af8e2a543d81b57575a2e457bd1e98ffc061b15639dac3be5ff6c2702b9ea1a done
#5 extracting sha256:14d68137d870fdc4bc849fbe5c5571ca3e6d9a7b2c5bf1c7e928ccf716dd3b4a 0.1s
#5 extracting sha256:14d68137d870fdc4bc849fbe5c5571ca3e6d9a7b2c5bf1c7e928ccf716dd3b4a 0.3s done
#5 extracting sha256:320659737461e071e3d809231861a7487daaa79805c57f97015ff020f34a939a
#5 extracting sha256:320659737461e071e3d809231861a7487daaa79805c57f97015ff020f34a939a 0.1s done
#5 extracting sha256:1dea7716181a505cea41832dc84691b858a8f64471cb352d8c29cec06e428aeb
#5 extracting sha256:1dea7716181a505cea41832dc84691b858a8f64471cb352d8c29cec06e428aeb 0.9s done
#5 extracting sha256:c9480a4b593fb5abc117a5e01c80d8495c9fd89e743757730c712545f425c9f3
#5 extracting sha256:c9480a4b593fb5abc117a5e01c80d8495c9fd89e743757730c712545f425c9f3 done
#5 extracting sha256:812216efaf878f5ff8086b6efdccec1cb090228107efd62db47d048c927c4f6a done
#5 extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 done
#5 extracting sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 0.1s
#5 extracting sha256:bb7a0c6ab9c129a399a3c8f034cd0de30406766d61a6694f150b4ab9e64c560c 0.9s done
#5 extracting sha256:2c0021742fe2b9491f62c3d89da10492c175081bd17d77efd0f22a298c37ad64
#5 extracting sha256:2c0021742fe2b9491f62c3d89da10492c175081bd17d77efd0f22a298c37ad64 done
#5 DONE 12.2s

#6 exporting to image
#6 exporting layers done
#6 writing image sha256:7acaf12f56850124672037dff3b0cbb60fabb7dc02731de40fc63e94c8e97787 done
#6 naming to docker.io/library/example_scala:test done
#6 DONE 0.0s
====================================================================
+/tmp/viash_test_example_scala_15438672778403597665/test_test/test_executable
warning: 1 deprecation
warning: 1 deprecation (since 2.13.3)
warning: 2 deprecations in total; re-run with -deprecation for details
>>> Create input test file
>>> Run executable
warning: 1 deprecation; re-run with -deprecation for details
Copying 'foo.txt' to 'bar.txt'.
>>> Check whether output file exists
>>> Check whether input and output file are the same
>>> Test finished successfully
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory

When running viash test, Viash will follow the following steps:

  1. Create a temporary directory
  2. Build the component into the main executable
  3. (Re-)build the Docker image for the component
  4. Iterate over all unit test scripts:
  1. Build the unit test into an executable
  2. Run the unit test, passing the main executable as an argument
  1. Return exit code 0 if all of the above steps succeed, otherwise 0

Bonus: unit testing all of the components

If you have multiple Viash components located in a directory called src/, what happens when you run the following?

viash ns test --parallel --src src/