Testing Components

To ensure that your components and namespaces works as expected during its development cycle, writing one or more tests is essential.
Viash supports unit testing, which is a software testing method by which individual units of source code are tested to determine whether they output data as expected.

Writing a unit test for components

To write a unit test for a component, you need two things: a definition in the config file and a script that runs the executable and verifies its output.
This tutorial uses the md_url_checker example component to explain how to write a unit test for Viash. To follow along, create a new folder named “testing” on your machine and download Testfile.md, config.vsh.yaml and script.sh from the URL below to it:

https://github.com/viash-io/viash_web/tree/main/static/examples/md_url_checker

To get started, open up config.vsh.yaml file and take a look at the end of the functionality dictionary, between the path: script.sh and platforms: lines:

  tests:
  - type: bash_script
    path: test.sh
  - path: Testfile.md

This test dictionary contains a reference to a test script and all of the files that need to be copied over in order to complete a test:

The type signifies what scripting language is used for performing the unit test. Note that this does not need to be the same language as the main script. The path here points to the test script.
Every file path added straight into the tests dictionary will be copied over next to the temporary test directory. Any file that is necessary for the test to work correctly can be added here.

In the case of this example, test.sh will be the test script and Testfile.md is necessary as an input markdown file for the script to function.
To write the test itself, create a new file named test.sh in the testing folder and add this as its content:

set -ex # exit the script when one of the checks fail and output all commands.

# check 1
echo ">>> Checking whether output is correct"

# run component with required input(s)
./md_url_checker --inputfile Testfile.md > test-output.txt

[[ ! -f test-output.txt ]] && echo "Test output file could not be found!" && exit 1
grep -q '1: https://www.google.com' test-output.txt # Did the script find the URL?
grep -q 'HTTP/2 404' test-output.txt  # Did the web request return a 404 for the page that doesn't exist?

# check 2
echo ">>> Checking whether an output file was created correctly"

[[ ! -f output.txt ]] && echo "Output file could not be found!" && exit 1
grep -q 'URL: https://www.google.com' output.txt # Was the URL written correctly in the report?
grep -q 'Status: ERROR! URL cannot be reached. Status code: HTTP/2 404' output.txt # Was the error written correctly in the report?
grep -q 'Link name: install viash here' output.txt # Was link name written correctly in the report?

echo ">>> Test finished successfully!"
exit 0 # don't forget to put this at the end

This bash script will run the component and perform several checks to its output. A successful test runs all the way down and exits with a 0 exit code, any other code means a failure:

set -ex will stop the script once any of the lines fail and will output all commands to the shell with a ‘+’ before it.
./md_url_checker --inputfile Testfile.md > test-output.txt runs the component and writes its output to a file.
[[ ! -f test-output.txt ]] && echo "Test output file could not be found!" && exit 1 checks is the output file exists, if it doesn’t exit with a 1 code.
All of the grep calls check if a certain piece of text could be found. Each of these calls can exit the script if the text wasn’t found.
If everything succeeded, exit with a 0 code. Make sure not to forget this final line in your own tests.

Make sure both the config and test files are saved, then run a test by running this command:

viash test config.vsh.yaml

When running tests, Viash will automatically build an executable and place it alongside the other defined resources in a temporary working directory.
The output will look like this:

Running tests in temporary directory: '/tmp/viash_test_md_url_checker5828773777380597444'
====================================================================
+/tmp/viash_test_md_url_checker5828773777380597444/build_executable/md_url_checker ---setup
====================================================================
+/tmp/viash_test_md_url_checker5828773777380597444/test_test.sh/test.sh
>>> Checking whether output is correct
+ echo '>>> Checking whether output is correct'
+ ./md_url_checker --inputfile Testfile.md
+ [[ ! -f test-output.txt ]]
+ grep -q '1: https://www.google.com' test-output.txt
+ grep -q 'HTTP/2 404' test-output.txt
>>> Checking whether an output file was created correctly
+ echo '>>> Checking whether an output file was created correctly'
+ [[ ! -f output.txt ]]
+ grep -q 'URL: https://www.google.com' output.txt
+ grep -q 'Status: ERROR! URL cannot be reached. Status code: HTTP/2 404' output.txt
+ grep -q 'Link name: install viash here' output.txt
+ echo '>>> Test finished successfully!'
>>> Test finished successfully!
+ exit 0
====================================================================
[32mSUCCESS! All 1 out of 1 test scripts succeeded![0m
Cleaning up temporary directory

If the test succeeds it simply writes the full output to the shell. If there are any issues, the script stops and an error message will appear in red. For more information on tests take a look at the viash test command page.

Testing Components

Writing a unit test for components #

Writing a unit test for components