To ensure that your components and namespaces works as expected during
its development cycle, writing one or more tests is essential.
Viash supports unit testing, which is a software testing method by which individual units of source code are tested to determine whether they output data as expected.
Writing a unit test for components
To write a unit test for a component, you need two things: a definition
in the config file and a script that runs
the executable and verifies its output.
This tutorial uses the md_url_checker example component to explain how to write a unit test for Viash. To follow along, create a new folder named “testing” on your machine and download Testfile.md, config.vsh.yaml and script.sh from the URL below to it:
To get started, open up config.vsh.yaml file and take a look at the
end of the functionality dictionary, between the
path: script.sh and
tests: - type: bash_script path: test.sh - path: Testfile.md
This test dictionary contains a reference to a test script and all of the files that need to be copied over in order to complete a test:
typesignifies what scripting language is used for performing the unit test. Note that this does not need to be the same language as the main script. The
pathhere points to the test script.
- Every file
pathadded straight into the
testsdictionary will be copied over next to the temporary test directory. Any file that is necessary for the test to work correctly can be added here.
In the case of this example, test.sh will be the test script and
Testfile.md is necessary as an input markdown file for the script to
To write the test itself, create a new file named test.sh in the testing folder and add this as its content:
set -ex # exit the script when one of the checks fail and output all commands. # check 1 echo ">>> Checking whether output is correct" # run component with required input(s) ./md_url_checker --inputfile Testfile.md > test-output.txt [[ ! -f test-output.txt ]] && echo "Test output file could not be found!" && exit 1 grep -q '1: https://www.google.com' test-output.txt # Did the script find the URL? grep -q 'HTTP/2 404' test-output.txt # Did the web request return a 404 for the page that doesn't exist? # check 2 echo ">>> Checking whether an output file was created correctly" [[ ! -f output.txt ]] && echo "Output file could not be found!" && exit 1 grep -q 'URL: https://www.google.com' output.txt # Was the URL written correctly in the report? grep -q 'Status: ERROR! URL cannot be reached. Status code: HTTP/2 404' output.txt # Was the error written correctly in the report? grep -q 'Link name: install viash here' output.txt # Was link name written correctly in the report? echo ">>> Test finished successfully!" exit 0 # don't forget to put this at the end
This bash script will run the component and perform several checks to its output. A successful test runs all the way down and exits with a 0 exit code, any other code means a failure:
set -exwill stop the script once any of the lines fail and will output all commands to the shell with a ‘+’ before it.
./md_url_checker --inputfile Testfile.md > test-output.txtruns the component and writes its output to a file.
[[ ! -f test-output.txt ]] && echo "Test output file could not be found!" && exit 1checks is the output file exists, if it doesn’t exit with a 1 code.
- All of the
grepcalls check if a certain piece of text could be found. Each of these calls can exit the script if the text wasn’t found.
- If everything succeeded, exit with a 0 code. Make sure not to forget this final line in your own tests.
Make sure both the config and test files are saved, then run a test by running this command:
viash test config.vsh.yaml
When running tests, Viash will automatically build an executable and
place it alongside the other defined resources in a temporary working
The output will look like this:
Running tests in temporary directory: '/tmp/viash_test_md_url_checker5828773777380597444' ==================================================================== +/tmp/viash_test_md_url_checker5828773777380597444/build_executable/md_url_checker ---setup ==================================================================== +/tmp/viash_test_md_url_checker5828773777380597444/test_test.sh/test.sh >>> Checking whether output is correct + echo '>>> Checking whether output is correct' + ./md_url_checker --inputfile Testfile.md + [[ ! -f test-output.txt ]] + grep -q '1: https://www.google.com' test-output.txt + grep -q 'HTTP/2 404' test-output.txt >>> Checking whether an output file was created correctly + echo '>>> Checking whether an output file was created correctly' + [[ ! -f output.txt ]] + grep -q 'URL: https://www.google.com' output.txt + grep -q 'Status: ERROR! URL cannot be reached. Status code: HTTP/2 404' output.txt + grep -q 'Link name: install viash here' output.txt + echo '>>> Test finished successfully!' >>> Test finished successfully! + exit 0 ==================================================================== [32mSUCCESS! All 1 out of 1 test scripts succeeded![0m Cleaning up temporary directory
If the test succeeds it simply writes the full output to the shell. If there are any issues, the script stops and an error message will appear in red. For more information on tests take a look at the viash test command page.