VDSL3: Only the first two elements from an event in a channel are now passed to a process. This avoids calculating cache entries based on arguments that are not used by the process, causing false-negative cache misses.

config_schema:

• Correct some incorrect markdown tags.
• Add project config.
• Correct documentation/markdown tags to the correct order.
• Add summary description and example for ‘resource’ and ‘argument’, to be used on the reference website.
• Add documentation for the Nextflow directives.

cli_schema: Correct documentation/markdown tags to the correct order.

Viash config: Previously deprecated fields are now removed.

• functionality.inputs: Use arguments or argument_groups instead.
• functionality.outputs: Use arguments or argument_groups instead.
• functionality.tests: Use test_resources instead. No functional difference.
• functionality.enabled: Use status: enabled instead.
• functionality.requirements.n_proc: Use cpus instead.
• platforms.DockerPlatform.privileged: Add a --privileged flag in run_args instead.
• platforms.DockerPlatform.apk: Use setup: [{ type: apk, packages: ... }] instead.
• platforms.DockerPlatform.apt: Use setup: [{ type: apt, packages: ... }] instead.
• platforms.DockerPlatform.yum: Use setup: [{ type: yum, packages: ... }] instead.
• platforms.DockerPlatform.r: Use setup: [{ type: r, packages: ... }] instead.
• platforms.DockerPlatform.python: Use setup: [{ type: python, packages: ... }] instead.
• platforms.DockerPlatform.docker: Use setup: [{ type: docker, run: ... }] instead.
• platforms.DockerPlatform.docker.setup.resources: Use setup: [{ type: docker, copy: ... }] instead.
• platforms.NextflowLegacy: Use the Nextflow VDSL3 platform instead.
• functionality.ArgumentGroups: No longer supports strings referring to arguments in the arguments: section. Instead directly put the arguments inside the argument groups.

viash_install: The bootstrap script has been reworked in line with the project config introduced in 0.6.4:

• The default location for installing the Viash executable is now ./viash (was: bin/viash).
• The new viash_install support --output and --tag.
• The various settings that existed in viash_install (organisation, tag, …) are moved to the project config.

Please note that this new viash_install bootstrap script can be run from the CLI using:

curl -fsSL dl.viash.io | bash

The old get.viash.io is still available but points to the version 0.6.7 version of this component and is deprecated.

WorkflowHelper: paramsToList, paramsToChannel and viashChannel are now deprecated and will be removed in a future release.

viash (ns) build: Change the default value of the namespace separator in a Docker platform from _ to /. Add ".platforms[.type == 'docker'].namespace_separator := '_'" to the project config _viash.yaml to revert to the previous behaviour.

Main: Capture build, setup and push errors and output an exit code.

File downloading: Add check to pre-emptively catch file errors (e.g. 404).

Scala: Updated to Scala 2.13 and updated several dependencies.

Main: Improve match completeness in some edge cases and throw exceptions where needed.

Changelog: Reformat the changelog to a more structured format. For every release, there is now a date, title, and summary. This both improves the changelog itself but can then also be used to postprocess the CHANGELOG programmatically.

VDSL3: Add a default value for id when running a VDSL3 module as a standalone pipeline.

TestBenches:

• Verify switching of Viash versions
• Prepare ConfigDeriver by copying base resources to the targetFolder. Use cases so far showed that it’s always required and it simplifies the usage.
• Remove some old & unmaintained IntelliJ Idea editor-fold tags. Given that the testbenches were split up, these were broken but also no longer needed.
• Add 2 testbenches for computational requirements when running viash run or viash test.
• Added tests for different values for the --id and --param_list parameters of VDSL3 modules.

viash test: Use test as a random tag during testing, instead of test plus a random string.

WorkflowHelper: fixed where passing a relative path as --param_list would cause incorrect resolving of input files.

Testbenches: Fix GitTest testbench to correctly increment temporary folder naming and dispose them after the test finishes.

viash xxx url: Fix passing a url to viash as the config file to process. Add a short testbench to test principle functionality.

Testbenches: Simplify testr container.

Main: Improve error reporting to the user in some cases where files or folders can’t be found. Depending on the thrown exception, more or less context was given.

VDSL3: Create parent directory of output files before starting the script.

NextflowPlatform: Create directories during a stub run when output path is a nested directory (#314).

Automatically generate a warning for deprecated parameters while parsing a .viash.yaml configuration file using the inline documentation deprecation annotations.

Add a “planned removal” field in the deprecation annotations.

Add testbenches to verify proper formatting of the deprecation versions and compare current version to the planned removal version so no deprecated parameters get to stick around beyond what was planned.

NextflowPlatform: Nextflow processes are created lazily; that is, only when running a Nextflow workflow (#321).

Config: Viash configs whose filenames start with a . are ignored (#291).

viash build: --write_meta/-m and --meta/-m arguments have been removed. Instead, the .config.vsh.yaml file is always created when building Viash components (#293).

FileArgument: Default setting of must_exist was changed from false to true. As such, the component will throw an error by default if an input file or output file is missing (#295).

Config merging: __inherits__ has been renamed to __merge__.

You can switch versions of Viash using the VIASH_VERSION environment variable (#304)! Example:

VIASH_VERSION=0.6.0 viash --version

Traceability: Running viash build and viash test creates a .config.vsh.yaml file by default, which contains the processed config of the component. As a side effect, this allows for reading in the .config.vsh.yaml from within the component to learn more about the component being tested (#291 and #293).

FileArgument: Added create_parent option, which will check if the directory of an output file exists and create it if necessary (#295).

viash run, viash test: When running or testing a component, Viash will add an extension to the temporary file that is created. Before: /tmp/viash-run-wdckjnce, now: /tmp/viash-run-wdckjnce.py (#302).

NextflowPlatform: Add DataflowHelper.nf as a retrievable resource in Viash (#301).

NextflowPlatform: During a stubrun, argument requirements are turned off and the publishDir, cpus, memory, and label directives are also removed from the process (#301).

NextflowPlatform: Added a filter processing argument to filter the incoming channel after the map, mapData, mapId and mapPassthrough have been applied (#296).

NextflowPlatform: Added the Viash config to the Nextflow module for later introspection (#296). For example:

include { foo } from "$targetDir/path/foo/main.nf"

foo.run(filter: { tup ->
  def preferredNormalization = foo.config.functionality.info.preferred_normalization
  tup.normalization_id == preferredNormalization
})

BUG FIXES

BashWrapper: Don’t overwrite meta values when trailing arguments are provided (#295).

Viash Project: Viash will automatically search for a _viash.yaml file in the directory of a component and its parent directories (#294).

Contents of _viash.yaml:

source: src
target: target
config_mods: |
  .platforms[.type == 'docker'].target_registry := 'ghcr.io'
  .platforms[.type == 'docker'].target_organization := 'viash-io'
  .platforms[.type == 'docker'].namespace_separator := '/'
  .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'

Config merging: Allow specifying the order in which Viash will merge configs (#289). If . is not in the list of inherited objects, it will be added at the end.

Contents of config.vsh.yaml:

functionality:
  name: foo
  arguments:
    - __merge__: obj_input.yaml
      name: "--one"
    - __merge__: [., obj_input.yaml]
      name: "--two"
    - __merge__: [obj_input.yaml, .]
      name: "--three"

Contents of obj_input.yaml:

type: file
name: --input
description: A h5ad file

Output of viash config view config.vsh.yaml (stripped irrelevant bits):

functionality:
  arguments:
  - type: "file"
    name: "--one"
    description: "A h5ad file"
  - type: "file"
    name: "--input"
    description: "A h5ad file"
  - type: "file"
    name: "--three"
    description: "A h5ad file"

Config: Made major internal changes w.r.t. how config files are read and at which point a platform (native, docker, nextflow) is applied to the functionality script. The only visible side effect is that viash ns list will output each config only once instead of multiple times.

Functionality: Structured annotation can be added to a functionality and its arguments using the info field. Example:

functionality:
  name: foo
  info:
    site: https://abc.xyz
    tags: [ one, two, three ]
  arguments:
    - name: --foo
      type: string
      info:
        foo: bar
        a:
          b:
            c

BashWrapper: Allow printing the executor command by adding ---verbose ---verbose to a viash run or an executable.

Testbenches: Rework MainBuildAuxiliaryNativeParameterCheck to create stimulus files and loop over the file from bash instead of looping natively. This prevents creating thousands of new processes which would only test a single parameter. Note this still calls the main script for each stimulus separately, but that was the case anyway, only much much worse.

Testbenches: Split some grouped test benches into slightly smaller test benches that group tested functionality better.

Annotations: Complete the config schema annotations. Make sure all arguments are documented. Added an annotation internalFunctionality and undocumented for arguments that should not be documented. Added a testbench that verifies that all arguments are in fact annotated, skipping those that are not in the class constructor. Adds a hierarchy field in the __this__ member to list the relation of the own and parent classes.

Testbenches: Add exit code to helper method testMainWithStdErr.

Testbenches: Add testbench to verify viash underscore components (viash_build, viash_install, viash_push, viash_skeleton, viash_test).

Testbenches: Update viash underscore component tests to use $meta_executable.

viash ns exec: Allow choosing whether the {platform} field should be filled in, based on the --apply_platform parameter.

DockerPlatform: Remove duplicate auto-mounts (#257).

Underscore component tests: Fix tests for viash_skeleton and viash_test components.

NextflowVDSL3Platform: Fix ‘Module scriptPath has not been defined yet’ error when Nextflow>=22.10 (#269).

config inject: Doesn’t work when must_exist == true (#273).

RScript: Fix compatibility issue where the new character escaping in r_script required R>=4.0 (#275). Escaping is now handled without using the new r'(foo)' notation.

DockerRequirements: The resources: setting has been deprecated and will be removed in Viash 0.7.0. Please use copy: instead.

DockerRequirements: The privileged: setting has been deprecated and will be removed in Viash 0.7.0. Please use run_args: "--privileged" instead.

Config: Any part of a Viash config can use inheritance to fill data (#271). For example: Contents of src/test/config.vsh.yaml:

__inherits__: ../api/base.yaml
functionality:
  name: test
  resources:
    - type: bash_script
      path: script.sh
      text: |
        echo Copying $par_input to $par_output
        cp $par_input $par_output

Contents of src/api/base.yaml:

functionality:
  arguments:
    - name: "--input"
      type: file
    - name: "--output"
      type: file
      direction: output

The resulting yaml will be:

functionality:
  name: test
  arguments:
    - name: "--input"
      type: file
    - name: "--output"
      type: file
      direction: output
  resources:
    - type: bash_script
      path: script.sh
      text: |
        echo Copying $par_input to $par_output
        cp $par_input $par_output

Git: Strip credentials from remote repositories when retrieving the path.

VDSL3: Allow optional output files to be null.

Deprecated usage resources_dir variable inside scripts, use meta["resources_dir"] instead (or $meta_resources_dir in Bash, or meta$resources_dir in R).

Deprecated meta["n_proc"] in favour for meta["cpus"].

viash ns exec: Added two more fields:

• {platform}: the platform name (if applicable)
• {namespace}: the namespace of the component

LongArgument: Added support for 64-bit integers with type: long as opposed to type: integer which are 32-bit integers.

meta["cpus"] is now an integer, meta["memory_*"] are now longs (#224).

DockerPlatform: Only store author names in the authors metadata.

NextflowPlatform: Only store author names in the authors metadata.

Argument[_]: Turn multiple_sep from Char into String.

All meta[...] variables are now processed similar to Argument[_]s, instead of using custom code to convert object types and detect Docker mounts.

Escaper: Make more generic Escaper helper class.

Hardcoded URLs pointing to viash.io in the documentation annotations were replaced with a new keyword system.

Replaced references to “DSL” with “Dynamic Config Modding” in the --help output.

Added an example for Ruby based Docker setups.

viash ns: Reverse exit code outputs, was returning 1 when everything was OK and 0 when errors were detected (#227).

viash config inject: Fix processing of arguments when argument groups are defined (#231).

Fixed a few typos in the CLI.

Fixed the formatting of ns exec documentation.

VDSL3: Fix stub functionality.

VDSL3: Fix error during error message.

viash test: Fix issue where VIASH_TEMP could not be a relative directory when running viash test (#242).

BashScript, CSharpScript, JavaScriptScript, PythonScript, RScript, ScalaScript: Fix quoting issues of certain characters (#113).

BashScript, CSharpScript, JavaScriptScript, PythonScript, RScript, ScalaScript: Implement more rigorous testing of which characters are escaped.

BashWrapper: Escape usage of multiple_sep. This fixes various checks and transformations not working when when multiple_sep is set to ";" (#235).

NextflowPlatform: variant: vdsl3 is now the default NextflowPlatform. variant: legacy has been deprecated.

Functionality: Fields .inputs and .outputs has been deprecated. Please use .argument_groups instead (#186). Before:

functionality:
  inputs:
    - name: "--foo"
  outputs:
    - name: "--bar"

Now:

functionality:
  argument_groups:
    - name: Inputs
      arguments:
        - name: "--foo"
          type: file
    - name: Outputs
      arguments:
        - name: "--bar"
          type: file
          direction: output

Passing strings to an argument group’s arguments has been deprecated. Please simply copy the argument itself into the argument group (#186). Before:

functionality:
  arguments:
    - name: "--foo"
      type: file
    - name: "--bar"
      type: file
      direction: output
  argument_groups:
    - name: Inputs
      arguments: [ foo ]
    - name: Outputs
      arguments: [ bar ]

Now:

functionality:
  argument_groups:
    - name: Inputs
      arguments:
        - name: "--foo"
          type: file
    - name: Outputs
      arguments:
        - name: "--bar"
          type: file
          direction: output

Allow setting the number of processes and memory limit from within the Viash config, as well as a list of required commands. Example:

functionality:
name: foo
requirements:
  cpus: 10
  memory: 10GB
  commands: [ bash, r, perl ]

You can override the default requirements at runtime:

• ./foo ---cpus 4 ---memory 100PB (for NativePlatform or DockerPlatform)
• By adding process.cpus = 4 and process.memory "100 PB" to a nextflow.config (for NextflowPlatform)

This results the following meta variables to be injected into a script:

• meta_cpus (in Bash) or meta["cpus"] (in any other language): Number of processes the script is allowed to spawn.
• meta_memory_b (in Bash) or meta["memory_b"] (in any other language): Amount of memory the script is allowed to allocate, in bytes.
• meta_memory_kb (in Bash) or meta["memory_kb"] (in any other language): Same but in kilobytes, rounded up.
• meta_memory_mb (in Bash) or meta["memory_mb"] (in any other language): Same but in megabytes, rounded up.
• meta_memory_gb (in Bash) or meta["memory_gb"] (in any other language): Same but in gigabytes, rounded up.
• meta_memory_tb (in Bash) or meta["memory_tb"] (in any other language): Same but in terabytes, rounded up.
• meta_memory_pb (in Bash) or meta["memory_pb"] (in any other language): Same but in petabytes, rounded up.

viash ns exec: Added a command for executing arbitrary commands for all found Viash components. The syntax of this command is inspired by find . -exec echo {} \;.

The following fields are automatically replaced:

• {} | {path}: path to the config file
• {abs-path}: absolute path to the config file
• {dir}: path to the parent directory of the config file
• {abs-dir}: absolute path to the directory of the config file
• {main-script}: path to the main script (if any)
• {abs-main-script}: absolute path to the main script (if any)
• {functionality-name}: name of the component

A command suffixed by \; (or nothing) will execute one command for each of the Viash components, whereas a command suffixed by + will execute one command for all Viash components.

ConfigMod: Added a del(...) config mod to be able to delete a value from the yaml. Example: del(.functionality.version).

Folder structure: Adjusted the folder structure to correctly reflect the the namespace change of viash from com.dataintuitive.viash to io.viash.

Functionality: Reworked the enabled field from boolean to a status field which can have the following statusses: enabled, disabled and deprecated. When parsing a config file which has the status field set to deprecated a warning message is displayed on stderr. Backwards for enabled is provided where enabled: true => status: enabled and enabled: false => status: false. The enabled field is marked deprecated.

Resources: Handle edge case when no resources are specified in the vsh.yaml config file and display a warning message.

BashWrapper: Add a warning when an argument containing flags (e.g. --foo) is not recognized and will be handled as a positional argument as this is likely a mistake.

Functionality: Add check to verify there are no double argument names or short names in the config vsh.yaml declarations.

BashWrapper: Add check to verify a parameter isn’t declared twice on the CLI, except in the case multiple: true is declared as then it’s a valid use case.

BashWrapper: For int min/max checking: use native bash functionality so there is no dependency to bc. For double min/max checking: add fallback code to use awk in case bc is not present on the system (most likely to happen when running tests in a docker container).

viash ns list/viash config view: Allow viewing the post-processed argument groups by passing the --parse_argument_groups parameter.

ConfigMod: Added unit tests for condition config mods.

MainTestDockerSuite: Derive config alternatives from the base vsh.yaml instead of adding the changes in separate files. This both reduces file clutter and prevents having to change several files when there are updates in the config format.

GitTest: Added unit tests for Git helper (#216).

csharp_script, javascript_script, python_script, r_script, scala_script: Make meta fields for memory and cpus optional.

NextflowVdsl3Platform: Don’t generate an error when --publish_dir is not defined and -profile no_publish is used.

Viash run: Viash now properly returns the exit code from the executed script.

Git: Fix incorrect metadata when git repository is empty (#216).

Testbenches: Add a testbench framework to test lots of character sequences, single or repeating to be tested in the yaml config. This can be used to later extend to other tests.

Testbenches::vdsl3: Add testbenches to verify functionality:

• Vdsl3’s param_list (yamlblob, yaml, json, csv).
• Nextflow’s own params-file.
• Vdsl3’s recalculating resource file paths to be relative to the param_list file instead of the workflow file (only available for yaml, json, csv).
• Vdsl3’s wrapping of modules to run these as a separate workflow automagically out of the box.

Main: Added viash --schema_export which outputs a schema of the Viash config file to console. This is to be used to automate populating the documentation website.

Helper: Split help message by argument group.

Helper: Remove unneeded arguments.

Functionality: Add default groups Inputs, Outputs and Arguments for all arguments missing from user-defined argument_groups.

WorkflowHelper::helpMessage: Rewrite to bring on par with Viash’s help message.

BooleanArguments: Renamed internal class names for BooleanArguments to be better in line with how they are named in the config yaml. BooleanArgumentRegular -> BooleanArgument (in line with boolean) BooleanArgumentTrue -> BooleanTrueArgument (in line with boolean_true) BooleanArgumentFalse -> BooleanFalseArgument (in line with boolean_false)

NextflowVdsl3Platform: Change how --id is processed when a VDSL3 module is called from the CLI.

NextflowVdsl3Platform: Fix error when param_list is null.

NextflowVdsl3Platform: Fix error when optional, multiple arguments are set to null.

Testbenches: Better capture expected error messages while running testbenches again. Code changes right before previous release re-introduced some of the messages.

NextflowVdsl3Platform: Fix issue where optional parameters aren’t removed when .run(args: [optarg: null]).

WorkflowHelper::readCsv: Treat empty values as undefined instead of throwing an error.

NextflowVdsl3Platform: Use $NXF_TEMP or $VIASH_TEMP as temporary directory if the container engine is not set to docker, podman or charlieengine, else set to /tmp.

Resources: When adding a resource folder, allow a trailing / at the end of the path. Previously this caused the target folder to be erased and the content of the resource folder to be written directly into the target folder.

Functionality: Allow specifying argument groups. Example:

functionality:
  ...
  argument_groups:
    - name: First group
      arguments: [foo, bar]
      description: Description

Addition of the viash_nxf_schema component for converting a Viash config (for a workflow) into a nextflow schema file.

NextflowVdsl3Platform: Use --param_list to initialise a Nextflow channel with multiple parameter sets. Possible formats are csv, json, yaml, or simply a yaml_blob. A csv should have column names which correspond to the different arguments of this pipeline. A json or a yaml file should be a list of maps, each of which has keys corresponding to the arguments of the pipeline. A yaml blob can also be passed directly as a parameter. Inside the Nextflow pipeline code, params.param_list can also be used to directly a list of parameter sets. When passing a csv, json or yaml, relative path names are relativized to the location of the parameter file.

Examples:

nextflow run "target/foo/bar/main.nf" --param_list '[{"id": "foo", "input": "/path/to/bar"}]'
nextflow run "target/foo/bar/main.nf" --param_list "params.csv" --reference "/path/to/ref"

NextflowVdsl3Platform: The functionality is now slurped from a json instead of manually taking care of the formatting in Groovy.

NextflowVdsl3Platform: The --help is auto-generated from the config.

NextflowVdsl3Platform: Allow both --publish_dir and --publishDir when auto.publish = true.

NextflowVdsl3Platform: Allow passing parameters with multiplicity > 1 from Nextflow CLI.

Main: Added viash --cli_export which outputs the internal cli construction information to console. This is to be used to automate populating the documentation website.

viash ns: Display success and failure summary statistics, printed to stderr.

DataObject: .alternatives is now a OneOrMore[String] instead of List[String], meaning you can now specify { type: string, name: "--foo", alternatives: "-f" } instead of { type: string, name: "--foo", alternatives: [ "-f" ] }

BashWrapper: Added metadata field meta_executable, which is a shorthand notation for meta_executable="$meta_resources_dir/$meta_functionality_name"

Arguments: Internal naming of functionality.arguments is changed from DataObject to Arguments. Change is also applied to child classes, e.g. StringObject -> StringArgument.

Script: Allow more control over where injected code ends up.

Restructure type system to allow type-specific arguments.

DockerPlatform: Change org.opencontainers.image.version annotation to functionality.version when set. Additionally fixed retrieving the git tag possibly returning fatal: No names found, cannot describe anything. or similar.

viash config inject: Fix config inject when .functionality.inputs or .functionality.outputs is used.

BashWrapper: Don’t add bc as dependency. Only perform integer/float min/max checks when bc is available, otherwise ignore.

DockerPlatform: Fix inputs & outputs arguments being present twice.

viash ns test: Silently skip Nextflow platforms as these don’t support tests and will always fail.

Testbenches: Better capture expected error messages while running testbenches. Having these show on the console could be confusing.

NextflowVdsl3Platform: Fix issue when running multiple VDSL3 modules concurrently on the same channel.

Functionality and viash ns: Added .enabled in functionality, set to true by default. Filter for disabled components in namespace commands.

DockerPlatform: Add org.opencontainers.image annotations to built docker images.

Functionality: when defining text resources, permit defining path instead of dest. If both dest and path are unset, use a default file name depending on the resource type, such as script.sh or text.txt.

viash build: Errors are printed in red.

NextflowVdsl3Platform: Undefined input files should not inject a VIASH_PAR_* variable when multiple: true.

NextflowVdsl3Platform: Make injected resources dir absolute.

NextflowVdsl3Platform: Fix escaping of triple single quotes.

NextflowVdsl3Platform: Also apply auto.simplifyInput to Lists.

DockerPlatform: added a test_setup that allows adding apt/apk/… setup requirements. These are only executed when running tests.

--help: Don’t print “my_component ” when no version is specified, but instead simply “my_component”.

NextflowVdsl3Platform: Set mode=copy for auto.publish and auto.transcript.

NextflowVdsl3Platform: When a module is used multiple times in the same workflow, don’t throw an error anymore, instead simply generate a warning.

NextflowVdsl3Platform: Throw an error when an input file was not found.

viash build: Indent auto-generated code according the indentation of VIASH START when found.

Main: Handle not finding the config file or resources in a config file better. Display a more helpful message instead of a stack trace.

BashWrapper: Add checks on parameters for valid integer, double and boolean values.

BashWrapper: Add option to limit string and integer values to specific choice values.

BashWrapper: Add option to set min and max values for integer and double values.

Dependencies:

• Scala was upgraded from 2.12.10 to 2.12.15
• sbt was upgraded from 1.3.4 to 1.6.1
• sbt-scoverage was upgraded from 1.5.1 to 1.9.3

viash_test: Add back --no_cache parameter to viash_test.

viash_test: Fix --append parameter for viash_test, was not getting passed through.

viash ns test: Fix --append parameter, actually start from a clean file if append is false.

viash_push: Fix component not being built during a release of Viash.

PythonRequirements: Fix packages being mentioned twice in a Dockerfile.

Main: Added support spaces in filenames of config files and resources

BashWrapper: Display a message when the last parsed argument would require more values than are still available. Now display a message that values are missing, used to silently crash the wrapper.

viash config inject: Fix error when file argument is must_exist: true.

DockerPlatform: Move description labels to the end of the Dockerfile to improve cross-component caching.

Functionality: Arguments where .multiple is true can now have lists as default and example.

viash_build: Added unit test for this component.

viash_test: Added unit test for this component.

PythonRequirements: Allow upgrading dependencies. Example: [ type: python. pypi: anndata, upgrade: true ].

NextflowLegacyPlatform: Remove annoying messages when building Nxf modules.

ConfigMods: Expanded the DSL to allow specifying at which point to apply a config mod. This functionality was necessary to allow for setting fields which alter the way configs are parsed. Example of when this is useful: <preparse> .platforms[.type == "nextflow"].variant := "vdsl3". Updating workflow of parsing a config file is:

• read Yaml from file
• apply preparse config mods
• parse resulting Json as Config, thereby instantiating default values etc.
• convert Config back to Json
• apply postparse config mods (original config mods)
• convert final Json back to Config

NextflowVdsl3Platform: A beta implementation of the next-generation Viash+Nextflow platform. See https://github.com/viash-io/viash/issues/82 for more information. You can access the previous Nextflow platform by using the variant parameter:

- type: nextflow
  variant: legacy
  separate_multiple_outputs: false

viash_build and viash_test: The query_name and query_namespace arguments were switched around. These arguments are now passed correctly.

BashScript, JavaScriptScript, PythonScript, RScript: Correctly escape ' (#113). Update unit tests accordingly.

CSharpScript, ScalaScript: Correctly escape " (#113). Update unit tests accordingly.

viash_build, viash_test, viash_push: Don’t try to remove log files if they don’t exist.

• Renamed otype to flags.
• Renamed oType to type
• Deprecated tag (unused feature).

• Added --log_prefix: This prefix is used to determine the path of the log files for viash_build, viash_test and viash_push.
• Added --organization: Id of the organisation to be used in the Docker image name, i.e. <registry>/<organization>/<namespace><namespace_sep><name>.
• Added --target_image_source: Url to the Git repo in which this project resides.
• Removed --log.

• Reduce code duplication by contructing the command with Bash Arrays.
• Renamed --platforms to --platform.
• Added --organization: Id of the organisation to be used in the Docker image name, i.e. <registry>/<organization>/<namespace><namespace_sep><name>.
• Added --target_image_source: Url to the Git repo in which this project resides.
• Changed default of --log from log.txt to .viash_build_log.txt.
• Added --verbose: Print out the underlying viash ns build command before running it.

• Reduce code duplication by contructing the command with Bash Arrays.
• Renamed --platforms to --platform.
• Added --organization: Id of the organisation to be used in the Docker image name, i.e. <registry>/<organization>/<namespace><namespace_sep><name>.
• Added --target_image_source: Url to the Git repo in which this project resides.
• Changed default of --log from log.txt to .viash_test_log.txt.
• Changed default of --tsv from log.tsv to .viash_test_log.tsv.
• Added --verbose: Print out the underlying viash ns test command before running it.

• Reduce code duplication by contructing the command with Bash Arrays.
• Added --organization: Id of the organisation to be used in the Docker image name, i.e. <registry>/<organization>/<namespace><namespace_sep><name>.
• Changed default of --log from log.txt to .viash_push_log.txt.
• Added --verbose: Print out the underlying viash ns build command before running it.

viash run: A long running Viash component can be interrupted by pressing CTRL-C or by sending it an INT or SIGINT signal.

DockerPlatform: Automatically add a few labels based on metadata to Dockerfile.

DockerPlatform: Added value target_image_source for setting the source of the target image. This is used for defining labels in the dockerfile. Example:

target_image_source: https://github.com/foo/bar

viash ns list: Added --format yaml/json argument to be able to return the output as a json as well. Useful for when jq is installed but yq is not. Example:

  viash ns list -p docker -f json | jq '.[] | .info.config'

viash config view: Same as above.

CLI: Deprecated -P flag use -p intead.

DockerPlatform: Deprecated version value.

DockerPlatform: Allow defining a container’s organisation. Example:

  - type: docker
    registry: ghcr.io
    organisation: viash-io
    image: viash
    tag: "1.0"
    target_registry: ghcr.io
    target_organization: viash-io

DockerRequirement: Add label instructions. Example: setup: [ [ type: docker, label: [ "foo BAR" ]]]

Config: In specific places, allow parsing a value as a list of values. Fixes #97. This mostly applies to list values in DockerPlatform, but also to author roles. Examples:

functionality:
  name: foo
  authors:
    - name: Alice
      role: author # can be a string or a list
platforms:
  - type: docker
    port: "80:80" # can be a string or a list
    setup:
      - type: r
        packages: incgraph # can be a string or a list

Functionality: Fix .functionality.add_resources_to_path not being picked up correctly.

AptRequirement: Set DEBIAN_FRONTEND=noninteractive by default. This can be turned off by specifying:

  - type: apt
    packages: [ foo, bar ]
    interactive: true

Main: Slightly better error messages when parsing of viash yaml file fails. Before:

$ viash test src/test/resources/testbash/config_failed_build.vsh.yaml 
Exception in thread "main" DecodingFailure(Unexpected field: [package]; valid fields: packages, interactive, type, List(DownField(apt), DownArray, DownField(platforms)))

After:

$ viash test src/test/resources/testbash/config_failed_build.vsh.yaml 
Error parsing 'file:///path/to/viash/src/test/resources/testbash/config_failed_build.vsh.yaml'. Details:
Unexpected field: [package]; valid fields: packages, interactive, type: DownField(apt),DownArray,DownField(platforms)

viash config inject: A command for inserting a Viash header into your script.

YumRequirement: Added a requirement setup for installing through yum. Example: setup: [ [ type: yum, packages: [ wget] ] ]

DockerRequirement: Allow using copy and add instructions. Example: setup: [ [ type: docker, add: [ "http://foo.bar ." ]]]

ViashTest: Fix verbosity passthrough.

--help: Fix repeated usage flag when printing the help.