Viash 0.6.4

Add backwards compability by supporting switching to older Viash versions

New Release
Author

Viash Team

Published

November 30, 2022

What’s new?

This release adds features related to managing Viash projects and allows for better runtime introspection of Nextflow VDSL3 modules.

The most notable changes are:

  • You can switch versions of Viash using the VIASH_VERSION environment variable! Example:

    VIASH_VERSION=0.6.0 viash --version

    More importantly, you can specify the version of Viash you want in a project config. See below for more info.

  • Introducing Viash project config files as an experimental feature. It allows storing project-related settings in a _viash.yaml config file which you should store at the root of your repository. Example:

    viash_version: 0.6.4
    source: src
    target: target
    config_mods: |
      .platforms[.type == 'docker'].target_registry := 'ghcr.io'
      .platforms[.type == 'docker'].target_organization := 'viash-io'
      .platforms[.type == 'docker'].namespace_separator := '/'
      .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'
  • It’s now possible to specify in which order Viash will merge Viash configs. Example:

    functionality:
      name: foo
      arguments:
        - __merge__: obj_input.yaml
          name: "--one"
        - __merge__: [., obj_input.yaml]
          name: "--two"
        - __merge__: [obj_input.yaml, .]
         name: "--three"

Please take note of the following breaking changes:

  • Passing non-existent paths to a Viash component will cause the component to generate an error when no file or folder is found. Set must_exist to false to revert to the previous behaviour.

  • The arguments --write_meta/-w and --meta/-m no longer exist, because every viash build/run/test run will generate a .config.vsh.yaml meta file.

Full changelog

BREAKING CHANGES

  • Config: Viash configs whose filenames start with a . are ignored (PR #291).

  • viash build: --write_meta/-m and --meta/-m arguments have been removed. Instead, the .config.vsh.yaml file is always created when building Viash components (PR #293).

  • FileArgument: Default setting of must_exist was changed from false to true. As such, the component will throw an error by default if an input file or output file is missing (PR #295).

  • Config merging: __inherits__ has been renamed to __merge__.

NEW FUNCTIONALITY

  • You can switch versions of Viash using the VIASH_VERSION environment variable (PR #304)! Example:

    VIASH_VERSION=0.6.0 viash --version
  • Traceability: Running viash build and viash test creates a .config.vsh.yaml file by default, which contains the processed config of the component. As a side effect, this allows for reading in the .config.vsh.yaml from within the component to learn more about the component being tested (PR #291 and PR #293).

  • FileArgument: Added create_parent option, which will check if the directory of an output file exists and create it if necessary (PR #295).

MINOR CHANGES

  • viash run, viash test: When running or testing a component, Viash will add an extension to the temporary file that is created. Before: /tmp/viash-run-wdckjnce, now: /tmp/viash-run-wdckjnce.py (PR #302).

  • NextflowPlatform: Add DataflowHelper.nf as a retrievable resource in Viash (PR #301).

  • NextflowPlatform: During a stubrun, argument requirements are turned off and the publishDir, cpus, memory, and label directives are also removed from the process (PR #301).

  • NextflowPlatform: Added a filter processing argument to filter the incoming channel after the map, mapData, mapId and mapPassthrough have been applied (PR #296).

  • NextflowPlatform: Added the Viash config to the Nextflow module for later introspection (PR #296). For example:

    include { foo } from "$targetDir/path/foo/main.nf"
    
    foo.run(filter: { tup ->
      def preferredNormalization = foo.config.functionality.info.preferred_normalization
      tup.normalization_id == preferredNormalization
    })

    BUG FIXES

  • BashWrapper: Don’t overwrite meta values when trailing arguments are provided (PR #295).

EXPERIMENTAL FEATURES

  • Viash Project: Viash will automatically search for a _viash.yaml file in the directory of a component and its parent directories (PR #294).

    Contents of _viash.yaml:

    source: src
    target: target
    config_mods: |
      .platforms[.type == 'docker'].target_registry := 'ghcr.io'
      .platforms[.type == 'docker'].target_organization := 'viash-io'
      .platforms[.type == 'docker'].namespace_separator := '/'
      .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'
  • Config merging: Allow specifying the order in which Viash will merge configs (PR #289). If . is not in the list of inherited objects, it will be added at the end.

    Contents of config.vsh.yaml:

    functionality:
      name: foo
      arguments:
        - __merge__: obj_input.yaml
          name: "--one"
        - __merge__: [., obj_input.yaml]
          name: "--two"
        - __merge__: [obj_input.yaml, .]
          name: "--three"

    Contents of obj_input.yaml:

    type: file
    name: --input
    description: A h5ad file

    Output of viash config view config.vsh.yaml (stripped irrelevant bits):

    functionality:
      arguments:
      - type: "file"
        name: "--one"
        description: "A h5ad file"
      - type: "file"
        name: "--input"
        description: "A h5ad file"
      - type: "file"
        name: "--three"
        description: "A h5ad file"