logo
SciPipe Docs
Implement scatter/gather workflows
Initializing search
    scipipe/scipipe
    scipipe/scipipe
    • Introduction
    • Installing
    • Basic Concepts
    • Writing Workflows
      • Running parts of workflows
      • Using parameters in commands and file names
      • Creating components in Go
      • Creating re-usable components
      • Globbing files
      • Splitting files
      • Joining multiple files
      • Creating combinations of files
      • Enable streaming between components
      • Creating sub-workflows
      • Implement scatter/gather workflows
      • Interacting with an HPC resource manager
      • Constrain resource usage
      • Plotting workflow graphs
      • Convert audit logs to other formats
    • Settings
    • Examples
    • Video tutorials
    • Other Resources
    • Acknowledgements
    • Related Tools
    • Contributing

    Implement scatter/gather workflows

    There is work going on to add better scatter/gather support in SciPipe (Issue #20). In the meanwhile, have a look at this example on GitHub which demonstrates one way of doing a scatter gather operation, using the Splitter component (see line 24) and two concatenator components (see lines 38-39) to do the scatter, and gather, operations respectively.

    Made with Material for MkDocs