Skip to main content

Experiment Manager

In the 'Experiment Manager' page you can compose and execute workflows.

Experiment Manager

Cells Catalog​

The cells catalog contains all the cells that have been containerized.

Cells Catalog

Parallelization​

To reduce workflow execution time, you can distribute tasks to multiple processors. For this two special components are available:

  • Splitter: This component is used to split an output list or array into batches and feed each batch to the next workflow component. That way the next component will be executed in parallel as many times as the number of batches. You can limit the number of branches created by the splitter by using the pencil icon on the splitter.

Max branches

  • Merger: This component is used to merge the output list or array of the workflow component parallelized by the splitter into a single list or array.

You can practice using the splitter and merger in exercise 2 in the Open Lab.

Sharing Cells​

To share a cell, click on

. You can select users, communities or virtual labs:

Cell sharing dialog

Once a cell is shared, you can click on

to update sharing permissions.

important

Cells shared with a community or virtual lab are visible to everyone. Please only share cells that are useful to others.

Cells that are shared with you show up directly in your catalogue:

Shared cell in user's catalogue

Cells shared within communities or virtual labs are only shown if they are relevant to the current virtual lab that you are running. For instance, when running the “Open Lab”, only cells shared with the “NaaVRE” community and the “Open Lab” lab are shown by default. To display cells from other communities or labs, click on the

:

Shared cells filter

Re-containerizing cells used in a workflow​

When re-containerizing a cell used in a workflow, you might need to update the workflow itself:

  • If you only updated the cell's source code or dependencies: the workflow automatically uses the new version of the cell.
  • If you changed the cell’s inputs, outputs or parameters: the workflow needs to be updated. Remove the cell from the workflow, and add the new version from the catalogue.
  • If you changed the cell’s title: a new cell is created in the catalogue. Both the old and new cell can be used. Workflows using the old cell don’t need to be updated.

Managing files in workflows​

To transfer files between containerized cells when running the workflow, they need to be placed in the /tmp/data/ repository. Files outside of this repository are not preserved from one containerized cell to the other.

The best practice for exchanging files between cells is to save the file in /tmp/data/, and pass the filename between the cells. Example:

# Write file
filename = '/tmp/data/my_file.csv'
with open(filename, 'w') as f:
    f.write('file content')
# Read file
with open(filename, 'r') as f:
    data = f.read()

Files workflow example