Creating a Cumulus Workflow
The Cumulus workflow module
To facilitate adding a workflows to your deployment Cumulus provides a workflow module.
In combination with the Cumulus message, the workflow module provides a way to easily turn a Step Function definition into a Cumulus workflow, complete with:
- AWS CloudWatch events that handle database updates for Cumulus objects (e.g.
executions,granules, etc) - Built-in integration with our throttling feature
Using the module also ensures that your workflows will continue to be compatible with future versions of Cumulus.
For more on the full set of current available options for the module, please consult the module README.
Adding a new Cumulus workflow to your deployment
To add a new Cumulus workflow to your deployment that is using the cumulus module, add a new workflow resource to your deployment directory, either in a new .tf file, or to an existing file.
The workflow should follow a syntax similar to:
module "my_workflow" {
source = "https://github.com/nasa/cumulus/releases/download/vx.x.x/terraform-aws-cumulus-workflow.zip"
prefix = "my-prefix"
name = "MyWorkflowName"
system_bucket = "my-internal-bucket"
workflow_config = module.cumulus.workflow_config
tags = { Deployment = var.prefix }
state_machine_definition = <<JSON
{}
JSON
}
In the above example, you would add your state_machine_definition using the Amazon States Language, using tasks you've developed and Cumulus core tasks that are made available as part of the cumulus terraform module.
Cumulus follows the convention of tagging resources with the prefix variable { Deployment = var.prefix } that you pass to the cumulus module. For resources defined outside of Core, it's recommended that you adopt this convention as it makes resources and/or deployment recovery scenarios much easier to manage.
Examples
For a functional example of a basic workflow, please take a look at the hello_world_workflow.
For more complete/advanced examples, please read the following cookbook entries/topics: