Getting Started
Overview | Quick Tutorials | Helpful Tips
Overview
This serves as a guide for new Cumulus users to deploy and learn how to use Cumulus. Here you will learn what you need in order to complete any prerequisites, what Cumulus is and how it works, and how to successfully navigate and deploy a Cumulus environment.
What is Cumulus
Cumulus is an open source set of components for creating cloud-based data ingest, archive, distribution and management designed for NASA's future Earth Science data streams.
Who uses Cumulus
Data integrators/developers and operators across projects not limited to NASA use Cumulus for their daily work functions.
Cumulus Roles
Integrator/Developer
Cumulus integrators/developers are those who work within Cumulus and AWS for deployments and to manage workflows.
Operator
Cumulus operators are those who work within Cumulus to ingest/archive data and manage collections.
Role Guides
As a developer, integrator, or operator, you will need to set up your environments to work in Cumulus. The following docs can get you started in your role specific activities.
- Developers - Cumulus Team Dev
- Integrators
- Operators
What is a Cumulus Data Type
In Cumulus, we have the following types of data that you can create and manage:
- Collections
- Granules
- Providers
- Rules
- Workflows
- Executions
- Reports
For details on how to create or manage data types go to Data Management Types.
Quick Tutorials
Deployment & Configuration
Cumulus is deployed to an AWS account, so you must have access to deploy resources to an AWS account to get started.
1. Set up Git Secrets
To ensure your AWS access keys and passwords are protected as you submit commits we recommend setting up Git Secrets.
2. Deploy Cumulus Core and Cumulus Dashboard to AWS
Follow the deployment instructions to deploy Cumulus to your AWS account.
3. Configure and Run the HelloWorld Workflow
If you have deployed using the cumulus-template-deploy repository, you have a HelloWorld workflow deployed to your Cumulus backend.
You can see your deployed workflows on the Workflows page of your Cumulus dashboard.
Configure a collection and provider using the setup guidance on the Cumulus dashboard.
Then create a rule to trigger your HelloWorld workflow. You can select a rule type of one time.
Navigate to the Executions page of the dashboard to check the status of your workflow execution.
4. Configure a Custom Workflow
See Developing a custom workflow documentation for adding a new workflow to your deployment.
There are plenty of workflow examples using Cumulus tasks here. The Data Cookbooks provide a more in-depth look at some of these more advanced workflows and their configurations.
There is a list of Cumulus tasks already included in your deployment here.
After configuring your workflow and redeploying, you can configure and run your workflow using the same steps as in step 2.
Helpful Tips
Here are some useful tips to keep in mind when deploying or working in Cumulus.
Integrator/Developer
- Versioning and Releases: This documentation gives information on our global versioning approach. We suggest upgrading to the supported version for Cumulus, Cumulus dashboard, and Thin Egress App (TEA).
- Cumulus Developer Documentation: We suggest that you read through and reference this resource for development best practices in Cumulus.
- Cumulus Deployment: We will guide you on how to manually deploy a new instance of Cumulus. In this reference, you will learn how to install Terraform, create an AWS S3 bucket, configure a compatible database, and create a Lambda layer.
- Terraform Best Practices: This will help guide you through your Terraform configuration and Cumulus deployment. For an introduction about Terraform, go to Terraform's official site.
- Integrator Common Use Cases: Scenarios to help integrators along in the Cumulus environment.
Operator
- Operator Common Use Cases: Scenarios to help operators along in the Cumulus environment.
Troubleshooting
Troubleshooting: Some suggestions to help you troubleshoot and solve issues you may encounter.