Svc::DpCatalog Component

Svc::DpCatalog Component

1 Introduction

DpCatalog is an active F´ component that manages the downlink of generate data products. The data products were previously generated by components into a project-specific set of directories. Upon command, DpCatalog will build a catalog of data existing products and start downloading them. They are downloaded based on a priority stored in the file and by generation time. DpCatalog also has commands to reprioritize and delete data products.

NOTE: This release has an early prototype that will do the following:

1. Reads the data product files from the specified directory.
2. Puts them in an unsorted list.
3. Sends requests to Svc/FileDownlink to downlink each file in the list.

It does not:

1. Sort the data products
2. Mark them as complete
3. Any of the future features like modifying/deleting data products.

Features and more extended unit testing will be added over time. Use at your own risk!

2 Requirements

Requirement	Description	Rationale	Verification Method
SVC-DPCAT-001	DpCatalog shall read a set of directories and build a catalog of data products.	DpCatalog needs to know at least one directory where data products reside	Test
SVC-DPCAT-002	DpCatalog shall sort data products first based on the internally recorded priority.	DpCatalog needs to downlink highest priority items first	Test
SVC-DPCAT-003	DpCatalog shall sort data products second based on the internally recorded time with oldest products as higher priority.	DpCatalog needs to downlink oldest items first	Test
SVC-DPCAT-004	DpCatalog shall sort data products third based on the internally recorded product ID with the lowest as higher priority.	DpCatalog needs to resolve case where priority and time match	Test
SVC-DPCAT-005	DpCatalog shall update the data product metadata once download is complete	DpCatalog needs to track completion status to avoid duplicate downloads	Test
SVC-DPCAT-006	DpCatalog shall implement a command and port to build the catalog.	DpCatalog should consume the resources to build the catalog only when needed	Test
SVC-DPCAT-007	DpCatalog shall implement a command and port to start downloading the catalog.	DpCatalog downloads should be timed for when communications are ready, and not consume comm window time building	Test
SVC-DPCAT-008	DpCatalog shall have filters for the download based on container ID, priority, and data size limit.	DpCatalog downloads should be tunable for available bandwidth and container metadata	Test
SVC-DPCAT-009	DpCatalog shall implement a way to insert newly-generated data products into the catalog after the catalog is built	DpCatalog should notice new products and not require a rebuild of the catalog	Test
SVC-DPCAT-010	DpCatalog shall implement commands to modify the priority of existing data products	Operators made change the priority to lower or raise priorities due to troubleshooting, etc	Test
SVC-DPCAT-011	DpCatalog shall implement commands to delete data products	Ground tools can use DpManager commands to automatically delete DPs	Test
SVC-DPCAT-012	DpCatalog shall implement a catalog data product that is a listing of existing data products and their metadata	Allow operators to know what products exist	Test
SVC-DPCAT-013	DpCatalog shall have filters for the catalog data product for priority, container ID, and time range	Allow operators to know what products exist	Test

3 Design

3.1 Assumptions

The design of DpCatalog assumes the following:

1. A file system exists to store the data product files.
2. The contents of the data product files match the data product specification.
3. The file downlink will acknowledge completion of each file

3.3 Ports

3.3.1 Role Ports

Name	Type	Role
timeCaller	Fw::Time	TimeGet
cmdIn	`Fw::Cmd`	Cmd
cmdRegOut	`Fw::CmdReg`	CmdReg
cmdResponseOut	`Fw::CmdResponse`	CmdResponse
tlmOut	`Fw::Tlm`	Telemetry
eventOut	`Fw::LogEvent`	LogEvent

3.3.2 Component-Specific Ports

Name	Type	Kind	Purpose
sendFile	SendFileRequest	output	Send next file to downlink
fileDone	SendFileComplete	async_input	Last requested file is complete
newDp	DpNotify	async_input	Notification that a new DP has been generated

3.4 Constants

DpCatalog can be statically configured with the following constants:

Constant	Purpose
MAX_DP_DIRS	Maximum directories that can be provided for DPs

3.5 Configuration

During initialization, the initialization function takes a set of parameters:

Parameter	Purpose
dpDirs	A set of strings up to MAX_DP_DIRS that are directory names where DPs are written
maxFiles	Specify the maximum number of files the catalog can track
allocator	Memory allocator for catalog records

SDD work will continue from here

Constants

3.6 State

3.6 Commands

4 Checklists

TODO

6 Unit Testing

TODO