NPR 7150 NASA Software Engineering Requirements#

NASA software must comply with the requirements enumarated in NPR 7150. See the NASA Software Engineering and Assurance Handbook for more information.

Intention is to expand the scope of use to also include its inclusion in Ground software tools that support mission planning or formulation; Ground software that operates a research, development, test, or evaluation laboratory (i.e., not a major engineering/research facility); or Ground software that provides decision support for non-mission-critical situations and airborn software whose anomalous behavior would cause or contribute to a failure of system function resulting in a minor failure condition for the airborne vehicle, or whose anomalous behavior would cause or contribute to a failure of system function with no effect on airborne vehicle operational capability or pilot workload. The classification for this increased scope is described below

Effective version: v1.4 (Est. Fall 2022)
Software Classification: Class-D (Basic Research and Technology Software)
Safety Criticality: Not Safety Critical
Assessment done by Christopher Teubert in April 26, 2022
Approved July 14, 2022 (Robert Duffy)

Previous Classification:

Software Classification: Class-E (Research Software)
Safety Criticality: Not Safety Critical
Assessment done by Christopher Teubert in March 2021

Safety Criticality Analysis#

The safety criticality assessment was made based on the responses to the following questions:

Does the software reside in a safety-critical system? NO
The software causes or contributes to a hazard? NO
The software provides control or mitigation for hazards? NO
The software controls safety-critical functions? NO
The software processes safety-critical commands or data? NO
The software detects and reports, or takes corrective action, if the system reaches a specific hazardous state? NO
The software mitigates damage if a hazard occurs? NO
The software resides on the same system (processor) as safety-critical software? NO
Does the software process data or analyze trends that lead directly to safety decisions (e.g., determining when to turn power off to a wind tunnel to prevent system destruction)? NO

Compliance Notation Legend#

FC: Fully Compliant
T: Tailored (Specific tailoring described in mitigation) SWE-121
PC: Partially Compliant
NC: Not Compliant
NA: Not Applicable

Compliance Matrix#

This matrix tracks the compliance of this software and the project’s software engineering practices with the requirements from 7150.2.

Note: for requirements with different evidence for the different softwares, they have a capital M (for ProgModels), A (for ProgAlgs) or S (for ProgServer) following the requirement.

Life Cycle Management#

SWE #	Description	Compliance	Evidence
033	Assess Aquisiton Options	FC	See section below
013	Maintain Software Plans	FC	This document
024a	Conformance with Project Plan	FC	Corrective actions are tracked as github issues
024b	Project Plan Configuration Mgmt	FC	Project plan is a tracked object in github
034	Software Acceptance Criteria	FC	See Release Checklist, above
036	Software Processes	FC	See notes below
037M	Document Milestones	FC	Milestones
037S	Document Milestones	FC	Milestones
039a	Monitor integration	FC	Integration status visible on github pull requests
039b	Review Verification Activities	FC	Verification status and results visible on github actions
039c	Review Trade Studies	FC	Trade studies will be documented in github issues
039d	Audit Development Processes	FC	Development processes are visible in pull requests and commits
039e	Software Reviews	FC	Software Assurance Officer gives final approval after reviews
040aM	Products	FC	Kept in Repo
040aS	Products	FC	Kept in Repo
040b	Tracability	FC	Maintained in PR and issue documentation
040c	Non-conformances	FC	See github issues
040dM	Change tracking	FC	See Commits
040dS	Change tracking	FC	See Commits
042	Electronic Accesss to Source	FC	See respective github repository
139	Comply with 7150	FC	This document
121	Tailored Reqs	NA	No tailoring
125	Compliance Matrix	FC	This document
029	Software Classification	FC	This document
027	COTS/GOTS/MOTS/OSS	FC	See dependencies at bottom of page

Note on software processes [SWE 034]:

The project manager shall establish and maintain the software processes, software documentation plans, list of developed electronic products, deliverables, and list of tasks for the software development that are required for the project’s software developers, as well as the action required (e.g., approval, review) of the Government upon receipt of each of the deliverables.

Processes are tracked in this document.
Documentation, electronic products, and deliverables are tracked in GitHub
Tasks are tracked in GitHub issues
Actions required are listed in the checklists above

Cost Estimation#

SWE #	Description	Compliance	Evidence
015	Maintain 1 cost estimate	NA	See note below
151	Cost Estimate Requirements	NA	See note Below
174	Submit Planning Parameters	NA	Specified Center measurement repo does not exist

Software Cost Estimation Note:

The Python Prognostics Packages are a collaborative product of multiple projects. As projects use the software they will implement features and fix bugs to accomplish the goals of their projects, with some input from the Project Manager as the chief software architect. These improvements are incorporated into the shared product for the all participating projects to benefit from.

Responsibility for cost accounting for these contributions is delegated to the project(s) conducting them. Cost of contributing to the Prognostics Python Packages should be represented in their project plans and other documents.

Schedules#

SWE #	Description	Compliance	Evidence
016M	Schedule Requirements	FC	Milestones
016S	Schedule Requirements	FC	Milestones
046	Maintain Schedule	FC	See Milestones (from SWE016, above)

Note: Release checklist includes confirming schedule exists for next release

Classification#

SWE #	Description	Compliance	Evidence
020	Software Classification	FC	This document
176	Software Classification	FC	This document

Software Assurance#

SWE #	Description	Compliance	Evidence
022	Software Assurance	FC	This document

See checklists at top of page for software assurance activities. Additionally, some software activities are enforced by github branch policies.

Safety Critical Software#

SWE #	Description	Compliance	Evidence
205	Safety Cricial Software	FC	See above
023	Safety Critical Reqs	NA	Not safety critical
134	Safety Critical Reqs	NA	Not safety critical
219	Safety Critical Reqs	NA	Not safety critical
220	Safety Critical Reqs	NA	Not safety critical

This software is not safety critical, see [NPR 7150 NASA Software Engineering Requirements](https://nasa.github.io/progpy/dev_guide.html#npr-7150-nasa-software-engineering-requirements) for more details

Automatic Generation of Source Code#

SWE #	Description	Compliance	Evidence
146	Autogen Software Reqs	NA	No autogen
206	Autogen Software	NA	No autogen

Reuse#

SWE #	Description	Compliance	Evidence
147	Reusability Requirements	FC	See notes below
148M	Software Catalog	FC	prog_models
148A	Software Catalog	FC	prog_algs
148S	Software Catalog	FC	Will be posted soon

Notes on SWE-147: Reusability requirements

This software is a research support software. As, such it is designed to be reusable and to support a wide variety of use-cases. The requirements and coding standards are specified to ensure re-usability.
Examples, Templates, Tutorials, sourcecode, and Documentation will help support reuse
Some support will be provided to help users re-use it.
The software will be released open-source, helping reduce the barriers to reuse.
Design decisions will support ease-of-use
Exceptions and warning will be implemented whenever appropriate to help users use the software properly
When deprecating a feature, notice is provided at least one version of the software prior, in the form of a warning.

Cybersecurity#

SWE #	Description	Compliance	Evidence
156	Perform CyberSecurity Assessment	FC	See below
154	Perform CyberSecurity Risks	FC	See below
157	Protect Against Unauthorized Access	FC	See below
159	Test CyberSecurity Mitigation	FC	See below
207	Secure Coding Practices	FC	Part of LGTM Static Analysis and Code Reviews
185	Static Analysis	FC	See Static Analysis Notes under Implementation

Cybersecurity risks were assessed, the identified cybersecurity threats and our mitigations are described below:

Code injection
- Risk: insertion of hazardous code into an open-source project by malicious actor
- Mitigation: Strict code review requirements in the repository. Static analysis/security alerts. Vetting for contributors. Branch rules to prohibit direct commits and unapproved additions
- Validation: Part of automated tests and confirmed in release review
Programmers Accidentally Introduce Security Risks
- Risk: Programmers accidentally introduce security risks into the codebase
- Mitigation: Automated Tests. Strict code review requirements in the repository. Static analysis/security alerts. Vetting for contributors. Branch rules to prohibit direct commits and unapproved additions
- Validation: Part of automated tests and confirmed in release review
Dependencies
- Risk: Dependencies could introduce cybersecurity vulnerabilities
- Mitigation: GitHub “dependabot” alerts will identify any known issues with package decencies. Also, the project is actively trying to limit the number of dependencies, and only use well-known packages from trusted developers.
- Validation: Alerts produced by dependabot system. Dependencies must be approved by Project Manager
Language
- Risk: Python itself may introduce cybersecurity vulnerabilities
- Mitigation: Python is a well-known language, this risk is low. To mitigate this we only support actively maintained versions.
- Validation: Will check with each release
Unauthorized Access to Hardware [SWE-157]
- Risk: Unauthorized access to hardware (GitHub Servers)
- Mitigation: Github is a trusted partner who has strict access control. Administrator rights are limited to Project Manager and NASA Org Administrators. Individuals not involved and vetted by the project cannot add to the repository directly (only through PR from fork)
- Validation: System configuration validated by PM 7/13/22

Bi-Directional Traceability#

SWE #	Description	Compliance	Evidence
052	Tracability	FC	See Tracability Notes, at bottom of page

Requirements#

SWE #	Description	Compliance	Evidence
050M	Software Requirements	FC	Enhancement Issues
050S	Software Requirements	FC	Enhancement Issues
053	Requirement Change Tracking	FC	Tracked in enhancement issues, see comment from SWE050, above
054	Track Inconsistencies	FC	Tracked in enhancement issues, see comment from SWE050, above
055	Requirement Validation	FC	See below

Note on SWE-55: Requirement Validation:

This is a Research Suffort Software. As such it has many different usages that the requirements must fulfill.
These usages will be implemented as examples, in the tutorial, or as tests. Running these and inspecting them (part of the release process) will help validate that the requirements (as implmeneted) are correct to enable each use case.

Implementation#

SWE #	Description	Compliance	Evidence
061	Coding Standards	FC	See Notes for Developers, above
135	Static Analysis	FC	See list of static analysis tools, below.
062	Unit Testing	FC	Unit tests are created with each enhancement, run automatically with each PR.
186	Unit Test Repeatability	FC	Unit tests are created with each enhancement, run automatically with each PR.
063M	Software Version Description	FC	See here
063S	Software Version Description	FC	See here

Static Analysis Methods Used:

CodeFactor.io (progpy, prog_server): Runs automatically in each PR. If issues are detected, they are noted in the PR chat.
LGTM (progpy, prog_server): Runs automatically in each PR. If issues are detected, they are noted in the PR chat.
Codecov (progpy, prog_server): Runs automatically in each PR. If issues are detected, they are noted in the PR chat.
CodeQL Scanning: Runs automatically in each PR. If issues are detected, they are noted in the PR chat.
Github Dependabot Alerts: Tracks dependencies, alerts of any issues.

Testing#

SWE #	Description	Compliance	Evidence
065a	Test Plan	FC	See this document.
065bM	Test Procedures	FC	See GitHub Actions Workflows.
065bS	Test Procedures	FC	See GitHub Actions Workflows.
065cM	Tests	FC	See tests directory.
065cS	Tests	FC	See tests directory.
065dM	Test Reports	FC	See Github Actions Results.
065dS	Test Reports	FC	See Github Actions Results.
066	Verification	FC	Each requirement has verification tests created before closing. Tests run using GitHub actions
068	Evaluate Test Results	FC	See notes below
071	Update Test Plans	FC	Workflow, tests, and this document are updated as requirements change
186M	Code Coverage	FC	See Codecov
186S	Code Coverage	FC	See Codecov
192	Reqs that Trace to Hazard	FC	When potential for hazard is identified, thorough tests are created to prevent the hazard

Notes on SWE-068: Evaluate Test Results:

All tests are conducted automatically, using github actions, using multiple versions of Python and both dev and release versions of compatable software.
Results of the test are compared with expected results automatically. If they do not match what is expected, the test fails and GitHub does not allow merger of the branch into dev/master
Whenever a requirement (i.e., feature issue) is implemented, it must include tests that properly cover the requirement. Ensuring that it covers the requirement is part of the code review checklist.
- Furthermore, the coverage data is automatically reported on a PR. This will provide additional information to ensure the requirement is tested
- Code coverage is also reviewed as part of the release process

Operations, Maintenance, and Retirement#

SWE #	Description	Compliance	Evidence
075	Ops, Maintenance, and Retirement	FC	See below
077	Record-Keeping	FC	Records maintained in repository and Sharepoint folder
194	Delivery Verification	FC	See Release Checklist and automated unit and verification tests
195	Maintainance Standards	FC	See this document, GitHub records
196	Retirement Plan	FC	See Below

Operations and Maintenance: During the operational phase of the software, the team will do the following:

Provide documentation, examples, tutorial, templates, and source code to users to help them use the software
Provide a issues page for users to report issues and suggest features
Provide a discussion page for further discussion on best practices and suggestions
Monitor and resolve security alerts from users, NASA organiztions, static analysis tools, and GitHub dependency checkers
Maintain the requirements (i.e., issues) and releases
Maintain and update software assurance and compliance documentation (e.g., this page)
Until retirement, maintain the software. This includes:
- Providing support for NASA projects using the software and limited support for external users.
- Fixing bugs, when identified
- Assisting in incorporating new features and enhancements
The project team is also responsible for conducting appropriate reviews (e.g., Release Reviews) to ensure the software is ready for use.

Retirement Plan:

On complete retirement the software, tests, documentation, and other artifacts will continue to be available on GitHub, but a message will be added to the ReadMe and the main page of the documentation indicating that the software is no longer maintained or updated by NASA
All internal documents about the software will be archived in the Diagnostics and Prognostics Group’s Teams OneDrive folder

Configuration Management#

SWE #	Description	Compliance	Evidence
079	Configuration Management Plan	FC	See this document
080M	Evaluate Sotware Product Changes	FC	See PRs
080S	Evaluate Sotware Product Changes	FC	See PRs
081	Identify Configuration Items	FC	See this document
082a	Levels of Control	FC	See this document
082b	Authorization Authority	FC	See this document
082c	Authorization Authority	FC	See this document
083M	Configuration Status	FC	See Branches
083S	Configuration Status	FC	See Branches
084	Configuration Audits	FC	See Note below
085	Release Procedures	FC	See this document

Note on SWE084- Configuration Audits: Configuration audits are conducted in parts throughout the lifecycle process

To a large degree- configuration audits are performed automatically by GitHub actions and branch restrictions. These check the following:
- That tests were run (they are automatically run), passed, and results are recorded
- That files conform with copyright rules
- That required code reviews were performed (Requirement for branch merging)
In performing a code review, the reviewing user confirms:
- That the change is linked to a requirement (i.e., feature issue) and the requirement was met or that it is linked to another issue (e.g., bug report)
- That appropriate tests exist
In performing a release review, the project manager confirms:
- That all issues are completed
- That all tests pass, have proper documentation
- That documentation has been updated and matches the code
- That a schedule exists for the next release and is in the proper place

Non-Conformances#

SWE #	Description	Compliance	Evidence
201M	Track non-conformances	FC	See Github Issues
201S	Track non-conformances	FC	See Github Issues

Transition to a Higher Class#

SWE #	Description	Compliance	Evidence
021	Transition to a higher class	FC	Plans have been updated to reflect the updated classification

Aquisition Options#

Assessed, there are some existing prognostics tools but no general Python package that can support model-based prognostics like we need and no general python package that can support model-based prognostics is an Service Oriented Architecture (SOA) like we need (prog_server).

Requirement Tracking#

Requirements are tracked as issues with the “Enhancement” label (See progpy, prog_server Enhancement Issues). An issue template is used to ensure that the requirement has the desired information. Issues are closed to indicate the requirement has been met. Closing a requirement issue is done with a pull request, which is linked to the relevant requirement, for tracability. Closing the requirement issue requires a code review (see above for details), and requires implementation of passing tests that test the requirement (i.e., verification tests). The tests are reviewed with the code implementing the requirement. Issues are assigned to a milestone (i.e., release) indicating the requirements for that release. Github automatically tracks any changes to the issues (i.e., requirements)

Dependencies#

The following dependencies are used in the project:

numpy
- Requirements met: Various mathematical and array functions
- Documentation: https://numpy.org
- Usage Rights: Released under the BSD 3-Clause License
- Future Support: expected- Numpy is a common tool still under development and actively supported
scipy
- Requirements met: Various mathematical and array functions
- Documentation: https://www.scipy.org
- Usage Rights: Released under the BSD 3-Clause License
- Future Support: expected- Scipy is a common tool still under development and actively supported
matplotlib
- Requirements met: Various figure generation methods
- Documentation: https://matplotlib.org
- Usage Rights: Released under the BSD 3-Clause License
- Future Support: expected- Matplotlib is a common tool still under development and actively supported
pandas
- Requirements met: Various data analysis methods (especially for datasets)
- Documentation: https://pandas.pydata.org
- Usage Rights: Released under the BSD 3-Clause License
- Future Support: expected- Pandas is a common tool still under development and actively supported
tensorflow
- Requirements met: Machine learning algorithms
- Documentation: https://www.tensorflow.org
- Usage Rights: Released under the Apache License 2.0
- Future Support: expected- Tensorflow is a common tool still under development and actively supported
chaospy
- Requirements met: Uncertainty quantification and polynomial chaos expansion logic
- Documentation: http://chaospy.readthedocs.io
- Usage Rights: Released under the MIT License
- Future Support: expected- Chaospy is a common tool still under development and actively supported
FilterPy
- Requirements met: Algorithms for state estimators and predictors
- Documentation: https://filterpy.readthedocs.io
- Usage Rights: Released under the MIT License
- Future Support: expected- FilterPy is a common tool still under development and actively supported
Requests (prog_server only)
- Requirements met: HTTP requests
- Documentation: https://requests.readthedocs.io
- Usage Rights: Released under the Apache License
- Future Support: expected- Requests is a common tool still under development and actively supported
Flask (prog_server only)
- Requirements met: Web server
- Documentation: https://flask.palletsprojects.com/en/1.1.x
- Usage Rights: Released under the BSD 3-Clause License
- Future Support: expected- Flask is a common tool still under development and actively supported
urllib3 (prog_server only)
- Requirements met: HTTP requests
- Documentation: https://urllib3.readthedocs.org
- Usage Rights: Released under the MIT License
- Future Support: expected- urllib3 is a common tool still under development and actively supported
fastdtw
- Requirements met: Dynamic time warping (for dtw metric used in calc_error)
- Documentation: slaypni/fastdtw
- Usage Rights: Released under the MIT License
- Future Support: Unknown- at the time of investigation, the last release was in 3.5 years ago and the last branch was updated 2 years ago. However, the code is simple and the tool is still used by many projects.
- Other notes: validated by comparing the output of fastdtw to dtaidistance and the algorithm from the ‘Towards Data Science’ page (https://towardsdatascience.com/dynamic-time-warping-3933f25fcdd). Additionally, the team inspected the output of the package and it seemed reasonable. Finally, a review of open issues on their github repository and a search for known vulnerabilities yielded no concerns. It’s only dependency is numpy, which is a trusted package.

Notes for all:

The function and performance of these tools are verified and validated as part of the automated tests and examples. [SWE-027 3.1.14.e]
GitHub dependency checker will periodically check for known issues and notify. [SWE-027 3.1.14.f]

Tracability Notes#

Hazards and non-conformances are tracked as issues with the label bug (See progpy, prog_server). In the template for a bug report, there is a section asking for relevant enhancement issues (i.e., requirements). This linking establishes tracability from hazards/non-conformances to the underlying requirement. These linkings are automatically marked by the github system in the requirement issue. Additionally, to close an enhancement issue (i.e., requirement), passing verification tests must be created and checked in. The PR where these tests are created and the implementation is completed is linked to the issue establishing tracability from requirement -> verification test. These tests run automatically at every change/PR.

Additionally, requirements are assigned to milestones/releases, establishing bi-directional tracability to these

Summary: The following tracabilities are maintained:

Hazard <-> Requirement
Non-conformance <-> Requirement
Requirement <-> Verification Test & Results
Requirement <-> Implementation
Release/Milestone <-> Requirement

For past bugs, enhancements, pull requests, etc. look at the previously used prog_models and prog_algs servers.

NPR 7150 NASA Software Engineering Requirements

Contents