State Estimation and Prediction Guide#

https://mybinder.org/badge_logo.svg

The Prognostic Python Package (progpy) is a python framework for prognostics (computation of remaining useful life or future states) of engineering systems. The package provides an extendable set of algorithms for state estimation and prediction, including uncertainty propagation. The package also include metrics, visualization, and analysis tools needed to measure the prognostic performance. The algorithms use prognostic models (from Modeling and Simulation Guide) to perform estimation and prediction functions. The package enables the rapid development of prognostics solutions for given models of components and systems. Different algorithms can be easily swapped to do comparative studies and evaluations of different algorithms to select the best for the application at hand.

Installing progpy#

The latest stable release of ProgPy is hosted on PyPi. For most users, this version will be adequate. To install via the command line, use the following command:

$ pip install progpy

Summary#

The structure of the packages is illustrated below:

_images/package_structure.png

Prognostics is performed using State Estimators and Predictors. State Estimators are resposible for estimating the current state of the modeled system using sensor data and a prognostics model (see: Modeling and Simulation Guide). The state estimator then produces an estimate of the system state with uncertainty in the form of an uncertain data object. This state estimate is used by the predictor to predict when events will occur (Time of Event, ToE - returned as an uncertain data object), and future system states (returned as a Prediction object).

Data Structures#

A few custom data structures are available for storing and manipulating prognostics data of various forms. These structures are listed below and desribed on their respective pages:

State Estimation#

State estimation is the process of estimating the internal model state (x) using input (i.e., loading), output (i.e., sensor data), and system parameters. State estimation is necessary for cases where model state isn’t directly measurable (i.e., hidden state) or where there is sensor noise. Most state estimators estimate the state with some representation of uncertainty.

The foundation of state estimators is the estimate method. The estimate method is called with a time, inputs, and outputs. Each time estimate is called, the internal state estimate is updated.

>>> estimator.estimate(time, inputs, outputs)

The internal state is stored in the estimators x property as a UncertainData subclass (see UncertainData). State is accessed like so x_est = estimator.x.

Extending#

New state estimator are created by extending the progpy.state_estimators.StateEstimator class.

See examples.new_state_estimator_example for an example of this approach.

Example#

Prediction#

Prediction is the process by which future states are estimated, given the initial state (e.g., from State Estimation), a model, and an estimate of future load. An algorithm used to do this is called a predictor. Prediction is often computationally expensive, especially for sample-based approaches with strict precision requirements (which therefore require large number of samples).

With this framework, there are a number of results that can be predicted. The exact prediction results are selected based on the needs of the end-user. The most common prediction results are Time of Event (ToE) and Time to Event (TtE). Time of Event at a specific prediction time (\(t_P\)) is defined as the time when the event is expected to occur (with uncertainty), or equivalently, the time where the event state for that event is zero. Time to Event is defined as the time to ToE (\(TtE = ToE - t_P\)). In prognostics, ToE and TtE are often referred to as End of Life (EOL) and Remaining Useful Life (RUL), respectively.

Beyond these, results of prediction can also include event state, outputs, performance metrics, and system states at different future times, including at ToE. For approaches that predict ToE with uncertainty, some users consider Probability of Success (PoS) or Probability of Failure (PoF). PoF is defined as the percentage of predictions that result in failure before the prognostics horizon (\(PoS \triangleq 1 - PoF\)).

A predictors predict method is used to perform prediction, generally defined below:

result = predictor.predict(x0, future_loading, **config)

Where x0 is the initial state as an UncertainData object (often the output of state estimation), future_loading is a function defining future loading as a function of state and time, and config is a dictionary of any additional configuration parameters, specific to the predictor being used. See Predictors for options available for each predictor

The result of the predict method is a named tuple with the following members:

  • times: array of times for each savepoint such that times[i] corresponds to inputs.snapshot(i)

  • inputs: progpy.predictors.Prediction object containing inputs used to perform prediction such that inputs.snapshot(i) corresponds to times[i]

  • outputs: progpy.predictors.Prediction object containing predicted outputs at each savepoint such that outputs.snapshot(i) corresponds to times[i]

  • event_states: progpy.predictors.Prediction object containing predicted event states at each savepoint such that event_states.snapshot(i) corresponds to times[i]

  • time_of_event: progpy.uncertain_data.UncertainData object containing the predicted Time of Event (ToE) for each event. Additionally, final state at time of event is saved at time_of_event.final_state -> progpy.uncertain_data.UncertainData for each event

The stepsize and times at which results are saved can be defined like in a simulation. See Simulation.

Extending#

New predictor are created by extending the progpy.predictors.Predictor class.

Analyzing Results#

State Estimation#

The results of the state estimation are stored in an object of type progpy.uncertain_data.UncertainData. This class contains a number of methods for analyzing a state estimate. This includes methods for obtaining statistics about the distribution, including the following:

  • mean: The mean value of the state estimate distribution.

  • median: The median value of the state estimate distribution.

  • cov: Covariance matrix (in same order as keys in mean)

  • metrics: A collection of various metrics about the distribution, inlcuding the ones above and percentiles of the state estimate

  • describe: Similar to metrics, but in human readable format

  • percentage_in_bounds: The percentage of the state estimate that is within defined bounds.

  • relative_accuracy: Relative accuracy is how close the mean of the distribution is to the ground truth, on relative terms

There are also a number of figures available to describe a state estimate, described below

See progpy.uncertain_data.UncertainData documentation for more details.

Predicted Future States#

Predicted future states, inputs, outputs, and event states come in the form of a progpy.predictors.Prediction object. Predictions store distributions of predicted future values at multiple future times. Predictions contain a number of tools for analyzing the results, some of which are described below:

  • mean: Estimate the mean value at each time. The result is a list of dictionaries such that prediction.mean[i] corresponds to times[i]

  • monotonicity: Given a single prediction, for each event: go through all predicted states and compare those to the next one.

    Calculates monotonicity for each event key using its associated mean value in UncertainData 4 3

Time of Event (ToE)#

Time of Event is also stored as an object of type progpy.uncertain_data.UncertainData, so the analysis functions described in State Estimation are also available for a ToE estimate. See State Estimation or progpy.uncertain_data.UncertainData documentation for details.

In addition to these standard UncertainData metrics, Probability of Success (PoS) is an important metric for prognostics. Probability of Success is the probability that a event will not occur before a defined time. For example, in aeronautics, PoS might be the probability that no failure will occur before end of mission.

Below is an example calculating probability of success:

>>> from progpy.metrics import prob_success
>>> ps = prob_success(some_distribution, end_of_mission)

ToE Prediction Profile#

A progpy.predictors.ToEPredictionProfile contains Time of Event (ToE) predictions performed at multiple points. ToEPredictionProfile is frequently used to evaluate the prognostic quality for a given prognostic solution. It contains a number of methods to help with this, including:

  • alpha_lambda: Whether the prediction falls within specified limits at particular times with respect to a performance measure 1 2

  • cumulate_relative_accuracy: The sum of the relative accuracies of each prediction, given a ground truth

  • monotonicity: The monotonicity of the prediction series 4 3

  • prognostic_horizon: The difference between a time \(t_i\), when the predictions meet specified performance criteria, and the time corresponding to the true Time of Event (ToE), for each event 1 2

A ToEPredictionProfile also contains a plot method (profile.plot(...)), which looks like this:

_images/alpha_chart.png

This chart shows the distribution of estimated RUL (y-axis) at different prediction times (x-axis) in red. The ground truth and an alpha bound around the ground truth is displayed in green.

Examples#

https://mybinder.org/badge_logo.svg

The best way to learn how to use progpy is through the tutorial. There are also a number of examples which show different aspects of the package, summarized and linked below:

References#

1(1,2)

Kai Goebel, Matthew John Daigle, Abhinav Saxena, Indranil Roychoudhury, Shankar Sankararaman, and José R Celaya. Prognostics: The science of making predictions. 2017

2(1,2)

Abhinav Saxena, José Celaya, Sankalita Saha, Bhaskar Saha, and Kai Goebel. Saxena, A., Celaya, J. Metrics for Offline Evaluation of Prognostic Performance. International Journal of Prognostics and Health Management, 1(1), 20. 2010.

3(1,2)

Jamie Coble et al. Identifying Optimal Prognostic Parameters from Data: A Genetic Algorithms Approach. Annual Conference of the PHM Society. http://www.papers.phmsociety.org/index.php/phmconf/article/view/1404, 2021

4(1,2)

Marcia Baptista, et. al.. Relation between prognostics predictor evaluation metrics and local interpretability SHAP values. Aritifical Intelligence, Volume 306. https://www.sciencedirect.com/science/article/pii/S0004370222000078, 2022