Custom Named-Entity Recognition
Available at: https://huggingface.co/NASA-AIML/MIKA_BERT_FMEA_NER
base-bert-uncased model first further pre-trained then fine-tuned for custom NER to extract failure-relevant entities from incident and accident reports. The model was trained on manually annotated NASA LLIS reports and evaluated on SAFECOM reports.
NER model training was for 4 epochs with:BertForTokenClassification.from_pretrained , learning_rate=2e-5, ` weight_decay=0.01,`
The model was trained to identify the following long-tailed entities:
CAU: failure cause
MOD: failure mode
EFF: failure effect
CON: control process
REC: recommendations
Performace:
Entity |
Precision |
Recall |
F-1 |
Support |
|---|---|---|---|---|
CAU |
0.31 |
0.19 |
0.23 |
1634 |
CON |
0.49 |
0.34 |
0.40 |
3859 |
EFF |
0.45 |
0.20 |
0.28 |
1959 |
MOD |
0.19 |
0.52 |
0.28 |
594 |
REC |
0.30 |
0.59 |
0.40 |
954 |
Average |
0.41 |
0.32 |
0.33 |
9000 |
More infomation on training data, evaluation, and intended use can be found in the original publication
Citation: S. R. Andrade and H. S. Walsh, “What Went Wrong: A Survey of Wildfire UAS Mishaps through Named Entity Recognition,” 2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC), Portsmouth, VA, USA, 2022, pp. 1-10, doi: 10.1109/DASC55683.2022.9925798. https://ieeexplore.ieee.org/abstract/document/9925798