RULE EXTRACTION TO UNDERSTAND CHANGES IN AN ADAPTIVE SYSTEM

Rule extraction as a specific technique can support the rigorous development and assurance needed before using adaptive systems in mission- and safety-critical applications. The ultimate goal and significant innovation of this research discussed in this chapter is to demonstrate that neural network rule extraction technology could be transferred into a practical software tool for neural network verification and validation and other purposes, such as monitoring the state of a neural network. Such a tool would accept as input a formal specification of the trained neural network and use neural network rule extraction algorithms to translate the neural network into an equivalent set of rules. These rule-based systems, which represent the neural network's knowledge, have a more visible and potentially human-readable decision logic that supports a robust set of verification techniques. Rule extraction technology in the form of a usable tool will dramatically increase the ability to verify and validate high assurance neural network systems.