Adaptive alarm filtering by causal correlation consideration in intrusion detection

One of the main difficulties in most modern Intrusion Detection Systems is the problem of massive alarms generated by the systems. The alarms may either be false alarms which are wrongly classified by a sensitive model, or duplicated alarms which may be issued by various intrusion detectors or be issued at different time for the same attack. We focus on learning-based alarm filtering system. The system takes alarms as the input which may include the alarms from several intrusion detectors, or the alarms issued in different time such as for multi-step attacks. The goal is to filter those alarms with high accuracy and enough representative capability so that the number of false alarms and duplicated alarms can be reduced and the efforts from alarm analysts can be significantly saved. To achieve that, we consider the causal correlation between relevant alarms in the temporal domain to re-label the alarm either to be a false alarm, a duplicated alarm, or a representative true alarm. To be more specific, recognizing the importance of causal correlation can also help us to find novel attacks. As another feature of our system, our system can deal with the frequent changes of network environment. The framework gives the judgment of attacks adaptively. An ensemble of classifiers is adopted for the purpose. Accordingly, we propose a system mainly consisting of two components: one is for alarm filtering to reduce the number of false alarms and duplicated alarms; and one is the ensemble-based adaptive learner which is capable of adapting to environment changes through automatic tuning given the expertise feedback. Two datasets are evaluated.