Practical Machine Learning (ML) and Deep Learning (DL) Application for Network Detection and Respons

Practical Machine Learning (ML) and Deep Learning (DL) Application for Network Detection and Response (NDR) in Intrusion Detection (IDS) and Prevention Systems (IPS)

The vendors of network sniffers and network monitoring bundles have modified their software to include specific behavioural detection rules and capture particular network traffic. Using a combination of machine learning (ML) and its subset of deep learning (DL) and other analytical techniques to network traffic helps enterprise organisations detect suspicious traffic that other security tools are missing in the cyber domain. In this report, NDR's available analytics approach through data gathering, modelling and algorithm expertise for outclassing the institution in the industry of understanding possibilities maximises opportunities. The report's goal is to improve and complement attack detection and enhance prevention and mitigate successful attack through testing in the lab environment.

The research proposal was comprehensive, noting the literature review portion and included in the request's vital part. It contains details about the previous contribution and establishes the gaps and questions to identify the problem domain's statement. The following section provides by detailing the aim and objects. The approach of methodology indicates the whole research mechanism to achieve the objects mentioned. The final section explains the work breakdown, and schedules carried out for the dissertation.

The fields of communication, wired and wireless and the internet have advanced rapidly over the years, with networks having significant and increasing influences and impact on human life in which the data are consequently leading to an increase in the size, volume, and velocity of network traffic and corresponding immense adverse data growth. The evolution of technology and industrialisation led with the increased use of communication networks sparked the risks and exploitation that have multiplied, with several novel attacks that are difficult to detect accurately being generated. It has made cybersecurity a vital field of research.

Further in the connected network environment, facilitated with hardware and software, there has been an increase in malicious entities whose objective is to launch a variety of passive and active attacks to compromise the network for obtaining or unauthorised access data; in which the data's confidentiality, integrity and availability well protected; however, these risks are not extracted, instead of being explored in past phase to the cybersecurity domain learning and development of next-gen Intrusion Detection Systems (IDS) in host-based (HIDS), in the network by monitoring inbound-outbound traffic  (NIDS), monitoring the packets by determining attack signatures (SIDS), an anomaly network traffic by observing bandwidth, protocols in use, system ports (AIDS) and  Intrusion Prevention Systems (IPS) residing in Firewall (FW) through with enhanced approach of DL and ML.

Challenges with IDS/IPS

At the beginning of the networking era, IDS/IPS tools are preferably installed next to the gateway aligning with Firewall (FW) that prevent internal and external attacks to networks by protecting the data conveying from cyber threats.  The trending collaborated eco-system in wireless access or/and mobile devices urgently act and bring mobile IDS/IPS' necessity to safeguard the security policy. By nature, mobile IDS/IPS tools reside in the portable device while scanning the mechanism deployed in the cloud; however, the mobiles by itself are no longer in the same business network.

Despite significant advances in IDS/IPS, these systems still face several challenges and have inherent limitations that exacerbate the problem of cybersecurity; IDS still has imitations of accurate detection, false alarms, and detecting novel intrusions as intruders become more sophisticated. These factors raise new challenges to network analysts in evaluating several false alarms and failing to detect significant threats posed by undetected novel intrusions. Attack variants frequently change as network environments change equally quickly, resulting in several novel attacks always emerging; further, IDS are mostly incapable of detecting unknown attacks; hence creating novel approaches to NDR.

This research proposal seeks to put forth the use of machine learning (ML) and deep learning (DL) as a novel approach to enhance NDR by improving the scope and functionality of IDS/IPS. An artificial intelligence ( AI) in the NDR preceptive enables the computer system or IDS/IPS capture. It mimics human intention and exploitation by learning through Machine learning (ML) without explicitly intensive human overhead in the network data streams. Upon receiving massive novel intrusion and attack variants data, compiling data science techniques leads and understands the higher-level detection rate.

In advancing technology in data science, the study of Deep Learning (DL) mechanism by utilising more extensive data sets for training and understanding extract patterns makes vendors/entrepreneurs an opportunity to implement well adapted IDS/IPS equipment for the next generation. Moreover, DL can automatically learn feature presentations from the numerical and categorical data source and generate results, making it a novel approach to enhancing NDR; DL has a marvel operational cycles and is highly practical because of algorithm related to several hidden layers for analysing the result.

A brief review of relevant literature follows this brief introduction of the proposal, and then the project aims and objectives are listed. It is followed by a proposed research methodology and a project plan on how the entire project will be accomplished.