How to test models trained on CICIDS2017 data will they be able to detect attacks from suricata logs?

Help
,
1

How can I develop an effective adaptive intrusion detection system that leverages machine learning techniques to automatically modify existing Suricata rules based on dynamic network traffic analysis, while minimizing false alarms and ensuring the safe implementation of these auto-generated rules?

Specifically, I’m looking for guidance on:

  1. Which features should be selected or engineered from the network traffic data and Suricata logs to train machine learning models that can accurately detect both known and novel threats in real-time, given that my current feature set leads to models classifying all traffic as benign even during simulated attacks?
  2. What machine learning architectures and training approaches are best suited for this adaptive IDS use case, considering the need to learn from an initial dataset like CICIDS2017 but also continuously adapt to my own network’s evolving traffic patterns and emerging threats?
  3. How can I design an integration pipeline that automatically extracts training data from Suricata, validates machine learning model outputs, generates optimized Suricata rules from high-confidence detections, and incorporates analyst feedback to iteratively improve the system’s accuracy and robustness over time?

The end goal is an intrusion detection system that can dynamically adjust to changing network conditions and provide reliable, actionable security alerts, without overburdening analysts with false positives or introducing misconfigured rules that potentially disrupt legitimate traffic. I would appreciate any insights or suggestions on the feature engineering, modeling techniques, system architecture, and automation mechanisms required to successfully implement such an adaptive, ML-powered IDS. Curently i use this features but models see all logs as BENIGN even i create DDoS atack or suricate create an alert. Models are trained on CICIDS2017 data.

selected_features = [ 
# Flow-based features
'Flow Duration', 'Flow Bytes/s', 'Flow Packets/s', 'Total Length of Fwd Packets', 'Total Length of Bwd Packets',


# Timing features
'Flow IAT Mean', 'Flow IAT Std', 'Flow IAT Max', 'Flow IAT Min',
'Fwd IAT Total', 'Bwd IAT Total',

# Packet characteristics
'Fwd Packet Length Max', 'Fwd Packet Length Min',
'Bwd Packet Length Max', 'Bwd Packet Length Min',
'Packet Length Mean', 'Packet Length Std', 'Packet Length Variance',

# TCP flags
'SYN Flag Count', 'FIN Flag Count', 'RST Flag Count',
'PSH Flag Count', 'ACK Flag Count', 'URG Flag Count',

# Additional features
'Total Fwd Packets', 'Total Backward Packets',
'Fwd Header Length', 'Bwd Header Length',
'Active Mean', 'Active Std', 'Idle Mean',
'Init_Win_bytes_forward', 'Init_Win_bytes_backward'
]

I tried to use this features but it didn’t work. I have created file to test this and I used these commands sudo hping3 -2 -p 80 -S --flood --rand-source 192.168.0.target
sudo hping3 -S -p 80 --flood 192.168.0.target
sudo hping3 -1 --flood 192.168.0.target
sudo nmap -sS 192.168.0.target
sudo nmap -sS -sV -p- 192.168.0.target
sudo nmap -sU 192.168.0.target
sudo nmap -sU 192.168.0.target
hydra -l user -P wordlist.txt 192.168.0.target ssh
hydra -l user -P wordlist.txt ftp://192.168.0.target
hydra -l admin -P wordlist.txt 192.168.0.target http-get /admin/
for i in {1…1000}; do curl http://192.168.0.target/; done
while true; do
curl http://192.168.0.target
sleep 10
done
for endpoint in / /about /contact /login; do
curl http://192.168.0.target$endpoint
sleep 5
done
to create a file.