Why reconnaissance
Attacks rarely start with the attack. They start with someone looking: which machines are up, which ports are open, which services answer. That scanning is quiet and easy to lose among normal traffic. Catch it, and you see the attacker before they find anything.
I have done this from the other side. My first live penetration test was in 2019, and reconnaissance is most of the job. This project asks whether a model can learn the shape of it.
The data
UNSW-NB15 is a public intrusion dataset built at the Cyber Range Lab of UNSW Canberra. Each row is one network flow, a single conversation between two machines, described by 49 features: duration, bytes and packets each way, load, protocol, service, connection state and more. Every flow is labelled normal or one of nine attack types, and Reconnaissance is one of them.
The method
- Clean the flows, one-hot encode protocol, service and state, and scale the numbers so bytes in millions do not drown out everything else.
- Train a feedforward neural network, with dropout and early stopping so it does not memorise the training set.
- Handle class imbalance head on. Normal traffic dwarfs reconnaissance, and a model that ignores that learns to say "normal" and look accurate.
How I will judge it
Not on accuracy. When most traffic is normal, accuracy rewards a model for missing attacks. I will report precision, recall and F1 for every class, a confusion matrix, and ROC curves. The question is simple: of the reconnaissance in the test set, how much does it catch, and how often does it cry wolf?
What I expect to go wrong
The rarest attack types, like worms, have very few examples, so the model will likely do badly on them. I will not hide that. Where a model fails is usually where the interesting part is.
Readings
None yet. Results go here when they are real, failures included.
Python, pandas, scikit-learn, TensorFlow and Keras.
Working on intrusion detection or network security data? Email shaw@shawsuraj.com.