138 lines
6.3 KiB
Markdown
138 lines
6.3 KiB
Markdown
|
# StreamLens - Ethernet Traffic Analyzer
|
|||
|
|
|
|||
|
|
Advanced TUI-based network traffic analyzer for pcap files and live streams with specialized protocol dissection for aviation and industrial networks. Features sigma-based outlier identification and real-time statistical analysis.
|
|||
|
|
|
|||
|
|
## Quick Start
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Install dependencies
|
|||
|
|
pip install scapy numpy
|
|||
|
|
|
|||
|
|
# Analyze pcap file with TUI (flows sorted by largest sigma outliers)
|
|||
|
|
python ethernet_analyzer_modular.py --pcap file.pcap
|
|||
|
|
|
|||
|
|
# Live capture with real-time statistics
|
|||
|
|
python ethernet_analyzer_modular.py --live --interface eth0
|
|||
|
|
|
|||
|
|
# Console output with outlier reporting
|
|||
|
|
python ethernet_analyzer_modular.py --pcap file.pcap --no-tui
|
|||
|
|
|
|||
|
|
# Generate comprehensive outlier report
|
|||
|
|
python ethernet_analyzer_modular.py --pcap file.pcap --report
|
|||
|
|
|
|||
|
|
# Get pcap file information
|
|||
|
|
python ethernet_analyzer_modular.py --pcap file.pcap --info
|
|||
|
|
|
|||
|
|
# Adjust outlier threshold (default: 3.0 sigma)
|
|||
|
|
python ethernet_analyzer_modular.py --pcap file.pcap --outlier-threshold 2.0
|
|||
|
|
|
|||
|
|
# With BPF filter for live capture
|
|||
|
|
python ethernet_analyzer_modular.py --live --filter "port 319 or port 320"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Features
|
|||
|
|
|
|||
|
|
### Enhanced TUI Interface
|
|||
|
|
- **Three-Panel Layout**: Flows list (top-left), flow details (top-right), timing visualization (bottom)
|
|||
|
|
- **Sigma-Based Flow Sorting**: Flows automatically sorted by largest outlier sigma deviation
|
|||
|
|
- **Real-time Navigation**: Arrow keys to navigate between flows with instant detail updates
|
|||
|
|
- **Protocol-aware Display**: Shows detected protocols in flow list and details
|
|||
|
|
- **Smart Protocol Detection**: Prioritizes specialized protocols (Chapter 10, PTP, IENA) over generic ones
|
|||
|
|
- **Detailed Outlier Analysis**: Individual rows showing frame numbers and exact time deltas for outlier packets
|
|||
|
|
- **Visual Timeline**: ASCII timeline showing frame timing deviations with outlier highlighting
|
|||
|
|
- **Live Statistics**: Real-time running averages and outlier detection during capture
|
|||
|
|
|
|||
|
|
### Core Analysis Engine
|
|||
|
|
- **Flow-based Analysis**: Groups packets by source-destination IP pairs with timing statistics
|
|||
|
|
- **Configurable Outlier Detection**: Adjustable sigma threshold (default: 3.0σ)
|
|||
|
|
- **Multi-layer Protocol Analysis**: Ethernet, IP, UDP, TCP with specialized dissectors
|
|||
|
|
- **Real-time Statistical Updates**: Running statistics for live capture mode
|
|||
|
|
- **High Jitter Flow Identification**: Coefficient of variation analysis
|
|||
|
|
|
|||
|
|
### Specialized Protocol Dissectors
|
|||
|
|
- **Chapter 10 (IRIG 106-17)**: Complete packet dissection including data types, timestamps, and payload analysis
|
|||
|
|
- **PTP (IEEE 1588-2019)**: Precision Time Protocol message parsing with sync, delay, and announce messages
|
|||
|
|
- **IENA (Airbus)**: Industrial Ethernet Network Architecture with P/D/N/M/Q message types
|
|||
|
|
|
|||
|
|
### Protocol Detection & Fallbacks
|
|||
|
|
- Automatic protocol identification based on port numbers and packet structure
|
|||
|
|
- Fallback to common protocols: HTTP, HTTPS, SSH, DNS, DHCP, NTP, SNMP, IGMP, ICMP
|
|||
|
|
- Multicast detection for aviation/industrial networks
|
|||
|
|
- Enhanced error handling and validation
|
|||
|
|
|
|||
|
|
## Installation
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# Clone or download the project
|
|||
|
|
cd streamlens
|
|||
|
|
|
|||
|
|
# Install dependencies
|
|||
|
|
pip install scapy numpy
|
|||
|
|
|
|||
|
|
# Run the analyzer
|
|||
|
|
python ethernet_analyzer_modular.py --help
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Key Features Highlights
|
|||
|
|
|
|||
|
|
### 🎯 Sigma-Based Flow Prioritization
|
|||
|
|
Flows are automatically sorted by their largest outlier sigma deviation, putting the most problematic flows at the top of the list for immediate attention.
|
|||
|
|
|
|||
|
|
### 📊 Real-time Statistics
|
|||
|
|
Live capture mode provides running averages and outlier detection as packets arrive, with TUI updates every 500ms.
|
|||
|
|
|
|||
|
|
### 🔍 Configurable Analysis
|
|||
|
|
Adjust outlier detection sensitivity with `--outlier-threshold` (default: 3.0σ) to fine-tune analysis for your specific network conditions.
|
|||
|
|
|
|||
|
|
### 📈 Comprehensive Reporting
|
|||
|
|
Generate detailed outlier reports with `--report` flag showing frame-by-frame sigma deviations and timing analysis.
|
|||
|
|
|
|||
|
|
## TUI Controls
|
|||
|
|
|
|||
|
|
- **↑↓**: Navigate between flows in main view
|
|||
|
|
- **d**: Switch to frame dissection view
|
|||
|
|
- **m** or **ESC**: Return to main view
|
|||
|
|
- **q**: Quit application
|
|||
|
|
|
|||
|
|
## Timeline Visualization
|
|||
|
|
|
|||
|
|
The bottom panel displays a visual timeline of the selected flow's timing behavior:
|
|||
|
|
|
|||
|
|
- **Horizontal axis**: Progression through packet sequence
|
|||
|
|
- **Vertical axis**: Deviation from average inter-arrival time (centered on average)
|
|||
|
|
- **Characters**: `·` = normal timing, `•`/`○` = moderate deviation, `█`/`▄` = outliers
|
|||
|
|
- **Scale**: Automatically adjusts to show full range of deviations
|
|||
|
|
- **Info bar**: Shows total frames, deviation range, and outlier count
|
|||
|
|
|
|||
|
|
## Project Structure
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
streamlens/
|
|||
|
|
├── ethernet_analyzer_modular.py # Main entry point
|
|||
|
|
├── analyzer/ # Core analysis package
|
|||
|
|
│ ├── main.py # CLI argument handling and main logic
|
|||
|
|
│ ├── analysis/ # Analysis engine
|
|||
|
|
│ │ ├── core.py # Main analyzer class
|
|||
|
|
│ │ ├── flow_manager.py # Flow tracking and management
|
|||
|
|
│ │ └── statistics.py # Statistical analysis and outlier detection
|
|||
|
|
│ ├── models/ # Data structures
|
|||
|
|
│ │ ├── flow_stats.py # Flow and frame type statistics
|
|||
|
|
│ │ └── analysis_results.py # Analysis result containers
|
|||
|
|
│ ├── protocols/ # Protocol dissectors
|
|||
|
|
│ │ ├── base.py # Base dissector interface
|
|||
|
|
│ │ ├── chapter10.py # IRIG106 telemetry protocol
|
|||
|
|
│ │ ├── ptp.py # IEEE 1588 Precision Time Protocol
|
|||
|
|
│ │ ├── iena.py # Airbus IENA protocol
|
|||
|
|
│ │ └── standard.py # Standard protocol detection
|
|||
|
|
│ ├── tui/ # Text User Interface
|
|||
|
|
│ │ ├── interface.py # Main TUI controller
|
|||
|
|
│ │ ├── navigation.py # Navigation handling
|
|||
|
|
│ │ └── panels/ # UI panel components
|
|||
|
|
│ │ ├── flow_list.py # Flow list panel
|
|||
|
|
│ │ ├── detail_panel.py # Flow details panel
|
|||
|
|
│ │ └── timeline.py # Timeline visualization panel
|
|||
|
|
│ └── utils/ # Utility modules
|
|||
|
|
│ ├── pcap_loader.py # PCAP file handling
|
|||
|
|
│ └── live_capture.py # Live network capture
|
|||
|
|
└── *.pcapng # Sample capture files
|
|||
|
|
```
|