112 lines
3.8 KiB
Markdown
112 lines
3.8 KiB
Markdown
# StreamLens Update Rate Optimization Summary
|
|
|
|
## Problem
|
|
The TUI was updating too frequently during PCAP parsing, causing:
|
|
- High CPU usage during parsing
|
|
- Choppy/stuttering interface
|
|
- Poor responsiveness due to excessive updates
|
|
|
|
## Solution
|
|
Slowed down all update rates across the parsing pipeline and TUI layers.
|
|
|
|
## Changes Made
|
|
|
|
### 1. Background Analyzer Updates (`background_analyzer.py`)
|
|
|
|
**Flow Update Batch Size:**
|
|
- **Before:** Every 10 packets (very frequent)
|
|
- **After:** Every 100 packets (10x slower)
|
|
- **Impact:** Reduces flow update callbacks from UI threads
|
|
|
|
**Progress Monitor Updates:**
|
|
- **Before:** Every 0.5 seconds
|
|
- **After:** Every 2.0 seconds (4x slower)
|
|
- **Impact:** Progress bar and parsing stats update less frequently
|
|
|
|
**Monitor Thread Sleep:**
|
|
- **Before:** 0.1 second sleep between checks
|
|
- **After:** 0.5 second sleep between checks (5x slower)
|
|
- **Impact:** Reduces background thread CPU usage
|
|
|
|
### 2. TUI Update Timers (`app_v2.py`)
|
|
|
|
**Metrics Timer:**
|
|
- **Before:** Every 2.0 seconds (0.5 Hz)
|
|
- **After:** Every 5.0 seconds (0.2 Hz)
|
|
- **Impact:** Flow counts, packet rates, outlier counts update less frequently
|
|
|
|
**Flow Timer:**
|
|
- **Before:** Every 5.0 seconds (0.2 Hz)
|
|
- **After:** Every 10.0 seconds (0.1 Hz)
|
|
- **Impact:** Flow table data refreshes less frequently
|
|
|
|
## Performance Benefits
|
|
|
|
### CPU Usage Reduction
|
|
- **Flow callbacks:** 10x reduction (100 packets vs 10 packets)
|
|
- **Progress updates:** 4x reduction (2.0s vs 0.5s intervals)
|
|
- **Monitor overhead:** 5x reduction (0.5s vs 0.1s sleep)
|
|
- **TUI metrics:** 2.5x reduction (5.0s vs 2.0s intervals)
|
|
- **TUI flows:** 2x reduction (10.0s vs 5.0s intervals)
|
|
|
|
### User Experience Improvements
|
|
- ✅ **Smoother parsing** - Less frequent UI interruptions
|
|
- ✅ **Lower CPU usage** - More resources for actual packet processing
|
|
- ✅ **Stable interface** - Buttons and data don't flicker/jump
|
|
- ✅ **Better responsiveness** - UI isn't blocked by constant updates
|
|
- ✅ **Maintained functionality** - All features still work, just update slower
|
|
|
|
## Update Timeline During Parsing
|
|
|
|
### Fast File (< 1000 packets)
|
|
- Progress updates every 2 seconds
|
|
- Flow updates every 100 packets (could be ~1-5 times total)
|
|
- TUI refreshes every 5-10 seconds
|
|
|
|
### Medium File (1000-10000 packets)
|
|
- Progress updates every 2 seconds
|
|
- Flow updates every 100 packets (~10-100 times total)
|
|
- TUI refreshes every 5-10 seconds
|
|
|
|
### Large File (> 10000 packets)
|
|
- Progress updates every 2 seconds
|
|
- Flow updates every 100 packets (100+ times total)
|
|
- TUI refreshes every 5-10 seconds
|
|
|
|
## Technical Details
|
|
|
|
### Thread Safety
|
|
- All update rate changes maintain thread safety
|
|
- Background parsing still uses proper locks and queues
|
|
- UI updates still use `call_from_thread()` for thread-safe UI updates
|
|
|
|
### Button Visibility Fix (Bonus)
|
|
- Buttons now pre-created at TUI initialization instead of dynamic creation
|
|
- Eliminates button creation delays during parsing
|
|
- Frame type buttons visible immediately: `[1. Overview] [2. CH10-Data] [3. UDP]` etc.
|
|
|
|
### Backward Compatibility
|
|
- All existing functionality preserved
|
|
- Same keyboard shortcuts (1-9,0 for filtering)
|
|
- Same data accuracy and completeness
|
|
- Same analysis features and reports
|
|
|
|
## Testing
|
|
|
|
Successfully tested with:
|
|
- ✅ Button creation at initialization
|
|
- ✅ Slower update rate configuration
|
|
- ✅ Real PCAP file parsing (`1 PTPGM.pcapng`)
|
|
- ✅ TUI responsiveness and button visibility
|
|
- ✅ Parsing completion and final results
|
|
|
|
## Conclusion
|
|
|
|
The PCAP parsing is now much smoother with significantly reduced CPU usage while maintaining all functionality. Users will experience:
|
|
|
|
- **10x fewer flow update interruptions**
|
|
- **4x fewer progress update interruptions**
|
|
- **5x less monitor thread overhead**
|
|
- **2-3x fewer TUI refresh interruptions**
|
|
|
|
The interface remains fully functional but is now much more pleasant to use during parsing operations. |