DP44/GLM5Analysis/Architecture.md

# DataPRO Architecture Document

## System Overview

DataPRO is a WPF-based desktop application for data acquisition system (DAS) management and sensor data analysis. It provides:

- **Hardware Management**: Configuration and control of DTS data acquisition hardware (SLICE, TDAS, Ribeye devices)
- **Sensor Configuration**: Management of sensor databases, calibrations, and channel setups
- **Test Setup Management**: Creation and management of test configurations
- **Real-time Data Collection**: Live streaming and visualization of sensor data
- **Data Review & Analysis**: Post-test data visualization, filtering, and export
- **Reporting**: PSD (Power Spectral Density) reports and pedestrian/head impact reports

The application follows a modular architecture using Prism Framework with Unity dependency injection, enabling loose coupling between components and supporting feature extensibility.

---

## Solution Structure

| Project | Path | Purpose |
|---------|------|---------|
| **DataPRO** | `DataPRO/DataPRO/` | Main WPF application (shell, bootstrapper, main window) |
| **DASFactory** | `DataPRO/DASFactory/` | Hardware discovery and DAS device management factory |
| **IService** | `DataPRO/IService/` | Core service interfaces and SLICE/TDAS service implementations |
| **DbAPI** | `DataPRO/DbAPI/` | Database API layer for SQL Server operations |
| **SensorDB** | `DataPRO/SensorDB/` | Sensor database models and calibration data |
| **DASFactoryDb** | `DataPRO/DASFactoryDb/` | Database wrapper for DAS factory operations |
| **ICommand** | `DataPRO/ICommand/` | Command pattern interfaces for hardware communication |
| **SLICECommands** | `DataPRO/SLICECommands/` | SLICE device command implementations |
| **TDASCommands** | `DataPRO/TDASCommands/` | TDAS device command implementations |
| **Reports** | `DataPRO/Reports/` | Report generation functionality |
| **DTS.Viewer** | `DTS Viewer/DTS.Viewer/` | Data visualization and analysis modules |
| **DTS.Common** | `Common/DTS.Common/` | Core interfaces, events, base classes |
| **DTS.Common.DataModel** | `Common/DTS.Common.DataModel/` | Data models (Group, Sensor, TestGraph) |
| **DTS.Common.Storage** | `Common/DTS.Common.Storage/` | Storage and serialization utilities |
| **DTS.Common.DAS.Concepts** | `Common/DTS.Common.DAS.Concepts/` | DAS domain concepts and interfaces |
| **[Module Projects]** | `DataPRO/Modules/*/` | 30+ Prism modules for UI features |

---

## Architecture Patterns

### MVVM (Model-View-ViewModel)

All UI modules follow strict MVVM pattern:

```
View (XAML) ←→ ViewModel (INotifyPropertyChanged) ←→ Model (Business Logic)
```

Example from `SensorsListModule.cs:42-43`:
```csharp
_unityContainer.RegisterType<ISensorsListView, SensorsListView>();
_unityContainer.RegisterType<ISensorsListViewModel, SensorsListViewModel>();
```

### Prism Framework

The application uses Prism 8.x for:

- **Modularity**: `IModule` implementations for feature isolation
- **Region Management**: Dynamic view composition in UI regions
- **Event Aggregation**: Pub/Sub messaging via `IEventAggregator`
- **Dependency Injection**: Unity container integration

Bootstrapper configuration (`Bootstrapper.cs:179-220`):
```csharp
protected override void ConfigureModuleCatalog(IModuleCatalog moduleCatalog)
{
    moduleCatalog.AddModule<StatusAndProgressBar.StatusAndProgressBarModule>();
    moduleCatalog.AddModule<DatabaseServices.DatabaseServicesModule>();
    moduleCatalog.AddModule<SensorsList.SensorsListModule>();
    // ... 30+ modules registered
}
```

### Unity Dependency Injection

Services and ViewModels are registered with Unity:

```csharp
// Singleton registration
_unityContainer.RegisterType<IMainViewModel, MainViewModel>(
    new ContainerControlledLifetimeManager());

// Transient registration
_unityContainer.RegisterType<ISensorsListView, SensorsListView>();
```

### Event Aggregation Pattern

Inter-module communication uses Prism's `PubSubEvent<T>`:

Defined in `Common/DTS.Common/Events/`:
- `TestEvent` - Test lifecycle events
- `SensorChangedEvent` - Sensor modifications
- `HardwareSavedEvent` - Hardware configuration saves
- `GroupChannelsChangedEvent` - Channel group updates
- `DASConfigurationEvent` - DAS configuration changes

Example event from `TestEvent.cs:11`:
```csharp
public class TestEvent : PubSubEvent<TestEventArg> { }
```

---

## Module Dependency Graph

```mermaid
graph TB
    subgraph "Main Application"
        DataPRO[DataPRO Shell]
    end
    
    subgraph "Core Services"
        DASFactory[DASFactory]
        IService[IService]
        DbAPI[DbAPI]
        SensorDB[SensorDB]
    end
    
    subgraph "Common Libraries"
        DTS_Common[DTS.Common]
        DTS_DataModel[DTS.Common.DataModel]
        DTS_Storage[DTS.Common.Storage]
        DTS_DASConcepts[DTS.Common.DAS.Concepts]
    end
    
    subgraph "Hardware Commands"
        SLICECmds[SLICECommands]
        TDASCmds[TDASCommands]
        RibeyeCmds[RibeyeCommands]
    end
    
    subgraph "UI Modules"
        SensorsList[SensorsList Module]
        HardwareList[HardwareList Module]
        GroupList[GroupList Module]
        TestSetupsList[TestSetupsList Module]
        RealtimeModule[RealtimeModule]
        DatabaseServices[DatabaseServices Module]
        DTSViewer[DTS.Viewer]
    end
    
    DataPRO --> DASFactory
    DataPRO --> IService
    DataPRO --> DbAPI
    DataPRO --> SensorsList
    DataPRO --> HardwareList
    DataPRO --> GroupList
    DataPRO --> TestSetupsList
    DataPRO --> RealtimeModule
    DataPRO --> DatabaseServices
    DataPRO --> DTSViewer
    
    DASFactory --> IService
    DASFactory --> SLICECmds
    DASFactory --> TDASCmds
    
    IService --> DTS_DASConcepts
    IService --> DTS_Storage
    IService --> SLICECmds
    IService --> TDASCmds
    
    DbAPI --> SensorDB
    
    SensorsList --> DTS_Common
    SensorsList --> DTS_DataModel
    HardwareList --> DTS_Common
    GroupList --> DTS_DataModel
    TestSetupsList --> DTS_Common
    
    DTS_Common --> DTS_DataModel
    DTS_DataModel --> DTS_Storage
    DTS_Storage --> DTS_DASConcepts
```

---

## Data Flow

### 1. Hardware Discovery Flow

```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  USB/       │────▶│  DASFactory  │────▶│  IService   │
│  Ethernet   │     │  Discovery   │     │  (Slice)    │
└─────────────┘     └──────────────┘     └─────────────┘
                           │                     │
                           ▼                     ▼
                    ┌──────────────┐     ┌─────────────┐
                    │ IEventAggr.  │────▶│   Modules   │
                    │  Publish     │     │  (Hardware) │
                    └──────────────┘     └─────────────┘
```

The `DASFactory` (`DASFactory.cs:110`) handles:
- USB device enumeration (WinUSB, CDC)
- Ethernet multicast discovery
- Serial port detection

### 2. Test Configuration Flow

```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Sensors   │────▶│    Groups    │────▶│  TestSetup  │
│   Module    │     │    Module    │     │   Module    │
└─────────────┘     └──────────────┘     └─────────────┘
      │                    │                    │
      ▼                    ▼                    ▼
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   SensorDB  │     │  Group.cs    │     │   DbAPI     │
│  (Models)   │     │ (DataModel)  │     │  (Persist)  │
└─────────────┘     └──────────────┘     └─────────────┘
```

### 3. Real-time Data Flow

```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   DAS HW    │────▶│   IService   │────▶│  Realtime   │
│  (SLICE)    │     │  (Stream)    │     │   Module    │
└─────────────┘     └──────────────┘     └─────────────┘
                                               │
                           ┌───────────────────┼───────────────────┐
                           ▼                   ▼                   ▼
                    ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
                    │  UDP Stream │     │   Charts    │     │   Storage   │
                    │  (TMATs)    │     │  (Display)  │     │   (DASDB)   │
                    └─────────────┘     └─────────────┘     └─────────────┘
```

### 4. Database Access Pattern

```
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Modules   │────▶│    DbAPI     │────▶│ SQL Server  │
│  (Views)    │     │ (Static API) │     │ (Database)  │
└─────────────┘     └──────────────┘     └─────────────┘
```

`DbAPI.cs:237-373` exposes static properties for each domain:
```csharp
public static IConnections Connections { get; }
public static IDataRecorders DAS { get; }
public static ISensors Sensors { get; }
public static IGroups Groups { get; }
public static ITestSetups TestSetups { get; }
```

---

## Extension Points

### Adding a New Module

1. Create project in `DataPRO/Modules/[Category]/[ModuleName]/`
2. Create Module class implementing `IModule`:
```csharp
[Module(ModuleName = "MyModule")]
public class MyModule : IModule
{
    public void RegisterTypes(IContainerRegistry containerRegistry)
    {
        containerRegistry.RegisterSingleton<IMyView, MyView>();
        containerRegistry.RegisterSingleton<IMyViewModel, MyViewModel>();
    }
}
```

3. Register in `Bootstrapper.cs:ConfigureModuleCatalog()`
4. Define region in `eAssemblyRegion` enum
5. Add module to solution folder structure

### Adding New Hardware Support

1. Create commands in `DataPRO/[Device]Commands/`
2. Add device handling in `DASFactory.cs:[Device].cs` partial class
3. Create service class in `IService/Classes/[Device]Service.cs`
4. Implement `IDASCommunication` interface

### Adding New Events

1. Create event class in `Common/DTS.Common/Events/`:
```csharp
public class MyEvent : PubSubEvent<MyEventArgs> { }
```

2. Create args class:
```csharp
public class MyEventArgs
{
    public string Data { get; set; }
}
```

3. Publish from ViewModel:
```csharp
_eventAggregator.GetEvent<MyEvent>().Publish(new MyEventArgs { Data = "value" });
```

4. Subscribe in target module:
```csharp
_eventAggregator.GetEvent<MyEvent>().Subscribe(OnMyEvent);
```

---

## Key Technologies

### Frameworks & Libraries

| Technology | Version | Purpose |
|------------|---------|---------|
| .NET Framework | 4.8 | Runtime |
| Prism | 8.x | MVVM framework, modularity |
| Unity | 2.x | Dependency injection container |
| Stateless | 4.2.1 | State machine for DAS operations |
| Newtonsoft.Json | Latest | JSON serialization |
| ComponentOne WPF | 4.0.20203 | Charts and UI controls |
| Xceed WPF Toolkit | 3.0 | Extended WPF controls |

### Communication Protocols

- **USB**: WinUSB, CDC for SLICE devices
- **Ethernet**: TCP/UDP, multicast discovery
- **Serial**: RS-232 for legacy TDAS
- **CAN**: CAN-FD for SLICE Pro

### Database

- **SQL Server**: Primary data storage
- **Stored Procedures**: Versioned SPs for DB compatibility
- **DASFactoryDb**: Local SQLite for device caching

---

## Gotchas and Tech Debt

### Architecture Issues

1. **Static DbAPI Access** (`DbAPI.cs:237-373`)
   - Static properties make testing difficult
   - No interface abstraction for database layer
   - Recommendation: Convert to injected `IDbAPI` service

2. **App.xaml.cs God Class** (4620 lines)
   - Contains licensing, test state, UI logic, database operations
   - Violates Single Responsibility Principle
   - Recommendation: Extract into separate services

3. **Mixed Module Initialization**
   - Some modules use `Initialize()`, others `RegisterTypes()`
   - Example: `SensorsListModule.cs:39` vs `DatabaseServicesModule.cs:38`

### Threading Concerns

1. **UI Thread Dependencies** (`App.xaml.cs:128-161`)
   - `StartTest()` method called from UI, affects global state
   - `RunTestVariables` static class for test configuration

2. **Device Queue Processing** (`DASFactory.cs:127`)
   ```csharp
   private BlockingCollection<Tuple<QueueActions, DeviceHandling>> _queueActionPerDevice;
   ```

### Database Versioning

- Stored procedures are versioned (`DbAPI.cs:34-97`)
- Client and server DB versions must be negotiated
- SP version caching implemented but fragile

### State Machine Complexity

IService uses Stateless for DAS operations (`IService/StateMachine/`):
- States: Prepare, Configure, Arm, Realtime, Download, Diagnose
- Complex state transitions can be difficult to debug

### Legacy Code Patterns

1. **WinForms Interop** in some modules
2. **BackgroundWorker** instead of async/await in places
3. **Mixed logging** - custom `APILogger` and `TextLogger` classes

### Missing Unit Tests

Limited test coverage in:
- `DataPRO/UnitTest/` - only database tests present
- No ViewModel tests
- No service layer tests

---

## Key File References

| Component | File | Lines |
|-----------|------|-------|
| Bootstrapper | `DataPRO/DataPRO/Bootstrapper.cs` | 223 |
| Application Entry | `DataPRO/DataPRO/App.xaml.cs` | 4620 |
| DAS Factory | `DataPRO/DASFactory/DASFactory.cs` | 1712 |
| SLICE Service | `DataPRO/IService/Classes/SLICEService/SLICE Service.cs` | 480+ |
| Database API | `DataPRO/DbAPI/DbAPI.cs` | 375 |
| Sensor Database | `DataPRO/SensorDB/SensorDB.cs` | 983 |
| Group Model | `Common/DTS.Common.DataModel/Group.cs` | 2452 |
| Events | `Common/DTS.Common/Events/*.cs` | 129 events |

---

## Module Summary

| Category | Modules |
|----------|---------|
| **Prepare** | SensorsList, HardwareList, GroupList, GroupChannelList, TestSetupsList, CachedItemsList, ChannelCodes |
| **Configure** | SensorSettingsModule, SoftwareFilters, AddEditHardware, GroupImport, Diagnostics |
| **System** | DatabaseServices, DBImportExport, RealtimeSettings, TestSettings, ISOSettings, QASettings, UISettings, TablesSettings, PowerAndBattery |
| **Realtime** | RealtimeModule, StatusAndProgressBar |
| **Viewer** | DTS.Viewer, Graph, GraphList, Filter, ChartOptions, TestModification, TestSummaryList, Navigation, AddCalculatedChannel, ViewerSettings |
| **Reports** | PSDReport, PSDReportSettings, PSDReportResults, PedestrianAndHeadReports |
| **ISO** | ExtraProperties, RegionOfInterestChannels |