DP44/enriched-partialglm/Common/DTS.Common.Serialization.md

---
source_files:
  - Common/DTS.Common.Serialization/StringWriterWithEncoding.cs
  - Common/DTS.Common.Serialization/TickEventHandler.cs
  - Common/DTS.Common.Serialization/InvariantStreamWriter.cs
  - Common/DTS.Common.Serialization/BadCRCBypass.cs
  - Common/DTS.Common.Serialization/IProgressAware.cs
  - Common/DTS.Common.Serialization/ToyotaCsv.File.cs
  - Common/DTS.Common.Serialization/EventHandlers.cs
  - Common/DTS.Common.Serialization/Test.IConvertable.cs
  - Common/DTS.Common.Serialization/TestSetup.TestObject.Channel.cs
  - Common/DTS.Common.Serialization/Test.Module.IConvertable.cs
  - Common/DTS.Common.Serialization/Test.Module.Channel.IConvertable.cs
  - Common/DTS.Common.Serialization/TestSetup.TestObject.DASHardware.cs
  - Common/DTS.Common.Serialization/TestSetup.Sensor.cs
  - Common/DTS.Common.Serialization/File.Writer.cs
  - Common/DTS.Common.Serialization/IReadable.cs
  - Common/DTS.Common.Serialization/TestSetup.Graph.cs
  - Common/DTS.Common.Serialization/FilteredData.cs
  - Common/DTS.Common.Serialization/File.Writer.CharacterCountingStreamWriter.cs
  - Common/DTS.Common.Serialization/Diadem.File.cs
  - Common/DTS.Common.Serialization/TestSetup.TestObject.DASHardware.DASChannel.cs
  - Common/DTS.Common.Serialization/File.cs
  - Common/DTS.Common.Serialization/File.Reader.cs
  - Common/DTS.Common.Serialization/TestSetup.Graph.Channel.cs
  - Common/DTS.Common.Serialization/EventInfoAggregate.cs
  - Common/DTS.Common.Serialization/IWriteable.cs
generated_at: "2026-04-16T11:37:25.731360+00:00"
model: "zai-org/GLM-5-FP8"
schema_version: 1
sha256: "2a2f1c17484ea7d0"
---

# DTS.Serialization Module Documentation

## 1. Purpose

The `DTS.Serialization` module provides a framework for serializing and deserializing test data to various file formats (Toyota CSV, Diadem, etc.). It defines abstract base classes (`File`, `File.Reader<T>`, `File.Writer<T>`) and interfaces (`IReadable<T>`, `IWritable<T>`, `IReader<T>`, `IWriter<T>`) that establish a consistent pattern for file import/export operations. The module also contains domain models for test setups (`TestSetup`, `Test`, `FilteredData`) and supports progress reporting during long-running serialization operations via event handlers.

---

## 2. Public Interface

### Core File Abstractions

#### `File` (abstract partial class)
- **Signature:** `public abstract partial class File : Exceptional`
- **Constructor:** `File(string formatName)` - Initializes with a format name.
- **Properties:**
  - `string FormatName` (get, private set) - The format name of this file.
  - `int DefaultEncoding` (get, set) - Defaults to `Encoding.UTF8.CodePage`.
  - `static string BaseExportDirectory` (get, set) - Static base path for exports.
  - `static bool UseLegacyTDCSoftwareFiltering` (get, set) - Controls filtered data sample adjustment.
  - `Common.Enums.IsoViewMode ISOViewMode` (get, set)
- **Methods:**
  - `virtual void SetEUData(string channelID, FilteredData fd)` - Stores EU data for linearized channels.
  - `virtual FilteredData GetEUData(string channelID)` - Retrieves EU data.
  - `virtual int GetChannelNumberFromChannelFileName(string channelFileName)` - Returns -1 by default; overridden in subclasses.
  - `protected string EnsureTrailingBackslashOnPathString(string path)` - Normalizes path strings.

#### `File.Reader<T>` (abstract class)
- **Signature:** `public abstract class Reader<T> : Exceptional, IReader where T : File`
- **Constructor:** `protected Reader(T fileType)` - Protected; readers created via File-level factory.
- **Properties:**
  - `protected T FileType` - The associated file type.
  - `protected ChannelFilenameComparer ChFileCompare` - Comparer for channel filenames.
- **Nested Class:** `ChannelFilenameComparer : IComparer<string>` - Compares channel filenames using `GetChannelNumberFromChannelFileName`.

#### `File.Writer<T>` (abstract partial class)
- **Signature:** `public abstract partial class Writer<T> : Exceptional, IWriter where T : File`
- **Constructor:** `public Writer(T fileType, int encoding)`
- **Properties:**
  - `protected T FileType` - The associated file type.
  - `virtual int DefaultEncoding` (get, set)
- **Nested Class:** `CharacterCountingStreamWriter : StreamWriter` - Counts characters written via `WriteLine(string)`. Property: `uint CharactersCounted`.

### Interfaces

#### `IReadable` / `IReadable<T>`
- Marker interfaces for readable file types.
- `IReadable<T>.Importer` returns `IReader<T>`.

#### `IWritable` / `IWritable<T>`
- Marker interfaces for writable file types.
- `IWritable<T>.Exporter` returns `IWriter<T>`.

#### `IReader<T>`
- **Method:** `void Read(string pathname, out T target)` - Deserializes file into target object.

#### `IWriter<T>`
- **Methods:**
  - `void Write(string pathname, string id, T target, bool bFiltering, bool includeGroupNameInISOExport, double minStartTime, int dataCollectionLength)`
  - `void Write(string pathname, string id, string dataFolder, T target, bool bFiltering, bool includeGroupNameInISOExport, FilteredData fd, Test.Module.Channel tmChannel, int channelNumber, BeginEventHandler onBeginEvent, CancelEventHandler onCancelEvent, EndEventHandler onEndEvent, TickEventHandler onTickEvent, ErrorEventHandler onErrorEvent, CancelRequested cancelRequested, double minStartTime, int dataCollectionLength)`
  - `void Initialize(...)` - Same parameters as Write minus minStartTime and dataCollectionLength.

#### `IProgressAware`
- **Events:** `OnBegin`, `OnEnd`, `OnTick`, `OnCancel`, `OnError`

### Event Delegates

- `delegate void TickEventHandler(object sender, double percentageComplete)`
- `delegate void BeginEventHandler(object sender, uint numberOfTicks)`
- `delegate void EndEventHandler(object sender)`
- `delegate void CancelEventHandler(object sender)`
- `delegate void ErrorEventHandler(object sender, Exception ex)`
- `delegate bool CancelRequested()`

### Utility Classes

#### `StringWriterWithEncoding`
- **Signature:** `public sealed class StringWriterWithEncoding : StringWriter`
- **Constructor:** `StringWriterWithEncoding(Encoding encoding)`
- **Property:** `override Encoding Encoding` - Returns the encoding passed to constructor.

#### `InvariantStreamWriter`
- **Signature:** `public class InvariantStreamWriter : System.IO.StreamWriter`
- **Constructors:** `InvariantStreamWriter(string path)`, `InvariantStreamWriter(string path, bool bAppend)`
- **Property:** `override IFormatProvider FormatProvider` - Always returns `CultureInfo.InvariantCulture`.

#### `FilteredData`
- **Signature:** `public class FilteredData : Exceptional, IComparable<FilteredData>`
- **Constructor:** `FilteredData(string filterDescription, double filterFrequencyHz, double[] data, int absoluteDisplayOrder)`
- **Properties:** `FilterDescription`, `FilterFrequencyHz`, `Data`, `AbsoluteDisplayOrder` (init-only)
- **Method:** `int CompareTo(FilteredData other)` - Compares by `AbsoluteDisplayOrder`.

### Domain Models

#### `TestSetup` (partial class hierarchy)
- `TestSetup : Exceptional`
  - `TestSetup.Sensor : Exceptional` - Properties: `FilterClass`, `Name`, `Polarity`, `Position`, `Range`, `Version`
  - `TestSetup.Graph : Exceptional` - Properties: `Channels`, `Name`, `HardwareChannelName`, `DisplayName`, `Version`, `Identifier` (Guid), `IsSingleChannelGraph`, `FirstChannel`, `FirstTestChannel`
    - `TestSetup.Graph.Channel : Exceptional` - Properties: `ChannelId`, `ChannelGroupName`, `Name`, `SensorName`, `AxisUnit`, `SerialNumber`, `ParentTestModule`, `TestChannel`
  - `TestSetup.TestObject : Exceptional`
    - `TestSetup.TestObject.TOChannel : Exceptional` - Properties: `Name`, `SensorName`, `Version`
    - `TestSetup.TestObject.DASHardware : Exceptional` - Properties: `DASChannels`, `SampleRate`, `SerialNumber`, `Version`
      - `TestSetup.TestObject.DASHardware.DASChannel : Exceptional` - Properties: `Location`, `MeasurementUnits`, `Name`, `NumberOfSamples`, `SensorName`, `SerialNumber`, `Version`

#### `Test` (partial class with nested interfaces)
- `Test.IConvertable` - Interface for converting to/from `Test` objects.
  - `Test ToDtsSerializationTest()`
  - `void FromDtsSerializationTest(Test test, ReportErrors reportErrors)`
- `Test.Module.IConvertable` - Interface for converting to/from `Test.Module`.
- `Test.Module.Channel.IConvertable` - Interface for converting to/from `Test.Module.Channel`.

#### `EventInfoAggregate`
- Aggregates event info from multiple DAS units.
- **Properties:** `EventId`, `EventDescription`, `DurationSeconds`, `GUID`, `HasBeenDownloaded`, `WasTriggered`, `NumberOfChannels`, `NumberOfSamples`, `NumberOfBytes`, `Faulted`, `EventNumber`
- **Methods:** `GetEvent()`, `GetEvent(bool bClear)`, `GetEventIndex(IDASCommunication idas)`, `GetDasList()`, `Add(DownloadReport.EventInfo newEvent)`

### File Format Implementations

#### `ToyotaCsv.File`
- **Signature:** `public partial class File : Serialization.File, IWritable<Test>`
- **Constructor:** `File()` - Base format name "Toyota CSV".
- **Properties:** `static string Extension => ".csv"`, `IWriter<Test> Exporter`

#### `Diadem.File`
- **Signature:** `public partial class File : Serialization.File, IWritable<Test>`
- **Constructor:** `File(bool bUseEVG20, TestPlan plan)`
- **Properties:** `static string Extension => ".dat"`, `IWriter<Test> Exporter`, `UseIsoCodeForDiadem200`, `UseZeroForUnfiltered`, `ChannelName200Option`, `UserComment201Option`, `Reserved1_301Option`, `Reserved2_302Option`
- **Enums:** `DiademOptions`, `DiademOptionsReserved1`, `DiademOptionsReserved2`

### UI Components

#### `BadCRCBypass` (Windows Forms)
- **Signature:** `public partial class BadCRCBypass : Form`
- **Properties:** `string FileName`, `bool RememberChoice`
- **Events:** `btnOK_Click`, `btnCancel_Click` - Set `DialogResult` and close.

---

## 3. Invariants

1. **Reader/Writer Construction:** `File.Reader<T>` and `File.Writer<T>` are intended to be constructed only by their hosting `File` class via factory methods or properties (Writer constructor is public, Reader constructor is protected).

2. **Encoding Consistency:** `InvariantStreamWriter` always uses `CultureInfo.InvariantCulture` regardless of system locale—this ensures consistent numeric formatting across different regional settings.

3. **Property<T> Pattern:** Domain model classes use `Property<T>` wrappers with namespace-qualified names for property tracking. These are initialized with default values in constructors.

4. **Version Defaults:** All `Version` properties in `TestSetup` nested classes default to `"1.0.0.0"`.

5. **Channel Filename Ordering:** `ChannelFilenameComparer` relies on `GetChannelNumberFromChannelFileName` returning -1 for non-SLICE/TDAS file types, resulting in standard string comparison.

6. **CharacterCountingStreamWriter:** Only `WriteLine(string)` is overridden; other write methods do not update `CharactersCounted`.

---

## 4. Dependencies

### External Dependencies (inferred from imports)
- `System.Text` - Encoding support
- `System.IO` - Stream/StreamWriter base classes
- `System` - Exception handling, IFormatProvider
- `System.Windows.Forms` - BadCRCBypass dialog
- `System.Collections.Generic` - List, Dictionary, IComparer
- `System.Linq` - LINQ operations
- `System.Globalization` - CultureInfo.InvariantCulture
- `System.Xml.Serialization` - XmlIgnoreAttribute

### Internal Dependencies
- `DTS.Common.Utilities` / `DTS.Utilities` - `Exceptional` base class
- `DTS.Common.Utilities.DotNetProgrammingConstructs` / `DTS.Utilities.DotNetProgrammingConstructs` - `Property<T>` wrapper class
- `DTS.Common.ISO` - ISO-related types (used in Diadem)
- `DTS.Common.Utilities.Logging` - `APILogger` (used in EventInfoAggregate)
- `DTS.DASLib.Service` - `IDASCommunication`, `DownloadReport` (used in EventInfoAggregate)
- `Slice.Control.Event` - Event class (used in EventInfoAggregate)

### Dependents (not shown in source)
- Unknown from source alone; likely consumed by export/import modules and test data management components.

---

## 5. Gotchas

1. **CharacterCountingStreamWriter Partial Implementation:** Only `WriteLine(string)` updates the character count. Using `Write()`, `WriteLine(char[])`, or other overloads will not be counted.

2. **IProgressAware Interface Visibility:** The `IProgressAware` interface has no access modifier explicitly shown, making it `internal` by default. It also references `DTS.Serialization.EventHandler` in XML comments, but the actual type is `TickEventHandler` (likely a documentation typo).

3. **Documentation Typos:** In `IProgressAware`, the XML comment for `OnTick` references `DTS.Seralization.EventHandler` (missing 'i' in Serialization).

4. **EventInfoAggregate Mismatched Data Handling:** When `Add()` is called with mismatched event data (different TestIDs, durations, sample counts), the code logs warnings but continues processing, potentially using `Math.Min` for some values. This could lead to data truncation.

5. **TestSetup.Graph.Channel Private Constructor:** Has a private parameterless constructor that sets `ChannelId = "Not Set"`, but public constructors require either a `Test.Module.Channel` or `channelId` string.

6. **FilteredData AbsoluteDisplayOrder:** The property is init-only with a default of -1, but the constructor parameter is required. The `CompareTo` method will use -1 if not properly initialized.

7. **Diadem.File WriterParent Cast:** The `Exporter` property casts `_Exporter` to `Writer` to set `WriterParent`, suggesting tight coupling between the File and Writer classes not expressed in interfaces.