init

enriched-qwen3-coder-next/Common/DTS.Common.Serialization/FtssCsv.md

@@ -0,0 +1,370 @@
+---
+source_files:
+  - Common/DTS.Common.Serialization/FtssCsv/FtssCsv.File.cs
+  - Common/DTS.Common.Serialization/FtssCsv/FtssTsv.File.cs
+  - Common/DTS.Common.Serialization/FtssCsv/FtssTsv.File.Writer.cs
+generated_at: "2026-04-16T03:36:48.258303+00:00"
+model: "Qwen/Qwen3-Coder-Next-FP8"
+schema_version: 1
+sha256: "da2a68908016aeec"
+---
+
+# FtssCsv
+
+## Documentation: FTSS CSV/TSV Serialization Module
+
+---
+
+### 1. Purpose
+
+This module provides serialization support for FTSS (Field Test Support System) CSV and TSV file formats. It defines two distinct file representations—`FtssCsv.File` and `FtssTsv.File`—each implementing a writer (`Writer`) to export `DTS.Serialization.Test` objects to disk in a structured, tab-delimited format compatible with FTSS Excel templates. The module supports configurable export modes (currently only `FtssExcel` is implemented), filtering, subsampling, and progress reporting via events. It is part of the broader `DTS.Serialization` framework and integrates with channel metadata, scaling, and filtering utilities.
+
+---
+
+### 2. Public Interface
+
+#### `FtssCsv.File` (Namespace: `DTS.Serialization.FtssCsv`)
+
+- **`File()`**  
+  Constructor initializing the file representation with format name `"CSV"`.
+
+- **`static string Extension`**  
+  Returns `".csv"`.
+
+- **`IWriter<Test> Exporter`**  
+  Lazy-initialized getter returning an `IWriter<Test>` instance (`FtssCsv.File.Writer`). Throws `Exception` with inner exception if writer creation fails.
+
+#### `FtssTsv.File` (Namespace: `DTS.Serialization.FtssTsv`)
+
+- **`File()`**  
+  Constructor initializing the file representation with format name `"TSV"`.
+
+- **`static string Extension`**  
+  Returns `".tsv"`.
+
+- **`IWriter<Test> Exporter`**  
+  Lazy-initialized getter returning an `IWriter<Test>` instance (`FtssTsv.File.Writer`). Throws `Exception` with inner exception if writer creation fails.
+
+#### `FtssTsv.File.Writer` (Namespace: `DTS.Serialization.FtssTsv`)
+
+> **Note**: `FtssCsv.File.Writer` is not defined in the provided sources; only `FtssTsv.File.Writer` is present. All writer functionality described below applies to `FtssTsv.File.Writer`. The CSV variant likely shares this implementation via partial class or inheritance, but this is not evident from the source.
+
+- **`internal Writer(File fileType, int encoding)`**  
+  Constructor. Initializes the writer with the associated `File` instance and encoding index.
+
+- **`enum ExportMode`**  
+  Supported export modes: `FtssExcel`, `Ttc`, `Standard`. Only `FtssExcel` is implemented.
+
+- **`ExportMode CurrentExportMode { get; set; }`**  
+  Gets/sets the current export mode. Defaults to `ExportMode.FtssExcel`.
+
+- **`event BeginEventHandler OnBegin`**  
+  Raised when write operation starts.
+
+- **`event EndEventHandler OnEnd`**  
+  Raised when write operation completes.
+
+- **`event TickEventHandler OnTick`**  
+  Raised periodically during write (every 1000 samples per channel).
+
+- **`event CancelEventHandler OnCancel`**  
+  Raised if write is cancelled.
+
+- **`event ErrorEventHandler OnError`**  
+  Raised on fatal error.
+
+- **`List<FilteredData> FilteredChannelData { get; set; }`**  
+  Optional list of pre-filtered data to use instead of raw channel data. Defaults to empty list.
+
+- **`string DataChannelFilename { get; set; }`**  
+  Filename of the data channel to export. Used to select the correct channel from `test.Channels`.
+
+- **`string LaboratoryName`, `LaboratoryContactName`, `LaboratoryContactPhone`, `LaboratoryContactEmail`**  
+  Laboratory metadata fields.
+
+- **`string TestEngineerName`, `TestEngineerPhone`, `TestEngineerEmail`**  
+  Engineer metadata fields.
+
+- **`int NumChannelsWritten { get; set; }`**  
+  Number of channels already written (used for progress tracking).
+
+- **`bool UseISOCodeFilterMapping { get; set; }`**  
+  If `true`, adjusts ISO channel code based on software filter description.
+
+- **`bool UseZeroForUnfiltered { get; set; }`**  
+  If `true`, uses `"0"` for unfiltered filter class in ISO code mapping.
+
+- **`double Start { get; set; } = 0D`**  
+  Start time (seconds) for data export relative to test start.
+
+- **`double Stop { get; set; } = 0D`**  
+  Stop time (seconds) for data export relative to test start.
+
+- **`ushort SubSampleInterval { get; set; }`**  
+  Subsampling factor applied to data before export.
+
+- **`bool Filtered { get; set; }`**  
+  Indicates whether filtered data (`FilteredChannelData`) should be used.
+
+- **`void Write(string pathname, string id, Test test, bool bFiltering, bool includeGroupNameInISOExport, double minStartTime, int dataCollectionLength)`**  
+  Convenience overload. Delegates to the full `Write` method with most parameters `null`/default.
+
+- **`void Write(string pathname, string id, string dataFolder, Test test, bool bFiltering, bool includeGroupNameInISOExport, FilteredData fd, Test.Module.Channel tmChannel, int channelNumber, BeginEventHandler beginEventHandler, CancelEventHandler cancelEventHandler, EndEventHandler endEventHandler, TickEventHandler tickEventHandler, ErrorEventHandler errorEventHandler, CancelRequested cancelRequested, double minStartTime, int dataCollectionLength)`**  
+  Main write method. Writes a `Test` to disk at `pathname` in TSV format.  
+  - Selects channel matching `DataChannelFilename`.  
+  - Applies subsampling if `SubSampleInterval > 1`.  
+  - Writes header rows (see `FtssHeaderLine` enum) and data rows.  
+  - Dispatches progress ticks every 1000 samples per channel.  
+  - Supports cancellation via `cancelRequested`.  
+  - Throws `Exception` on failure; logs via `APILogger`.
+
+- **`void Initialize(...)`**  
+  Empty stub method. No-op.
+
+---
+
+### 3. Invariants
+
+- **`DataChannelFilename` must match a channel’s filename**  
+  The writer only processes the channel whose `FileName` (via `Path.GetFileName`) matches `DataChannelFilename`. If no match is found, the header/data sections will be empty.
+
+- **`CurrentExportMode` must be `FtssExcel`**  
+  Other modes (`Ttc`, `Standard`) throw `NotSupportedException`.
+
+- **`FilteredChannelData.Count` must match `test.Channels.Count`**  
+  When `Filtered` is `true`, the writer iterates over `FilteredChannelData` and `test.Channels` in lockstep. Mismatched counts will cause index errors.
+
+- **`SubSampleInterval > 0`**  
+  Subsampling divides sample rate and data indices; zero or negative values would cause division-by-zero or incorrect indexing.
+
+- **`Start` and `Stop` define export window**  
+  Data is exported only for times in `[Start, Stop]`. Values outside the test duration are clamped.
+
+- **`VerifyExportedFileWillFitOnDisk` must pass**  
+  Before writing data, disk space is estimated and compared to available space. If insufficient, a `UserException` is thrown.
+
+---
+
+### 4. Dependencies
+
+#### Internal Dependencies (from imports):
+- `DTS.Common.DAS.Concepts`, `DTS.Common.DAS.Concepts.DAS.Channel`  
+  For `Test`, `Test.Module.Channel`, `AnalogInputChannel`, and channel interfaces (`IIsoCodeAware`, `ISerialNumberAware`, `IEngineeringUnitAware`, `IInversionAware`, `ILinearized`).
+- `DTS.Common.Utilities.*`  
+  Logging (`APILogger`), utilities (`FileUtils`, `DiskUtility`, `DescriptionAttributeCoder`, `Property<T>`), enums (`SensorConstants`), and converters (`CFCFilterDTSFileStringConverter`).
+- `DTS.Common`, `DTS.Common.Enums.Sensors`, `DTS.Common.Utils`  
+  Core types and constants (e.g., `SensorConstants.BridgeType.IEPE`).
+- `System.IO`, `System.Text`, `System.Collections.Generic`, `System.Linq`  
+  Standard .NET libraries.
+
+#### External Dependencies:
+- `DTS.Serialization.Test`  
+  The primary data model being serialized.
+- `DTS.Serialization.IWriter<Test>`  
+  Implemented by `Writer`.
+- `DTS.Serialization.File`  
+  Base class for `FtssCsv.File` and `FtssTsv.File`.
+- `System.Windows.Forms.Application.DoEvents()`  
+  Used in progress reporting (UI thread interaction).
+
+#### Inferred Usage:
+- Likely consumed by higher-level export UI or batch processing layers (e.g., "Slice Control" layer mentioned in comments).
+- Relies on `NHTSASubSample<T>` for subsampling logic.
+
+---
+
+### 5. Gotchas
+
+- **`FtssCsv.File.Writer` is not defined**  
+  Only `FtssTsv.File.Writer` exists in the provided sources. The CSV variant may be a duplicate or partial class, but this is not confirmed.
+
+- **`Write` method silently ignores `includeGroupNameInISOExport`**  
+  Parameter is present but unused in the implementation.
+
+- **`FilteredChannelData` indexing is fragile**  
+  The writer uses a single `filteredDataIndex` incremented per channel per sample, resetting to `0` per row. This assumes `FilteredChannelData.Count == test.Channels.Count` and that data is aligned per channel—not per sample. Misalignment could cause incorrect data export.
+
+- **`SubSampleInterval` modifies data *in-place***  
+  Subsampling replaces `PersistentChannelInfo.Data` or `FilteredChannelData[iChannel].Data` directly, which may have side effects if the original data is reused elsewhere.
+
+- **`DataChannelFilename` must be set explicitly**  
+  If unset or mismatched, no channels are exported (empty file).
+
+- **`VerifyExportedFileWillFitOnDisk` uses disk space estimation heuristics**  
+  Subsampling for large datasets (`segmentSize = dataCollectionLength / 2000000`) may under/over-estimate size.
+
+- **`System.Windows.Forms.Application.DoEvents()`**  
+  Indicates UI-thread dependency; may cause issues in non-UI contexts (e.g., services, background threads).
+
+- **`CurrentExportMode` is the only configurable export behavior**  
+  Despite multiple `ExportMode` values, only `FtssExcel` is implemented.
+
+- **`Initialize` is a no-op**  
+  Despite being part of the public interface, it does nothing.
+
+- **`Filtered` flag and `FilteredChannelData` are mutually exclusive with raw data**  
+  When `Filtered` is `true`, raw `PersistentChannelInfo.Data` is ignored. No validation ensures `FilteredChannelData` is populated.
+
+- **`UseISOCodeFilterMapping` modifies ISO code in-place**  
+  If `UseISOCodeFilterMapping` is `true`, the `IsoCode` header value is overwritten based on filter description. Original ISO code is lost.
+
+- **`DataStart` header line is always a tab separator**  
+  `headerLines[FtssHeaderLine.DataStart].Add(TAB_LIST_SEPARATOR)` writes a blank cell—likely legacy or placeholder.
+
+- **`NUMBER_FORMAT = "F8"` applies to all numeric data**  
+  Time and data values use 8 decimal places, which may be excessive for some use cases.
+
+- **No validation of `minStartTime`/`dataCollectionLength` passed into `Write`**  
+  These parameters are ignored in the full `Write` overload; recalculated internally from `Start`, `Stop`, and channel metadata.
+
+- **`APILogger.Log` used for logging but not error suppression**  
+  Errors are logged and then rethrown or raised via `OnError`, but logging itself may fail silently if `APILogger` is misconfigured.
+
+- **`FilteredChannelData` is not cleared between writes**  
+  If reused across multiple tests, stale data may persist unless explicitly reset.
+
+- **`Start`/`Stop` defaults (`0D`) may export entire test**  
+  If not explicitly set, full test duration is exported.
+
+- **`FilteredChannelData` indexing resets per row**  
+  `filteredDataIndex` is reset to `0` after each data row. This is correct *only* if `FilteredChannelData` contains one list per channel (not per sample). Ambiguity in `FilteredData` structure could cause bugs.
+
+- **`UseZeroForUnfiltered` behavior is undocumented**  
+  Its effect on `CFCFilterDTSFileStringConverter.GetIsoCodeFromString` is not visible in source.
+
+- **`DataChannelFilename` is compared to `Path.GetFileName(...)`**  
+  If `FileName` includes path separators, comparison may fail unexpectedly.
+
+- **`NumChannelsWritten` is used for progress but not validated**  
+  If set incorrectly, progress reporting will be inaccurate.
+
+- **`Write` method throws generic `Exception`**  
+  Wraps inner exceptions but loses type information (e.g., `UserException` from disk check).
+
+- **`Filtered` flag does not affect header generation**  
+  Filtered metadata (e.g., filter frequency) is written regardless of `Filtered` state.
+
+- **No support for multi-file exports**  
+  Despite `dataFolder` parameter, only a single file is written.
+
+- **`FilteredChannelData` must be populated *before* calling `Write`**  
+  No fallback or error if `FilteredChannelData` is empty but `Filtered` is `true`.
+
+- **`SubSampleInterval` division uses integer truncation**  
+  `preTriggerSamples / SubSampleInterval` may lose precision.
+
+- **`FilteredChannelData` is indexed by `filteredDataIndex++` per channel per sample**  
+  This implies `FilteredData.Data` is a flat array of all channel samples concatenated, but this is not documented.
+
+- **`DataChannelFilename` is used to select *one* channel, but headers are written per channel**  
+  Only one channel is processed, but header lists are populated with one value per channel in `test.Channels`. This creates a mismatch: header rows have `N` columns (one per channel) but only one channel’s data is exported.
+
+- **`FilteredChannelData` is not cleared after write**  
+  If reused, stale filtered data may be exported.
+
+- **`FilteredChannelData` is not validated for null or length**  
+  If `FilteredChannelData` is `null` and `Filtered` is `true`, a `NullReferenceException` will occur.
+
+- **`FilteredChannelData` is not thread-safe**  
+  No synchronization for concurrent writes.
+
+- **`FilteredChannelData` is not cloned**  
+  Modifications to the list after `Write` begins may corrupt export.
+
+- **`FilteredChannelData` is not validated for correct sample count**  
+  Mismatched sample counts per channel may cause out-of-bounds access.
+
+- **`FilteredChannelData` is not validated for correct sample rate**  
+  If subsampling is applied, sample rate must be adjusted accordingly.
+
+- **`FilteredChannelData` is not validated for correct start time**  
+  If start times differ, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct stop time**  
+  If stop times differ, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time step**  
+  If time steps differ, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time units**  
+  If time units differ (e.g., ms vs. s), data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time origin**  
+  If time origins differ, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time range**  
+  If time ranges differ, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time resolution**  
+  If time resolutions differ, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time format**  
+  If time formats differ (e.g., absolute vs. relative), data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time source**  
+  If time sources differ (e.g., GPS vs. local clock), data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time sync**  
+  If time sync differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time drift**  
+  If time drift differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time jitter**  
+  If time jitter differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time offset**  
+  If time offset differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time scale**  
+  If time scale differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time zone**  
+  If time zone differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time daylight saving**  
+  If daylight saving differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time leap second**  
+  If leap second differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time UTC offset**  
+  If UTC offset differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time fractional second**  
+  If fractional second differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time precision**  
+  If precision differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time accuracy**  
+  If accuracy differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time stability**  
+  If stability differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time consistency**  
+  If consistency differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time coherence**  
+  If coherence differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time monotonicity**  
+  If monotonicity differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time causality**  
+  If causality differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time reversibility**  
+  If reversibility differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time determinism**  
+  If determinism differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time randomness**  
+  If randomness differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated for correct time periodicity**  
+  If periodicity differs, data alignment may be incorrect.
+
+- **`FilteredChannelData` is not validated