DP44/enriched-qwen3-coder-next/Common/DTS.Common.Import/Parsers/CSV.md

---
source_files:
  - Common/DTS.Common.Import/Parsers/CSV/AbstractCSVParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/Version0CSVTestParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/CSVFile.cs
  - Common/DTS.Common.Import/Parsers/CSV/Version6CSVTestParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/Version3CSVSensorParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/Version5CSVTestParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/Version4CSVSensorParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/CSVGroupImport.cs
  - Common/DTS.Common.Import/Parsers/CSV/DTSCSVTestSetupParser.cs
  - Common/DTS.Common.Import/Parsers/CSV/DTSCSVSensorsParser.cs
generated_at: "2026-04-16T02:08:24.981291+00:00"
model: "Qwen/Qwen3-Coder-Next-FP8"
schema_version: 1
sha256: "2bff5b23b8f73415"
---

# CSV Import Module Documentation

## 1. Purpose

This module provides a versioned CSV parsing infrastructure for importing sensor and test setup data into the DTS system. It supports two distinct import workflows: **sensor data import** (handled by `DTSCSVSensorsParser`) and **test setup data import** (handled by `DTSCSVTestSetupParser`). The module uses an abstract base class (`AbstractCSVParser`) and concrete version-specific parsers to handle evolving CSV formats across versions 0 through 6. It decouples parsing logic from business logic, delegates group creation to `CSVGroupImport`, and integrates with calibration, sensor, and hardware data models to construct `TestTemplate` and `ImportObject` instances for downstream processing.

## 2. Public Interface

### `AbstractCSVParser` (abstract base class)
- **`int Version { get; }`**  
  Returns the CSV format version supported by this parser (e.g., 3, 4).
- **`void Initialize(ICalibrationImport import, ZeroMethodOptions zmOptions, IImportNotification importNotification, bool importCreateDynamicGroups, bool useISOCodeFilterMapping, bool useZeroForUnfiltered)`**  
  Injects dependencies and configuration flags required for parsing. Must be called before `ParseVersion`.
- **`abstract void ParseVersion(CSVImportTags.Tags field, string val, ParseParameters pp)`**  
  Parses a single field (`field`) from a sensor CSV row, using value `val`. Errors are appended to `pp.Errors`. Behavior is version-specific.

### `Version0CSVTestParser`
- **`int Version => 0`**
- **`void ParseVersion(CsvReader csvReader, TestSetupImportData tsid)`**  
  Parses test setup metadata (e.g., `PostTriggerSec`, `SampleRate`, `RecordingMode`) from a CSV with a simple header-value pair format. Reads two rows: header names and values.

### `Version3CSVSensorParser`
- **`int Version => 3`**
- **`void ParseVersion(CSVImportTags.Tags field, string sVal, ParseParameters pp)`**  
  Handles `GroupName` and `GroupType` fields. Enforces non-empty values, uniqueness of sensor serial numbers in group lookups, and consistency of `TestObject` per group (throws `Exception("Parse error")` on conflict if `ImportCreateDynamicGroups` is false).

### `Version4CSVSensorParser`
- **`int Version => 4`**
- **`void ParseVersion(CSVImportTags.Tags field, string sVal, ParseParameters pp)`**  
  Parses DAS-related (`DASSerialNumber`, `DASChannelIndex`), streaming (`StreamProfile`, `UDPAddress`, `TimeChannelId`, `DataChannelId`, `TmNSConfig`, `IRIGTimeDataPacketIntervalMS`, `TMATSIntervalMS`), UART (`BaudRate`, `DataBits`, `StopBits`, `Parity`, `DataFormat`), and user-defined fields (`TestUserCode`, `TestUserChannelName`, `TestIsoCode`, `TestIsoChannelName`). Updates `pp.SensorData` and lookup dictionaries.

### `Version5CSVTestParser`
- **`int Version => 5`**
- **`void ParseVersion(CsvReader csvReader, TestSetupImportData tsid)`**  
  Parses clock synchronization settings (e.g., `ClockMasterInputType`, `ClockMasterOutputType`, `ManageClocksOutsideDPMaster`) from a header-value row format.

### `Version6CSVTestParser`
- **`int Version => 6`**
- **`void ParseVersion(CsvReader csvReader, TestSetupImportData tsid)`**  
  Parses per-DAS hardware configuration (e.g., `DASSerial`, `DASSampleRate`, `PTPDomainId`, `ClockMaster`) in a table format. Reads header row, then iterates rows until an empty line or non-matching field is encountered.

### `CSVFile`
- **`static bool IsInUse(string filename)`**  
  Returns `true` if the file is locked (IOException on `File.ReadAllLines`), else `false`.
- **`static bool IsCSVFileForTestSetupImport(string filename, CsvImportOptions csvImportOptions)`**  
  Returns `true` if the first field of the first row is `"Version"`.
- **`static int GetCsvVersion(string filename)`**  
  Returns the integer value in the first column of the second row if the first row starts with `"Version"`, else `0`.
- **`int LineCount { get; }`**  
  Returns the number of lines in the file (0 on error or null filename).

### `CSVGroupImport`
- **`ParseParameters ParseParameters { get; set; }`**
- **`Tuple<TestTemplate, List<IGroup>> CreateGroups(List<SensorData> sensors, Dictionary<string, List<TsetSetupImportSensorInfo>> groupSensorLookup, TestTemplate testTemplate, bool createDynamicGroups, List<IGroup> staticGroups, Action<double> setProgress)`**  
  Creates `IGroup` instances from `groupSensorLookup`, assigns sensor/channel properties (ISO/user codes, DAS mapping, calibration defaults), and adds groups to `testTemplate`. Updates `setProgress`.
- **`Dictionary<string, List<TsetSetupImportSensorInfo>> GetGroupSensorLookup(List<SensorData> sensors, Dictionary<string, string> sensorGroupNameLookup, Dictionary<string, List<string>> groupNameSensorListLookup)`**  
  Builds a mapping from group name to list of `TsetSetupImportSensorInfo` (containing sensor/DAS parameters) using either `sensorGroupNameLookup` or `groupNameSensorListLookup`.

### `DTSCSVTestSetupParser`
- **`void Parse(ref ImportObject importObject)`**  
  Orchestrates test setup import: validates CSV format, parses test setup data via `ParseTestSetup`, creates `TestTemplate`, assigns hardware, clock sync, and tags, then delegates group creation to `AssignGroupsToTestSetup`. Adds result to `importObject`.

### `DTSCSVSensorsParser`
- **`void Parse(ref ImportObject importObject)`**  
  Orchestrates sensor import: reads CSV, populates `SensorData` and `SensorCalibration` objects via `PopulateSensor`, handles sensitivity unit corrections, and adds sensors/calibrations to `importObject`. Reports errors and warnings.

## 3. Invariants

- **CSV Format Validation**:  
  - A CSV is recognized as a test setup file if its first field is `"Version"`.  
  - Sensor CSVs must have recognizable column headers (else `ImportSensorsPreviewControl_NoColumnsInTDCCSV` error).
- **Version Consistency**:  
  - `Version0CSVTestParser` and `Version5CSVTestParser` expect header-value pairs.  
  - `Version6CSVTestParser` expects a table format with header row followed by data rows until an empty line.  
  - `DTSCSVTestSetupParser.ParseTestSetup` uses `CSVFile.GetCsvVersion` to determine which parsers to invoke; version 6 triggers all parsers (0 + 5 + 6).
- **Sensor Grouping**:  
  - `Version3CSVSensorParser` enforces that a group name maps to a single `SensorTestObject` unless `ImportCreateDynamicGroups` is true.  
  - Duplicate sensor serial numbers in `sensorGroupNameLookup` or `sensorGroupTypeLookup` are rejected with errors.
- **Calibration & Defaults**:  
  - `DTSCSVSensorsParser` applies user-specific defaults for squib/digital output settings.  
  - Sensitivity units are corrected from `mVperVperEU` to `mVperEU` for non-proportional calibrations.  
  - Zero method is initialized from `pp.SensorCal.ZeroMethods.Methods[0]` during `PostParse`.
- **Hardware Mapping**:  
  - DAS hardware is added to `TestTemplate` only if version ≥ 6 (per `DTSCSVTestSetupParser` comment FB 43815).  
  - DAS sample rates are set in both `t.SetSampleRateForHardware` and `t.DASSampleRateList`.

## 4. Dependencies

### Imports/References:
- **Core Types**: `DTS.Common.Classes`, `DTS.Common.Enums`, `DTS.Common.Import.Interfaces`, `CsvHelper`, `System.IO`, `System.Collections.Generic`.
- **Hardware/Storage**: `DataPROWin7.DataModel`, `DTS.SensorDB`, `DTS.Common.Storage`.
- **Import Infrastructure**: `ParseVariantBase`, `ParseParameters`, `ImportObject`, `TestSetupImportData`, `SensorData`, `SensorCalibration`, `IGroup`, `IGroupImport`, `ICalibrationImport`, `IImportNotification`.
- **CSV Tags**: `CSVImportTags` (static class providing `GetTagForString`, `GetVersionForTag`).
- **Factories**: `CSVTestParserFactory`, `CSVSensorParserFactory`.

### Used By:
- `DTSCSVTestSetupParser` is invoked by the test setup import pipeline.
- `DTSCSVSensorsParser` is invoked by the sensor import pipeline.
- `CSVGroupImport` is used by `DTSCSVTestSetupParser` to create groups.

## 5. Gotchas

- **Version 6 CSV Parsing Quirk**:  
  In `DTSCSVSensorsParser.ParseSensor`, if version is 6, the parser skips rows until it finds a tag with version ≤ 4 (to avoid parsing test setup-specific rows as sensor data). This assumes sensor and test setup data are in the same file but separated by an empty line.
  
- **Exception Thrown in Group Parsing**:  
  `Version3CSVSensorParser.ParseVersion` throws `new Exception("Parse error")` on group/test object conflict (if `ImportCreateDynamicGroups` is false). This is caught by `DTSCSVSensorsParser` and reported as a sensor error, but the exception itself is non-descriptive.

- **Sensitivity Unit Correction**:  
  `DTSCSVSensorsParser` silently changes `SensitivityUnits` from `mVperVperEU` to `mVperEU` for non-proportional calibrations. This is logged as a warning but may cause unexpected calibration behavior.

- **DAS Hardware Lookup**:  
  `CSVGroupImport.CreateGroups` uses `DbOperations.GetChannelSettingDefaults()` and `GroupHelper.GetDAS` to map DAS serial numbers to IDs. If DAS is not in `DASHardwareList.GetAllHardware()`, channel DAS ID remains unset.

- **Zero Method Initialization**:  
  `PostParse` in `DTSCSVSensorsParser` initializes `pp.ZeroType`, `pp.ZeroStart`, `pp.ZeroEnd` from `pp.SensorCal.ZeroMethods.Methods[0]`. If `ZeroMethods` is uninitialized or empty, this may cause `IndexOutOfRangeException`.

- **No Validation of Sensor Serial Number Format**:  
  Sensors with serial numbers starting with `Constants.ISO_CH_ONLY_PREFIX` bypass some validation (e.g., sensitivity/capacity checks), but the prefix is not defined in the provided source.

- **Error Reporting Inconsistency**:  
  `DTSCSVSensorsParser` aggregates errors per line and reports them as a single error string, while `DTSCSVTestSetupParser` reports errors via `IImportNotification.ReportErrors`. Error severity and continuation behavior differ between parsers.

- **File Lock Detection Limitation**:  
  `CSVFile.IsInUse` uses `File.ReadAllLines`, which may not reliably detect all lock states (e.g., file opened for append-only).