DP44/enriched-partialglm/Common/DTS.Common.Import/Parsers/CSV.md

source_files, generated_at, model, schema_version, sha256

source_files	generated_at	model	schema_version	sha256
Common/DTS.Common.Import/Parsers/CSV/AbstractCSVParser.cs Common/DTS.Common.Import/Parsers/CSV/Version0CSVTestParser.cs Common/DTS.Common.Import/Parsers/CSV/CSVFile.cs Common/DTS.Common.Import/Parsers/CSV/Version6CSVTestParser.cs Common/DTS.Common.Import/Parsers/CSV/Version3CSVSensorParser.cs Common/DTS.Common.Import/Parsers/CSV/Version5CSVTestParser.cs Common/DTS.Common.Import/Parsers/CSV/Version4CSVSensorParser.cs Common/DTS.Common.Import/Parsers/CSV/CSVGroupImport.cs Common/DTS.Common.Import/Parsers/CSV/DTSCSVTestSetupParser.cs Common/DTS.Common.Import/Parsers/CSV/DTSCSVSensorsParser.cs	2026-04-16T11:47:30.258239+00:00	zai-org/GLM-5-FP8	1	2bff5b23b8f73415

CSV Import Module Documentation

1. Purpose

This module provides a versioned parser architecture for importing DTS sensor configurations and test setups from CSV files. It transforms raw CSV data into domain objects (SensorData, SensorCalibration, TestTemplate) using a strategy pattern where specific parser versions (e.g., Version0CSVTestParser, Version4CSVSensorParser) handle different schema evolutions of the CSV format. It orchestrates the creation of groups, channel mappings, and hardware associations required for test setup imports.

2. Public Interface

AbstractCSVParser

Base class for sensor parsers.

• int Version { get; } - Abstract property indicating the parser version.
• void Initialize(ICalibrationImport import, ZeroMethodOptions zmOptions, IImportNotification importNotification, bool importCreateDynamicGroups, bool useISOCodeFilterMapping, bool useZeroForUnfiltered) - Injects dependencies and configuration flags required for parsing.
• void ParseVersion(CSVImportTags.Tags field, string val, ParseParameters pp) - Abstract method to parse a specific field/value pair into the ParseParameters state object.

CSVFile

Utility class for file inspection.

• CSVFile(string filename) - Constructor.
• static bool IsInUse(string filename) - Attempts to read the file to check for locks; returns true if IOException occurs.
• static bool IsCSVFileForTestSetupImport(string filename, CsvImportOptions csvImportOptions) - Validates if the file starts with the "Version" header.
• static int GetCsvVersion(string filename) - Extracts the integer version number from the second line of the CSV.
• int LineCount { get; } - Returns the total line count of the file.

Versioned Test Parsers

Classes implementing IParseCSVTest:

• Version0CSVTestParser
• Version5CSVTestParser
• Version6CSVTestParser

Common Members:

• int Version { get; } - Returns the specific version number (0, 5, or 6).
• void ParseVersion(CsvReader csvReader, TestSetupImportData tsid) - Reads the CSV stream and populates TestSetupImportData.

Versioned Sensor Parsers

Classes inheriting AbstractCSVParser:

• Version3CSVSensorParser: Handles GroupName, GroupType.
• Version4CSVSensorParser: Handles DAS mapping (DASSerialNumber, DASChannelIndex), UDP streaming (UDPAddress, StreamProfile), UART settings (BaudRate, Parity), and user codes.

CSVGroupImport

Implements IGroupImport.

• ParseParameters ParseParameters { get; set; }
• Tuple<TestTemplate, List<IGroup>> CreateGroups(List<SensorData> sensors, Dictionary<string, List<TsetSetupImportSensorInfo>> groupSensorLookup, TestTemplate testTemplate, bool createDynamicGroups, List<IGroup> staticGroups, Action<double> setProgress) - Constructs group hierarchies and channels within a TestTemplate.
• Dictionary<string, List<TsetSetupImportSensorInfo>> GetGroupSensorLookup(...) - Builds a lookup dictionary mapping group names to sensor import info.

DTSCSVTestSetupParser

Main orchestrator for test setup imports.

• DTSCSVTestSetupParser(IImportNotification importNotification, CsvImportOptions csvImportOptions, TestSetupImportData testSetupImportData, IGroupImport groupImport, bool createDynamicGroups)
• void Parse(ref ImportObject importObject) - Entry point. Validates file, parses test setup, creates groups, assigns calibrations.
• static TestTemplate CreateTestSetup(TestSetupImportData tsid, DASHardware[] allDAS) - Factory method to create a TestTemplate from parsed data.
• static void UpdateDASSampleRate(TestTemplate t, TestSetupImportData tsid, DASHardware[] allDAS) - Updates sample rates on the template.

DTSCSVSensorsParser

Main orchestrator for sensor imports.

• DTSCSVSensorsParser(IImportNotification importNotification, User user, CsvImportOptions csvImportOptions, ICalibrationImport calibrationImport, ZeroMethodOptions zeroMethodOptions)
• bool ImportCreateDynamicGroups { get; set; }
• bool UseISOCodeFilterMapping { get; set; }
• bool UseZeroForUnfiltered { get; set; }
• void Parse(ref ImportObject importObject) - Entry point. Reads headers, iterates rows, populates SensorData and SensorCalibration.

3. Invariants

• Version Header: A valid CSV file for import must contain a "Version" header in the first row, with the version integer in the second row (enforced by CSVFile.IsCSVFileForTestSetupImport and GetCsvVersion).
• Parser Selection: DTSCSVTestSetupParser only invokes parsers for versions 1 through 5 if the file version is exactly 6. Otherwise, it defaults to running only the Version 0 parser.
• Group Uniqueness: In Version3CSVSensorParser, a sensor serial number cannot appear twice in the SensorGroupNameLookup or SensorGroupTypeLookup; duplicates trigger an error.
• Version 6 Tag Filtering: Version6CSVTestParser and DTSCSVSensorsParser explicitly check CSVImportTags.GetVersionForTag and skip tags that do not match their expected version (e.g., Version 6 parser skips Version 5 tags).
• Hardware Association: In DTSCSVTestSetupParser, hardware is only added to the ImportObject if the CSV version is 6 or greater.

4. Dependencies

Internal Dependencies (Namespaces referenced):

• DTS.Common.Classes, DTS.Common.Classes.Sensors: Core domain objects (SensorData, SensorCalibration).
• DTS.Common.Enums, DTS.Common.Enums.Sensors: Enumeration types (RecordingModes, BridgeType, SensUnits).
• DTS.Common.Import.Interfaces: Interfaces IParseCSVSensor, IParseCSVTest, IGroupImport.
• DTS.Common.Import.ImportOptions: CsvImportOptions, ZeroMethodOptions.
• DTS.SensorDB: Database operations and storage types.
• DataPROWin7.DataModel: Hardware data models (DASHardware).
• DTS.Common.Storage: DbOperations.

External Libraries:

• CsvHelper: Used via CsvReader for reading CSV records.

Factory Dependencies (referenced but not defined in source):

• CSVTestParserFactory: Creates the array of test parsers.
• CSVSensorParserFactory: Creates the dictionary of sensor parsers.

5. Gotchas

• Silent Exception Swallowing: CSVFile.IsInUse catches all non-IOExceptions and returns true (indicating the file is in use/unavailable), which may mask permission errors or corrupt file issues.
• Hardcoded Version Logic: DTSCSVTestSetupParser.ParseTestSetup contains logic that specifically checks if (tsid.Version == 6) to determine whether to iterate through higher-version parsers. Adding a Version 7 would require modifying this specific method.
• Generic Exception Throwing: Version3CSVSensorParser throws a generic System.Exception("Parse error") when a group maps to multiple test objects while dynamic groups are disabled. This is caught in DTSCSVTestSetupParser.AssignGroupsToTestSetup and reported generically.
• Sensitivity Unit Mutation: DTSCSVSensorsParser silently changes SensitivityUnits from mVperVperEU to mVperEU for non-proportional sensors, reporting it only as a non-blocking notification after processing.
• ISO Channel Prefix: Sensors with serial numbers starting with Constants.ISO_CH_ONLY_PREFIX are excluded from the standard sensor/calibration lookup dictionaries in DTSCSVSensorsParser,