DP44/docs/ai/Common/DTS.Common.Import/Parsers/CSV.md

source_files, generated_at, model, schema_version, sha256

source_files	generated_at	model	schema_version	sha256
Common/DTS.Common.Import/Parsers/CSV/AbstractCSVParser.cs Common/DTS.Common.Import/Parsers/CSV/Version0CSVTestParser.cs Common/DTS.Common.Import/Parsers/CSV/CSVFile.cs Common/DTS.Common.Import/Parsers/CSV/Version6CSVTestParser.cs Common/DTS.Common.Import/Parsers/CSV/Version3CSVSensorParser.cs Common/DTS.Common.Import/Parsers/CSV/Version5CSVTestParser.cs Common/DTS.Common.Import/Parsers/CSV/Version4CSVSensorParser.cs Common/DTS.Common.Import/Parsers/CSV/CSVGroupImport.cs Common/DTS.Common.Import/Parsers/CSV/DTSCSVTestSetupParser.cs Common/DTS.Common.Import/Parsers/CSV/DTSCSVSensorsParser.cs	2026-04-17T15:31:35.696653+00:00	zai-org/GLM-5-FP8	1	82eec24579c17b93

CSV Import Parsers Documentation

1. Purpose

This module provides CSV parsing functionality for importing DTS sensor configurations and test setups. It implements a versioned parser architecture where different CSV format versions (0-6) are handled by specialized parser classes. The module reads CSV files containing sensor metadata, calibration data, group assignments, and test setup configurations, transforming them into domain objects for the DTS data acquisition system. The architecture separates sensor parsing from test setup parsing, with both feeding into a group import system that creates the final test template structure.

---

2. Public Interface

AbstractCSVParser (Abstract Base Class)

Member	Signature	Description
Version	public abstract int Version { get; }	Returns the parser version number this class handles
Initialize	public void Initialize(ICalibrationImport import, ZeroMethodOptions zmOptions, IImportNotification importNotification, bool importCreateDynamicGroups, bool useISOCodeFilterMapping, bool useZeroForUnfiltered)	Initializes the parser with dependencies and configuration flags
ParseVersion	public abstract void ParseVersion(CSVImportTags.Tags field, string val, ParseParameters pp)	Parses a specific field value for this parser's version

Version0CSVTestParser

Member	Signature	Description
Version	public int Version => 0	Returns version 0
ParseVersion	public void ParseVersion(CsvReader csvReader, TestSetupImportData tsid)	Parses version 0 test setup data including PostTriggerSec, PreTriggerSec, RecordingMode, SampleRate, Description, Name, Version, and Tags

Version5CSVTestParser

Member	Signature	Description
Version	public int Version => 5	Returns version 5
ParseVersion	public void ParseVersion(CsvReader csvReader, TestSetupImportData tsid)	Parses version 5 test setup data including clock configuration (ClockMasterInputType, ClockMasterOutputType, ClockSlaveInputType, ClockSlaveOutputType, ManageClocksOutsideDPMaster, ManageClocksOutsideDPSlave)

Version6CSVTestParser

Member	Signature	Description
Version	public int Version => 6	Returns version 6
ParseVersion	public void ParseVersion(CsvReader csvReader, TestSetupImportData tsid)	Parses version 6 test setup data including DAS hardware info (DASSerial, DASSampleRate, PTPDomainId, ClockMaster) with multi-row body parsing

Version3CSVSensorParser

Member	Signature	Description
Version	public override int Version => 3	Returns version 3
ParseVersion	public override void ParseVersion(CSVImportTags.Tags field, string sVal, ParseParameters pp)	Parses GroupName and GroupType fields with duplicate detection and test object validation

Version4CSVSensorParser

Member	Signature	Description
Version	public override int Version => 4	Returns version 4
ParseVersion	public override void ParseVersion(CSVImportTags.Tags field, string sVal, ParseParameters pp)	Parses DAS assignment (DASSerialNumber, DASChannelIndex), UDP streaming (StreamProfile, UDPAddress, TimeChannelId, DataChannelId, TmNSConfig, IRIGTimeDataPacketIntervalMS, TMATSIntervalMS), UART configuration (BaudRate, DataBits, StopBits, Parity, DataFormat), and user codes (TestUserCode, TestUserChannelName, TestIsoCode, TestIsoChannelName)

CSVFile

Member	Signature	Description
Constructor	public CSVFile(string filename)	Creates instance with specified filename
IsInUse	public static bool IsInUse(string filename)	Returns true if file is locked/in use by another process
IsCSVFileForTestSetupImport	public static bool IsCSVFileForTestSetupImport(string filename, CsvImportOptions csvImportOptions)	Validates if CSV file is formatted for test setup import (checks for "Version" header)
GetCsvVersion	public static int GetCsvVersion(string filename)	Extracts and returns the version number from CSV file, returns 0 on error
LineCount	public int LineCount { get; }	Returns the number of lines in the file

CSVGroupImport

Member	Signature	Description
ParseParameters	public ParseParameters ParseParameters { get; set; }	Gets or sets the parse parameters for import
CreateGroups	public Tuple<TestTemplate, List<IGroup>> CreateGroups(List<SensorData> sensors, Dictionary<string, List<TsetSetupImportSensorInfo>> groupSensorLookup, TestTemplate testTemplate, bool createDynamicGroups, List<IGroup> staticGroups, Action<double> setProgress)	Creates groups and channels from sensor data, populating the test template
GetGroupSensorLookup	public Dictionary<string, List<TsetSetupImportSensorInfo>> GetGroupSensorLookup(List<SensorData> sensors, Dictionary<string, string> sensorGroupNameLookup, Dictionary<string, List<string>> groupNameSensorListLookup)	Builds lookup dictionary mapping group names to sensor info lists

DTSCSVTestSetupParser

Member	Signature	Description
Constructor	public DTSCSVTestSetupParser(IImportNotification importNotification, CsvImportOptions csvImportOptions, TestSetupImportData testSetupImportData, IGroupImport groupImport, bool createDynamicGroups)	Initializes parser with dependencies
CreateTestSetup	public static TestTemplate CreateTestSetup(TestSetupImportData tsid, DASHardware[] allDAS)	Creates a TestTemplate from parsed import data
UpdateDASSampleRate	public static void UpdateDASSampleRate(TestTemplate t, TestSetupImportData tsid, DASHardware[] allDAS)	Updates sample rates for DAS hardware in the test template
Parse	public override void Parse(ref ImportObject importObject)	Main entry point - parses CSV file and populates ImportObject with test setup data

DTSCSVSensorsParser

Member	Signature	Description
Constructor	public DTSCSVSensorsParser(IImportNotification importNotification, User user, CsvImportOptions csvImportOptions, ICalibrationImport calibrationImport, ZeroMethodOptions zeroMethodOptions)	Initializes parser with dependencies
ImportCreateDynamicGroups	public bool ImportCreateDynamicGroups { get; set; }	Flag for dynamic group creation during import
UseISOCodeFilterMapping	public bool UseISOCodeFilterMapping { get; set; }	Flag for ISO code filter mapping behavior
UseZeroForUnfiltered	public bool UseZeroForUnfiltered { get; set; }	Flag for using zero values for unfiltered data
Parse	public override void Parse(ref ImportObject importObject)	Main entry point - parses CSV file and populates ImportObject with sensor and calibration data

---

3. Invariants

1. Version Header Requirement: CSV files must have "Version" as the first token in the first row to be recognized as valid for import.

2. Version-Based Parser Selection: The CSVImportTags.GetVersionForTag() determines which parser handles each field. Parsers only process fields matching their version.

3. Version 6 Multi-Parser Execution: When tsid.Version == 6, all parsers from index 1 through the array length are invoked sequentially (see DTSCSVTestSetupParser.ParseTestSetup).

4. Duplicate Serial Number Detection: Version3CSVSensorParser enforces unique sensor serial numbers within SensorGroupNameLookup and SensorGroupTypeLookup dictionaries.

5. Non-Empty Group Names: Group names must be non-empty and non-whitespace; empty values generate errors.

6. Test Setup Name Uniqueness: Import skips test setups with names that already exist in importObject.TestSetups().

7. Calibration Assignment Order: Calibrations are sorted and the most recent (last element after sort) is assigned to each sensor.

8. Analog Sensor Validation: Analog sensors require non-zero sensitivity and capacity >= 1; violations generate errors.

9. Excitation Validation: Full/Half/Quarter bridge sensors require defined excitation voltage (not Undefined).

---

4. Dependencies

External Dependencies (Imports)

Namespace	Purpose
CsvHelper	CSV file reading via CsvReader
DTS.Common.Classes.Sensors	SensorData, SensorCalibration, ParseParameters
DTS.Common.Enums.Sensors	Sensor enumerations including BridgeType, SensUnits, ExcitationVoltageOptions
DTS.Common.Import.ImportOptions	CsvImportOptions, ZeroMethodOptions
DTS.Common.Import.Interfaces	IParseCSVSensor, IParseCSVTest, ICalibrationImport, IImportNotification, IGroupImport
DTS.Common.Import.Factories	CSVTestParserFactory, CSVSensorParserFactory
DTS.Common.Import.Parsers	ParseVariantBase base class
DTS.Common.Storage	DbOperations for database access
DTS.SensorDB	Database operations
DataPROWin7.DataModel.Classes.Hardware	DASHardwareList for hardware enumeration
System.IO.Ports	StopBits, Parity enumerations for UART configuration
DTS.Slice.Users	User class for current user context
DTS.Common.Utilities.Logging	APILogger for error logging

Consumers (Inferred)

The module implements interfaces (IParseCSVSensor, IParseCSVTest, IGroupImport) and extends ParseVariantBase, suggesting consumption via:

• Factory patterns (CSVTestParserFactory, CSVSensorParserFactory)
• Dependency injection through interface types
• The ImportObject pattern which aggregates parsed results

---

5. Gotchas

1. Version 6 Special Execution Path: Unlike other versions, version 6 triggers iteration through all parsers starting at index 1:

   if (tsid.Version == 6)
   {
       for (var i = 1; i < parsers.Length; i++)
       {
           parsers[i].ParseVersion(parser, tsid);
       }
   }
   

   This means version 6 files are processed by multiple parsers, not just Version6CSVTestParser.

2. Silent Sensitivity Unit Correction: Non-proportional sensors with SensUnits.mVperVperEU are silently corrected to SensUnits.mVperEU. Users are notified via _importNotification.ReportErrors but parsing continues.

3. ISO Channel-Only Prefix Bypass: Sensors with serial numbers starting with Constants.ISO_CH_ONLY_PREFIX are excluded from AddSensorLookup and AddCalibrationLookup calls, but still added via AddSensor and AddCalibration.

4. Exception Swallowing in CSVFile:

   • IsInUse() suppresses all non-IOException exceptions
   • IsCSVFileForTestSetupImport() and GetCsvVersion() catch all exceptions and return false/0

5. Generic Exception in Version3CSVSensorParser: Throws new Exception("Parse error") when a group maps to multiple test objects and ImportCreateDynamicGroups is false. This breaks the pattern of adding errors to pp.Errors.

6. Dual Sample Rate Storage: The comment in UpdateDASSampleRate explicitly notes uncertainty about why sample rates are stored in both t.SetSampleRateForHardware() and t.DASSampleRateList[].

7. Historical Bug References: Comments reference "FB" tracking numbers (e.g., "FB 40598", "FB 36879", "FB 43815") indicating accumulated fixes. These may indicate areas with historical issues.

8. Range Assignment Complexity: The range assignment logic in CSVGroupImport.CreateGroups has specific conditions involving ImportContainedSensorRanges, RangeHigh, RangeLow, RangeMedium, and Capacity that may produce unexpected results if not fully understood.

9. Version 6 Header Search Loop: Version6CSVTestParser and Version5CSVTestParser both skip rows until finding a version-appropriate tag, but use different version checks (6 vs 5). This could cause confusion if tags overlap.