DP44/GLM5Analysis/PatternLibrary/ImportExportPattern.md

# Import/Export Pattern

## When to Use
- Importing sensor data from external files (CSV, XML, SIF, etc.)
- Importing test setups and group configurations
- Exporting data to external formats
- Migrating data between database versions

## Architecture Overview

```
┌────────────────────────────────────────────────────────────────────────┐
│                      Import Wizard Flow                                 │
│                                                                         │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐              │
│  │   Options    │───>│   Preview    │───>│   Import     │              │
│  │    View      │    │    View      │    │    View      │              │
│  └──────────────┘    └──────────────┘    └──────────────┘              │
│         │                   │                   │                       │
│         v                   v                   v                       │
│  ┌──────────────────────────────────────────────────────────┐          │
│  │              Shared Import ViewModel                      │          │
│  │  - SourceFiles                                            │          │
│  │  - ParseSourceFiles()                                     │          │
│  │  - Validate()                                             │          │
│  │  - Import()                                               │          │
│  └──────────────────────────────────────────────────────────┘          │
│         │                                                               │
│         v                                                               │
│  ┌──────────────────────────────────────────────────────────┐          │
│  │              EventAggregator Events                       │          │
│  │  - TTSImportReadFileFinishedEvent                         │          │
│  │  - TTSImportTestSetupChangedEvent                         │          │
│  │  - CustomChannelImportEvent                               │          │
│  └──────────────────────────────────────────────────────────┘          │
└────────────────────────────────────────────────────────────────────────┘
```

## Files to Create/Modify

### Structure
```
Modules/{Module}/
├── {Module}Import/
│   ├── {Module}ImportModule.cs
│   ├── View/
│   │   ├── {Module}ImportOptionsView.xaml
│   │   ├── {Module}ImportPreviewView.xaml
│   │   └── {Module}ImportImportView.xaml
│   └── ViewModel/
│       └── {Module}ImportViewModel.cs

Common/DTS.CommonCore/
├── Events/{Module}Import/
│   └── {Event}Events.cs
└── Interface/{Module}/
    └── I{Module}ImportViewModel.cs
```

## Code Templates

### 1. Import ViewModel
```csharp
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.Composition;
using System.IO;
using System.Linq;
using Prism.Events;
using Prism.Regions;
using Unity;
using DTS.Common.Interface.{Module};
using DTS.Common.Interactivity;
using DTS.Common.Events;

namespace {Module}Import
{
    [Export(typeof(I{Module}ImportOptionsView))]
    [Export(typeof(I{Module}ImportPreviewView))]
    [Export(typeof(I{Module}ImportImportView))]
    [PartCreationPolicy(CreationPolicy.Shared)]
    public class {Module}ImportViewModel : I{Module}ImportViewModel
    {
        #region Views
        public I{Module}ImportOptionsView ImportOptionsView { get; set; }
        public I{Module}ImportPreviewView ImportPreviewView { get; set; }
        public I{Module}ImportImportView ImportView { get; set; }
        #endregion

        private IEventAggregator _eventAggregator { get; set; }
        private IRegionManager _regionManager;
        private IUnityContainer UnityContainer { get; set; }

        public InteractionRequest<Notification> NotificationRequest { get; private set; }
        public InteractionRequest<Confirmation> ConfirmationRequest { get; private set; }

        #region Properties
        public string[] SourceFiles { get; set; }
        public List<ImportItem> ParsedItems { get; set; }
        public List<ImportError> Errors { get; set; }
        public bool HasErrors => Errors?.Any() == true;
        #endregion

        public {Module}ImportViewModel(
            I{Module}ImportOptionsView optionsView,
            I{Module}ImportPreviewView previewView,
            I{Module}ImportImportView importView,
            IRegionManager regionManager,
            IEventAggregator eventAggregator,
            IUnityContainer unityContainer)
        {
            ImportOptionsView = optionsView;
            ImportOptionsView.DataContext = this;
            ImportPreviewView = previewView;
            ImportPreviewView.DataContext = this;
            ImportView = importView;
            ImportView.DataContext = this;

            NotificationRequest = new InteractionRequest<Notification>();
            ConfirmationRequest = new InteractionRequest<Confirmation>();

            _eventAggregator = eventAggregator;
            UnityContainer = unityContainer;
            _regionManager = regionManager;

            _eventAggregator.GetEvent<RaiseNotification>().Subscribe(OnRaiseNotification);
        }

        /// <summary>
        /// Parse source files and extract data
        /// </summary>
        public void ParseSourceFiles()
        {
            ParsedItems = new List<ImportItem>();
            Errors = new List<ImportError>();

            foreach (var file in SourceFiles ?? new string[0])
            {
                try
                {
                    var lines = File.ReadAllLines(file);
                    for (var i = 0; i < lines.Length; i++)
                    {
                        var line = lines[i];
                        if (string.IsNullOrWhiteSpace(line)) continue;

                        var item = ParseLine(line, file, i);
                        if (item != null)
                        {
                            ParsedItems.Add(item);
                        }
                    }
                }
                catch (Exception ex)
                {
                    Errors.Add(new ImportError
                    {
                        File = file,
                        Message = ex.Message
                    });
                }
            }

            OnPropertyChanged("ParsedItems");
            OnPropertyChanged("HasErrors");
        }

        private ImportItem ParseLine(string line, string file, int lineNumber)
        {
            var fields = line.Split(',');

            // Validate required fields
            if (fields.Length < 3)
            {
                Errors.Add(new ImportError
                {
                    File = file,
                    Line = lineNumber,
                    Message = "Insufficient fields"
                });
                return null;
            }

            return new ImportItem
            {
                Name = fields[0],
                Value = fields[1],
                Type = fields[2]
            };
        }

        /// <summary>
        /// Execute the import
        /// </summary>
        public void Import()
        {
            var successCount = 0;
            var errors = new List<string>();

            foreach (var item in ParsedItems)
            {
                try
                {
                    // Call DbAPI to insert
                    var result = DbAPI.DbAPI.{Entity}.{Entity}Add(
                        CurrentUser.User,
                        ConnectionManager.CurrentConnection,
                        item.ToRecord(),
                        out int newId);

                    if (result == ErrorCodes.ERROR_SUCCESS)
                    {
                        successCount++;
                    }
                    else
                    {
                        errors.Add($"Failed to import {item.Name}: Error {result}");
                    }
                }
                catch (Exception ex)
                {
                    errors.Add($"Exception importing {item.Name}: {ex.Message}");
                }
            }

            if (errors.Any())
            {
                NotificationRequest.Raise(new Notification
                {
                    Title = "Import Completed with Errors",
                    Content = $"{successCount} items imported successfully.\n{errors.Count} errors occurred."
                });
            }
            else
            {
                NotificationRequest.Raise(new Notification
                {
                    Title = "Import Successful",
                    Content = $"{successCount} items imported successfully."
                });
            }
        }

        public event PropertyChangedEventHandler PropertyChanged;
        public void OnPropertyChanged(string propertyName) =>
            PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));

        // IBaseViewModel implementation...
        public bool IsBusy { get; set; }
        public bool IsDirty => false;
        public bool IsMenuIncluded { get; set; }
        public bool IsNavigationIncluded { get; set; }
        public void Initialize() { }
        public void Initialize(object parameter) { }
        public Task InitializeAsync() => Task.CompletedTask;
        public Task InitializeAsync(object parameter) => Task.CompletedTask;
        public void Activated() { }
        public void Cleanup() { }
        public Task CleanupAsync() => Task.CompletedTask;
    }
}
```

### 2. Import Event Definitions
**File:** `Common/DTS.CommonCore/Events/{Module}Import/{Event}Event.cs`
```csharp
using Prism.Events;
using DTS.Common.Base;

namespace DTS.Common.Events.{Module}Import
{
    /// <summary>
    /// Event fired when import file parsing is complete
    /// </summary>
    public class {Module}ImportFinishedEvent : PubSubEvent<IImportData> { }

    /// <summary>
    /// Event fired when an item is imported
    /// </summary>
    public class {Module}ItemImportedEvent : PubSubEvent<ImportedItemEventArgs> { }

    public class ImportedItemEventArgs
    {
        public string Name { get; set; }
        public int NewId { get; set; }
        public bool Success { get; set; }
        public string Error { get; set; }
    }
}
```

### 3. File Parser Pattern
```csharp
namespace DTS.Common.Import.Parsers
{
    public interface IParseImport
    {
        string[] SupportedExtensions { get; }
        bool CanParse(string filePath);
        IImportData Parse(string filePath);
    }

    public class CSVParser : IParseImport
    {
        public string[] SupportedExtensions => new[] { ".csv" };

        public bool CanParse(string filePath) =>
            Path.GetExtension(filePath).Equals(".csv", StringComparison.OrdinalIgnoreCase);

        public IImportData Parse(string filePath)
        {
            var data = new ImportData();
            var lines = File.ReadAllLines(filePath);

            // Skip header if present
            for (int i = 1; i < lines.Length; i++)
            {
                var fields = lines[i].Split(',');
                // Parse fields...
            }

            return data;
        }
    }
}
```

## Examples from Codebase

### Example 1: GroupImportViewModel
**File:** `DataPRO/Modules/Groups/GroupImport/ViewModel/GroupImportViewModel.cs:36`
```csharp
[Export(typeof(IGroupImportOptionsView))]
[Export(typeof(IGroupImportPreviewView))]
[Export(typeof(IGroupImportImportView))]
[PartCreationPolicy(CreationPolicy.Shared)]
public class GroupImportViewModel : IGroupImportViewModel
{
    public IGroupImportOptionsView ImportOptionsView { get; set; }
    public IGroupImportPreviewView ImportPreviewView { get; set; }
    public IGroupImportImportView ImportView { get; set; }

    public GroupImportViewModel(
        IGroupImportOptionsView optionsView,
        IGroupImportPreviewView previewView,
        IGroupImportImportView importView,
        IRegionManager regionManager,
        IEventAggregator eventAggregator,
        IUnityContainer unityContainer)
    {
        ImportView = importView;
        ImportView.DataContext = this;
        // ...
    }

    public void ParseSourceFiles(string userTags)
    {
        var groups = new List<GroupGRPImportGroup>();
        foreach (var file in SourceFiles)
        {
            var lines = System.IO.File.ReadAllLines(file);
            // Parse group data...
        }
    }
}
```

### Example 2: TTS Import Events
**File:** `Common/DTS.CommonCore/Events/TTSImport/TTSImportReadFileFinishedEvent.cs:13`
```csharp
public class TTSImportReadFileFinishedEvent : CompositePresentationEvent<ITTSSetup> { }
```

**File:** `Common/DTS.CommonCore/Events/TTSImport/TTSImportTestSetupChangedEvent.cs:13`
```csharp
public class TTSImportTestSetupChangedEvent : CompositePresentationEvent<ITTSSetup> { }
```

### Example 3: Sensor Importer Base Class
**File:** `DataPRO/DataPRO/Controls/Sensors and models/Classes/SensorTestSetupImporter.cs:40`
```csharp
public abstract class SensorTestSetupImporter : BasePropertyChanged, ISensorTestSetupImporter
{
    public abstract string[] SupportedExtensions { get; }
    public abstract string FileFilter { get; }
    
    public abstract ISensorData[] ParseImport(string[] files, ImportOptions options);
    public abstract void ImportSensors(ISensorData[] sensors, IConnectionDetails connection);
}
```

### Example 4: CSVImporter Implementation
**File:** `DataPRO/DataPRO/Controls/Sensors and models/Classes/CSVImporter.cs:30`
```csharp
public class CSVImporter : SensorTestSetupImporter
{
    public override string[] SupportedExtensions => new[] { ".csv" };
    public override string FileFilter => "CSV Files (*.csv)|*.csv";

    public override ISensorData[] ParseImport(string[] files, ImportOptions options)
    {
        var sensors = new List<ISensorData>();
        foreach (var file in files)
        {
            var lines = File.ReadAllLines(file);
            for (int i = 1; i < lines.Length; i++) // Skip header
            {
                var fields = lines[i].Split(',');
                sensors.Add(new SensorData
                {
                    SerialNumber = fields[0],
                    Description = fields[1],
                    // ...
                });
            }
        }
        return sensors.ToArray();
    }
}
```

## Common Mistakes to Avoid

1. **Not validating before import** - Always parse and preview before executing import
2. **Missing error collection** - Track errors per line/item, not just global exceptions
3. **Blocking UI during import** - Use async/await or background threads
4. **Not handling file encoding** - Use proper encoding (UTF-8, ANSI) for international files
5. **Hardcoded column indexes** - Use header row to find columns by name
6. **Not reporting progress** - For large imports, report progress percentage
7. **Missing transaction rollback** - Consider wrapping imports in transactions
8. **Not checking for duplicates** - Validate against existing database records
9. **Silent failures** - Always notify user of failed imports with specific error messages
10. **Not supporting cancel** - Allow user to cancel long-running imports