squad/implementation.md

# Implementation Architecture

## Executive Summary
Software Development Insight & Release Management Platform with modular microservices architecture, containerized deployment, and modern web frontend.

## System Architecture Overview

```mermaid
graph TB
    subgraph "Frontend Layer"
        UI[React TypeScript Frontend]
        UI --> SG[SignalR Client]
    end
    
    subgraph "API Gateway"
        GW[API Gateway Service<br/>Authentication & Routing]
        HUB[SignalR Hub<br/>Real-time Updates]
    end
    
    subgraph "Core Services"
        PS[Project Service<br/>CC.NET Integration]
        MS[Manuscript Service<br/>FogBugz Integration]
        DS[Document Service<br/>PDF Generation]
        SS[Storage Service<br/>Cloud Abstraction]
        HS[Help Center Service<br/>Article Updates]
        PUB[Publishing Service<br/>Workflow Orchestration]
    end
    
    subgraph "Background Processing"
        HF[Hangfire<br/>Background Jobs]
        HFD[Hangfire Dashboard]
    end
    
    subgraph "Data Layer"
        DB[(PostgreSQL<br/>Database)]
    end
    
    subgraph "External Integrations"
        CC[CruiseControl.NET]
        FB[FogBugz/Manuscript]
        S3[AWS S3]
        BOX[Box.com]
        FTP[FTP Server]
        ZD[Zendesk]
        SF[Salesforce Service]
    end
    
    UI --> GW
    SG --> HUB
    GW --> PS
    GW --> MS
    GW --> DS
    GW --> SS
    GW --> HS
    GW --> PUB
    
    PS --> CC
    MS --> FB
    SS --> S3
    SS --> BOX
    SS --> FTP
    HS --> ZD
    HS --> SF
    
    PUB --> HF
    HF --> DS
    HF --> SS
    HF --> HS
    
    PS --> DB
    MS --> DB
    DS --> DB
    SS --> DB
    HS --> DB
    PUB --> DB
    HF --> DB
    
    classDef frontend fill:#e1f5fe
    classDef gateway fill:#fff3e0
    classDef service fill:#f3e5f5
    classDef background fill:#e8f5e8
    classDef database fill:#fce4ec
    classDef external fill:#f5f5f5
    
    class UI,SG frontend
    class GW,HUB gateway
    class PS,MS,DS,SS,HS,PUB service
    class HF,HFD background
    class DB database
    class CC,FB,S3,BOX,FTP,ZD,SF external
```

## Technology Stack

### Backend Services (.NET 8)
- **API Gateway**: ASP.NET Core Web API with SignalR hubs
- **Database**: PostgreSQL with Entity Framework Core
- **Background Jobs**: Hangfire for document generation and publishing workflows
- **Logging**: Serilog with structured logging
- **Documentation**: OpenAPI/Swagger

### Frontend (React/TypeScript)
- **Framework**: React 18+ with TypeScript
- **State Management**: React Query + Context API
- **UI Library**: Material-UI or Ant Design
- **Real-time**: SignalR client for live updates
- **Build**: Vite for fast development

### Infrastructure
- **Containerization**: Docker Compose for development, Docker for production
- **Reverse Proxy**: Nginx for production
- **Service Discovery**: Docker network with service names

## Service Architecture

### Core Services

#### 1. API Gateway Service (`api-gateway`)
- **Purpose**: Single entry point, authentication, SignalR hub
- **Technology**: ASP.NET Core Web API + SignalR
- **Responsibilities**:
  - User authentication/authorization (JWT)
  - Route requests to appropriate services
  - Real-time notifications via SignalR
  - CORS configuration for React frontend

#### 2. Project Service (`project-service`)
- **Purpose**: CruiseControl.NET integration and project management
- **Technology**: ASP.NET Core Web API
- **Responsibilities**:
  - Parse CC.NET log files
  - Cache build history and status
  - Project configuration management
  - Build status monitoring

#### 3. Manuscript Service (`manuscript-service`)
- **Purpose**: FogBugz/Manuscript integration
- **Technology**: ASP.NET Core Web API
- **Responsibilities**:
  - Developer comment extraction
  - Feature timing data aggregation
  - Release note data compilation

#### 4. Document Service (`document-service`)
- **Purpose**: PDF generation and document management
- **Technology**: ASP.NET Core + iText7 or QuestPDF
- **Responsibilities**:
  - Release note PDF generation
  - Template management
  - Document versioning

#### 5. Storage Service (`storage-service`)
- **Purpose**: Cloud storage abstraction layer
- **Technology**: ASP.NET Core Web API
- **Storage Providers**:
  - **AWS S3**: AWS SDK for .NET
  - **Box.com**: Box .NET SDK
  - **FTP**: FluentFTP library
- **Pattern**: Strategy pattern for provider abstraction

#### 6. Help Center Service (`helpcenter-service`)
- **Purpose**: Support article updates
- **Technology**: ASP.NET Core Web API
- **Integrations**:
  - **Zendesk**: Zendesk API v2
  - **Salesforce Service**: Salesforce REST API
- **Pattern**: Strategy pattern for provider abstraction

#### 7. Publishing Service (`publishing-service`)
- **Purpose**: Orchestrate complete publishing workflows
- **Technology**: ASP.NET Core + Hangfire
- **Responsibilities**:
  - Coordinate document generation
  - Manage cloud uploads
  - Update help center articles
  - Workflow status tracking

### Database Design

```mermaid
erDiagram
    Users {
        int Id PK
        string Username
        string PasswordHash
        string Role
        datetime CreatedAt
        datetime LastLogin
    }
    
    Projects {
        int Id PK
        string Name
        string Description
        string CCNetProjectName
        string Status
        datetime CreatedAt
        datetime UpdatedAt
    }
    
    Builds {
        int Id PK
        int ProjectId FK
        string BuildNumber
        string Status
        datetime StartTime
        datetime EndTime
        string LogPath
    }
    
    Packages {
        int Id PK
        string Name
        int ProjectId FK
        json CloudStorageConfig
        json HelpCenterConfig
        datetime CreatedAt
    }
    
    Publications {
        int Id PK
        int PackageId FK
        string Version
        string Status
        datetime PublishedAt
        string ReleaseNotesPath
    }
    
    StorageProviders {
        int Id PK
        string Name
        string Type
        json Configuration
        boolean IsActive
    }
    
    HelpCenterProviders {
        int Id PK
        string Name
        string Type
        json Configuration
        boolean IsActive
    }
    
    Projects ||--o{ Builds : "has builds"
    Projects ||--o{ Packages : "has packages"
    Packages ||--o{ Publications : "has publications"
```

#### Core Entities
```sql
-- Projects from CruiseControl.NET
Projects (Id, Name, Description, CCNetProjectName, Status, CreatedAt, UpdatedAt)

-- Build information
Builds (Id, ProjectId, BuildNumber, Status, StartTime, EndTime, LogPath)

-- Software packages configuration
Packages (Id, Name, ProjectId, CloudStorageConfig, HelpCenterConfig, CreatedAt)

-- Publishing history
Publications (Id, PackageId, Version, Status, PublishedAt, ReleaseNotesPath)

-- User management
Users (Id, Username, PasswordHash, Role, CreatedAt, LastLogin)

-- Provider configurations
StorageProviders (Id, Name, Type, Configuration, IsActive)
HelpCenterProviders (Id, Name, Type, Configuration, IsActive)
```

## Interface Abstractions

### Cloud Storage Interface
```csharp
public interface ICloudStorageProvider
{
    Task<string> UploadAsync(string containerName, string fileName, Stream content);
    Task<bool> DeleteAsync(string containerName, string fileName);
    Task<Stream> DownloadAsync(string containerName, string fileName);
}
```

### Help Center Interface
```csharp
public interface IHelpCenterProvider
{
    Task<bool> UpdateArticleAsync(string articleId, string content, string title);
    Task<string> CreateArticleAsync(string content, string title, string categoryId);
    Task<bool> PublishArticleAsync(string articleId);
}
```

## Development Environment

### Docker Compose Structure
```yaml
services:
  postgres:
    image: postgres:15
    
  api-gateway:
    build: ./src/ApiGateway
    ports: ["5000:80"]
    
  project-service:
    build: ./src/ProjectService
    
  manuscript-service:
    build: ./src/ManuscriptService
    
  document-service:
    build: ./src/DocumentService
    
  storage-service:
    build: ./src/StorageService
    
  helpcenter-service:
    build: ./src/HelpCenterService
    
  publishing-service:
    build: ./src/PublishingService
    
  react-frontend:
    build: ./src/Frontend
    ports: ["3000:80"]
    
  hangfire-dashboard:
    ports: ["5001:80"]
```

### Project Structure
```mermaid
graph TD
    subgraph "Solution Structure"
        ROOT[Solution Root]
        
        subgraph "Backend Services"
            API[ApiGateway/]
            PROJ[ProjectService/]
            MAN[ManuscriptService/]
            DOC[DocumentService/]
            STOR[StorageService/]
            HELP[HelpCenterService/]
            PUB[PublishingService/]
        end
        
        subgraph "Shared Components"
            SHARED[Shared/]
            DB[Database/]
        end
        
        subgraph "Frontend"
            UI[Frontend/]
        end
        
        subgraph "Infrastructure"
            DOCKER[docker-compose.yml]
            ENV[.env files]
        end
    end
    
    ROOT --> API
    ROOT --> PROJ
    ROOT --> MAN
    ROOT --> DOC
    ROOT --> STOR
    ROOT --> HELP
    ROOT --> PUB
    ROOT --> SHARED
    ROOT --> DB
    ROOT --> UI
    ROOT --> DOCKER
    ROOT --> ENV
    
    classDef service fill:#e3f2fd
    classDef shared fill:#f3e5f5
    classDef frontend fill:#e8f5e8
    classDef infra fill:#fff3e0
    
    class API,PROJ,MAN,DOC,STOR,HELP,PUB service
    class SHARED,DB shared
    class UI frontend
    class DOCKER,ENV infra
```

```
src/
├── ApiGateway/          # Main API + SignalR hub
├── ProjectService/      # CC.NET integration
├── ManuscriptService/   # FogBugz integration  
├── DocumentService/     # PDF generation
├── StorageService/      # Cloud storage abstraction
├── HelpCenterService/   # Article update service
├── PublishingService/   # Workflow orchestration
├── Shared/              # Common models, interfaces
├── Frontend/            # React TypeScript app
└── Database/            # EF migrations, seed data
```

## Key Architectural Decisions

### 1. Microservices with Service Mesh
- **Rationale**: Clean separation of concerns, independent scaling
- **Communication**: HTTP REST + SignalR for real-time updates
- **Service Discovery**: Docker network DNS resolution

### 2. Hangfire for Background Processing
- **Benefits**: .NET native, web dashboard, persistent queues
- **Use Cases**: Document generation, cloud uploads, batch processing
- **Storage**: PostgreSQL for job persistence

### 3. Strategy Pattern for External Integrations
- **Cloud Storage**: Pluggable providers (S3, Box, FTP)
- **Help Centers**: Extensible article update system
- **Benefits**: Easy to add new providers without core changes

### 4. SignalR for Real-time Updates
- **Use Cases**: Build status, publishing progress, job completion
- **Frontend Integration**: React SignalR client with auto-reconnection

### 5. Entity Framework with Repository Pattern
- **Database**: Code-first migrations
- **Testing**: In-memory provider for unit tests
- **Performance**: Query optimization with projection

## Security Considerations

### Authentication & Authorization
- JWT tokens with refresh mechanism
- Role-based access control (Admin, User, ReadOnly)
- Secure password hashing (bcrypt)

### API Security
- HTTPS only in production
- CORS properly configured
- API rate limiting
- Input validation and sanitization

### Secrets Management
- Docker secrets for production
- Environment variables for configuration
- No hardcoded credentials

## Deployment Strategy

### Development
- Docker Compose with hot-reload volumes
- Shared PostgreSQL instance
- Frontend dev server proxy to API

### Production
- Individual Docker containers
- Nginx reverse proxy
- PostgreSQL with persistent volumes
- Health checks and restart policies

## Questions for Next Phase

1. **CC.NET Integration**: What format are the log files? XML, plain text, or proprietary?
2. **Manuscript API**: Is there existing API documentation or will we need to reverse-engineer?
3. **Document Templates**: Do you have existing release note templates or should we create standard ones?
4. **Cloud Storage**: Any specific bucket/container naming conventions?
5. **Help Center**: Do you have existing article templates or categories to work with?

## Publishing Workflow

```mermaid
sequenceDiagram
    participant User
    participant UI as React Frontend
    participant GW as API Gateway
    participant PUB as Publishing Service
    participant DOC as Document Service
    participant STOR as Storage Service
    participant HELP as Help Center Service
    participant HF as Hangfire
    participant SH as SignalR Hub
    
    User->>UI: Select Package & Initiate Publishing
    UI->>GW: POST /api/publish/package/{id}
    GW->>PUB: Queue Publishing Job
    PUB->>HF: Enqueue Background Job
    
    Note over HF: Background Processing Starts
    
    HF->>DOC: Generate Release Notes PDF
    DOC-->>HF: PDF Generated
    
    HF->>STOR: Upload Package + PDF to Cloud
    STOR-->>HF: Upload Complete
    
    HF->>HELP: Update Help Center Articles
    HELP-->>HF: Articles Updated
    
    HF->>SH: Notify Completion
    SH->>UI: Real-time Status Update
    UI->>User: Show Completion Status
```

## Data Flow Architecture

```mermaid
flowchart LR
    subgraph "External Sources"
        CC[CruiseControl.NET<br/>Build Logs]
        FB[FogBugz/Manuscript<br/>Developer Data]
    end
    
    subgraph "Data Ingestion"
        PS[Project Service<br/>Log Parser]
        MS[Manuscript Service<br/>API Client]
    end
    
    subgraph "Data Storage"
        DB[(PostgreSQL<br/>Centralized Data)]
    end
    
    subgraph "Processing Layer"
        DS[Document Service<br/>PDF Generator]
        PUB[Publishing Service<br/>Orchestrator]
    end
    
    subgraph "Output Channels"
        STOR[Storage Service<br/>Multi-Provider]
        HELP[Help Center Service<br/>Multi-Platform]
    end
    
    subgraph "User Interface"
        UI[React Frontend<br/>Management Console]
    end
    
    CC --> PS
    FB --> MS
    PS --> DB
    MS --> DB
    DB --> DS
    DB --> PUB
    DS --> STOR
    PUB --> STOR
    PUB --> HELP
    DB --> UI
    
    classDef external fill:#ffebee
    classDef ingestion fill:#e3f2fd
    classDef storage fill:#f3e5f5
    classDef processing fill:#e8f5e8
    classDef output fill:#fff3e0
    classDef ui fill:#fce4ec
    
    class CC,FB external
    class PS,MS ingestion
    class DB storage
    class DS,PUB processing
    class STOR,HELP output
    class UI ui
```

## Next Steps

1. Set up development environment with Docker Compose
2. Create solution structure and shared contracts
3. Implement core database models and Entity Framework setup
4. Build API Gateway with authentication
5. Develop individual services in parallel
6. Create React frontend with SignalR integration
7. Integration testing and deployment scripts