squad/implementation.md

Implementation Architecture

Executive Summary

Software Development Insight & Release Management Platform with modular microservices architecture, containerized deployment, and modern web frontend.

System Architecture Overview

graph TB
    subgraph "Frontend Layer"
        UI[React TypeScript Frontend]
        UI --> SG[SignalR Client]
    end
    
    subgraph "API Gateway"
        GW[API Gateway Service<br/>Authentication & Routing]
        HUB[SignalR Hub<br/>Real-time Updates]
    end
    
    subgraph "Core Services"
        PS[Project Service<br/>CC.NET Integration]
        MS[Manuscript Service<br/>FogBugz Integration]
        DS[Document Service<br/>PDF Generation]
        SS[Storage Service<br/>Cloud Abstraction]
        HS[Help Center Service<br/>Article Updates]
        PUB[Publishing Service<br/>Workflow Orchestration]
    end
    
    subgraph "Background Processing"
        HF[Hangfire<br/>Background Jobs]
        HFD[Hangfire Dashboard]
    end
    
    subgraph "Data Layer"
        DB[(PostgreSQL<br/>Database)]
    end
    
    subgraph "External Integrations"
        CC[CruiseControl.NET]
        FB[FogBugz/Manuscript]
        S3[AWS S3]
        BOX[Box.com]
        FTP[FTP Server]
        ZD[Zendesk]
        SF[Salesforce Service]
    end
    
    UI --> GW
    SG --> HUB
    GW --> PS
    GW --> MS
    GW --> DS
    GW --> SS
    GW --> HS
    GW --> PUB
    
    PS --> CC
    MS --> FB
    SS --> S3
    SS --> BOX
    SS --> FTP
    HS --> ZD
    HS --> SF
    
    PUB --> HF
    HF --> DS
    HF --> SS
    HF --> HS
    
    PS --> DB
    MS --> DB
    DS --> DB
    SS --> DB
    HS --> DB
    PUB --> DB
    HF --> DB
    
    classDef frontend fill:#e1f5fe
    classDef gateway fill:#fff3e0
    classDef service fill:#f3e5f5
    classDef background fill:#e8f5e8
    classDef database fill:#fce4ec
    classDef external fill:#f5f5f5
    
    class UI,SG frontend
    class GW,HUB gateway
    class PS,MS,DS,SS,HS,PUB service
    class HF,HFD background
    class DB database
    class CC,FB,S3,BOX,FTP,ZD,SF external

Technology Stack

Backend Services (.NET 8)

• API Gateway: ASP.NET Core Web API with SignalR hubs
• Database: PostgreSQL with Entity Framework Core
• Background Jobs: Hangfire for document generation and publishing workflows
• Logging: Serilog with structured logging
• Documentation: OpenAPI/Swagger

Frontend (React/TypeScript)

• Framework: React 18+ with TypeScript
• State Management: React Query + Context API
• UI Library: Material-UI or Ant Design
• Real-time: SignalR client for live updates
• Build: Vite for fast development

Infrastructure

• Containerization: Docker Compose for development, Docker for production
• Reverse Proxy: Nginx for production
• Service Discovery: Docker network with service names

Service Architecture

Core Services

1. API Gateway Service (api-gateway)

• Purpose: Single entry point, authentication, SignalR hub
• Technology: ASP.NET Core Web API + SignalR
• Responsibilities:
  • User authentication/authorization (JWT)
  • Route requests to appropriate services
  • Real-time notifications via SignalR
  • CORS configuration for React frontend

2. Project Service (project-service)

• Purpose: CruiseControl.NET integration and project management
• Technology: ASP.NET Core Web API
• Responsibilities:
  • Parse CC.NET log files
  • Cache build history and status
  • Project configuration management
  • Build status monitoring

3. Manuscript Service (manuscript-service)

• Purpose: FogBugz/Manuscript integration
• Technology: ASP.NET Core Web API
• Responsibilities:
  • Developer comment extraction
  • Feature timing data aggregation
  • Release note data compilation

4. Document Service (document-service)

• Purpose: PDF generation and document management
• Technology: ASP.NET Core + iText7 or QuestPDF
• Responsibilities:
  • Release note PDF generation
  • Template management
  • Document versioning

5. Storage Service (storage-service)

• Purpose: Cloud storage abstraction layer
• Technology: ASP.NET Core Web API
• Storage Providers:
  • AWS S3: AWS SDK for .NET
  • Box.com: Box .NET SDK
  • FTP: FluentFTP library
• Pattern: Strategy pattern for provider abstraction

6. Help Center Service (helpcenter-service)

• Purpose: Support article updates
• Technology: ASP.NET Core Web API
• Integrations:
  • Zendesk: Zendesk API v2
  • Salesforce Service: Salesforce REST API
• Pattern: Strategy pattern for provider abstraction

7. Publishing Service (publishing-service)

• Purpose: Orchestrate complete publishing workflows
• Technology: ASP.NET Core + Hangfire
• Responsibilities:
  • Package creation from successful builds
  • Coordinate document generation
  • Manage cloud uploads
  • Update help center articles
  • Workflow status tracking

Database Design

erDiagram
    Users {
        int Id PK
        string Username
        string PasswordHash
        string Role
        datetime CreatedAt
        datetime LastLogin
    }
    
    Packages {
        int Id PK
        string Name
        string Version
        string Description
        int SourceBuildId FK
        json CloudStorageConfig
        json HelpCenterConfig
        string Status
        datetime CreatedAt
        datetime UpdatedAt
    }
    
    Projects {
        int Id PK
        string Name
        string Description
        string CCNetProjectName
        string Status
        datetime CreatedAt
        datetime UpdatedAt
    }
    
    Builds {
        int Id PK
        int ProjectId FK
        string BuildNumber
        string Status
        datetime StartTime
        datetime EndTime
        string LogPath
        string ArtifactPath
    }
    
    Publications {
        int Id PK
        int PackageId FK
        string Status
        datetime PublishedAt
        string ReleaseNotesPath
        json PublicationDetails
    }
    
    StorageProviders {
        int Id PK
        string Name
        string Type
        json Configuration
        boolean IsActive
    }
    
    HelpCenterProviders {
        int Id PK
        string Name
        string Type
        json Configuration
        boolean IsActive
    }
    
    Projects ||--o{ Builds : "produces builds"
    Builds ||--o| Packages : "creates package"
    Packages ||--o{ Publications : "has publications"

Core Entities

-- Packages: Top-level finalized software releases
Packages (Id, Name, Version, Description, SourceBuildId, CloudStorageConfig, HelpCenterConfig, Status, CreatedAt, UpdatedAt)

-- Projects from CruiseControl.NET
Projects (Id, Name, Description, CCNetProjectName, Status, CreatedAt, UpdatedAt)

-- Build information
Builds (Id, ProjectId, BuildNumber, Status, StartTime, EndTime, LogPath, ArtifactPath)

-- Publishing history
Publications (Id, PackageId, Status, PublishedAt, ReleaseNotesPath, PublicationDetails)

-- User management
Users (Id, Username, PasswordHash, Role, CreatedAt, LastLogin)

-- Provider configurations
StorageProviders (Id, Name, Type, Configuration, IsActive)
HelpCenterProviders (Id, Name, Type, Configuration, IsActive)

Interface Abstractions

Cloud Storage Interface

public interface ICloudStorageProvider
{
    Task<string> UploadAsync(string containerName, string fileName, Stream content);
    Task<bool> DeleteAsync(string containerName, string fileName);
    Task<Stream> DownloadAsync(string containerName, string fileName);
}

Help Center Interface

public interface IHelpCenterProvider
{
    Task<bool> UpdateArticleAsync(string articleId, string content, string title);
    Task<string> CreateArticleAsync(string content, string title, string categoryId);
    Task<bool> PublishArticleAsync(string articleId);
}

Development Environment

Docker Compose Structure

services:
  postgres:
    image: postgres:15
    
  api-gateway:
    build: ./src/ApiGateway
    ports: ["5000:80"]
    
  project-service:
    build: ./src/ProjectService
    
  manuscript-service:
    build: ./src/ManuscriptService
    
  document-service:
    build: ./src/DocumentService
    
  storage-service:
    build: ./src/StorageService
    
  helpcenter-service:
    build: ./src/HelpCenterService
    
  publishing-service:
    build: ./src/PublishingService
    
  react-frontend:
    build: ./src/Frontend
    ports: ["3000:80"]
    
  hangfire-dashboard:
    ports: ["5001:80"]

Project Structure

graph TD
    subgraph "Solution Structure"
        ROOT[Solution Root]
        
        subgraph "Backend Services"
            API[ApiGateway/]
            PROJ[ProjectService/]
            MAN[ManuscriptService/]
            DOC[DocumentService/]
            STOR[StorageService/]
            HELP[HelpCenterService/]
            PUB[PublishingService/]
        end
        
        subgraph "Shared Components"
            SHARED[Shared/]
            DB[Database/]
        end
        
        subgraph "Frontend"
            UI[Frontend/]
        end
        
        subgraph "Infrastructure"
            DOCKER[docker-compose.yml]
            ENV[.env files]
        end
    end
    
    ROOT --> API
    ROOT --> PROJ
    ROOT --> MAN
    ROOT --> DOC
    ROOT --> STOR
    ROOT --> HELP
    ROOT --> PUB
    ROOT --> SHARED
    ROOT --> DB
    ROOT --> UI
    ROOT --> DOCKER
    ROOT --> ENV
    
    classDef service fill:#e3f2fd
    classDef shared fill:#f3e5f5
    classDef frontend fill:#e8f5e8
    classDef infra fill:#fff3e0
    
    class API,PROJ,MAN,DOC,STOR,HELP,PUB service
    class SHARED,DB shared
    class UI frontend
    class DOCKER,ENV infra

src/
├── ApiGateway/          # Main API + SignalR hub
├── ProjectService/      # CC.NET integration
├── ManuscriptService/   # FogBugz integration  
├── DocumentService/     # PDF generation
├── StorageService/      # Cloud storage abstraction
├── HelpCenterService/   # Article update service
├── PublishingService/   # Workflow orchestration
├── Shared/              # Common models, interfaces
├── Frontend/            # React TypeScript app
└── Database/            # EF migrations, seed data

Key Architectural Decisions

1. Microservices with Service Mesh

• Rationale: Clean separation of concerns, independent scaling
• Communication: HTTP REST + SignalR for real-time updates
• Service Discovery: Docker network DNS resolution

2. Hangfire for Background Processing

• Benefits: .NET native, web dashboard, persistent queues
• Use Cases: Document generation, cloud uploads, batch processing
• Storage: PostgreSQL for job persistence

3. Strategy Pattern for External Integrations

• Cloud Storage: Pluggable providers (S3, Box, FTP)
• Help Centers: Extensible article update system
• Benefits: Easy to add new providers without core changes

4. SignalR for Real-time Updates

• Use Cases: Build status, publishing progress, job completion
• Frontend Integration: React SignalR client with auto-reconnection

5. Entity Framework with Repository Pattern

• Database: Code-first migrations
• Testing: In-memory provider for unit tests
• Performance: Query optimization with projection

Security Considerations

Authentication & Authorization

• JWT tokens with refresh mechanism
• Role-based access control (Admin, User, ReadOnly)
• Secure password hashing (bcrypt)

API Security

• HTTPS only in production
• CORS properly configured
• API rate limiting
• Input validation and sanitization

Secrets Management

• Docker secrets for production
• Environment variables for configuration
• No hardcoded credentials

Deployment Strategy

Development

• Docker Compose with hot-reload volumes
• Shared PostgreSQL instance
• Frontend dev server proxy to API

Production

• Individual Docker containers
• Nginx reverse proxy
• PostgreSQL with persistent volumes
• Health checks and restart policies

Questions for Next Phase

1. CC.NET Integration: What format are the log files? XML, plain text, or proprietary?
2. Manuscript API: Is there existing API documentation or will we need to reverse-engineer?
3. Document Templates: Do you have existing release note templates or should we create standard ones?
4. Cloud Storage: Any specific bucket/container naming conventions?
5. Help Center: Do you have existing article templates or categories to work with?

Publishing Workflow

sequenceDiagram
    participant User
    participant UI as React Frontend
    participant GW as API Gateway
    participant PUB as Publishing Service
    participant DOC as Document Service
    participant STOR as Storage Service
    participant HELP as Help Center Service
    participant HF as Hangfire
    participant SH as SignalR Hub
    
    Note over User,UI: Package Creation Phase
    User->>UI: Select Build & Create Package
    UI->>GW: POST /api/packages/create-from-build
    GW->>PUB: Create Package from Build
    PUB->>SH: Package Created Notification
    SH->>UI: Real-time Package Status
    
    Note over User,UI: Publishing Phase
    User->>UI: Select Package & Initiate Publishing
    UI->>GW: POST /api/publish/package/{id}
    GW->>PUB: Queue Publishing Job
    PUB->>HF: Enqueue Background Job
    
    Note over HF: Background Processing Starts
    
    HF->>DOC: Generate Release Notes PDF
    DOC-->>HF: PDF Generated
    
    HF->>STOR: Upload Package + PDF to Cloud
    STOR-->>HF: Upload Complete
    
    HF->>HELP: Update Help Center Articles
    HELP-->>HF: Articles Updated
    
    HF->>SH: Notify Completion
    SH->>UI: Real-time Publication Status
    UI->>User: Show Completion Status

Data Flow Architecture

flowchart TD
    subgraph "External Sources"
        CC[CruiseControl.NET<br/>Build Logs]
        FB[FogBugz/Manuscript<br/>Developer Data]
    end
    
    subgraph "Data Ingestion"
        PS[Project Service<br/>Log Parser]
        MS[Manuscript Service<br/>API Client]
    end
    
    subgraph "Core Data Model"
        PROJ[Projects<br/>CC.NET Projects]
        BUILDS[Builds<br/>From Projects]
        PKG[Packages<br/>Finalized Releases]
        PUB_DATA[Publications<br/>Publishing History]
    end
    
    subgraph "Processing Layer"
        DS[Document Service<br/>PDF Generator]
        PUB[Publishing Service<br/>Package Creator & Orchestrator]
    end
    
    subgraph "Output Channels"
        STOR[Storage Service<br/>Multi-Provider]
        HELP[Help Center Service<br/>Multi-Platform]
    end
    
    subgraph "User Interface"
        UI[React Frontend<br/>Package Management Console]
    end
    
    CC --> PS
    FB --> MS
    PS --> PROJ
    MS --> PROJ
    PROJ --> BUILDS
    BUILDS --> PKG
    PKG --> PUB_DATA
    
    UI --> PKG
    PKG --> PUB
    PUB --> DS
    PUB --> STOR
    PUB --> HELP
    PUB --> PUB_DATA
    
    classDef external fill:#ffebee
    classDef ingestion fill:#e3f2fd
    classDef coredata fill:#e8f5e8
    classDef processing fill:#fff3e0
    classDef output fill:#f3e5f5
    classDef ui fill:#fce4ec
    
    class CC,FB external
    class PS,MS ingestion
    class PROJ,BUILDS,PKG,PUB_DATA coredata
    class DS,PUB processing
    class STOR,HELP output
    class UI ui

Next Steps

1. Set up development environment with Docker Compose
2. Create solution structure and shared contracts
3. Implement core database models and Entity Framework setup
4. Build API Gateway with authentication
5. Develop individual services in parallel
6. Create React frontend with SignalR integration
7. Integration testing and deployment scripts