# Design Document

## Overview

This design addresses critical issues in the SamCloud dashboard and enhances it into a comprehensive Mac Mini server management platform. The solution focuses on fixing broken functionality, improving real-time data accuracy, and adding advanced monitoring capabilities while maintaining the existing Apple-styled UI aesthetic.

## Architecture

### System Architecture
```
┌─────────────────────────────────────────────────────────────┐
│                    SamCloud Dashboard                       │
├─────────────────────────────────────────────────────────────┤
│  Frontend (Templates + JavaScript)                         │
│  ├── Real-time Updates (WebSocket/SSE)                     │
│  ├── File Management Interface                             │
│  └── System Monitoring Widgets                             │
├─────────────────────────────────────────────────────────────┤
│  Flask Backend                                              │
│  ├── API Routes                                             │
│  ├── Authentication & Session Management                   │
│  ├── File Operations & Share Management                    │
│  └── System Information Collectors                         │
├─────────────────────────────────────────────────────────────┤
│  Data Layer                                                 │
│  ├── SQLite Database (Users, Shares, Logs, Access History) │
│  ├── In-Memory Cache (System Stats, Active Sessions)       │
│  └── File System Integration                               │
├─────────────────────────────────────────────────────────────┤
│  External Integrations                                      │
│  ├── FileBrowser Service (Port 8833)                       │
│  ├── OrbStack API                                           │
│  ├── macOS System APIs                                      │
│  └── Weather Service (Open-Meteo)                          │
└─────────────────────────────────────────────────────────────┘
```

### Data Flow Architecture
```
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│   Browser    │◄──►│    Flask     │◄──►│   System     │
│   Client     │    │   Backend    │    │   APIs       │
└──────────────┘    └──────────────┘    └──────────────┘
       │                     │                   │
       │                     ▼                   │
       │            ┌──────────────┐             │
       │            │   SQLite     │             │
       │            │  Database    │             │
       │            └──────────────┘             │
       │                     │                   │
       ▼                     ▼                   ▼
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│ FileBrowser  │    │  In-Memory   │    │  OrbStack    │
│  Service     │    │    Cache     │    │    API       │
└──────────────┘    └──────────────┘    └──────────────┘
```

## Components and Interfaces

### 1. Bulk Download System

**Component**: `ZipDownloadManager`
- **Purpose**: Handle creation and delivery of ZIP archives for shared files
- **Interface**: 
  ```python
  class ZipDownloadManager:
      def create_zip_async(self, files: List[str], job_id: str) -> None
      def get_job_status(self, job_id: str) -> Dict
      def cleanup_old_zips(self) -> None
  ```

**Frontend Integration**:
- Add "Download All as ZIP" button to share pages
- Implement progress tracking with WebSocket updates
- Handle download initiation and error states

### 2. Share Management Enhancement

**Component**: `ShareAccessTracker`
- **Purpose**: Track and log share access with visitor information
- **Database Schema**:
  ```sql
  CREATE TABLE share_access_log (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      share_id TEXT NOT NULL,
      visitor_ip TEXT NOT NULL,
      user_agent TEXT,
      access_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
      action TEXT DEFAULT 'view',
      FOREIGN KEY (share_id) REFERENCES user_shares (share_id)
  );
  ```

**Real-time Updates**:
- Use Server-Sent Events (SSE) for live share list updates
- Implement WebSocket connection for instant share creation feedback

### 3. System Information Collectors

**Component**: `MacSystemInfoCollector`
- **Purpose**: Gather accurate Mac Mini system information
- **Methods**:
  ```python
  class MacSystemInfoCollector:
      def get_mac_uptime(self) -> Dict  # Real Mac uptime, not container
      def get_orbstack_uptime(self) -> Dict  # OrbStack specific uptime
      def get_accurate_storage_info(self) -> Dict
      def get_hardware_specs(self) -> Dict
      def get_macos_version_info(self) -> Dict
  ```

**Data Sources**:
- `uptime` command for Mac system uptime
- `system_profiler` for hardware information
- `sw_vers` for macOS version details
- `df` and `diskutil` for storage information
- OrbStack CLI for container uptime

### 4. Widget System Redesign

**Component**: `SystemWidgetManager`
- **Purpose**: Manage and render system monitoring widgets
- **Widget Types**:
  - CPU Usage (existing, improved)
  - Memory Usage (existing, improved)  
  - Temperature Monitoring (existing, improved)
  - **NEW**: Disk I/O Activity
  - **NEW**: Network Interface Statistics
  - **NEW**: Top Processes Monitor

**Widget Configuration**:
```javascript
const widgetConfig = {
    diskIO: {
        title: "Disk I/O",
        updateInterval: 5000,
        dataSource: "/api/system/disk-io",
        chartType: "line"
    },
    networkStats: {
        title: "Network Activity", 
        updateInterval: 5000,
        dataSource: "/api/system/network",
        chartType: "area"
    }
};
```

### 5. OrbStack Integration

**Component**: `OrbStackMonitor`
- **Purpose**: Monitor OrbStack containers and services
- **Interface**:
  ```python
  class OrbStackMonitor:
      def get_containers(self) -> List[Dict]
      def get_container_stats(self, container_id: str) -> Dict
      def get_orbstack_status(self) -> Dict
  ```

**Implementation Strategy**:
- Use OrbStack CLI commands: `orb list`, `orb stats`
- Parse JSON output for container information
- Cache results for 30 seconds to reduce system load
- Handle OrbStack not installed scenario gracefully

### 6. Network Device Detection

**Component**: `NetworkDeviceScanner`
- **Purpose**: Detect and monitor connected network devices
- **Detection Methods**:
  - ARP table scanning for local network devices
  - Active connection monitoring (lsof for ports 22, 445, 5900)
  - Network interface statistics
  - DHCP lease file parsing (if accessible)

**Data Structure**:
```python
@dataclass
class NetworkDevice:
    ip_address: str
    hostname: str
    mac_address: str
    connection_type: str  # SMB, SSH, VNC, etc.
    status: str
    last_seen: datetime
    icon: str
```

## Data Models

### Enhanced Share Model
```sql
-- Extend existing user_shares table
ALTER TABLE user_shares ADD COLUMN access_count INTEGER DEFAULT 0;
ALTER TABLE user_shares ADD COLUMN last_accessed TIMESTAMP;

-- New access tracking table
CREATE TABLE share_access_log (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    share_id TEXT NOT NULL,
    visitor_ip TEXT NOT NULL,
    user_agent TEXT,
    access_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    action TEXT DEFAULT 'view',
    file_path TEXT,
    FOREIGN KEY (share_id) REFERENCES user_shares (share_id)
);
```

### System Monitoring Data
```sql
CREATE TABLE system_metrics (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    metric_type TEXT NOT NULL,
    metric_value REAL NOT NULL,
    timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    metadata TEXT  -- JSON for additional data
);

CREATE TABLE process_history (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    process_name TEXT NOT NULL,
    pid INTEGER NOT NULL,
    cpu_percent REAL,
    memory_percent REAL,
    timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```

### Network Device Tracking
```sql
CREATE TABLE network_devices (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    ip_address TEXT UNIQUE NOT NULL,
    hostname TEXT,
    mac_address TEXT,
    device_type TEXT,
    first_seen TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    last_seen TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    status TEXT DEFAULT 'online'
);
```

## Error Handling

### File Operations
- **ZIP Creation Failures**: Implement retry mechanism with exponential backoff
- **Large File Handling**: Stream processing for files > 100MB
- **Disk Space**: Check available space before ZIP creation
- **Permission Errors**: Graceful fallback with user notification

### System Information Collection
- **Command Timeouts**: 5-second timeout for system commands
- **Missing Tools**: Fallback methods when system tools unavailable
- **Permission Issues**: Handle sudo requirements gracefully
- **Container Environment**: Detect and adapt to containerized environment

### Network Operations
- **FileBrowser Unavailable**: Seamless fallback to internal file browser
- **Network Scanning**: Handle network interface changes
- **External API Failures**: Cache last known good data

## Testing Strategy

### Unit Tests
- **System Information Collectors**: Mock system commands and test parsing
- **ZIP Creation**: Test with various file types and sizes
- **Share Access Tracking**: Verify logging accuracy and privacy
- **Network Device Detection**: Mock network interfaces and connections

### Integration Tests
- **FileBrowser Integration**: Test fallback mechanisms
- **Database Operations**: Verify share and access log integrity
- **Real-time Updates**: Test WebSocket/SSE functionality
- **OrbStack Integration**: Test with and without OrbStack installed

### Performance Tests
- **Large File ZIP Creation**: Test with files up to 1GB
- **Concurrent Share Access**: Simulate multiple users accessing shares
- **System Monitoring Load**: Verify minimal impact on system performance
- **Memory Usage**: Monitor for memory leaks in long-running processes

### Browser Compatibility Tests
- **Modern Browsers**: Chrome, Firefox, Safari, Edge
- **Mobile Browsers**: iOS Safari, Android Chrome
- **JavaScript Features**: WebSocket, Server-Sent Events, File API

## Security Considerations

### Share Access Security
- **IP Logging**: Store visitor IPs for security auditing
- **Rate Limiting**: Prevent abuse of share endpoints
- **Access Token Validation**: Verify share tokens haven't expired
- **File Path Validation**: Prevent directory traversal attacks

### System Information Security
- **Sensitive Data**: Filter out sensitive system information
- **Command Injection**: Sanitize all system command inputs
- **Process Information**: Limit process details to non-sensitive data
- **Network Scanning**: Respect network privacy boundaries

### Authentication & Authorization
- **Session Management**: Secure session handling with proper timeouts
- **Admin Functions**: Restrict system monitoring to admin users
- **API Endpoints**: Ensure all endpoints require proper authentication
- **CSRF Protection**: Implement CSRF tokens for state-changing operations

## Performance Optimizations

### Caching Strategy
- **System Metrics**: Cache for 5 seconds to reduce system load
- **Network Devices**: Cache for 30 seconds, background refresh
- **File Listings**: Cache directory contents for 10 seconds
- **Weather Data**: Cache for 30 minutes

### Background Processing
- **ZIP Creation**: Asynchronous processing with job queue
- **System Monitoring**: Background threads for data collection
- **Log Cleanup**: Scheduled cleanup of old access logs and metrics
- **Device Scanning**: Background network scanning every 60 seconds

### Database Optimization
- **Indexes**: Add indexes on frequently queried columns
- **Cleanup Jobs**: Regular cleanup of old metrics and logs
- **Connection Pooling**: Efficient database connection management
- **Query Optimization**: Use prepared statements and efficient queries

## Deployment Considerations

### Environment Variables
```bash
FLASK_SECRET_KEY=your-secret-key
FILEBROWSER_URL=http://127.0.0.1:8833
ORBSTACK_ENABLED=true
NETWORK_SCANNING_ENABLED=true
SYSTEM_MONITORING_INTERVAL=5
```

### File Permissions
- **ZIP Directory**: Ensure write permissions for ZIP creation
- **Log Files**: Proper permissions for log file creation
- **System Commands**: Handle sudo requirements for system monitoring

### Resource Requirements
- **Memory**: Additional 50-100MB for caching and background processes
- **CPU**: Minimal impact with optimized system monitoring
- **Disk**: Space for ZIP files and log storage
- **Network**: Bandwidth for real-time updates and external integrations