Engineering

System Architecture Flow

graph TD
    %% Data Sources Layer
    subgraph DS["Data Sources"]
        WHOIS["whois.ts"]
        URLINFO["urlinfo.ts"]
        OTHER["other providers"]
    end

    %% Configuration Layer
    EXPLORE["explore.vue<br/>Config Manager"]

    %% Validation System
    VALIDATORS["validators.ts<br/>Input Processor"]

    %% Main Controller
    subgraph SCAN["scan.vue"]
        UI["UI"]
        CTRL["Controller"]
    end

    %% Core Processing Components
    subgraph CORE["Core Components"]
        GS["graph_source.ts<br/>State Manager"]
        SE["source_executor.ts<br/>Command Runner"]
        CHART["chart.vue<br/>Visualization"]
    end

    %% Layout Engine
    ENGINE["engine.ts<br/>Layout Calc"]

    %% Data Flow
    DS --> EXPLORE
    EXPLORE -->|"config"| SCAN
    UI -->|"input"| VALIDATORS
    VALIDATORS -->|"formats"| SE
    CTRL -->|"commands"| SE
    SE -->|"calls"| DS
    DS -->|"results"| SE
    SE -->|"batches"| CTRL
    CTRL -->|"process"| GS
    GS -->|"graph data"| CHART
    CHART -->|"calc"| ENGINE
    ENGINE -->|"props"| CHART
    CHART -->|"interactions"| CTRL

    %% Dark mode styling
    classDef provider fill:#1e3a8a,stroke:#3b82f6,stroke-width:2px,color:#e5e7eb
    classDef config fill:#7c2d12,stroke:#ea580c,stroke-width:2px,color:#e5e7eb
    classDef controller fill:#14532d,stroke:#22c55e,stroke-width:2px,color:#e5e7eb
    classDef core fill:#581c87,stroke:#a855f7,stroke-width:2px,color:#e5e7eb
    classDef engine fill:#9f1239,stroke:#f43f5e,stroke-width:2px,color:#e5e7eb

    class WHOIS,URLINFO,OTHER,DS provider
    class EXPLORE,VALIDATORS config
    class SCAN,UI,CTRL controller
    class GS,SE,CHART,CORE core
    class ENGINE engine

Component Flow:

1. Data Sources - Define available commands and how to process their results
2. explore.vue - Configures the system for specific use cases (discovery, incident response)
3. validators.ts - Transforms user input into formats that different commands can use
4. scan.vue - Main controller that manages UI and coordinates all components
5. Core components - Handle data storage, command execution, and visualization

The Components

1. Data Sources (like whois.ts, urlinfo.ts)

What it does: Defines what commands you can run and how to process the results.

Each data source file contains:

• A command definition (what to run)
• How to process the response data
• What type of input it needs (domain, email, etc.)

Example:

// whois.ts defines:
cmds['whois'] = {
  type: QueryType.DOMAIN,     // Only works with domains
  query: (query) => `whois(${query})`,  // The actual command
  category: "Tools"           // UI grouping
};

2. explore.vue

What it does: Sets up the explorer for different use cases (like “discovery” or “incident response”).

Functions:

• Loads all available data sources
• Decides which commands to show based on your use case
• Sets up the UI components
• Passes everything to scan.vue to actually run

Example: For incident response, it auto-runs certain commands and shows different navigation options than general discovery.

3. validators.ts

What it does: Detects input types and converts them to different formats for various commands.

Examples:

• Input example.com → Creates domain: "example.com" and url: "https://example.com/"
• Input [email protected] → Creates email: "[email protected]", domain: "example.com", and name: "user"

Purpose: Different commands need different formats. Whois needs just the domain, while URL scanners need the full URL.

4. scan.vue

What it does: Main controller that manages the UI and coordinates all components.

Manages:

• UI elements (buttons, dropdowns, graph display)
• Command execution when you click “Search”
• Graph data storage
• Loading states

Controls three core components:

• graph_source.ts - Stores nodes and connections
• source_executor.ts - Runs commands
• chart.vue - Displays the visual graph

5. graph_source.ts

What it does: Stores graph data and processes new results.

Stores:

• Nodes - Individual data pieces (“google.com”, “192.168.1.1”)
• Edges - Connections between nodes (“google.com connects to 192.168.1.1”)
• Modal Data - Additional details shown when clicking nodes

Node types:

• ROOT - Starting points (initial search terms)
• DATA - Discovered information (IPs, emails)
• GROUP - Collections of related items

6. source_executor.ts

What it does: Executes commands and handles results in batches.

Process:

1. User selects nodes and clicks “Search”
2. Runs appropriate commands for selected nodes
3. Collects results every few seconds
4. Sends result batches back to scan.vue
5. Handles timeouts and errors

Batch processing: Instead of waiting for all commands to complete, results are delivered as they become available.

7. chart.vue

What it does: Displays the interactive graph visualization.

User interactions:

• Click nodes to select them or view details
• Hover over nodes to highlight connections
• Drag nodes to reposition them
• Zoom and pan around the graph

Visual features:

• Color-coded node types
• Node sizes based on connection count
• Loading animations during searches
• Dark/light mode support

8. engine.ts

What it does: Calculates node sizes and positioning for optimal graph layout.

Calculations:

• Node sizes - Larger sizes for highly connected nodes
• Positioning - Node placement to minimize overlap
• Visual properties - Graph organization and appearance

Current implementation: Only calculates node sizes. Positioning is handled by the chart library.

Detailed Process Flows

Application Initialization Flow

sequenceDiagram
    participant DS as Data Sources
    participant EX as explore.vue
    participant SC as scan.vue
    participant GS as graph_source.ts
    participant CH as chart.vue

    DS->>EX: Export command definitions & processors
    EX->>EX: Determine use case (discovery/incident)
    EX->>EX: Build configuration objects
    EX->>SC: Pass complete configuration
    SC->>GS: Initialize GraphSource
    SC->>SC: Initialize SourceExecutor
    SC->>CH: Initialize Chart component
    CH->>CH: Render empty graph

Search and Command Execution Flow

sequenceDiagram
    participant UI as User Interface
    participant VA as validators.ts
    participant SE as source_executor.ts
    participant DS as Data Sources
    participant GS as graph_source.ts
    participant CH as chart.vue

    UI->>VA: User input: "example.com"
    VA->>VA: Detect type: DOMAIN
    VA->>SE: Transformed formats: {domain, url}
    SE->>DS: Execute whois(example.com)
    SE->>DS: Execute urlcheck(https://example.com)
    DS->>SE: Return raw results
    SE->>SE: Batch results every 3-5 seconds
    SE->>GS: Process batch with provider configs
    GS->>GS: Extract nodes & edges from results
    GS->>CH: Update graph with new data
    CH->>CH: Re-render visualization

User Interaction Flow

flowchart TD
    A["User clicks node"] --> B["chart.vue detects click"]
    B --> C{"Node has modal data?"}
    C -->|Yes| D["Show node details in modal"]
    C -->|No| E["Add to selection"]
    E --> F["User selects commands"]
    F --> G["User clicks Search"]
    G --> H["Repeat search flow for selected nodes"]
    D --> I["User can close modal"]
    I --> E

Adding New Features

Want to add a new command (like “ping”)?

1. Create a new file like providers/ping.ts
2. Define what input it needs (IP address, domain, etc.)
3. Write how to process the results
4. Register it in the main providers list

Want to create a new use case?

1. Add it to explore.vue (like “malware-analysis”)
2. Choose which commands should be available
3. Set up which commands should run automatically
4. Create custom components if needed

Want to support a new input type?

1. Add it to validators.ts
2. Write a regex pattern to detect it
3. Define how to transform it for different commands

What Could Go Wrong (Error Handling)

Commands fail or timeout:

• The system tracks failed commands
• Shows you error messages
• Continues with other commands that worked

Invalid input:

• validators.ts checks if your input makes sense
• Warns you if something looks wrong
• Prevents running commands that won’t work

Too much data:

• Results come in batches so you don’t wait forever
• Loading indicators show progress
• You can start exploring while more data loads

Performance Tips

For large graphs:

• Only shows updates when needed
• Calculates node sizes efficiently
• Handles thousands of nodes without freezing

For memory:

• Removes duplicate nodes automatically
• Cleans up when components are destroyed
• Batches operations to avoid overload