Universal Request Analyzer - Development Guide

Table of Contents

  1. Development Setup
  2. Project Structure
  3. Building the Extension
  4. Testing
  5. Code Style & Standards
  6. Contributing
  7. Debugging
  8. Release Process

Development Setup

Prerequisites

Initial Setup

  1. Clone the Repository 
    git clone https://github.com/ModernaCyber/Universal-Request-Analyzer.git
cd Universal-Request-Analyzer
  2. Install Dependencies 
    npm install
  3. Build the Extension 
    npm run build

  4. Load in Browser

    Chrome/Edge:

    • Navigate to chrome://extensions/
    • Enable “Developer mode”
    • Click “Load unpacked”
    • Select the dist folder

    Firefox:

    • Navigate to about:debugging#/runtime/this-firefox
    • Click “Load Temporary Add-on”
    • Select any file in the dist folder

Project Structure

Universal-Request-Analyzer/
├── src/                          # Source code
│   ├── background/               # Background service worker
│   │   ├── database/            # Database layer (medallion + star schema)
│   │   ├── capture/             # Request capture logic
│   │   ├── messaging/           # Message handlers
│   │   └── background.js        # Main entry point
│   │
│   ├── popup/                   # Popup UI
│   │   ├── components/          # UI components
│   │   ├── popup.html           # Main HTML
│   │   └── popup.js             # Main script
│   │
│   ├── options/                 # Options/Dashboard page
│   │   ├── components/          # UI components
│   │   ├── options.html         # Main HTML
│   │   └── options.js           # Main script
│   │
│   ├── devtools/                # DevTools panel
│   │   ├── devtools.html        # Panel HTML
│   │   └── devtools.js          # Panel script
│   │
│   ├── content/                 # Content scripts
│   │   └── content.js           # Injected into pages
│   │
│   ├── lib/                     # Shared library
│   │   ├── core/               # Core classes (DataManager, etc.)
│   │   ├── ui/                 # UI components (BaseComponent, ChartManager)
│   │   ├── managers/           # Feature managers (ExportManager, etc.)
│   │   └── utils/              # Utility functions
│   │
│   ├── assets/                  # Static assets
│   │   ├── icons/              # Extension icons
│   │   ├── images/             # Images
│   │   └── wasm/               # WebAssembly files (sql.js)
│   │
│   └── manifest.json            # Extension manifest
│
├── docs/                        # Documentation
│   ├── ARCHITECTURE.md          # Technical architecture
│   ├── USER_GUIDE.md           # User documentation
│   └── DEVELOPMENT.md          # This file
│
├── webpack.*.js                 # Webpack configurations
├── babel.config.js              # Babel configuration
├── jest.config.js               # Jest test configuration
├── .eslintrc.js                # ESLint configuration
└── package.json                # Dependencies and scripts

Key Directories

Building the Extension

Available Build Commands

# Development build with watch mode
npm run dev

# Production build (optimized and minified)
npm run build

# Clean build artifacts
npm run clean

# Clean and rebuild
npm run clean && npm run build

Build Outputs

All builds output to the dist/ directory:

dist/
├── background.js              # Service worker
├── popup.html & popup.js      # Popup interface
├── options.html & options.js  # Options page
├── devtools.html & devtools.js # DevTools panel
├── content.js                 # Content script
├── assets/                    # Copied static assets
└── manifest.json              # Processed manifest

Development Mode

npm run dev

This starts webpack in watch mode:

Note: After rebuild, you must reload the extension in your browser:

Production Mode

npm run build

Production builds include:

Testing

Automated Testing

# Run all tests
npm test

# Run tests in watch mode
npm run test:watch

# Generate coverage report
npm run test:coverage

Test Structure

Tests are located in src/tests/:

src/tests/
├── unit/              # Unit tests
├── integration/       # Integration tests
└── mocks/            # Mock data and utilities

Writing Tests

// Example test
import { DatabaseManager } from '@/background/database/db-manager-medallion.js';

describe('DatabaseManager', () => {
  let dbManager;
  
  beforeEach(() => {
    dbManager = new DatabaseManager();
  });
  
  test('should initialize database', async () => {
    await dbManager.initialize();
    expect(dbManager.isInitialized()).toBe(true);
  });
});

Manual Testing

Testing the Extension

  1. Build the Extension 
    npm run dev

  2. Load in Browser (see Initial Setup)

  3. Open Developer Tools
    • For popup: Right-click popup → Inspect
    • For options: Right-click options page → Inspect
    • For background: Go to chrome://extensions/ → Click “service worker”
  4. Test Features
    • Navigate to websites and verify request capture
    • Open popup and verify metrics display
    • Open DevTools panel and test filters
    • Open Dashboard and verify charts render

Visual Testing Checklist

See TESTING_GUIDE.md for comprehensive visual testing checklist covering:

Linting

# Check for linting errors
npm run lint

# Auto-fix linting errors
npm run lint:fix

Code Style & Standards

JavaScript Style

We use ESLint with standard configuration. Key rules:

File Organization

// 1. Imports
import { BaseComponent } from '@/lib/ui/BaseComponent.js';
import { formatBytes } from '@/lib/utils/helpers.js';

// 2. Constants
const DEFAULT_REFRESH_INTERVAL = 5000;
const MAX_RETRIES = 3;

// 3. Class definition
class MyComponent extends BaseComponent {
  constructor() {
    super();
    this.state = {};
  }
  
  // Public methods first
  async initialize() {
    // ...
  }
  
  // Private methods last
  _privateMethod() {
    // ...
  }
}

// 4. Export
export default MyComponent;

Comments

/**
 * Processes requests from Bronze to Silver layer
 * @param {Array} requests - Raw request objects
 * @param {Object} options - Processing options
 * @returns {Promise<number>} Number of processed requests
 */
async function processRequests(requests, options) {
  // Complex logic here warrants a comment
  const filtered = requests.filter(r => {
    // Only process requests within retention period
    return Date.now() - r.timestamp < options.retentionMs;
  });
  
  return filtered.length;
}

Database Conventions

Git Commit Messages

Follow conventional commits:

type(scope): subject

body (optional)

footer (optional)

Types:

Examples:

feat(popup): add request type filter

fix(database): resolve SCD Type 2 version conflict

docs(user-guide): update performance metrics section

refactor(lib): extract chart logic to ChartManager

Contributing

Before You Start

  1. Check existing issues for duplicates
  2. Discuss major changes in an issue first
  3. Read CONTRIBUTING.md

Development Workflow

  1. Fork and Clone 
    git clone https://github.com/YOUR_USERNAME/Universal-Request-Analyzer.git
  2. Create Feature Branch 
    git checkout -b feat/my-new-feature
  3. Make Changes
    • Write code
    • Add/update tests
    • Update documentation if needed
  4. Test Your Changes 
    npm run lint
npm test
npm run build
  5. Commit Changes 
    git add .
git commit -m "feat(scope): description"
  6. Push to Fork 
    git push origin feat/my-new-feature
  7. Create Pull Request
    • Go to GitHub
    • Click “New Pull Request”
    • Fill in PR template
    • Request review

Pull Request Guidelines

Debugging

Background Service Worker

// In background.js or any background file
console.log('Debug info:', data);
console.error('Error occurred:', error);

View logs:

Popup/Options/DevTools

Right-click on the interface → Inspect

Database Debugging

Use the Advanced tab in options page:

  1. Open Options → Advanced
  2. Execute SQL queries: 
    -- View all bronze requests
SELECT * FROM bronze_requests LIMIT 10;
   
-- Check layer counts
SELECT 'bronze' as layer, COUNT(*) FROM bronze_requests
UNION ALL
SELECT 'silver', COUNT(*) FROM silver_requests
UNION ALL
SELECT 'gold', COUNT(*) FROM gold_daily_analytics;
   
-- View star schema dimensions
SELECT * FROM dim_domain WHERE is_current = 1;
  3. Use debug tools:
    • Inspect Schema: View all tables and columns
    • Test Connection: Verify database is working
    • Force Processing: Trigger data processing manually

Network Debugging

Monitor extension’s network activity:

  1. Open DevTools in extension page
  2. Go to Network tab
  3. Trigger actions
  4. Verify API calls (if any)

Common Issues

Extension Not Loading

Database Errors

Build Failures

Release Process

Version Numbering

We use Semantic Versioning (semver):

Creating a Release

  1. Update Version 
    # Update version in package.json and manifest.json
npm version patch  # or minor, or major
  2. Update Changelog
    • Document all changes since last release
    • Categorize: Added, Changed, Fixed, Removed
  3. Build Production Version 
    npm run build
  4. Test Thoroughly
    • Load production build in browser
    • Test all major features
    • Verify no console errors
  5. Create Git Tag 
    git tag -a v1.0.1 -m "Release version 1.0.1"
git push origin v1.0.1
  6. Create GitHub Release
    • Go to GitHub → Releases
    • Create new release from tag
    • Upload dist.zip
    • Add changelog to description
  7. Submit to Browser Stores
    • Chrome Web Store: Upload dist.zip
    • Firefox Add-ons: Upload dist.zip
    • Follow store-specific review process

Pre-Release Checklist

Additional Resources

Project Documentation

External Resources

Getting Help

Happy Coding! 🚀