Testing Guide¶

Golden Age Hub maintains a comprehensive, isolated test suite that verifies our cryptographic pipelines, API schema validation, rate-limiting, and ledger operations. This guide explains how to execute tests locally, leverage our Dockerized test runners, and configure test database states to maintain our 100% pass rate.

Local Testing¶

Prerequisites¶

1. Python Environment: Python 3.11+ with uv
2. Database: PostgreSQL/TimescaleDB running in Docker
3. Redis: For message queue testing
4. Services: Gateway and pipeline services running

Quick Start¶

# Install dependencies
make install-dev

# Start all services
make dev

# Run all tests
make test

# Run tests with verbose output
make test-verbose

# Run comprehensive test suite
make test-comprehensive

# Run only ledger tests
make test-ledger

# Run tests with coverage
make test-coverage

Dockerized Test Runners (Recommended for Isolation)¶

For automated or isolated testing, you can use the pre-configured bash scripts located at the repository root. These scripts boot up the required services (Postgres, Redis, Gateway), verify health status, execute tests, and clean up.

1. Quick Runner (run_tests_simple.sh)¶

A simple, robust runner that handles setup, health validation, test execution, and cleanup.

# Run tests with automatic service boot and cleanup
./run_tests_simple.sh

2. Advanced Runner (run_test_suite.sh)¶

A feature-rich script providing detailed logging, coloring, and configurable parameters.

# Basic usage
./run_test_suite.sh

# Clean docker volumes after test execution
./run_test_suite.sh --clean-volumes

# Keep containers running (useful for inspecting database/logs post-test)
./run_test_suite.sh --no-cleanup

# Show advanced options
./run_test_suite.sh --help

Test Environment Setup¶

For local testing with the full stack:

# Start test database and services
make dev

# Check service health
curl http://localhost:8000/health

# Run tests
make test

Test Structure¶

backend/tests/
├── conftest.py          # Test configuration and fixtures
├── test_events.py       # Event ingestion tests
├── test_proof_of_care.py # Merkle tree tests
├── test_security.py     # Security validation tests
├── test_performance.py  # Performance tests
├── test_integration.py  # Integration tests
└── test_api_integration.py # API integration tests

Test Categories¶

Event Ingestion Tests (make test-events)¶

• Schema validation (JSON Schema Draft 7)
• Field validation (UUIDs, timestamps, device IDs)
• Error handling and edge cases
• Rate limiting and replay protection

Proof-of-Care Tests (make test-ledger)¶

• Merkle tree verification endpoints
• Block management and pagination
• Event inclusion proofs
• Ledger status and statistics

Security Tests (make test-security)¶

• Cryptographic signature verification
• Device authentication
• Rate limiting validation
• Audit logging verification

Performance Tests (make test-performance)¶

• Response time benchmarks
• Concurrent load testing
• Throughput validation
• Memory usage stability

Integration Tests (make test-api)¶

• End-to-end pipeline validation
• Cross-component data consistency
• System resilience testing

Running Specific Tests¶

# Run specific test files
uv run pytest tests/test_events.py -v
uv run pytest tests/test_proof_of_care.py -v

# Run with markers
uv run pytest tests/ -m "integration" -v
uv run pytest tests/ -m "security" -v
uv run pytest tests/ -m "performance" -v

# Run with coverage
uv run pytest tests/ --cov=backend --cov-report=html --cov-report=term-missing

# Run in parallel (faster)
uv run pytest tests/ -n auto --tb=short

Test Configuration¶

Tests are configured to run against the live docker compose services:

• API Service: http://localhost:8000 (gateway)
• Database: Direct connection to PostgreSQL container with ga_user/ga_pass/ga_db
• Redis: Live Redis instance for queue testing
• Async Mode: Full async testing with proper fixtures

Environment Variables for Testing¶

# Database configuration
DATABASE_URL=postgresql+asyncpg://ga_user:ga_pass@localhost:5432/ga_db
TEST_DATABASE_URL=postgresql+asyncpg://ga_user:ga_pass@localhost:5432/ga_db

# Redis configuration
REDIS_URL=redis://localhost:6379/0

# Test configuration
ENV=testing
LOG_LEVEL=INFO

CI/CD Testing¶

GitHub Actions Workflow¶

The project uses GitHub Actions for automated testing:

# .github/workflows/test-isolated.yml
- Comprehensive test suite execution
- Security scanning with Bandit
- Code quality checks with ruff
- Type checking with mypy
- Coverage reporting

Test Categories in CI¶

1. Health and Basic Integration Tests
2. Schema Validation Tests
3. Security and Cryptography Tests
4. Performance and Load Tests
5. Integration Tests

Quality Gates¶

All code must pass these quality gates:

• Test Coverage: Minimum 80% required
• Security Tests: No security test failures allowed
• Performance Tests: All benchmarks must pass
• Integration Tests: All components must work together

Test Results¶

Current Status: All Tests Passing¶

• Total Tests: 90
• Passing: 90
• Failing: 0
• Skipped: 0
• Coverage: ≥80%

Expected Test Output¶

🚀 Starting Golden Age Hub Phase 1 Comprehensive Testing Suite
=================================================================
[SUCCESS] API service is accessible
[SUCCESS] Database service is accessible

=================================================================
1. HEALTH AND BASIC INTEGRATION TESTS
=================================================================
[SUCCESS] Health and basic integration tests PASSED

=================================================================
2. SCHEMA VALIDATION TESTS
=================================================================
[SUCCESS] Schema validation tests PASSED

... (all test categories)

🎉 All Phase 1 comprehensive tests completed successfully!
🏆 Golden Age Hub Phase 1 is ready for production deployment!

Writing Tests¶

Test Structure¶

import pytest
from httpx2 import AsyncClient

@pytest.mark.asyncio
async def test_event_ingestion():
    """Test event ingestion with proper validation"""
    async with AsyncClient(base_url="http://localhost:8000") as client:
        response = await client.post("/v1/events/ingest", json=test_event)
        assert response.status_code == 200

Test Fixtures¶

The project uses comprehensive test fixtures in conftest.py:

@pytest.fixture
async def test_client():
    """Create test client for API testing"""
    async with AsyncClient(base_url="http://localhost:8000") as client:
        yield client

@pytest.fixture
async def db_session():
    """Create database session for testing"""
    # Database setup and cleanup

Test Markers¶

Use pytest markers for categorizing tests:

@pytest.mark.integration
async def test_full_pipeline():
    """Integration test for full pipeline"""
    pass

@pytest.mark.security
async def test_signature_verification():
    """Security test for signature verification"""
    pass

@pytest.mark.performance
async def test_load_testing():
    """Performance test for load testing"""
    pass

Troubleshooting Tests¶

Common Issues¶

Services not running:

# Check service status
make health

# Start services
make dev

# Check logs
make logs

Database connection issues:

# Check database connectivity
make shell-postgres

# Reset database
make db-reset

# Run migrations
make db-migrate

Test failures:

# Run with verbose output
make test-verbose

# Run specific test
uv run pytest tests/test_events.py::test_specific_function -v -s

# Check test environment
curl http://localhost:8000/health

Debugging Tests¶

# Run with debugging
uv run pytest tests/test_events.py -v -s --pdb

# Run specific test with debugging
uv run pytest tests/test_events.py::test_event_ingestion -v -s --pdb

# Check test configuration
uv run python -c "from tests.conftest import *; print('Test config loaded')"

Performance Testing¶

Load Testing¶

# Run performance tests
make test-performance

# Run with specific load parameters
uv run pytest tests/test_performance.py -v -k "load_test"

# Run benchmark tests
uv run pytest tests/test_performance.py -v -k "benchmark"

Performance Benchmarks¶

• API Response Time: <200ms for endpoints
• Database Queries: <50ms for core operations
• Throughput: >100 events/second
• Memory Usage: Stable under load

Security Testing¶

Security Validation¶

# Run security tests
make test-security

# Run cryptographic tests
uv run pytest tests/test_security.py -v -k "crypto"

# Run authentication tests
uv run pytest tests/test_security.py -v -k "auth"

Security Test Coverage¶

• Signature Verification: Ed25519 signature validation
• Replay Protection: Sequence ID validation
• Rate Limiting: Token bucket algorithm testing
• Input Validation: Schema validation and SQL injection prevention
• Audit Logging: Comprehensive audit trail testing

This guide covers the comprehensive testing strategy for Golden Age Hub Phase 1. All tests are passing and the system is production-ready. ├── test_ledger.py # Comprehensive ledger tests ├── test_events.py # Event-related tests └── test_health.py # Health check tests

### Test Categories

#### 1. Unit Tests
- **Merkle Tree Operations**: `test_merkle_tree_creation`, `test_merkle_inclusion_proof`
- **Cryptographic Functions**: `test_canonical_hashing`
- **Utility Functions**: Hash generation and validation

#### 2. Integration Tests
- **Database Persistence**: `test_ledger_event_persistence`
- **API Endpoints**: Full request/response cycle testing
- **Merkle Root Progression**: `test_merkle_root_progression`

#### 3. Performance Tests
- **Large Dataset Operations**: `test_large_tree_operations`
- **Scalability Testing**: Multiple events and consistency proofs

#### 4. Error Handling Tests
- **Validation Errors**: `test_append_ledger_event_invalid_data`
- **Edge Cases**: Empty ledger, invalid ranges, malformed data

## CI/CD Pipeline Testing

### Automated Testing

The CI/CD pipeline runs tests automatically on:

- **Push to main/dev branches**
- **Pull requests to main/dev branches**
- **Tag pushes** (for releases)

### Pipeline Stages

#### 1. Backend Tests
- **Environment**: Ubuntu Latest with Python 3.11
- **Database**: TimescaleDB service container
- **Tests**: Full test suite with coverage reporting
- **Coverage**: XML and HTML reports uploaded to Codecov

#### 2. Security Checks
- **Bandit**: Python security linting
- **Safety**: Dependency vulnerability scanning
- **Reports**: JSON artifacts for analysis

#### 3. Performance Tests
- **Trigger**: Only on main branch pushes
- **Focus**: Ledger performance with larger datasets
- **Database**: Dedicated PostgreSQL instance

#### 4. Docker Build
- **Build**: Multi-stage Docker build with caching
- **Test**: Container functionality validation
- **Cache**: GitHub Actions cache for faster builds

### Test Execution in CI

```yaml
# Example from ci.yml
- name: Run backend tests
  run: |
    uv run pytest tests/ -v --tb=short --cov=backend --cov-report=xml --cov-report=term-missing
  env:
    DATABASE_URL: postgresql+asyncpg://test_user:test_password@localhost:5432/goldenage_test

Coverage Reporting¶

Local Coverage¶

# Generate HTML coverage report
make test-coverage

# View report
open backend/htmlcov/index.html

CI Coverage¶

• Codecov Integration: Automatic upload of coverage reports
• Badge: Coverage status displayed in README
• Thresholds: Configurable coverage requirements

Debugging Tests¶

Verbose Output¶

# Detailed test output
pytest tests/ -v -s

# Show test duration
pytest tests/ --durations=10

# Debug mode
pytest tests/ -v --pdb

Database Debugging¶

# Run specific test with database logging
DATABASE_URL="postgresql+asyncpg://test_user:test_password@localhost:5433/goldenage_test" \
  pytest tests/test_ledger.py::TestLedgerIntegration::test_ledger_event_persistence -v -s

Test Data¶

Fixtures¶

Test fixtures are defined in conftest.py:

@pytest.fixture
def sample_ledger_event():
    return {
        "event_type": "check_in",
        "patient_id": "550e8400-e29b-41d4-a716-446655440001",
        "caregiver_id": "550e8400-e29b-41d4-a716-446655440002",
        "event_data": {"note": "Patient check-in"},
        "metadata": {"source": "test"}
    }

Test Database¶

• Isolation: Each test runs in a clean database state
• Cleanup: Automatic rollback after each test
• Performance: Fast test execution with proper indexing

Best Practices¶

Writing Tests¶

1. Async Testing: Use pytest.mark.asyncio for async functions
2. Database Isolation: Each test should be independent
3. Descriptive Names: Clear test method names explaining intent
4. Fixtures: Use fixtures for reusable test data
5. Assertions: Clear and specific assertions

Test Organization¶

class TestFeatureName:
    """Group related tests in classes."""

    def test_specific_functionality(self):
        """Test a specific feature."""

    @pytest.mark.asyncio
    async def test_async_functionality(self):
        """Test async functionality."""

Troubleshooting¶

Common Issues¶

Database Connection Errors¶

# Ensure database is running
docker-compose -f docker-compose.test.yml up -d

# Check database logs
docker-compose -f docker-compose.test.yml logs test-db

Import Errors¶

# Install dependencies
uv sync

# Check Python path
python -c "import sys; print(sys.path)"

Async Test Issues¶

# Ensure pytest-asyncio is installed
uv add --dev pytest-asyncio

# Use proper async markers
@pytest.mark.asyncio
async def test_async_function():
    pass

Performance Issues¶

Slow Tests¶

• Check database query performance
• Use test database with proper indexing
• Avoid unnecessary database operations in tests

Memory Issues¶

• Monitor test resource usage
• Use fixtures for expensive setup
• Clean up resources in test teardown

Continuous Integration¶

Branch Protection¶

The following branches have protection rules: - main: Requires passing tests and approved reviews - dev: Requires passing tests

Pull Request Requirements¶

• ✅ All tests must pass
• ✅ Security checks must pass
• ✅ Code coverage maintained
• ✅ At least one approval required

Release Process¶

1. Feature Development: Develop on feature branches
2. Testing: All tests pass in CI
3. Review: Code review and approval
4. Merge: Merge to main branch
5. Tag: Create release tag (e.g., v1.0.0)
6. Deploy: Automatic deployment pipeline triggers

Monitoring and Alerts¶

CI/CD Status¶

• GitHub Actions: View pipeline status in Actions tab
• Coverage: Codecov reports and badges
• Security: Dependabot alerts for vulnerabilities

Notifications¶

• Slack: CI/CD status notifications (configurable)
• Email: Failure notifications for maintainers
• GitHub: Status checks on pull requests

Contributing¶

When contributing to the project:

1. Run Tests Locally: Ensure all tests pass before submitting PR
2. Add Tests: Include tests for new features
3. Update Documentation: Keep testing guide current
4. Performance: Consider test performance impact

Next Steps¶

Once your tests are passing, review how these verification steps integrate with our release workflow:

• CI/CD Pipeline: Learn about the automated quality gates, linting checks, and security scans run on every PR.
• Deployment Guide: Understand the steps for deploying to staging and production U.S.-sovereign regions.

Resources¶

• pytest Documentation
• pytest-asyncio
• FastAPI Testing
• SQLAlchemy Testing