storycove/DUAL_WRITE_USAGE.md

Dual-Write Search Engine Usage Guide

This guide explains how to use the dual-write functionality during the search engine migration.

🎛️ Configuration Options

Environment Variables

# Search engine selection
SEARCH_ENGINE=typesense          # or 'opensearch'
SEARCH_DUAL_WRITE=false          # or 'true'

# OpenSearch connection (required when using OpenSearch)
OPENSEARCH_PASSWORD=your_password

Migration Flow

Phase 1: Initial Setup

SEARCH_ENGINE=typesense          # Keep using Typesense
SEARCH_DUAL_WRITE=false          # Single-write to Typesense only

Phase 2: Enable Dual-Write

SEARCH_ENGINE=typesense          # Still reading from Typesense
SEARCH_DUAL_WRITE=true           # Now writing to BOTH engines

Phase 3: Switch to OpenSearch

SEARCH_ENGINE=opensearch         # Now reading from OpenSearch
SEARCH_DUAL_WRITE=true           # Still writing to both (safety)

Phase 4: Complete Migration

SEARCH_ENGINE=opensearch         # Reading from OpenSearch
SEARCH_DUAL_WRITE=false          # Only writing to OpenSearch

🔧 Admin API Endpoints

Check Current Status

GET /api/admin/search/status

Response:
{
  "primaryEngine": "typesense",
  "dualWrite": false,
  "typesenseAvailable": true,
  "openSearchAvailable": true
}

Switch Configuration

POST /api/admin/search/configure
Content-Type: application/json

{
  "engine": "opensearch",
  "dualWrite": true
}

Quick Actions

# Enable dual-write
POST /api/admin/search/dual-write/enable

# Disable dual-write
POST /api/admin/search/dual-write/disable

# Switch to OpenSearch
POST /api/admin/search/switch/opensearch

# Switch back to Typesense
POST /api/admin/search/switch/typesense

# Emergency rollback (Typesense only, no dual-write)
POST /api/admin/search/emergency-rollback

🚀 Migration Process Example

Step 1: Verify OpenSearch is Ready

# Check if OpenSearch is available
curl http://localhost:8080/api/admin/search/status

# Should show openSearchAvailable: true

Step 2: Enable Dual-Write

# Enable writing to both engines
curl -X POST http://localhost:8080/api/admin/search/dual-write/enable

# Or via configuration
curl -X POST http://localhost:8080/api/admin/search/configure \
  -H "Content-Type: application/json" \
  -d '{"engine": "typesense", "dualWrite": true}'

Step 3: Populate OpenSearch

At this point, any new/updated stories will be written to both engines. For existing data, you may want to trigger a reindex.

Step 4: Switch to OpenSearch

# Switch primary engine to OpenSearch
curl -X POST http://localhost:8080/api/admin/search/switch/opensearch

# Check it worked
curl http://localhost:8080/api/admin/search/status
# Should show: primaryEngine: "opensearch", dualWrite: true

Step 5: Test & Validate

• Test search functionality
• Verify results match expectations
• Monitor for any errors

Step 6: Complete Migration

# Disable dual-write (OpenSearch only)
curl -X POST http://localhost:8080/api/admin/search/dual-write/disable

🚨 Emergency Procedures

Immediate Rollback

If OpenSearch has issues:

curl -X POST http://localhost:8080/api/admin/search/emergency-rollback

This immediately switches to Typesense-only mode.

Partial Rollback

Switch back to Typesense but keep dual-write:

curl -X POST http://localhost:8080/api/admin/search/switch/typesense

📊 Monitoring

Check Current Configuration

curl http://localhost:8080/api/admin/search/status

Application Logs

Watch for dual-write success/failure messages:

# Successful operations
2025-09-18 08:00:00 DEBUG SearchMigrationManager - Successfully indexed story 123 in OpenSearch
2025-09-18 08:00:00 DEBUG SearchMigrationManager - Successfully indexed story 123 in Typesense

# Failed operations (non-critical in dual-write mode)
2025-09-18 08:00:00 ERROR SearchMigrationManager - Failed to index story 123 in OpenSearch

⚠️ Important Notes

1. Dual-write errors are non-critical - if one engine fails, the other continues
2. Read operations only use primary engine - no dual-read
3. Configuration updates take effect immediately - no restart required
4. Emergency rollback is always available - safe to experiment
5. Both engines must be available for dual-write to work optimally

🔄 Typical Migration Timeline

Step	Duration	Configuration	Purpose
1	-	typesense + dual:false	Current state
2	1 day	typesense + dual:true	Populate OpenSearch
3	1 day	opensearch + dual:true	Test OpenSearch
4	-	opensearch + dual:false	Complete migration

Total migration time: ~2 days of gradual transition

---

Note: This dual-write mechanism is temporary and will be removed once the migration is complete. See CLEANUP_CHECKLIST.md for removal instructions.