188 lines
4.9 KiB
Markdown
188 lines
4.9 KiB
Markdown
# Dual-Write Search Engine Usage Guide
|
|
|
|
This guide explains how to use the dual-write functionality during the search engine migration.
|
|
|
|
## 🎛️ Configuration Options
|
|
|
|
### Environment Variables
|
|
|
|
```bash
|
|
# Search engine selection
|
|
SEARCH_ENGINE=typesense # or 'opensearch'
|
|
SEARCH_DUAL_WRITE=false # or 'true'
|
|
|
|
# OpenSearch connection (required when using OpenSearch)
|
|
OPENSEARCH_PASSWORD=your_password
|
|
```
|
|
|
|
### Migration Flow
|
|
|
|
#### Phase 1: Initial Setup
|
|
```bash
|
|
SEARCH_ENGINE=typesense # Keep using Typesense
|
|
SEARCH_DUAL_WRITE=false # Single-write to Typesense only
|
|
```
|
|
|
|
#### Phase 2: Enable Dual-Write
|
|
```bash
|
|
SEARCH_ENGINE=typesense # Still reading from Typesense
|
|
SEARCH_DUAL_WRITE=true # Now writing to BOTH engines
|
|
```
|
|
|
|
#### Phase 3: Switch to OpenSearch
|
|
```bash
|
|
SEARCH_ENGINE=opensearch # Now reading from OpenSearch
|
|
SEARCH_DUAL_WRITE=true # Still writing to both (safety)
|
|
```
|
|
|
|
#### Phase 4: Complete Migration
|
|
```bash
|
|
SEARCH_ENGINE=opensearch # Reading from OpenSearch
|
|
SEARCH_DUAL_WRITE=false # Only writing to OpenSearch
|
|
```
|
|
|
|
## 🔧 Admin API Endpoints
|
|
|
|
### Check Current Status
|
|
```bash
|
|
GET /api/admin/search/status
|
|
|
|
Response:
|
|
{
|
|
"primaryEngine": "typesense",
|
|
"dualWrite": false,
|
|
"typesenseAvailable": true,
|
|
"openSearchAvailable": true
|
|
}
|
|
```
|
|
|
|
### Switch Configuration
|
|
```bash
|
|
POST /api/admin/search/configure
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"engine": "opensearch",
|
|
"dualWrite": true
|
|
}
|
|
```
|
|
|
|
### Quick Actions
|
|
```bash
|
|
# Enable dual-write
|
|
POST /api/admin/search/dual-write/enable
|
|
|
|
# Disable dual-write
|
|
POST /api/admin/search/dual-write/disable
|
|
|
|
# Switch to OpenSearch
|
|
POST /api/admin/search/switch/opensearch
|
|
|
|
# Switch back to Typesense
|
|
POST /api/admin/search/switch/typesense
|
|
|
|
# Emergency rollback (Typesense only, no dual-write)
|
|
POST /api/admin/search/emergency-rollback
|
|
```
|
|
|
|
## 🚀 Migration Process Example
|
|
|
|
### Step 1: Verify OpenSearch is Ready
|
|
```bash
|
|
# Check if OpenSearch is available
|
|
curl http://localhost:8080/api/admin/search/status
|
|
|
|
# Should show openSearchAvailable: true
|
|
```
|
|
|
|
### Step 2: Enable Dual-Write
|
|
```bash
|
|
# Enable writing to both engines
|
|
curl -X POST http://localhost:8080/api/admin/search/dual-write/enable
|
|
|
|
# Or via configuration
|
|
curl -X POST http://localhost:8080/api/admin/search/configure \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"engine": "typesense", "dualWrite": true}'
|
|
```
|
|
|
|
### Step 3: Populate OpenSearch
|
|
At this point, any new/updated stories will be written to both engines.
|
|
For existing data, you may want to trigger a reindex.
|
|
|
|
### Step 4: Switch to OpenSearch
|
|
```bash
|
|
# Switch primary engine to OpenSearch
|
|
curl -X POST http://localhost:8080/api/admin/search/switch/opensearch
|
|
|
|
# Check it worked
|
|
curl http://localhost:8080/api/admin/search/status
|
|
# Should show: primaryEngine: "opensearch", dualWrite: true
|
|
```
|
|
|
|
### Step 5: Test & Validate
|
|
- Test search functionality
|
|
- Verify results match expectations
|
|
- Monitor for any errors
|
|
|
|
### Step 6: Complete Migration
|
|
```bash
|
|
# Disable dual-write (OpenSearch only)
|
|
curl -X POST http://localhost:8080/api/admin/search/dual-write/disable
|
|
```
|
|
|
|
## 🚨 Emergency Procedures
|
|
|
|
### Immediate Rollback
|
|
If OpenSearch has issues:
|
|
```bash
|
|
curl -X POST http://localhost:8080/api/admin/search/emergency-rollback
|
|
```
|
|
This immediately switches to Typesense-only mode.
|
|
|
|
### Partial Rollback
|
|
Switch back to Typesense but keep dual-write:
|
|
```bash
|
|
curl -X POST http://localhost:8080/api/admin/search/switch/typesense
|
|
```
|
|
|
|
## 📊 Monitoring
|
|
|
|
### Check Current Configuration
|
|
```bash
|
|
curl http://localhost:8080/api/admin/search/status
|
|
```
|
|
|
|
### Application Logs
|
|
Watch for dual-write success/failure messages:
|
|
```bash
|
|
# Successful operations
|
|
2025-09-18 08:00:00 DEBUG SearchMigrationManager - Successfully indexed story 123 in OpenSearch
|
|
2025-09-18 08:00:00 DEBUG SearchMigrationManager - Successfully indexed story 123 in Typesense
|
|
|
|
# Failed operations (non-critical in dual-write mode)
|
|
2025-09-18 08:00:00 ERROR SearchMigrationManager - Failed to index story 123 in OpenSearch
|
|
```
|
|
|
|
## ⚠️ Important Notes
|
|
|
|
1. **Dual-write errors are non-critical** - if one engine fails, the other continues
|
|
2. **Read operations only use primary engine** - no dual-read
|
|
3. **Configuration updates take effect immediately** - no restart required
|
|
4. **Emergency rollback is always available** - safe to experiment
|
|
5. **Both engines must be available** for dual-write to work optimally
|
|
|
|
## 🔄 Typical Migration Timeline
|
|
|
|
| Step | Duration | Configuration | Purpose |
|
|
|------|----------|---------------|---------|
|
|
| 1 | - | typesense + dual:false | Current state |
|
|
| 2 | 1 day | typesense + dual:true | Populate OpenSearch |
|
|
| 3 | 1 day | opensearch + dual:true | Test OpenSearch |
|
|
| 4 | - | opensearch + dual:false | Complete migration |
|
|
|
|
**Total migration time: ~2 days of gradual transition**
|
|
|
|
---
|
|
|
|
**Note:** This dual-write mechanism is temporary and will be removed once the migration is complete. See `CLEANUP_CHECKLIST.md` for removal instructions. |