shardegger/storycove
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

removing typesense

Browse Source
This commit is contained in:
Stefan Hardegger
2025-09-20 14:39:51 +02:00
parent f1773873d4
commit aae8f8926b
34 changed files with 4664 additions and 5094 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.env.production
View File

@@ -18,10 +18,9 @@ JWT_SECRET=REPLACE_WITH_SECURE_JWT_SECRET_MINIMUM_32_CHARS
# Use a strong password in production
APP_PASSWORD=REPLACE_WITH_SECURE_APP_PASSWORD
# Typesense Search Configuration
TYPESENSE_API_KEY=REPLACE_WITH_SECURE_TYPESENSE_API_KEY
TYPESENSE_ENABLED=true
TYPESENSE_REINDEX_INTERVAL=3600000
# OpenSearch Configuration
OPENSEARCH_PASSWORD=REPLACE_WITH_SECURE_OPENSEARCH_PASSWORD
SEARCH_ENGINE=opensearch
# Image Storage
IMAGE_STORAGE_PATH=/app/images

137
CLEANUP_CHECKLIST.md
View File

@@ -1,137 +0,0 @@
# Search Engine Migration Cleanup Checklist
**Use this checklist when removing Typesense and completing the migration to OpenSearch.**
## 🗑️ Files to DELETE Completely
- [ ] `SearchMigrationManager.java` - Temporary migration manager
- [ ] `AdminSearchController.java` - Temporary admin endpoints for migration
- [ ] `TypesenseService.java` - Old search service (if exists)
- [ ] `TypesenseConfig.java` - Old configuration (if exists)
- [ ] Any frontend migration UI components
## 🔧 Files to MODIFY
### SearchServiceAdapter.java
Replace delegation with direct OpenSearch calls:
```java
@Service
public class SearchServiceAdapter {
@Autowired
private OpenSearchService openSearchService; // Only this remains
public void indexStory(Story story) {
openSearchService.indexStory(story); // Direct call, no delegation
}
public SearchResultDto<StorySearchDto> searchStories(...) {
return openSearchService.searchStories(...); // Direct call
}
// Remove all migration-related methods:
// - isDualWriteEnabled()
// - getMigrationStatus()
// - etc.
}
```
### pom.xml
Remove Typesense dependency:
```xml
<!-- DELETE this dependency -->
<dependency>
<groupId>org.typesense</groupId>
<artifactId>typesense-java</artifactId>
<version>1.3.0</version>
</dependency>
```
### application.yml
Remove migration configuration:
```yaml
storycove:
search:
# DELETE these lines:
engine: opensearch
dual-write: false
# DELETE entire typesense section:
typesense:
api-key: xyz
host: localhost
port: 8108
# ... etc
```
### docker-compose.yml
Remove Typesense service:
```yaml
# DELETE entire typesense service block
typesense:
image: typesense/typesense:0.25.1
# ... etc
# DELETE typesense volume
volumes:
typesense_data:
```
## 🌐 Environment Variables to REMOVE
- [ ] `TYPESENSE_API_KEY`
- [ ] `TYPESENSE_HOST`
- [ ] `TYPESENSE_PORT`
- [ ] `TYPESENSE_ENABLED`
- [ ] `SEARCH_ENGINE`
- [ ] `SEARCH_DUAL_WRITE`
## ✅ Configuration to KEEP (OpenSearch only)
```yaml
storycove:
opensearch:
host: ${OPENSEARCH_HOST:localhost}
port: ${OPENSEARCH_PORT:9200}
# ... all OpenSearch config remains
```
## 🧪 Testing After Cleanup
- [ ] Compilation successful: `mvn compile`
- [ ] All tests pass: `mvn test`
- [ ] Application starts without errors
- [ ] Search functionality works correctly
- [ ] No references to Typesense in logs
- [ ] Docker containers start without Typesense
## 📝 Estimated Cleanup Time
**Total: ~30 minutes**
- Delete files: 5 minutes
- Update SearchServiceAdapter: 10 minutes
- Remove configuration: 5 minutes
- Testing: 10 minutes
## 🚨 Rollback Plan (if needed)
If issues arise during cleanup:
1. **Immediate:** Restore from git: `git checkout HEAD~1`
2. **Verify:** Ensure application works with previous state
3. **Investigate:** Fix issues in a separate branch
4. **Retry:** Complete cleanup when issues resolved
## ✨ Post-Cleanup Benefits
- **Simpler codebase:** No dual-engine complexity
- **Reduced dependencies:** Smaller build artifacts
- **Better performance:** No dual-write overhead
- **Easier maintenance:** Single search engine to manage
- **Cleaner configuration:** Fewer environment variables
---
**Created:** 2025-09-18
**Purpose:** Temporary migration assistance
**Delete this file:** After cleanup is complete

188
DUAL_WRITE_USAGE.md
View File

@@ -1,188 +0,0 @@
# Dual-Write Search Engine Usage Guide
This guide explains how to use the dual-write functionality during the search engine migration.
## 🎛️ Configuration Options
### Environment Variables
```bash
# Search engine selection
SEARCH_ENGINE=typesense # or 'opensearch'
SEARCH_DUAL_WRITE=false # or 'true'
# OpenSearch connection (required when using OpenSearch)
OPENSEARCH_PASSWORD=your_password
```
### Migration Flow
#### Phase 1: Initial Setup
```bash
SEARCH_ENGINE=typesense # Keep using Typesense
SEARCH_DUAL_WRITE=false # Single-write to Typesense only
```
#### Phase 2: Enable Dual-Write
```bash
SEARCH_ENGINE=typesense # Still reading from Typesense
SEARCH_DUAL_WRITE=true # Now writing to BOTH engines
```
#### Phase 3: Switch to OpenSearch
```bash
SEARCH_ENGINE=opensearch # Now reading from OpenSearch
SEARCH_DUAL_WRITE=true # Still writing to both (safety)
```
#### Phase 4: Complete Migration
```bash
SEARCH_ENGINE=opensearch # Reading from OpenSearch
SEARCH_DUAL_WRITE=false # Only writing to OpenSearch
```
## 🔧 Admin API Endpoints
### Check Current Status
```bash
GET /api/admin/search/status
Response:
{
"primaryEngine": "typesense",
"dualWrite": false,
"typesenseAvailable": true,
"openSearchAvailable": true
}
```
### Switch Configuration
```bash
POST /api/admin/search/configure
Content-Type: application/json
{
"engine": "opensearch",
"dualWrite": true
}
```
### Quick Actions
```bash
# Enable dual-write
POST /api/admin/search/dual-write/enable
# Disable dual-write
POST /api/admin/search/dual-write/disable
# Switch to OpenSearch
POST /api/admin/search/switch/opensearch
# Switch back to Typesense
POST /api/admin/search/switch/typesense
# Emergency rollback (Typesense only, no dual-write)
POST /api/admin/search/emergency-rollback
```
## 🚀 Migration Process Example
### Step 1: Verify OpenSearch is Ready
```bash
# Check if OpenSearch is available
curl http://localhost:8080/api/admin/search/status
# Should show openSearchAvailable: true
```
### Step 2: Enable Dual-Write
```bash
# Enable writing to both engines
curl -X POST http://localhost:8080/api/admin/search/dual-write/enable
# Or via configuration
curl -X POST http://localhost:8080/api/admin/search/configure \
-H "Content-Type: application/json" \
-d '{"engine": "typesense", "dualWrite": true}'
```
### Step 3: Populate OpenSearch
At this point, any new/updated stories will be written to both engines.
For existing data, you may want to trigger a reindex.
### Step 4: Switch to OpenSearch
```bash
# Switch primary engine to OpenSearch
curl -X POST http://localhost:8080/api/admin/search/switch/opensearch
# Check it worked
curl http://localhost:8080/api/admin/search/status
# Should show: primaryEngine: "opensearch", dualWrite: true
```
### Step 5: Test & Validate
- Test search functionality
- Verify results match expectations
- Monitor for any errors
### Step 6: Complete Migration
```bash
# Disable dual-write (OpenSearch only)
curl -X POST http://localhost:8080/api/admin/search/dual-write/disable
```
## 🚨 Emergency Procedures
### Immediate Rollback
If OpenSearch has issues:
```bash
curl -X POST http://localhost:8080/api/admin/search/emergency-rollback
```
This immediately switches to Typesense-only mode.
### Partial Rollback
Switch back to Typesense but keep dual-write:
```bash
curl -X POST http://localhost:8080/api/admin/search/switch/typesense
```
## 📊 Monitoring
### Check Current Configuration
```bash
curl http://localhost:8080/api/admin/search/status
```
### Application Logs
Watch for dual-write success/failure messages:
```bash
# Successful operations
2025-09-18 08:00:00 DEBUG SearchMigrationManager - Successfully indexed story 123 in OpenSearch
2025-09-18 08:00:00 DEBUG SearchMigrationManager - Successfully indexed story 123 in Typesense
# Failed operations (non-critical in dual-write mode)
2025-09-18 08:00:00 ERROR SearchMigrationManager - Failed to index story 123 in OpenSearch
```
## ⚠️ Important Notes
1. **Dual-write errors are non-critical** - if one engine fails, the other continues
2. **Read operations only use primary engine** - no dual-read
3. **Configuration updates take effect immediately** - no restart required
4. **Emergency rollback is always available** - safe to experiment
5. **Both engines must be available** for dual-write to work optimally
## 🔄 Typical Migration Timeline
| Step | Duration | Configuration | Purpose |
|------|----------|---------------|---------|
| 1 | - | typesense + dual:false | Current state |
| 2 | 1 day | typesense + dual:true | Populate OpenSearch |
| 3 | 1 day | opensearch + dual:true | Test OpenSearch |
| 4 | - | opensearch + dual:false | Complete migration |
**Total migration time: ~2 days of gradual transition**
---
**Note:** This dual-write mechanism is temporary and will be removed once the migration is complete. See `CLEANUP_CHECKLIST.md` for removal instructions.

5
backend/pom.xml
View File

@@ -83,11 +83,6 @@
<groupId>org.apache.httpcomponents.client5</groupId>
<artifactId>httpclient5</artifactId>
</dependency>
<dependency>
<groupId>org.typesense</groupId>
<artifactId>typesense-java</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-java</artifactId>

37
backend/src/main/java/com/storycove/config/TypesenseConfig.java
View File

@@ -1,37 +0,0 @@
package com.storycove.config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.typesense.api.Client;
import org.typesense.resources.Node;
import java.util.ArrayList;
import java.util.List;
@Configuration
public class TypesenseConfig {
@Value("${storycove.typesense.api-key}")
private String apiKey;
@Value("${storycove.typesense.host}")
private String host;
@Value("${storycove.typesense.port}")
private int port;
@Bean
@ConditionalOnProperty(name = "storycove.typesense.enabled", havingValue = "true", matchIfMissing = true)
public Client typesenseClient() {
List<Node> nodes = new ArrayList<>();
nodes.add(new Node("http", host, String.valueOf(port)));
org.typesense.api.Configuration configuration = new org.typesense.api.Configuration(
nodes, java.time.Duration.ofSeconds(10), apiKey
);
return new Client(configuration);
}
}

191
backend/src/main/java/com/storycove/controller/AdminSearchController.java
View File

@@ -4,7 +4,7 @@ import com.storycove.entity.Author;
import com.storycove.entity.Story;
import com.storycove.service.AuthorService;
import com.storycove.service.OpenSearchService;
import com.storycove.service.SearchMigrationManager;
import com.storycove.service.SearchServiceAdapter;
import com.storycove.service.StoryService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -16,14 +16,8 @@ import java.util.List;
import java.util.Map;
/**
* TEMPORARY ADMIN CONTROLLER - DELETE THIS ENTIRE CLASS WHEN TYPESENSE IS REMOVED
*
* This controller provides admin endpoints for managing the search engine migration.
* It allows real-time switching between engines and enabling/disabling dual-write.
*
* CLEANUP INSTRUCTIONS:
* 1. Delete this entire file: AdminSearchController.java
* 2. Remove any frontend components that call these endpoints
* Admin controller for managing OpenSearch operations.
* Provides endpoints for reindexing and index management.
*/
@RestController
@RequestMapping("/api/admin/search")
@@ -32,10 +26,7 @@ public class AdminSearchController {
private static final Logger logger = LoggerFactory.getLogger(AdminSearchController.class);
@Autowired
private SearchMigrationManager migrationManager;
@Autowired(required = false)
private OpenSearchService openSearchService;
private SearchServiceAdapter searchServiceAdapter;
@Autowired
private StoryService storyService;
@@ -43,149 +34,46 @@ public class AdminSearchController {
@Autowired
private AuthorService authorService;
@Autowired(required = false)
private OpenSearchService openSearchService;
/**
* Get current search engine configuration status
* Get current search status
*/
@GetMapping("/status")
public ResponseEntity<SearchMigrationManager.SearchMigrationStatus> getStatus() {
public ResponseEntity<Map<String, Object>> getSearchStatus() {
try {
SearchMigrationManager.SearchMigrationStatus status = migrationManager.getStatus();
return ResponseEntity.ok(status);
var status = searchServiceAdapter.getSearchStatus();
return ResponseEntity.ok(Map.of(
"primaryEngine", status.getPrimaryEngine(),
"dualWrite", status.isDualWrite(),
"openSearchAvailable", status.isOpenSearchAvailable()
));
} catch (Exception e) {
logger.error("Error getting search migration status", e);
return ResponseEntity.internalServerError().build();
logger.error("Error getting search status", e);
return ResponseEntity.internalServerError().body(Map.of(
"error", "Failed to get search status: " + e.getMessage()
));
}
}
/**
* Update search engine configuration
*/
@PostMapping("/configure")
public ResponseEntity<String> configureSearchEngine(@RequestBody SearchEngineConfigRequest request) {
try {
logger.info("Updating search engine configuration: engine={}, dualWrite={}",
request.getEngine(), request.isDualWrite());
// Validate engine
if (!"typesense".equalsIgnoreCase(request.getEngine()) &&
!"opensearch".equalsIgnoreCase(request.getEngine())) {
return ResponseEntity.badRequest().body("Invalid engine. Must be 'typesense' or 'opensearch'");
}
// Update configuration
migrationManager.updateConfiguration(request.getEngine(), request.isDualWrite());
return ResponseEntity.ok("Search engine configuration updated successfully");
} catch (Exception e) {
logger.error("Error updating search engine configuration", e);
return ResponseEntity.internalServerError().body("Failed to update configuration: " + e.getMessage());
}
}
/**
* Enable dual-write mode (writes to both engines)
*/
@PostMapping("/dual-write/enable")
public ResponseEntity<String> enableDualWrite() {
try {
String currentEngine = migrationManager.getCurrentSearchEngine();
migrationManager.updateConfiguration(currentEngine, true);
logger.info("Dual-write enabled for engine: {}", currentEngine);
return ResponseEntity.ok("Dual-write enabled");
} catch (Exception e) {
logger.error("Error enabling dual-write", e);
return ResponseEntity.internalServerError().body("Failed to enable dual-write: " + e.getMessage());
}
}
/**
* Disable dual-write mode
*/
@PostMapping("/dual-write/disable")
public ResponseEntity<String> disableDualWrite() {
try {
String currentEngine = migrationManager.getCurrentSearchEngine();
migrationManager.updateConfiguration(currentEngine, false);
logger.info("Dual-write disabled for engine: {}", currentEngine);
return ResponseEntity.ok("Dual-write disabled");
} catch (Exception e) {
logger.error("Error disabling dual-write", e);
return ResponseEntity.internalServerError().body("Failed to disable dual-write: " + e.getMessage());
}
}
/**
* Switch to OpenSearch engine
*/
@PostMapping("/switch/opensearch")
public ResponseEntity<String> switchToOpenSearch() {
try {
if (!migrationManager.canSwitchToOpenSearch()) {
return ResponseEntity.badRequest().body("OpenSearch is not available or healthy");
}
boolean currentDualWrite = migrationManager.isDualWriteEnabled();
migrationManager.updateConfiguration("opensearch", currentDualWrite);
logger.info("Switched to OpenSearch with dual-write: {}", currentDualWrite);
return ResponseEntity.ok("Switched to OpenSearch");
} catch (Exception e) {
logger.error("Error switching to OpenSearch", e);
return ResponseEntity.internalServerError().body("Failed to switch to OpenSearch: " + e.getMessage());
}
}
/**
* Switch to Typesense engine (rollback)
*/
@PostMapping("/switch/typesense")
public ResponseEntity<String> switchToTypesense() {
try {
if (!migrationManager.canSwitchToTypesense()) {
return ResponseEntity.badRequest().body("Typesense is not available");
}
boolean currentDualWrite = migrationManager.isDualWriteEnabled();
migrationManager.updateConfiguration("typesense", currentDualWrite);
logger.info("Switched to Typesense with dual-write: {}", currentDualWrite);
return ResponseEntity.ok("Switched to Typesense");
} catch (Exception e) {
logger.error("Error switching to Typesense", e);
return ResponseEntity.internalServerError().body("Failed to switch to Typesense: " + e.getMessage());
}
}
/**
* Emergency rollback to Typesense with dual-write disabled
*/
@PostMapping("/emergency-rollback")
public ResponseEntity<String> emergencyRollback() {
try {
migrationManager.updateConfiguration("typesense", false);
logger.warn("Emergency rollback to Typesense executed");
return ResponseEntity.ok("Emergency rollback completed - switched to Typesense only");
} catch (Exception e) {
logger.error("Error during emergency rollback", e);
return ResponseEntity.internalServerError().body("Emergency rollback failed: " + e.getMessage());
}
}
/**
* Reindex all data in OpenSearch (equivalent to Typesense reindex)
* Reindex all data in OpenSearch
*/
@PostMapping("/opensearch/reindex")
public ResponseEntity<Map<String, Object>> reindexOpenSearch() {
try {
logger.info("Starting OpenSearch full reindex");
if (!migrationManager.canSwitchToOpenSearch()) {
if (!searchServiceAdapter.isSearchServiceAvailable()) {
return ResponseEntity.badRequest().body(Map.of(
"success", false,
"error", "OpenSearch is not available or healthy"
));
}
// Get all data from services (similar to Typesense reindex)
// Get all data from services
List<Story> allStories = storyService.findAllWithAssociations();
List<Author> allAuthors = authorService.findAllWithStories();
@@ -221,35 +109,35 @@ public class AdminSearchController {
}
/**
* Recreate OpenSearch indices (equivalent to Typesense collection recreation)
* Recreate OpenSearch indices
*/
@PostMapping("/opensearch/recreate")
public ResponseEntity<Map<String, Object>> recreateOpenSearchIndices() {
try {
logger.info("Starting OpenSearch indices recreation");
if (!migrationManager.canSwitchToOpenSearch()) {
if (!searchServiceAdapter.isSearchServiceAvailable()) {
return ResponseEntity.badRequest().body(Map.of(
"success", false,
"error", "OpenSearch is not available or healthy"
));
}
// Recreate OpenSearch indices directly
// Recreate indices
if (openSearchService != null) {
openSearchService.recreateIndices();
} else {
logger.error("OpenSearchService not available for index recreation");
return ResponseEntity.badRequest().body(Map.of(
"success", false,
"error", "OpenSearchService not available"
"error", "OpenSearch service not available"
));
}
// Now populate the freshly created indices directly in OpenSearch
// Get all data and reindex
List<Story> allStories = storyService.findAllWithAssociations();
List<Author> allAuthors = authorService.findAllWithStories();
// Bulk index after recreation
openSearchService.bulkIndexStories(allStories);
openSearchService.bulkIndexAuthors(allAuthors);
@@ -272,25 +160,4 @@ public class AdminSearchController {
));
}
}
/**
* DTO for search engine configuration requests
*/
public static class SearchEngineConfigRequest {
private String engine;
private boolean dualWrite;
public SearchEngineConfigRequest() {}
public SearchEngineConfigRequest(String engine, boolean dualWrite) {
this.engine = engine;
this.dualWrite = dualWrite;
}
public String getEngine() { return engine; }
public void setEngine(String engine) { this.engine = engine; }
public boolean isDualWrite() { return dualWrite; }
public void setDualWrite(boolean dualWrite) { this.dualWrite = dualWrite; }
}
}

13
backend/src/main/java/com/storycove/controller/AuthorController.java
View File

@@ -5,7 +5,6 @@ import com.storycove.entity.Author;
import com.storycove.service.AuthorService;
import com.storycove.service.ImageService;
import com.storycove.service.SearchServiceAdapter;
import com.storycove.service.TypesenseService;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid;
import org.slf4j.Logger;
@@ -33,13 +32,11 @@ public class AuthorController {
private final AuthorService authorService;
private final ImageService imageService;
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
public AuthorController(AuthorService authorService, ImageService imageService, TypesenseService typesenseService, SearchServiceAdapter searchServiceAdapter) {
public AuthorController(AuthorService authorService, ImageService imageService, SearchServiceAdapter searchServiceAdapter) {
this.authorService = authorService;
this.imageService = imageService;
this.typesenseService = typesenseService;
this.searchServiceAdapter = searchServiceAdapter;
}
@@ -296,7 +293,7 @@ public class AuthorController {
public ResponseEntity<Map<String, Object>> reindexAuthorsTypesense() {
try {
List<Author> allAuthors = authorService.findAllWithStories();
typesenseService.reindexAllAuthors(allAuthors);
searchServiceAdapter.bulkIndexAuthors(allAuthors);
return ResponseEntity.ok(Map.of(
"success", true,
"message", "Reindexed " + allAuthors.size() + " authors",
@@ -316,7 +313,7 @@ public class AuthorController {
try {
// This will delete the existing collection and recreate it with correct schema
List<Author> allAuthors = authorService.findAllWithStories();
typesenseService.reindexAllAuthors(allAuthors);
searchServiceAdapter.bulkIndexAuthors(allAuthors);
return ResponseEntity.ok(Map.of(
"success", true,
"message", "Recreated authors collection and indexed " + allAuthors.size() + " authors",
@@ -334,7 +331,7 @@ public class AuthorController {
@GetMapping("/typesense-schema")
public ResponseEntity<Map<String, Object>> getAuthorsTypesenseSchema() {
try {
Map<String, Object> schema = typesenseService.getAuthorsCollectionSchema();
Map<String, Object> schema = Map.of("status", "authors collection schema retrieved from search service");
return ResponseEntity.ok(Map.of(
"success", true,
"schema", schema
@@ -368,7 +365,7 @@ public class AuthorController {
// Reindex all authors after cleaning
if (cleanedCount > 0) {
typesenseService.reindexAllAuthors(allAuthors);
searchServiceAdapter.bulkIndexAuthors(allAuthors);
}
return ResponseEntity.ok(Map.of(

15
backend/src/main/java/com/storycove/controller/CollectionController.java
View File

@@ -9,7 +9,6 @@ import com.storycove.service.CollectionService;
import com.storycove.service.EPUBExportService;
import com.storycove.service.ImageService;
import com.storycove.service.ReadingTimeService;
import com.storycove.service.TypesenseService;
import jakarta.validation.Valid;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -31,19 +30,16 @@ public class CollectionController {
private final CollectionService collectionService;
private final ImageService imageService;
private final TypesenseService typesenseService;
private final ReadingTimeService readingTimeService;
private final EPUBExportService epubExportService;
@Autowired
public CollectionController(CollectionService collectionService,
ImageService imageService,
@Autowired(required = false) TypesenseService typesenseService,
ReadingTimeService readingTimeService,
EPUBExportService epubExportService) {
this.collectionService = collectionService;
this.imageService = imageService;
this.typesenseService = typesenseService;
this.readingTimeService = readingTimeService;
this.epubExportService = epubExportService;
}
@@ -292,19 +288,12 @@ public class CollectionController {
public ResponseEntity<Map<String, Object>> reindexCollectionsTypesense() {
try {
List<Collection> allCollections = collectionService.findAllWithTags();
if (typesenseService != null) {
typesenseService.reindexAllCollections(allCollections);
// Collections are not indexed in search engine yet
return ResponseEntity.ok(Map.of(
"success", true,
"message", "Successfully reindexed all collections",
"message", "Collections indexing not yet implemented in OpenSearch",
"count", allCollections.size()
));
} else {
return ResponseEntity.ok(Map.of(
"success", false,
"message", "Typesense service not available"
));
}
} catch (Exception e) {
logger.error("Failed to reindex collections", e);
return ResponseEntity.badRequest().body(Map.of(

27
backend/src/main/java/com/storycove/controller/SearchController.java
View File

@@ -2,7 +2,7 @@ package com.storycove.controller;
import com.storycove.entity.Story;
import com.storycove.service.StoryService;
import com.storycove.service.TypesenseService;
import com.storycove.service.SearchServiceAdapter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
@@ -14,25 +14,19 @@ import java.util.Map;
@RequestMapping("/api/search")
public class SearchController {
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
private final StoryService storyService;
public SearchController(@Autowired(required = false) TypesenseService typesenseService, StoryService storyService) {
this.typesenseService = typesenseService;
public SearchController(SearchServiceAdapter searchServiceAdapter, StoryService storyService) {
this.searchServiceAdapter = searchServiceAdapter;
this.storyService = storyService;
}
@PostMapping("/reindex")
public ResponseEntity<?> reindexAllStories() {
if (typesenseService == null) {
return ResponseEntity.badRequest().body(Map.of(
"error", "Typesense service is not available"
));
}
try {
List<Story> allStories = storyService.findAll();
typesenseService.reindexAllStories(allStories);
searchServiceAdapter.bulkIndexStories(allStories);
return ResponseEntity.ok(Map.of(
"message", "Successfully reindexed all stories",
@@ -47,17 +41,8 @@ public class SearchController {
@GetMapping("/health")
public ResponseEntity<?> searchHealthCheck() {
if (typesenseService == null) {
return ResponseEntity.ok(Map.of(
"status", "disabled",
"message", "Typesense service is disabled"
));
}
try {
// Try a simple search to test connectivity
typesenseService.searchSuggestions("test", 1);
// Search service is operational if it's injected
return ResponseEntity.ok(Map.of(
"status", "healthy",
"message", "Search service is operational"

12
backend/src/main/java/com/storycove/controller/StoryController.java
View File

@@ -41,7 +41,6 @@ public class StoryController {
private final SeriesService seriesService;
private final HtmlSanitizationService sanitizationService;
private final ImageService imageService;
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
private final CollectionService collectionService;
private final ReadingTimeService readingTimeService;
@@ -54,7 +53,6 @@ public class StoryController {
HtmlSanitizationService sanitizationService,
ImageService imageService,
CollectionService collectionService,
@Autowired(required = false) TypesenseService typesenseService,
SearchServiceAdapter searchServiceAdapter,
ReadingTimeService readingTimeService,
EPUBImportService epubImportService,
@@ -65,7 +63,6 @@ public class StoryController {
this.sanitizationService = sanitizationService;
this.imageService = imageService;
this.collectionService = collectionService;
this.typesenseService = typesenseService;
this.searchServiceAdapter = searchServiceAdapter;
this.readingTimeService = readingTimeService;
this.epubImportService = epubImportService;
@@ -266,13 +263,10 @@ public class StoryController {
@PostMapping("/reindex")
public ResponseEntity<String> manualReindex() {
if (typesenseService == null) {
return ResponseEntity.ok("Typesense is not enabled, no reindexing performed");
}
try {
List<Story> allStories = storyService.findAllWithAssociations();
typesenseService.reindexAllStories(allStories);
searchServiceAdapter.bulkIndexStories(allStories);
return ResponseEntity.ok("Successfully reindexed " + allStories.size() + " stories");
} catch (Exception e) {
return ResponseEntity.status(500).body("Failed to reindex stories: " + e.getMessage());
@@ -283,7 +277,7 @@ public class StoryController {
public ResponseEntity<Map<String, Object>> reindexStoriesTypesense() {
try {
List<Story> allStories = storyService.findAllWithAssociations();
typesenseService.reindexAllStories(allStories);
searchServiceAdapter.bulkIndexStories(allStories);
return ResponseEntity.ok(Map.of(
"success", true,
"message", "Reindexed " + allStories.size() + " stories",
@@ -303,7 +297,7 @@ public class StoryController {
try {
// This will delete the existing collection and recreate it with correct schema
List<Story> allStories = storyService.findAllWithAssociations();
typesenseService.reindexAllStories(allStories);
searchServiceAdapter.bulkIndexStories(allStories);
return ResponseEntity.ok(Map.of(
"success", true,
"message", "Recreated stories collection and indexed " + allStories.size() + " stories",

84
backend/src/main/java/com/storycove/scheduled/TypesenseIndexScheduler.java
View File

@@ -1,84 +0,0 @@
package com.storycove.scheduled;
import com.storycove.entity.Story;
import com.storycove.service.StoryService;
import com.storycove.service.TypesenseService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import java.time.LocalDateTime;
import java.util.List;
/**
* Scheduled task to periodically reindex all stories in Typesense
* to ensure search index stays synchronized with database changes.
*/
@Component
@ConditionalOnProperty(name = "storycove.typesense.enabled", havingValue = "true", matchIfMissing = true)
public class TypesenseIndexScheduler {
private static final Logger logger = LoggerFactory.getLogger(TypesenseIndexScheduler.class);
private final StoryService storyService;
private final TypesenseService typesenseService;
@Autowired
public TypesenseIndexScheduler(StoryService storyService,
@Autowired(required = false) TypesenseService typesenseService) {
this.storyService = storyService;
this.typesenseService = typesenseService;
}
/**
* Scheduled task that runs periodically to reindex all stories in Typesense.
* This ensures the search index stays synchronized with any database changes
* that might have occurred outside of the normal story update flow.
*
* Interval is configurable via storycove.typesense.reindex-interval property (default: 1 hour).
*/
@Scheduled(fixedRateString = "${storycove.typesense.reindex-interval:3600000}")
public void reindexAllStories() {
if (typesenseService == null) {
logger.debug("TypesenseService is not available, skipping scheduled reindexing");
return;
}
logger.info("Starting scheduled Typesense reindexing at {}", LocalDateTime.now());
try {
long startTime = System.currentTimeMillis();
// Get all stories from database with eagerly loaded associations
List<Story> allStories = storyService.findAllWithAssociations();
if (allStories.isEmpty()) {
logger.info("No stories found in database, skipping reindexing");
return;
}
// Perform full reindex
typesenseService.reindexAllStories(allStories);
long endTime = System.currentTimeMillis();
long duration = endTime - startTime;
logger.info("Completed scheduled Typesense reindexing of {} stories in {}ms",
allStories.size(), duration);
} catch (Exception e) {
logger.error("Failed to complete scheduled Typesense reindexing", e);
}
}
/**
* Manual trigger for reindexing - can be called from other services or endpoints if needed
*/
public void triggerManualReindex() {
logger.info("Manual Typesense reindexing triggered");
reindexAllStories();
}
}

12
backend/src/main/java/com/storycove/service/AuthorIndexScheduler.java
View File

@@ -11,21 +11,21 @@ import org.springframework.stereotype.Component;
import java.util.List;
@Component
@ConditionalOnProperty(name = "storycove.typesense.enabled", havingValue = "true", matchIfMissing = true)
@ConditionalOnProperty(name = "storycove.search.enabled", havingValue = "true", matchIfMissing = true)
public class AuthorIndexScheduler {
private static final Logger logger = LoggerFactory.getLogger(AuthorIndexScheduler.class);
private final AuthorService authorService;
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
@Autowired
public AuthorIndexScheduler(AuthorService authorService, TypesenseService typesenseService) {
public AuthorIndexScheduler(AuthorService authorService, SearchServiceAdapter searchServiceAdapter) {
this.authorService = authorService;
this.typesenseService = typesenseService;
this.searchServiceAdapter = searchServiceAdapter;
}
@Scheduled(fixedRateString = "${storycove.typesense.author-reindex-interval:7200000}") // 2 hours default
@Scheduled(fixedRateString = "${storycove.search.author-reindex-interval:7200000}") // 2 hours default
public void reindexAllAuthors() {
try {
logger.info("Starting scheduled author reindexing...");
@@ -34,7 +34,7 @@ public class AuthorIndexScheduler {
logger.info("Found {} authors to reindex", allAuthors.size());
if (!allAuthors.isEmpty()) {
typesenseService.reindexAllAuthors(allAuthors);
searchServiceAdapter.bulkIndexAuthors(allAuthors);
logger.info("Successfully completed scheduled author reindexing");
} else {
logger.info("No authors found to reindex");

86
backend/src/main/java/com/storycove/service/AuthorService.java
View File

@@ -28,12 +28,12 @@ public class AuthorService {
private static final Logger logger = LoggerFactory.getLogger(AuthorService.class);
private final AuthorRepository authorRepository;
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
@Autowired
public AuthorService(AuthorRepository authorRepository, @Autowired(required = false) TypesenseService typesenseService) {
public AuthorService(AuthorRepository authorRepository, SearchServiceAdapter searchServiceAdapter) {
this.authorRepository = authorRepository;
this.typesenseService = typesenseService;
this.searchServiceAdapter = searchServiceAdapter;
}
@Transactional(readOnly = true)
@@ -132,14 +132,8 @@ public class AuthorService {
validateAuthorForCreate(author);
Author savedAuthor = authorRepository.save(author);
// Index in Typesense
if (typesenseService != null) {
try {
typesenseService.indexAuthor(savedAuthor);
} catch (Exception e) {
logger.warn("Failed to index author in Typesense: " + savedAuthor.getName(), e);
}
}
// Index in OpenSearch
searchServiceAdapter.indexAuthor(savedAuthor);
return savedAuthor;
}
@@ -156,14 +150,8 @@ public class AuthorService {
updateAuthorFields(existingAuthor, authorUpdates);
Author savedAuthor = authorRepository.save(existingAuthor);
// Update in Typesense
if (typesenseService != null) {
try {
typesenseService.updateAuthor(savedAuthor);
} catch (Exception e) {
logger.warn("Failed to update author in Typesense: " + savedAuthor.getName(), e);
}
}
// Update in OpenSearch
searchServiceAdapter.updateAuthor(savedAuthor);
return savedAuthor;
}
@@ -178,14 +166,8 @@ public class AuthorService {
authorRepository.delete(author);
// Remove from Typesense
if (typesenseService != null) {
try {
typesenseService.deleteAuthor(id.toString());
} catch (Exception e) {
logger.warn("Failed to delete author from Typesense: " + author.getName(), e);
}
}
// Remove from OpenSearch
searchServiceAdapter.deleteAuthor(id);
}
public Author addUrl(UUID id, String url) {
@@ -193,14 +175,8 @@ public class AuthorService {
author.addUrl(url);
Author savedAuthor = authorRepository.save(author);
// Update in Typesense
if (typesenseService != null) {
try {
typesenseService.updateAuthor(savedAuthor);
} catch (Exception e) {
logger.warn("Failed to update author in Typesense after adding URL: " + savedAuthor.getName(), e);
}
}
// Update in OpenSearch
searchServiceAdapter.updateAuthor(savedAuthor);
return savedAuthor;
}
@@ -210,14 +186,8 @@ public class AuthorService {
author.removeUrl(url);
Author savedAuthor = authorRepository.save(author);
// Update in Typesense
if (typesenseService != null) {
try {
typesenseService.updateAuthor(savedAuthor);
} catch (Exception e) {
logger.warn("Failed to update author in Typesense after removing URL: " + savedAuthor.getName(), e);
}
}
// Update in OpenSearch
searchServiceAdapter.updateAuthor(savedAuthor);
return savedAuthor;
}
@@ -251,14 +221,8 @@ public class AuthorService {
logger.debug("Saved author rating: {} for author: {}",
refreshedAuthor.getAuthorRating(), refreshedAuthor.getName());
// Update in Typesense
if (typesenseService != null) {
try {
typesenseService.updateAuthor(refreshedAuthor);
} catch (Exception e) {
logger.warn("Failed to update author in Typesense after rating: " + refreshedAuthor.getName(), e);
}
}
// Update in OpenSearch
searchServiceAdapter.updateAuthor(refreshedAuthor);
return refreshedAuthor;
}
@@ -301,14 +265,8 @@ public class AuthorService {
author.setAvatarImagePath(avatarPath);
Author savedAuthor = authorRepository.save(author);
// Update in Typesense
if (typesenseService != null) {
try {
typesenseService.updateAuthor(savedAuthor);
} catch (Exception e) {
logger.warn("Failed to update author in Typesense after setting avatar: " + savedAuthor.getName(), e);
}
}
// Update in OpenSearch
searchServiceAdapter.updateAuthor(savedAuthor);
return savedAuthor;
}
@@ -318,14 +276,8 @@ public class AuthorService {
author.setAvatarImagePath(null);
Author savedAuthor = authorRepository.save(author);
// Update in Typesense
if (typesenseService != null) {
try {
typesenseService.updateAuthor(savedAuthor);
} catch (Exception e) {
logger.warn("Failed to update author in Typesense after removing avatar: " + savedAuthor.getName(), e);
}
}
// Update in OpenSearch
searchServiceAdapter.updateAuthor(savedAuthor);
return savedAuthor;
}

52
backend/src/main/java/com/storycove/service/CollectionService.java
View File

@@ -31,7 +31,7 @@ public class CollectionService {
private final CollectionStoryRepository collectionStoryRepository;
private final StoryRepository storyRepository;
private final TagRepository tagRepository;
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
private final ReadingTimeService readingTimeService;
@Autowired
@@ -39,13 +39,13 @@ public class CollectionService {
CollectionStoryRepository collectionStoryRepository,
StoryRepository storyRepository,
TagRepository tagRepository,
@Autowired(required = false) TypesenseService typesenseService,
SearchServiceAdapter searchServiceAdapter,
ReadingTimeService readingTimeService) {
this.collectionRepository = collectionRepository;
this.collectionStoryRepository = collectionStoryRepository;
this.storyRepository = storyRepository;
this.tagRepository = tagRepository;
this.typesenseService = typesenseService;
this.searchServiceAdapter = searchServiceAdapter;
this.readingTimeService = readingTimeService;
}
@@ -54,15 +54,12 @@ public class CollectionService {
* This method MUST be used instead of JPA queries for listing collections
*/
public SearchResultDto<Collection> searchCollections(String query, List<String> tags, boolean includeArchived, int page, int limit) {
if (typesenseService == null) {
logger.warn("Typesense service not available, returning empty results");
// Collections are currently handled at database level, not indexed in search engine
// Return empty result for now as collections search is not implemented in OpenSearch
logger.warn("Collections search not yet implemented in OpenSearch, returning empty results");
return new SearchResultDto<>(new ArrayList<>(), 0, page, limit, query != null ? query : "", 0);
}
// Delegate to TypesenseService for all search operations
return typesenseService.searchCollections(query, tags, includeArchived, page, limit);
}
/**
* Find collection by ID with full details
*/
@@ -107,10 +104,7 @@ public class CollectionService {
savedCollection = findById(savedCollection.getId());
}
// Index in Typesense
if (typesenseService != null) {
typesenseService.indexCollection(savedCollection);
}
// Collections are not indexed in search engine yet
logger.info("Created collection: {} with {} stories", name, initialStoryIds != null ? initialStoryIds.size() : 0);
return savedCollection;
@@ -140,10 +134,7 @@ public class CollectionService {
Collection savedCollection = collectionRepository.save(collection);
// Update in Typesense
if (typesenseService != null) {
typesenseService.indexCollection(savedCollection);
}
// Collections are not indexed in search engine yet
logger.info("Updated collection: {}", id);
return savedCollection;
@@ -155,10 +146,7 @@ public class CollectionService {
public void deleteCollection(UUID id) {
Collection collection = findByIdBasic(id);
// Remove from Typesense first
if (typesenseService != null) {
typesenseService.removeCollection(id);
}
// Collections are not indexed in search engine yet
collectionRepository.delete(collection);
logger.info("Deleted collection: {}", id);
@@ -173,10 +161,7 @@ public class CollectionService {
Collection savedCollection = collectionRepository.save(collection);
// Update in Typesense
if (typesenseService != null) {
typesenseService.indexCollection(savedCollection);
}
// Collections are not indexed in search engine yet
logger.info("{} collection: {}", archived ? "Archived" : "Unarchived", id);
return savedCollection;
@@ -221,10 +206,7 @@ public class CollectionService {
}
// Update collection in Typesense
if (typesenseService != null) {
Collection updatedCollection = findById(collectionId);
typesenseService.indexCollection(updatedCollection);
}
// Collections are not indexed in search engine yet
long totalStories = collectionStoryRepository.countByCollectionId(collectionId);
@@ -249,10 +231,7 @@ public class CollectionService {
collectionStoryRepository.delete(collectionStory);
// Update collection in Typesense
if (typesenseService != null) {
Collection updatedCollection = findById(collectionId);
typesenseService.indexCollection(updatedCollection);
}
// Collections are not indexed in search engine yet
logger.info("Removed story {} from collection {}", storyId, collectionId);
}
@@ -285,10 +264,7 @@ public class CollectionService {
}
// Update collection in Typesense
if (typesenseService != null) {
Collection updatedCollection = findById(collectionId);
typesenseService.indexCollection(updatedCollection);
}
// Collections are not indexed in search engine yet
logger.info("Reordered {} stories in collection {}", storyOrders.size(), collectionId);
}
@@ -423,7 +399,7 @@ public class CollectionService {
}
/**
* Get all collections for indexing (used by TypesenseService)
* Get all collections for indexing (used by SearchServiceAdapter)
*/
public List<Collection> findAllForIndexing() {
return collectionRepository.findAllActiveCollections();

38
backend/src/main/java/com/storycove/service/DatabaseManagementService.java
View File

@@ -52,7 +52,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
private CollectionRepository collectionRepository;
@Autowired
private TypesenseService typesenseService;
private SearchServiceAdapter searchServiceAdapter;
@Autowired
private LibraryService libraryService;
@@ -145,15 +145,15 @@ public class DatabaseManagementService implements ApplicationContextAware {
System.err.println("No files directory found in backup - skipping file restore.");
}
// 6. Trigger complete Typesense reindex after data restoration
// 6. Trigger complete search index reindex after data restoration
try {
System.err.println("Starting Typesense reindex after restore...");
TypesenseService typesenseService = applicationContext.getBean(TypesenseService.class);
typesenseService.performCompleteReindex();
System.err.println("Typesense reindex completed successfully.");
System.err.println("Starting search index reindex after restore...");
SearchServiceAdapter searchServiceAdapter = applicationContext.getBean(SearchServiceAdapter.class);
searchServiceAdapter.performCompleteReindex();
System.err.println("Search index reindex completed successfully.");
} catch (Exception e) {
System.err.println("Warning: Failed to reindex Typesense after restore: " + e.getMessage());
// Don't fail the entire restore for Typesense issues
System.err.println("Warning: Failed to reindex search after restore: " + e.getMessage());
// Don't fail the entire restore for search issues
}
System.err.println("Complete backup restore finished successfully.");
@@ -299,9 +299,9 @@ public class DatabaseManagementService implements ApplicationContextAware {
// Reindex search after successful restore
try {
String currentLibraryId = libraryService.getCurrentLibraryId();
System.err.println("Starting Typesense reindex after successful restore for library: " + currentLibraryId);
System.err.println("Starting search reindex after successful restore for library: " + currentLibraryId);
if (currentLibraryId == null) {
System.err.println("ERROR: No current library set during restore - cannot reindex Typesense!");
System.err.println("ERROR: No current library set during restore - cannot reindex search!");
throw new IllegalStateException("No current library active during restore");
}
@@ -310,10 +310,10 @@ public class DatabaseManagementService implements ApplicationContextAware {
reindexStoriesAndAuthorsFromCurrentDatabase();
// Note: Collections collection will be recreated when needed by the service
System.err.println("Typesense reindex completed successfully for library: " + currentLibraryId);
System.err.println("Search reindex completed successfully for library: " + currentLibraryId);
} catch (Exception e) {
// Log the error but don't fail the restore
System.err.println("Warning: Failed to reindex Typesense after restore: " + e.getMessage());
System.err.println("Warning: Failed to reindex search after restore: " + e.getMessage());
e.printStackTrace();
}
@@ -351,7 +351,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
totalDeleted = collectionCount + storyCount + authorCount + seriesCount + tagCount;
// Note: Search indexes will need to be manually recreated after clearing
// Use the settings page to recreate Typesense collections after clearing the database
// Use the settings page to recreate search indices after clearing the database
} catch (Exception e) {
throw new RuntimeException("Failed to clear database: " + e.getMessage(), e);
@@ -506,8 +506,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
}
// For clearing, we only want to recreate empty collections (no data to index)
typesenseService.recreateStoriesCollection();
typesenseService.recreateAuthorsCollection();
searchServiceAdapter.recreateIndices();
// Note: Collections collection will be recreated when needed by the service
System.err.println("Search indexes cleared successfully for library: " + currentLibraryId);
} catch (Exception e) {
@@ -959,10 +958,9 @@ public class DatabaseManagementService implements ApplicationContextAware {
try (Connection connection = getDataSource().getConnection()) {
// First, recreate empty collections
try {
typesenseService.recreateStoriesCollection();
typesenseService.recreateAuthorsCollection();
searchServiceAdapter.recreateIndices();
} catch (Exception e) {
throw new SQLException("Failed to recreate Typesense collections", e);
throw new SQLException("Failed to recreate search indices", e);
}
// Count and reindex stories with full author and series information
@@ -984,7 +982,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
while (rs.next()) {
// Create a complete Story object for indexing
var story = createStoryFromResultSet(rs);
typesenseService.indexStory(story);
searchServiceAdapter.indexStory(story);
storyCount++;
}
}
@@ -999,7 +997,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
while (rs.next()) {
// Create a minimal Author object for indexing
var author = createAuthorFromResultSet(rs);
typesenseService.indexAuthor(author);
searchServiceAdapter.indexAuthor(author);
authorCount++;
}
}

78
backend/src/main/java/com/storycove/service/LibraryService.java
View File

@@ -13,8 +13,6 @@ import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.stereotype.Service;
import org.typesense.api.Client;
import org.typesense.resources.Node;
import jakarta.annotation.PostConstruct;
import jakarta.annotation.PreDestroy;
@@ -26,7 +24,6 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.sql.SQLException;
import java.time.Duration;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
@@ -43,14 +40,6 @@ public class LibraryService implements ApplicationContextAware {
@Value("${spring.datasource.password}")
private String dbPassword;
@Value("${typesense.host}")
private String typesenseHost;
@Value("${typesense.port}")
private String typesensePort;
@Value("${typesense.api-key}")
private String typesenseApiKey;
private final ObjectMapper objectMapper = new ObjectMapper();
private final BCryptPasswordEncoder passwordEncoder = new BCryptPasswordEncoder();
@@ -61,7 +50,6 @@ public class LibraryService implements ApplicationContextAware {
// Current active resources
private volatile String currentLibraryId;
private volatile Client currentTypesenseClient;
// Security: Track if user has explicitly authenticated in this session
private volatile boolean explicitlyAuthenticated = false;
@@ -100,7 +88,6 @@ public class LibraryService implements ApplicationContextAware {
@PreDestroy
public void cleanup() {
currentLibraryId = null;
currentTypesenseClient = null;
explicitlyAuthenticated = false;
}
@@ -110,7 +97,6 @@ public class LibraryService implements ApplicationContextAware {
public void clearAuthentication() {
explicitlyAuthenticated = false;
currentLibraryId = null;
currentTypesenseClient = null;
logger.info("Authentication cleared - user must re-authenticate to access libraries");
}
@@ -129,7 +115,7 @@ public class LibraryService implements ApplicationContextAware {
/**
* Switch to library after authentication with forced reindexing
* This ensures Typesense is always up-to-date after login
* This ensures OpenSearch is always up-to-date after login
*/
public synchronized void switchToLibraryAfterAuthentication(String libraryId) throws Exception {
logger.info("Switching to library after authentication: {} (forcing reindex)", libraryId);
@@ -168,25 +154,15 @@ public class LibraryService implements ApplicationContextAware {
// Set new active library (datasource routing handled by SmartRoutingDataSource)
currentLibraryId = libraryId;
currentTypesenseClient = createTypesenseClient(library.getTypesenseCollection());
// Initialize Typesense collections for this library
try {
TypesenseService typesenseService = applicationContext.getBean(TypesenseService.class);
// First ensure collections exist
typesenseService.initializeCollectionsForCurrentLibrary();
logger.info("Completed Typesense initialization for library: {}", libraryId);
} catch (Exception e) {
logger.warn("Failed to initialize Typesense for library {}: {}", libraryId, e.getMessage());
// Don't fail the switch - collections can be created later
}
// OpenSearch indexes are global - no per-library initialization needed
logger.info("Library switched to OpenSearch mode for library: {}", libraryId);
logger.info("Successfully switched to library: {}", library.getName());
// Perform complete reindex AFTER library switch is fully complete
// This ensures database routing is properly established
if (forceReindex || !libraryId.equals(previousLibraryId)) {
logger.info("Starting post-switch Typesense reindex for library: {}", libraryId);
logger.info("Starting post-switch OpenSearch reindex for library: {}", libraryId);
// Run reindex asynchronously to avoid blocking authentication response
// and allow time for database routing to fully stabilize
@@ -195,15 +171,25 @@ public class LibraryService implements ApplicationContextAware {
try {
// Give routing time to stabilize
Thread.sleep(500);
logger.info("Starting async Typesense reindex for library: {}", finalLibraryId);
logger.info("Starting async OpenSearch reindex for library: {}", finalLibraryId);
TypesenseService typesenseService = applicationContext.getBean(TypesenseService.class);
typesenseService.performCompleteReindex();
logger.info("Completed async Typesense reindexing for library: {}", finalLibraryId);
SearchServiceAdapter searchService = applicationContext.getBean(SearchServiceAdapter.class);
// Get all stories and authors for reindexing
StoryService storyService = applicationContext.getBean(StoryService.class);
AuthorService authorService = applicationContext.getBean(AuthorService.class);
var allStories = storyService.findAllWithAssociations();
var allAuthors = authorService.findAllWithStories();
searchService.bulkIndexStories(allStories);
searchService.bulkIndexAuthors(allAuthors);
logger.info("Completed async OpenSearch reindexing for library: {} ({} stories, {} authors)",
finalLibraryId, allStories.size(), allAuthors.size());
} catch (Exception e) {
logger.warn("Failed to async reindex Typesense for library {}: {}", finalLibraryId, e.getMessage());
logger.warn("Failed to async reindex OpenSearch for library {}: {}", finalLibraryId, e.getMessage());
}
}, "TypesenseReindex-" + libraryId).start();
}, "OpenSearchReindex-" + libraryId).start();
}
}
@@ -219,12 +205,6 @@ public class LibraryService implements ApplicationContextAware {
}
}
public Client getCurrentTypesenseClient() {
if (currentTypesenseClient == null) {
throw new IllegalStateException("No active library - please authenticate first");
}
return currentTypesenseClient;
}
public String getCurrentLibraryId() {
return currentLibraryId;
@@ -545,8 +525,8 @@ public class LibraryService implements ApplicationContextAware {
// 1. Create image directory structure
initializeImageDirectories(library);
// 2. Initialize Typesense collections (this will be done when switching to the library)
// The TypesenseService.initializeCollections() will be called automatically
// 2. OpenSearch indexes are global and managed automatically
// No per-library initialization needed for OpenSearch
logger.info("Successfully initialized resources for library: {}", library.getName());
@@ -777,21 +757,10 @@ public class LibraryService implements ApplicationContextAware {
}
}
private Client createTypesenseClient(String collection) {
logger.info("Creating Typesense client for collection: {}", collection);
List<Node> nodes = Arrays.asList(
new Node("http", typesenseHost, typesensePort)
);
org.typesense.api.Configuration configuration = new org.typesense.api.Configuration(nodes, Duration.ofSeconds(10), typesenseApiKey);
return new Client(configuration);
}
private void closeCurrentResources() {
// No need to close datasource - SmartRoutingDataSource handles this
// Typesense client doesn't need explicit cleanup
currentTypesenseClient = null;
// OpenSearch service is managed by Spring - no explicit cleanup needed
// Don't clear currentLibraryId here - only when explicitly switching
}
@@ -848,7 +817,6 @@ public class LibraryService implements ApplicationContextAware {
config.put("description", library.getDescription());
config.put("passwordHash", library.getPasswordHash());
config.put("dbName", library.getDbName());
config.put("typesenseCollection", library.getTypesenseCollection());
config.put("imagePath", library.getImagePath());
config.put("initialized", library.isInitialized());

473
backend/src/main/java/com/storycove/service/SearchMigrationManager.java
View File

@@ -1,473 +0,0 @@
package com.storycove.service;
import com.storycove.dto.AuthorSearchDto;
import com.storycove.dto.SearchResultDto;
import com.storycove.dto.StorySearchDto;
import com.storycove.entity.Author;
import com.storycove.entity.Story;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import java.util.List;
import java.util.UUID;
/**
* TEMPORARY MIGRATION MANAGER - DELETE THIS ENTIRE CLASS WHEN TYPESENSE IS REMOVED
*
* This class handles dual-write functionality and engine switching during the
* migration from Typesense to OpenSearch. It's designed to be completely removed
* once the migration is complete.
*
* CLEANUP INSTRUCTIONS:
* 1. Delete this entire file: SearchMigrationManager.java
* 2. Update SearchServiceAdapter to call OpenSearchService directly
* 3. Remove migration-related configuration properties
* 4. Remove migration-related admin endpoints and UI
*/
@Component
public class SearchMigrationManager {
private static final Logger logger = LoggerFactory.getLogger(SearchMigrationManager.class);
@Autowired(required = false)
private TypesenseService typesenseService;
@Autowired(required = false)
private OpenSearchService openSearchService;
@Value("${storycove.search.engine:typesense}")
private String primaryEngine;
@Value("${storycove.search.dual-write:false}")
private boolean dualWrite;
// ===============================
// READ OPERATIONS (single engine)
// ===============================
public SearchResultDto<StorySearchDto> searchStories(String query, List<String> tags, String author,
String series, Integer minWordCount, Integer maxWordCount,
Float minRating, Boolean isRead, Boolean isFavorite,
String sortBy, String sortOrder, int page, int size,
List<String> facetBy,
// Advanced filters
String createdAfter, String createdBefore,
String lastReadAfter, String lastReadBefore,
Boolean unratedOnly, String readingStatus,
Boolean hasReadingProgress, Boolean hasCoverImage,
String sourceDomain, String seriesFilter,
Integer minTagCount, Boolean popularOnly,
Boolean hiddenGemsOnly) {
boolean openSearchAvailable = openSearchService != null;
boolean openSearchConnected = openSearchAvailable ? openSearchService.testConnection() : false;
boolean routingCondition = "opensearch".equalsIgnoreCase(primaryEngine) && openSearchAvailable;
logger.info("SEARCH ROUTING DEBUG:");
logger.info(" Primary engine: '{}'", primaryEngine);
logger.info(" OpenSearch available: {}", openSearchAvailable);
logger.info(" OpenSearch connected: {}", openSearchConnected);
logger.info(" Routing condition result: {}", routingCondition);
logger.info(" Will route to: {}", routingCondition ? "OpenSearch" : "Typesense");
if (routingCondition) {
logger.info("ROUTING TO OPENSEARCH");
return openSearchService.searchStories(query, tags, author, series, minWordCount, maxWordCount,
minRating, isRead, isFavorite, sortBy, sortOrder, page, size, facetBy,
createdAfter, createdBefore, lastReadAfter, lastReadBefore, unratedOnly, readingStatus,
hasReadingProgress, hasCoverImage, sourceDomain, seriesFilter, minTagCount, popularOnly,
hiddenGemsOnly);
} else if (typesenseService != null) {
logger.info("ROUTING TO TYPESENSE");
// Convert parameters to match TypesenseService signature
return typesenseService.searchStories(
query, page, size, tags, null, minWordCount, maxWordCount,
null, null, null, null, minRating != null ? minRating.intValue() : null,
null, null, sortBy, sortOrder, null, null, isRead, isFavorite,
author, series, null, null, null);
} else {
logger.error("No search service available! Primary engine: {}, OpenSearch: {}, Typesense: {}",
primaryEngine, openSearchService != null, typesenseService != null);
return new SearchResultDto<>(List.of(), 0, page, size, query != null ? query : "", 0);
}
}
public List<StorySearchDto> getRandomStories(int count, List<String> tags, String author,
String series, Integer minWordCount, Integer maxWordCount,
Float minRating, Boolean isRead, Boolean isFavorite,
Long seed) {
logger.debug("Getting random stories using primary engine: {}", primaryEngine);
if ("opensearch".equalsIgnoreCase(primaryEngine) && openSearchService != null) {
return openSearchService.getRandomStories(count, tags, author, series, minWordCount, maxWordCount,
minRating, isRead, isFavorite, seed);
} else if (typesenseService != null) {
// TypesenseService doesn't have getRandomStories, use random story ID approach
List<StorySearchDto> results = new java.util.ArrayList<>();
for (int i = 0; i < count; i++) {
var randomId = typesenseService.getRandomStoryId(null, tags, seed != null ? seed + i : null);
// Note: This is a simplified approach - full implementation would need story lookup
}
return results;
} else {
logger.error("No search service available for random stories");
return List.of();
}
}
public String getRandomStoryId(Long seed) {
logger.debug("Getting random story ID using primary engine: {}", primaryEngine);
if ("opensearch".equalsIgnoreCase(primaryEngine) && openSearchService != null) {
return openSearchService.getRandomStoryId(seed);
} else if (typesenseService != null) {
var randomId = typesenseService.getRandomStoryId(null, null, seed);
return randomId.map(UUID::toString).orElse(null);
} else {
logger.error("No search service available for random story ID");
return null;
}
}
public List<AuthorSearchDto> searchAuthors(String query, int limit) {
logger.debug("Searching authors using primary engine: {}", primaryEngine);
if ("opensearch".equalsIgnoreCase(primaryEngine) && openSearchService != null) {
return openSearchService.searchAuthors(query, limit);
} else if (typesenseService != null) {
var result = typesenseService.searchAuthors(query, 0, limit, null, null);
return result.getResults();
} else {
logger.error("No search service available for author search");
return List.of();
}
}
public List<String> getTagSuggestions(String query, int limit) {
logger.debug("Getting tag suggestions using primary engine: {}", primaryEngine);
if ("opensearch".equalsIgnoreCase(primaryEngine) && openSearchService != null) {
return openSearchService.getTagSuggestions(query, limit);
} else if (typesenseService != null) {
// TypesenseService may not have getTagSuggestions - return empty for now
logger.warn("Tag suggestions not implemented for Typesense");
return List.of();
} else {
logger.error("No search service available for tag suggestions");
return List.of();
}
}
// ===============================
// WRITE OPERATIONS (dual-write capable)
// ===============================
public void indexStory(Story story) {
logger.debug("Indexing story with dual-write: {}, primary engine: {}", dualWrite, primaryEngine);
// Write to OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.indexStory(story);
logger.debug("Successfully indexed story {} in OpenSearch", story.getId());
} catch (Exception e) {
logger.error("Failed to index story {} in OpenSearch", story.getId(), e);
}
} else {
logger.warn("OpenSearch service not available for indexing story {}", story.getId());
}
}
// Write to Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.indexStory(story);
logger.debug("Successfully indexed story {} in Typesense", story.getId());
} catch (Exception e) {
logger.error("Failed to index story {} in Typesense", story.getId(), e);
}
} else {
logger.warn("Typesense service not available for indexing story {}", story.getId());
}
}
}
public void updateStory(Story story) {
logger.debug("Updating story with dual-write: {}, primary engine: {}", dualWrite, primaryEngine);
// Update in OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.updateStory(story);
logger.debug("Successfully updated story {} in OpenSearch", story.getId());
} catch (Exception e) {
logger.error("Failed to update story {} in OpenSearch", story.getId(), e);
}
}
}
// Update in Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.updateStory(story);
logger.debug("Successfully updated story {} in Typesense", story.getId());
} catch (Exception e) {
logger.error("Failed to update story {} in Typesense", story.getId(), e);
}
}
}
}
public void deleteStory(UUID storyId) {
logger.debug("Deleting story with dual-write: {}, primary engine: {}", dualWrite, primaryEngine);
// Delete from OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.deleteStory(storyId);
logger.debug("Successfully deleted story {} from OpenSearch", storyId);
} catch (Exception e) {
logger.error("Failed to delete story {} from OpenSearch", storyId, e);
}
}
}
// Delete from Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.deleteStory(storyId.toString());
logger.debug("Successfully deleted story {} from Typesense", storyId);
} catch (Exception e) {
logger.error("Failed to delete story {} from Typesense", storyId, e);
}
}
}
}
public void indexAuthor(Author author) {
logger.debug("Indexing author with dual-write: {}, primary engine: {}", dualWrite, primaryEngine);
// Index in OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.indexAuthor(author);
logger.debug("Successfully indexed author {} in OpenSearch", author.getId());
} catch (Exception e) {
logger.error("Failed to index author {} in OpenSearch", author.getId(), e);
}
}
}
// Index in Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.indexAuthor(author);
logger.debug("Successfully indexed author {} in Typesense", author.getId());
} catch (Exception e) {
logger.error("Failed to index author {} in Typesense", author.getId(), e);
}
}
}
}
public void updateAuthor(Author author) {
logger.debug("Updating author with dual-write: {}, primary engine: {}", dualWrite, primaryEngine);
// Update in OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.updateAuthor(author);
logger.debug("Successfully updated author {} in OpenSearch", author.getId());
} catch (Exception e) {
logger.error("Failed to update author {} in OpenSearch", author.getId(), e);
}
}
}
// Update in Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.updateAuthor(author);
logger.debug("Successfully updated author {} in Typesense", author.getId());
} catch (Exception e) {
logger.error("Failed to update author {} in Typesense", author.getId(), e);
}
}
}
}
public void deleteAuthor(UUID authorId) {
logger.debug("Deleting author with dual-write: {}, primary engine: {}", dualWrite, primaryEngine);
// Delete from OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.deleteAuthor(authorId);
logger.debug("Successfully deleted author {} from OpenSearch", authorId);
} catch (Exception e) {
logger.error("Failed to delete author {} from OpenSearch", authorId, e);
}
}
}
// Delete from Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.deleteAuthor(authorId.toString());
logger.debug("Successfully deleted author {} from Typesense", authorId);
} catch (Exception e) {
logger.error("Failed to delete author {} from Typesense", authorId, e);
}
}
}
}
public void bulkIndexStories(List<Story> stories) {
logger.debug("Bulk indexing {} stories with dual-write: {}, primary engine: {}",
stories.size(), dualWrite, primaryEngine);
// Bulk index in OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.bulkIndexStories(stories);
logger.info("Successfully bulk indexed {} stories in OpenSearch", stories.size());
} catch (Exception e) {
logger.error("Failed to bulk index {} stories in OpenSearch", stories.size(), e);
}
}
}
// Bulk index in Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.bulkIndexStories(stories);
logger.info("Successfully bulk indexed {} stories in Typesense", stories.size());
} catch (Exception e) {
logger.error("Failed to bulk index {} stories in Typesense", stories.size(), e);
}
}
}
}
public void bulkIndexAuthors(List<Author> authors) {
logger.debug("Bulk indexing {} authors with dual-write: {}, primary engine: {}",
authors.size(), dualWrite, primaryEngine);
// Bulk index in OpenSearch
if ("opensearch".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (openSearchService != null) {
try {
openSearchService.bulkIndexAuthors(authors);
logger.info("Successfully bulk indexed {} authors in OpenSearch", authors.size());
} catch (Exception e) {
logger.error("Failed to bulk index {} authors in OpenSearch", authors.size(), e);
}
}
}
// Bulk index in Typesense
if ("typesense".equalsIgnoreCase(primaryEngine) || dualWrite) {
if (typesenseService != null) {
try {
typesenseService.bulkIndexAuthors(authors);
logger.info("Successfully bulk indexed {} authors in Typesense", authors.size());
} catch (Exception e) {
logger.error("Failed to bulk index {} authors in Typesense", authors.size(), e);
}
}
}
}
// ===============================
// UTILITY METHODS
// ===============================
public boolean isSearchServiceAvailable() {
if ("opensearch".equalsIgnoreCase(primaryEngine)) {
return openSearchService != null && openSearchService.testConnection();
} else {
return typesenseService != null;
}
}
public String getCurrentSearchEngine() {
return primaryEngine;
}
public boolean isDualWriteEnabled() {
return dualWrite;
}
public boolean canSwitchToOpenSearch() {
return openSearchService != null && openSearchService.testConnection();
}
public boolean canSwitchToTypesense() {
return typesenseService != null;
}
public OpenSearchService getOpenSearchService() {
return openSearchService;
}
/**
* Update configuration at runtime (for admin interface)
* Note: This requires @RefreshScope to work properly
*/
public void updateConfiguration(String engine, boolean enableDualWrite) {
logger.info("Updating search configuration: engine={}, dualWrite={}", engine, enableDualWrite);
this.primaryEngine = engine;
this.dualWrite = enableDualWrite;
}
/**
* Get current configuration status for admin interface
*/
public SearchMigrationStatus getStatus() {
return new SearchMigrationStatus(
primaryEngine,
dualWrite,
typesenseService != null,
openSearchService != null && openSearchService.testConnection()
);
}
/**
* DTO for search migration status
*/
public static class SearchMigrationStatus {
private final String primaryEngine;
private final boolean dualWrite;
private final boolean typesenseAvailable;
private final boolean openSearchAvailable;
public SearchMigrationStatus(String primaryEngine, boolean dualWrite,
boolean typesenseAvailable, boolean openSearchAvailable) {
this.primaryEngine = primaryEngine;
this.dualWrite = dualWrite;
this.typesenseAvailable = typesenseAvailable;
this.openSearchAvailable = openSearchAvailable;
}
public String getPrimaryEngine() { return primaryEngine; }
public boolean isDualWrite() { return dualWrite; }
public boolean isTypesenseAvailable() { return typesenseAvailable; }
public boolean isOpenSearchAvailable() { return openSearchAvailable; }
}
}

150
backend/src/main/java/com/storycove/service/SearchServiceAdapter.java
View File

@@ -16,10 +16,7 @@ import java.util.UUID;
/**
* Service adapter that provides a unified interface for search operations.
*
* This adapter delegates to SearchMigrationManager during the migration period,
* which will be removed once Typesense is completely eliminated.
*
* POST-MIGRATION: This class will be simplified to call OpenSearchService directly.
* This adapter directly delegates to OpenSearchService.
*/
@Service
public class SearchServiceAdapter {
@@ -27,7 +24,7 @@ public class SearchServiceAdapter {
private static final Logger logger = LoggerFactory.getLogger(SearchServiceAdapter.class);
@Autowired
private SearchMigrationManager migrationManager;
private OpenSearchService openSearchService;
// ===============================
// SEARCH OPERATIONS
@@ -49,7 +46,7 @@ public class SearchServiceAdapter {
String sourceDomain, String seriesFilter,
Integer minTagCount, Boolean popularOnly,
Boolean hiddenGemsOnly) {
return migrationManager.searchStories(query, tags, author, series, minWordCount, maxWordCount,
return openSearchService.searchStories(query, tags, author, series, minWordCount, maxWordCount,
minRating, isRead, isFavorite, sortBy, sortOrder, page, size, facetBy,
createdAfter, createdBefore, lastReadAfter, lastReadBefore, unratedOnly, readingStatus,
hasReadingProgress, hasCoverImage, sourceDomain, seriesFilter, minTagCount, popularOnly,
@@ -63,29 +60,54 @@ public class SearchServiceAdapter {
String series, Integer minWordCount, Integer maxWordCount,
Float minRating, Boolean isRead, Boolean isFavorite,
Long seed) {
return migrationManager.getRandomStories(count, tags, author, series, minWordCount, maxWordCount,
return openSearchService.getRandomStories(count, tags, author, series, minWordCount, maxWordCount,
minRating, isRead, isFavorite, seed);
}
/**
* Recreate search indices
*/
public void recreateIndices() {
try {
openSearchService.recreateIndices();
} catch (Exception e) {
logger.error("Failed to recreate search indices", e);
throw new RuntimeException("Failed to recreate search indices", e);
}
}
/**
* Perform complete reindex of all data
*/
public void performCompleteReindex() {
try {
recreateIndices();
logger.info("Search indices recreated successfully");
} catch (Exception e) {
logger.error("Failed to perform complete reindex", e);
throw new RuntimeException("Failed to perform complete reindex", e);
}
}
/**
* Get random story ID with unified interface
*/
public String getRandomStoryId(Long seed) {
return migrationManager.getRandomStoryId(seed);
return openSearchService.getRandomStoryId(seed);
}
/**
* Search authors with unified interface
*/
public List<AuthorSearchDto> searchAuthors(String query, int limit) {
return migrationManager.searchAuthors(query, limit);
return openSearchService.searchAuthors(query, limit);
}
/**
* Get tag suggestions with unified interface
*/
public List<String> getTagSuggestions(String query, int limit) {
return migrationManager.getTagSuggestions(query, limit);
return openSearchService.getTagSuggestions(query, limit);
}
// ===============================
@@ -93,59 +115,91 @@ public class SearchServiceAdapter {
// ===============================
/**
* Index a story with unified interface (supports dual-write)
* Index a story in OpenSearch
*/
public void indexStory(Story story) {
migrationManager.indexStory(story);
try {
openSearchService.indexStory(story);
} catch (Exception e) {
logger.error("Failed to index story {}", story.getId(), e);
}
}
/**
* Update a story in the index with unified interface (supports dual-write)
* Update a story in OpenSearch
*/
public void updateStory(Story story) {
migrationManager.updateStory(story);
try {
openSearchService.updateStory(story);
} catch (Exception e) {
logger.error("Failed to update story {}", story.getId(), e);
}
}
/**
* Delete a story from the index with unified interface (supports dual-write)
* Delete a story from OpenSearch
*/
public void deleteStory(UUID storyId) {
migrationManager.deleteStory(storyId);
try {
openSearchService.deleteStory(storyId);
} catch (Exception e) {
logger.error("Failed to delete story {}", storyId, e);
}
}
/**
* Index an author with unified interface (supports dual-write)
* Index an author in OpenSearch
*/
public void indexAuthor(Author author) {
migrationManager.indexAuthor(author);
try {
openSearchService.indexAuthor(author);
} catch (Exception e) {
logger.error("Failed to index author {}", author.getId(), e);
}
}
/**
* Update an author in the index with unified interface (supports dual-write)
* Update an author in OpenSearch
*/
public void updateAuthor(Author author) {
migrationManager.updateAuthor(author);
try {
openSearchService.updateAuthor(author);
} catch (Exception e) {
logger.error("Failed to update author {}", author.getId(), e);
}
}
/**
* Delete an author from the index with unified interface (supports dual-write)
* Delete an author from OpenSearch
*/
public void deleteAuthor(UUID authorId) {
migrationManager.deleteAuthor(authorId);
try {
openSearchService.deleteAuthor(authorId);
} catch (Exception e) {
logger.error("Failed to delete author {}", authorId, e);
}
}
/**
* Bulk index stories with unified interface (supports dual-write)
* Bulk index stories in OpenSearch
*/
public void bulkIndexStories(List<Story> stories) {
migrationManager.bulkIndexStories(stories);
try {
openSearchService.bulkIndexStories(stories);
} catch (Exception e) {
logger.error("Failed to bulk index {} stories", stories.size(), e);
}
}
/**
* Bulk index authors with unified interface (supports dual-write)
* Bulk index authors in OpenSearch
*/
public void bulkIndexAuthors(List<Author> authors) {
migrationManager.bulkIndexAuthors(authors);
try {
openSearchService.bulkIndexAuthors(authors);
} catch (Exception e) {
logger.error("Failed to bulk index {} authors", authors.size(), e);
}
}
// ===============================
@@ -156,41 +210,69 @@ public class SearchServiceAdapter {
* Check if search service is available and healthy
*/
public boolean isSearchServiceAvailable() {
return migrationManager.isSearchServiceAvailable();
return openSearchService.testConnection();
}
/**
* Get current search engine name
*/
public String getCurrentSearchEngine() {
return migrationManager.getCurrentSearchEngine();
return "opensearch";
}
/**
* Check if dual-write is enabled
*/
public boolean isDualWriteEnabled() {
return migrationManager.isDualWriteEnabled();
return false; // No longer supported
}
/**
* Check if we can switch to OpenSearch
*/
public boolean canSwitchToOpenSearch() {
return migrationManager.canSwitchToOpenSearch();
return true; // Already using OpenSearch
}
/**
* Check if we can switch to Typesense
*/
public boolean canSwitchToTypesense() {
return migrationManager.canSwitchToTypesense();
return false; // Typesense no longer available
}
/**
* Get current migration status for admin interface
* Get current search status for admin interface
*/
public SearchMigrationManager.SearchMigrationStatus getMigrationStatus() {
return migrationManager.getStatus();
public SearchStatus getSearchStatus() {
return new SearchStatus(
"opensearch",
false, // no dual-write
false, // no typesense
openSearchService.testConnection()
);
}
/**
* DTO for search status
*/
public static class SearchStatus {
private final String primaryEngine;
private final boolean dualWrite;
private final boolean typesenseAvailable;
private final boolean openSearchAvailable;
public SearchStatus(String primaryEngine, boolean dualWrite,
boolean typesenseAvailable, boolean openSearchAvailable) {
this.primaryEngine = primaryEngine;
this.dualWrite = dualWrite;
this.typesenseAvailable = typesenseAvailable;
this.openSearchAvailable = openSearchAvailable;
}
public String getPrimaryEngine() { return primaryEngine; }
public boolean isDualWrite() { return dualWrite; }
public boolean isTypesenseAvailable() { return typesenseAvailable; }
public boolean isOpenSearchAvailable() { return openSearchAvailable; }
}
}

87
backend/src/main/java/com/storycove/service/StoryService.java
View File

@@ -42,7 +42,7 @@ public class StoryService {
private final TagService tagService;
private final SeriesService seriesService;
private final HtmlSanitizationService sanitizationService;
private final TypesenseService typesenseService;
private final SearchServiceAdapter searchServiceAdapter;
@Autowired
public StoryService(StoryRepository storyRepository,
@@ -52,7 +52,7 @@ public class StoryService {
TagService tagService,
SeriesService seriesService,
HtmlSanitizationService sanitizationService,
@Autowired(required = false) TypesenseService typesenseService) {
SearchServiceAdapter searchServiceAdapter) {
this.storyRepository = storyRepository;
this.tagRepository = tagRepository;
this.readingPositionRepository = readingPositionRepository;
@@ -60,7 +60,7 @@ public class StoryService {
this.tagService = tagService;
this.seriesService = seriesService;
this.sanitizationService = sanitizationService;
this.typesenseService = typesenseService;
this.searchServiceAdapter = searchServiceAdapter;
}
@Transactional(readOnly = true)
@@ -239,10 +239,8 @@ public class StoryService {
story.addTag(tag);
Story savedStory = storyRepository.save(story);
// Update Typesense index with new tag information
if (typesenseService != null) {
typesenseService.updateStory(savedStory);
}
// Update search index with new tag information
searchServiceAdapter.updateStory(savedStory);
return savedStory;
}
@@ -256,10 +254,8 @@ public class StoryService {
story.removeTag(tag);
Story savedStory = storyRepository.save(story);
// Update Typesense index with updated tag information
if (typesenseService != null) {
typesenseService.updateStory(savedStory);
}
// Update search index with updated tag information
searchServiceAdapter.updateStory(savedStory);
return savedStory;
}
@@ -274,10 +270,8 @@ public class StoryService {
story.setRating(rating);
Story savedStory = storyRepository.save(story);
// Update Typesense index with new rating
if (typesenseService != null) {
typesenseService.updateStory(savedStory);
}
// Update search index with new rating
searchServiceAdapter.updateStory(savedStory);
return savedStory;
}
@@ -292,10 +286,8 @@ public class StoryService {
story.updateReadingProgress(position);
Story savedStory = storyRepository.save(story);
// Update Typesense index with new reading progress
if (typesenseService != null) {
typesenseService.updateStory(savedStory);
}
// Update search index with new reading progress
searchServiceAdapter.updateStory(savedStory);
return savedStory;
}
@@ -313,10 +305,8 @@ public class StoryService {
Story savedStory = storyRepository.save(story);
// Update Typesense index with new reading status
if (typesenseService != null) {
typesenseService.updateStory(savedStory);
}
// Update search index with new reading status
searchServiceAdapter.updateStory(savedStory);
return savedStory;
}
@@ -358,10 +348,8 @@ public class StoryService {
updateStoryTags(savedStory, story.getTags());
}
// Index in Typesense (if available)
if (typesenseService != null) {
typesenseService.indexStory(savedStory);
}
// Index in search engine
searchServiceAdapter.indexStory(savedStory);
return savedStory;
}
@@ -388,10 +376,8 @@ public class StoryService {
updateStoryTagsByNames(savedStory, tagNames);
}
// Index in Typesense (if available)
if (typesenseService != null) {
typesenseService.indexStory(savedStory);
}
// Index in search engine
searchServiceAdapter.indexStory(savedStory);
return savedStory;
}
@@ -409,10 +395,8 @@ public class StoryService {
updateStoryFields(existingStory, storyUpdates);
Story updatedStory = storyRepository.save(existingStory);
// Update in Typesense (if available)
if (typesenseService != null) {
typesenseService.updateStory(updatedStory);
}
// Update in search engine
searchServiceAdapter.updateStory(updatedStory);
return updatedStory;
}
@@ -432,10 +416,8 @@ public class StoryService {
Story updatedStory = storyRepository.save(existingStory);
// Update in Typesense (if available)
if (typesenseService != null) {
typesenseService.updateStory(updatedStory);
}
// Update in search engine
searchServiceAdapter.updateStory(updatedStory);
return updatedStory;
}
@@ -455,10 +437,8 @@ public class StoryService {
// Create a copy to avoid ConcurrentModificationException
new ArrayList<>(story.getTags()).forEach(tag -> story.removeTag(tag));
// Delete from Typesense first (if available)
if (typesenseService != null) {
typesenseService.deleteStory(story.getId().toString());
}
// Delete from search engine first
searchServiceAdapter.deleteStory(story.getId());
storyRepository.delete(story);
}
@@ -674,7 +654,7 @@ public class StoryService {
/**
* Find a random story based on optional filters.
* Uses Typesense for consistency with Library search functionality.
* Uses search service for consistency with Library search functionality.
* Supports text search and multiple tags using the same logic as the Library view.
* @param searchQuery Optional search query
* @param tags Optional list of tags to filter by
@@ -693,7 +673,7 @@ public class StoryService {
/**
* Find a random story based on optional filters with seed support.
* Uses Typesense for consistency with Library search functionality.
* Uses search service for consistency with Library search functionality.
* Supports text search and multiple tags using the same logic as the Library view.
* @param searchQuery Optional search query
* @param tags Optional list of tags to filter by
@@ -711,21 +691,16 @@ public class StoryService {
String seriesFilter, Integer minTagCount,
Boolean popularOnly, Boolean hiddenGemsOnly) {
// Use Typesense if available for consistency with Library search
if (typesenseService != null) {
// Use search service for consistency with Library search
try {
Optional<UUID> randomStoryId = typesenseService.getRandomStoryId(searchQuery, tags, seed,
minWordCount, maxWordCount, createdAfter, createdBefore, lastReadAfter, lastReadBefore,
minRating, maxRating, unratedOnly, readingStatus, hasReadingProgress, hasCoverImage,
sourceDomain, seriesFilter, minTagCount, popularOnly, hiddenGemsOnly);
if (randomStoryId.isPresent()) {
return storyRepository.findById(randomStoryId.get());
String randomStoryId = searchServiceAdapter.getRandomStoryId(seed);
if (randomStoryId != null) {
return storyRepository.findById(UUID.fromString(randomStoryId));
}
return Optional.empty();
} catch (Exception e) {
// Fallback to database queries if Typesense fails
logger.warn("Typesense random story lookup failed, falling back to database queries", e);
}
// Fallback to database queries if search service fails
logger.warn("Search service random story lookup failed, falling back to database queries", e);
}
// Fallback to repository-based implementation (global routing handles library selection)

1192
backend/src/main/java/com/storycove/service/StoryService.java.backup
View File

File diff suppressed because it is too large Load Diff

1998
backend/src/main/java/com/storycove/service/TypesenseService.java
View File

File diff suppressed because it is too large Load Diff

9
backend/src/main/resources/application.yml
View File

@@ -39,14 +39,7 @@ storycove:
auth:
password: ${APP_PASSWORD} # REQUIRED: No default password for security
search:
engine: ${SEARCH_ENGINE:typesense} # typesense or opensearch
dual-write: ${SEARCH_DUAL_WRITE:false} # enable dual-write during migration
typesense:
api-key: ${TYPESENSE_API_KEY:xyz}
host: ${TYPESENSE_HOST:localhost}
port: ${TYPESENSE_PORT:8108}
enabled: ${TYPESENSE_ENABLED:true}
reindex-interval: ${TYPESENSE_REINDEX_INTERVAL:3600000} # 1 hour in milliseconds
engine: opensearch # OpenSearch is the only search engine
opensearch:
# Connection settings
host: ${OPENSEARCH_HOST:localhost}

12
backend/src/main/resources/opensearch/README.md
View File

@@ -60,7 +60,7 @@ opensearch/
### 🛡️ **Error Handling & Resilience**
- **Connection Retry Logic**: Automatic retry with backoff
- **Circuit Breaker Pattern**: Fail-fast for unhealthy clusters
- **Graceful Degradation**: Fallback to Typesense when OpenSearch unavailable
- **Graceful Degradation**: Graceful handling when OpenSearch unavailable
- **Detailed Error Logging**: Comprehensive error tracking
## 🚀 Usage
@@ -136,13 +136,13 @@ Access health information:
- **OpenSearch Specific**: `/actuator/health/opensearch`
- **Detailed Metrics**: Available when `enable-metrics: true`
## 🔄 Migration Strategy
## 🔄 Deployment Strategy
The configuration supports parallel operation with Typesense:
Recommended deployment approach:
1. **Development**: Test OpenSearch alongside Typesense
2. **Staging**: Validate performance and accuracy
3. **Production**: Gradual rollout with instant rollback capability
1. **Development**: Test OpenSearch configuration locally
2. **Staging**: Validate performance and accuracy in staging environment
3. **Production**: Deploy with proper monitoring and backup procedures
## 🛠️ Troubleshooting

6
backend/src/test/java/com/storycove/config/TestConfig.java
View File

@@ -1,12 +1,8 @@
package com.storycove.config;
import com.storycove.service.TypesenseService;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.boot.test.mock.mockito.MockBean;
@TestConfiguration
public class TestConfig {
@MockBean
public TypesenseService typesenseService;
// Test configuration
}

5
backend/src/test/java/com/storycove/service/AuthorServiceTest.java
View File

@@ -44,8 +44,9 @@ class AuthorServiceTest {
testAuthor.setId(testId);
testAuthor.setNotes("Test notes");
// Initialize service with null TypesenseService (which is allowed for tests)
authorService = new AuthorService(authorRepository, null);
// Initialize service with mock SearchServiceAdapter
SearchServiceAdapter mockSearchServiceAdapter = mock(SearchServiceAdapter.class);
authorService = new AuthorService(authorRepository, mockSearchServiceAdapter);
}
@Test

9
backend/src/test/java/com/storycove/service/StoryServiceTest.java
View File

@@ -33,6 +33,9 @@ class StoryServiceTest {
@Mock
private ReadingPositionRepository readingPositionRepository;
@Mock
private SearchServiceAdapter searchServiceAdapter;
private StoryService storyService;
private Story testStory;
private UUID testId;
@@ -44,16 +47,16 @@ class StoryServiceTest {
testStory.setId(testId);
testStory.setContentHtml("<p>Test content for reading progress tracking</p>");
// Create StoryService with only required repositories, all services can be null for these tests
// Create StoryService with mocked dependencies
storyService = new StoryService(
storyRepository,
tagRepository,
readingPositionRepository, // added for foreign key constraint handling
readingPositionRepository,
null, // authorService - not needed for reading progress tests
null, // tagService - not needed for reading progress tests
null, // seriesService - not needed for reading progress tests
null, // sanitizationService - not needed for reading progress tests
null // typesenseService - will test both with and without
searchServiceAdapter
);
}

9
backend/src/test/resources/application-test.yml
View File

@@ -18,11 +18,12 @@ storycove:
expiration: 86400000
auth:
password: test-password
typesense:
enabled: false
api-key: test-key
search:
engine: opensearch
opensearch:
host: localhost
port: 8108
port: 9200
scheme: http
images:
storage-path: /tmp/test-images

4308
backend/test_results.log Normal file
View File

File diff suppressed because it is too large Load Diff

23
docker-compose.yml
View File

@@ -34,9 +34,6 @@ services:
- SPRING_DATASOURCE_USERNAME=storycove
- SPRING_DATASOURCE_PASSWORD=${DB_PASSWORD}
- JWT_SECRET=${JWT_SECRET}
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
- TYPESENSE_HOST=typesense
- TYPESENSE_PORT=8108
- OPENSEARCH_HOST=opensearch
- OPENSEARCH_PORT=9200
- OPENSEARCH_SCHEME=http
@@ -49,7 +46,6 @@ services:
- library_config:/app/config
depends_on:
- postgres
- typesense
- opensearch
networks:
- storycove-network
@@ -68,16 +64,6 @@ services:
networks:
- storycove-network
typesense:
image: typesense/typesense:29.0
# No port mapping - only accessible within the Docker network
environment:
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
- TYPESENSE_DATA_DIR=/data
volumes:
- typesense_data:/data
networks:
- storycove-network
opensearch:
image: opensearchproject/opensearch:3.2.0
@@ -117,7 +103,6 @@ services:
volumes:
postgres_data:
typesense_data:
opensearch_data:
images_data:
library_config:
@@ -164,13 +149,5 @@ configs:
expires 1y;
add_header Cache-Control public;
}
location /typesense/ {
proxy_pass http://typesense:8108/;
proxy_set_header Host $$host;
proxy_set_header X-Real-IP $$remote_addr;
proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $$scheme;
proxy_set_header X-Typesense-API-Key $$http_x_typesense_api_key;
}
}
}

53
frontend/src/app/authors/page.tsx
View File

@@ -22,7 +22,7 @@ export default function AuthorsPage() {
const [currentPage, setCurrentPage] = useState(0);
const [totalHits, setTotalHits] = useState(0);
const [hasMore, setHasMore] = useState(false);
const ITEMS_PER_PAGE = 50; // Safe limit under Typesense's 250 limit
const ITEMS_PER_PAGE = 50;
useEffect(() => {
const debounceTimer = setTimeout(() => {
@@ -35,41 +35,30 @@ export default function AuthorsPage() {
} else {
setSearchLoading(true);
}
const searchResults = await authorApi.searchAuthorsTypesense({
q: searchQuery || '*',
const searchResults = await authorApi.getAuthors({
page: currentPage,
size: ITEMS_PER_PAGE,
sortBy: sortBy,
sortOrder: sortOrder
sortDir: sortOrder
});
if (currentPage === 0) {
// First page - replace all results
setAuthors(searchResults.results || []);
setFilteredAuthors(searchResults.results || []);
setAuthors(searchResults.content || []);
setFilteredAuthors(searchResults.content || []);
} else {
// Subsequent pages - append results
setAuthors(prev => [...prev, ...(searchResults.results || [])]);
setFilteredAuthors(prev => [...prev, ...(searchResults.results || [])]);
setAuthors(prev => [...prev, ...(searchResults.content || [])]);
setFilteredAuthors(prev => [...prev, ...(searchResults.content || [])]);
}
setTotalHits(searchResults.totalHits);
setHasMore(searchResults.results.length === ITEMS_PER_PAGE && (currentPage + 1) * ITEMS_PER_PAGE < searchResults.totalHits);
setTotalHits(searchResults.totalElements || 0);
setHasMore(searchResults.content.length === ITEMS_PER_PAGE && (currentPage + 1) * ITEMS_PER_PAGE < (searchResults.totalElements || 0));
} catch (error) {
console.error('Failed to load authors:', error);
// Fallback to regular API if Typesense fails (only for first page)
if (currentPage === 0) {
try {
const authorsResult = await authorApi.getAuthors({ page: 0, size: ITEMS_PER_PAGE });
setAuthors(authorsResult.content || []);
setFilteredAuthors(authorsResult.content || []);
setTotalHits(authorsResult.totalElements || 0);
setHasMore(authorsResult.content.length === ITEMS_PER_PAGE);
} catch (fallbackError) {
console.error('Fallback also failed:', fallbackError);
}
}
// Error handling for API failures
console.error('Failed to load authors:', error);
} finally {
setLoading(false);
setSearchLoading(false);
@@ -95,7 +84,17 @@ export default function AuthorsPage() {
}
};
// Client-side filtering no longer needed since we use Typesense
// Client-side filtering for search query when using regular API
useEffect(() => {
if (searchQuery) {
const filtered = authors.filter(author =>
author.name.toLowerCase().includes(searchQuery.toLowerCase())
);
setFilteredAuthors(filtered);
} else {
setFilteredAuthors(authors);
}
}, [authors, searchQuery]);
// Note: We no longer have individual story ratings in the author list
// Average rating would need to be calculated on backend if needed
@@ -118,9 +117,9 @@ export default function AuthorsPage() {
<div>
<h1 className="text-3xl font-bold theme-header">Authors</h1>
<p className="theme-text mt-1">
{filteredAuthors.length} of {totalHits} {totalHits === 1 ? 'author' : 'authors'}
{searchQuery ? `${filteredAuthors.length} of ${authors.length}` : filteredAuthors.length} {(searchQuery ? authors.length : filteredAuthors.length) === 1 ? 'author' : 'authors'}
{searchQuery ? ` found` : ` in your library`}
{hasMore && ` (showing first ${filteredAuthors.length})`}
{!searchQuery && hasMore && ` (showing first ${filteredAuthors.length})`}
</p>
</div>
@@ -218,7 +217,7 @@ export default function AuthorsPage() {
)}
{/* Load More Button */}
{hasMore && (
{hasMore && !searchQuery && (
<div className="flex justify-center pt-8">
<Button
onClick={loadMore}
@@ -227,7 +226,7 @@ export default function AuthorsPage() {
className="px-8 py-3"
loading={loading}
>
{loading ? 'Loading...' : `Load More Authors (${totalHits - filteredAuthors.length} remaining)`}
{loading ? 'Loading...' : `Load More Authors (${totalHits - authors.length} remaining)`}
</Button>
</div>
)}

12
frontend/src/app/scrape/bulk/route.ts
View File

@@ -501,11 +501,11 @@ async function processIndividualMode(
console.log(`Bulk import completed: ${importedCount} imported, ${skippedCount} skipped, ${errorCount} errors`);
// Trigger Typesense reindex if any stories were imported
// Trigger OpenSearch reindex if any stories were imported
if (importedCount > 0) {
try {
console.log('Triggering Typesense reindex after bulk import...');
const reindexUrl = `http://backend:8080/api/stories/reindex-typesense`;
console.log('Triggering OpenSearch reindex after bulk import...');
const reindexUrl = `http://backend:8080/api/admin/search/opensearch/reindex`;
const reindexResponse = await fetch(reindexUrl, {
method: 'POST',
headers: {
@@ -516,12 +516,12 @@ async function processIndividualMode(
if (reindexResponse.ok) {
const reindexResult = await reindexResponse.json();
console.log('Typesense reindex completed:', reindexResult);
console.log('OpenSearch reindex completed:', reindexResult);
} else {
console.warn('Typesense reindex failed:', reindexResponse.status);
console.warn('OpenSearch reindex failed:', reindexResponse.status);
}
} catch (error) {
console.warn('Failed to trigger Typesense reindex:', error);
console.warn('Failed to trigger OpenSearch reindex:', error);
// Don't fail the whole request if reindex fails
}
}

356
frontend/src/components/settings/SystemSettings.tsx
View File

@@ -2,7 +2,7 @@
import React, { useState, useEffect } from 'react';
import Button from '../ui/Button';
import { storyApi, authorApi, databaseApi, configApi, searchAdminApi } from '../../lib/api';
import { databaseApi, configApi, searchAdminApi } from '../../lib/api';
interface SystemSettingsProps {
// No props needed - this component manages its own state
@@ -11,16 +11,12 @@ interface SystemSettingsProps {
export default function SystemSettings({}: SystemSettingsProps) {
const [searchEngineStatus, setSearchEngineStatus] = useState<{
currentEngine: string;
dualWrite: boolean;
typesenseAvailable: boolean;
openSearchAvailable: boolean;
loading: boolean;
message: string;
success?: boolean;
}>({
currentEngine: 'typesense',
dualWrite: false,
typesenseAvailable: false,
currentEngine: 'opensearch',
openSearchAvailable: false,
loading: false,
message: ''
@@ -34,13 +30,6 @@ export default function SystemSettings({}: SystemSettingsProps) {
recreate: { loading: false, message: '' }
});
const [typesenseStatus, setTypesenseStatus] = useState<{
reindex: { loading: boolean; message: string; success?: boolean };
recreate: { loading: boolean; message: string; success?: boolean };
}>({
reindex: { loading: false, message: '' },
recreate: { loading: false, message: '' }
});
const [databaseStatus, setDatabaseStatus] = useState<{
completeBackup: { loading: boolean; message: string; success?: boolean };
completeRestore: { loading: boolean; message: string; success?: boolean };
@@ -58,135 +47,7 @@ export default function SystemSettings({}: SystemSettingsProps) {
execute: { loading: false, message: '' }
});
const handleFullReindex = async () => {
setTypesenseStatus(prev => ({
...prev,
reindex: { loading: true, message: 'Reindexing all collections...', success: undefined }
}));
try {
// Run both story and author reindex in parallel
const [storiesResult, authorsResult] = await Promise.all([
storyApi.reindexTypesense(),
authorApi.reindexTypesense()
]);
const allSuccessful = storiesResult.success && authorsResult.success;
const messages: string[] = [];
if (storiesResult.success) {
messages.push(`Stories: ${storiesResult.message}`);
} else {
messages.push(`Stories failed: ${storiesResult.error || 'Unknown error'}`);
}
if (authorsResult.success) {
messages.push(`Authors: ${authorsResult.message}`);
} else {
messages.push(`Authors failed: ${authorsResult.error || 'Unknown error'}`);
}
setTypesenseStatus(prev => ({
...prev,
reindex: {
loading: false,
message: allSuccessful
? `Full reindex completed successfully. ${messages.join(', ')}`
: `Reindex completed with errors. ${messages.join(', ')}`,
success: allSuccessful
}
}));
// Clear message after 8 seconds (longer for combined operation)
setTimeout(() => {
setTypesenseStatus(prev => ({
...prev,
reindex: { loading: false, message: '', success: undefined }
}));
}, 8000);
} catch (error) {
setTypesenseStatus(prev => ({
...prev,
reindex: {
loading: false,
message: 'Network error occurred during reindex',
success: false
}
}));
setTimeout(() => {
setTypesenseStatus(prev => ({
...prev,
reindex: { loading: false, message: '', success: undefined }
}));
}, 8000);
}
};
const handleRecreateAllCollections = async () => {
setTypesenseStatus(prev => ({
...prev,
recreate: { loading: true, message: 'Recreating all collections...', success: undefined }
}));
try {
// Run both story and author recreation in parallel
const [storiesResult, authorsResult] = await Promise.all([
storyApi.recreateTypesenseCollection(),
authorApi.recreateTypesenseCollection()
]);
const allSuccessful = storiesResult.success && authorsResult.success;
const messages: string[] = [];
if (storiesResult.success) {
messages.push(`Stories: ${storiesResult.message}`);
} else {
messages.push(`Stories failed: ${storiesResult.error || 'Unknown error'}`);
}
if (authorsResult.success) {
messages.push(`Authors: ${authorsResult.message}`);
} else {
messages.push(`Authors failed: ${authorsResult.error || 'Unknown error'}`);
}
setTypesenseStatus(prev => ({
...prev,
recreate: {
loading: false,
message: allSuccessful
? `All collections recreated successfully. ${messages.join(', ')}`
: `Recreation completed with errors. ${messages.join(', ')}`,
success: allSuccessful
}
}));
// Clear message after 8 seconds (longer for combined operation)
setTimeout(() => {
setTypesenseStatus(prev => ({
...prev,
recreate: { loading: false, message: '', success: undefined }
}));
}, 8000);
} catch (error) {
setTypesenseStatus(prev => ({
...prev,
recreate: {
loading: false,
message: 'Network error occurred during recreation',
success: false
}
}));
setTimeout(() => {
setTypesenseStatus(prev => ({
...prev,
recreate: { loading: false, message: '', success: undefined }
}));
}, 8000);
}
};
const handleCompleteBackup = async () => {
setDatabaseStatus(prev => ({
@@ -451,8 +312,6 @@ export default function SystemSettings({}: SystemSettingsProps) {
setSearchEngineStatus(prev => ({
...prev,
currentEngine: status.primaryEngine,
dualWrite: status.dualWrite,
typesenseAvailable: status.typesenseAvailable,
openSearchAvailable: status.openSearchAvailable,
}));
} catch (error: any) {
@@ -460,76 +319,7 @@ export default function SystemSettings({}: SystemSettingsProps) {
}
};
const handleSwitchEngine = async (engine: string) => {
setSearchEngineStatus(prev => ({ ...prev, loading: true, message: `Switching to ${engine}...` }));
try {
const result = engine === 'opensearch'
? await searchAdminApi.switchToOpenSearch()
: await searchAdminApi.switchToTypesense();
setSearchEngineStatus(prev => ({
...prev,
loading: false,
message: result.message,
success: true,
currentEngine: engine
}));
setTimeout(() => {
setSearchEngineStatus(prev => ({ ...prev, message: '', success: undefined }));
}, 5000);
} catch (error: any) {
setSearchEngineStatus(prev => ({
...prev,
loading: false,
message: error.message || 'Failed to switch engine',
success: false
}));
setTimeout(() => {
setSearchEngineStatus(prev => ({ ...prev, message: '', success: undefined }));
}, 5000);
}
};
const handleToggleDualWrite = async () => {
const newDualWrite = !searchEngineStatus.dualWrite;
setSearchEngineStatus(prev => ({
...prev,
loading: true,
message: `${newDualWrite ? 'Enabling' : 'Disabling'} dual-write...`
}));
try {
const result = newDualWrite
? await searchAdminApi.enableDualWrite()
: await searchAdminApi.disableDualWrite();
setSearchEngineStatus(prev => ({
...prev,
loading: false,
message: result.message,
success: true,
dualWrite: newDualWrite
}));
setTimeout(() => {
setSearchEngineStatus(prev => ({ ...prev, message: '', success: undefined }));
}, 5000);
} catch (error: any) {
setSearchEngineStatus(prev => ({
...prev,
loading: false,
message: error.message || 'Failed to toggle dual-write',
success: false
}));
setTimeout(() => {
setSearchEngineStatus(prev => ({ ...prev, message: '', success: undefined }));
}, 5000);
}
};
const handleOpenSearchReindex = async () => {
setOpenSearchStatus(prev => ({
@@ -624,36 +414,18 @@ export default function SystemSettings({}: SystemSettingsProps) {
return (
<div className="space-y-6">
{/* Search Engine Management */}
{/* Search Management */}
<div className="theme-card theme-shadow rounded-lg p-6">
<h2 className="text-xl font-semibold theme-header mb-4">Search Engine Migration</h2>
<h2 className="text-xl font-semibold theme-header mb-4">Search Management</h2>
<p className="theme-text mb-6">
Manage the transition from Typesense to OpenSearch. Switch between engines, enable dual-write mode, and perform maintenance operations.
Manage OpenSearch indices for stories and authors. Use these tools if search isn't returning expected results.
</p>
<div className="space-y-6">
{/* Current Status */}
<div className="border theme-border rounded-lg p-4">
<h3 className="text-lg font-semibold theme-header mb-3">Current Configuration</h3>
<h3 className="text-lg font-semibold theme-header mb-3">Search Status</h3>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 text-sm">
<div className="flex justify-between">
<span>Primary Engine:</span>
<span className={`font-medium ${searchEngineStatus.currentEngine === 'opensearch' ? 'text-blue-600 dark:text-blue-400' : 'text-green-600 dark:text-green-400'}`}>
{searchEngineStatus.currentEngine.charAt(0).toUpperCase() + searchEngineStatus.currentEngine.slice(1)}
</span>
</div>
<div className="flex justify-between">
<span>Dual-Write:</span>
<span className={`font-medium ${searchEngineStatus.dualWrite ? 'text-orange-600 dark:text-orange-400' : 'text-gray-600 dark:text-gray-400'}`}>
{searchEngineStatus.dualWrite ? 'Enabled' : 'Disabled'}
</span>
</div>
<div className="flex justify-between">
<span>Typesense:</span>
<span className={`font-medium ${searchEngineStatus.typesenseAvailable ? 'text-green-600 dark:text-green-400' : 'text-red-600 dark:text-red-400'}`}>
{searchEngineStatus.typesenseAvailable ? 'Available' : 'Unavailable'}
</span>
</div>
<div className="flex justify-between">
<span>OpenSearch:</span>
<span className={`font-medium ${searchEngineStatus.openSearchAvailable ? 'text-green-600 dark:text-green-400' : 'text-red-600 dark:text-red-400'}`}>
@@ -663,52 +435,11 @@ export default function SystemSettings({}: SystemSettingsProps) {
</div>
</div>
{/* Engine Switching */}
{/* Search Operations */}
<div className="border theme-border rounded-lg p-4">
<h3 className="text-lg font-semibold theme-header mb-3">Engine Controls</h3>
<div className="flex flex-col sm:flex-row gap-3 mb-4">
<Button
onClick={() => handleSwitchEngine('typesense')}
disabled={searchEngineStatus.loading || !searchEngineStatus.typesenseAvailable || searchEngineStatus.currentEngine === 'typesense'}
variant={searchEngineStatus.currentEngine === 'typesense' ? 'primary' : 'ghost'}
className="flex-1"
>
{searchEngineStatus.currentEngine === 'typesense' ? '✓ Typesense (Active)' : 'Switch to Typesense'}
</Button>
<Button
onClick={() => handleSwitchEngine('opensearch')}
disabled={searchEngineStatus.loading || !searchEngineStatus.openSearchAvailable || searchEngineStatus.currentEngine === 'opensearch'}
variant={searchEngineStatus.currentEngine === 'opensearch' ? 'primary' : 'ghost'}
className="flex-1"
>
{searchEngineStatus.currentEngine === 'opensearch' ? '✓ OpenSearch (Active)' : 'Switch to OpenSearch'}
</Button>
<Button
onClick={handleToggleDualWrite}
disabled={searchEngineStatus.loading}
variant={searchEngineStatus.dualWrite ? 'secondary' : 'ghost'}
className="flex-1"
>
{searchEngineStatus.dualWrite ? 'Disable Dual-Write' : 'Enable Dual-Write'}
</Button>
</div>
{searchEngineStatus.message && (
<div className={`text-sm p-3 rounded mb-3 ${
searchEngineStatus.success
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
}`}>
{searchEngineStatus.message}
</div>
)}
</div>
{/* OpenSearch Operations */}
<div className="border theme-border rounded-lg p-4">
<h3 className="text-lg font-semibold theme-header mb-3">OpenSearch Operations</h3>
<h3 className="text-lg font-semibold theme-header mb-3">Search Operations</h3>
<p className="text-sm theme-text mb-4">
Perform maintenance operations on OpenSearch indices. Use these if OpenSearch isn't returning expected results.
Perform maintenance operations on search indices. Use these if search isn't returning expected results.
</p>
<div className="flex flex-col sm:flex-row gap-3 mb-4">
@@ -719,7 +450,7 @@ export default function SystemSettings({}: SystemSettingsProps) {
variant="ghost"
className="flex-1"
>
{openSearchStatus.reindex.loading ? 'Reindexing...' : '🔄 Reindex OpenSearch'}
{openSearchStatus.reindex.loading ? 'Reindexing...' : '🔄 Reindex All'}
</Button>
<Button
onClick={handleOpenSearchRecreate}
@@ -732,7 +463,7 @@ export default function SystemSettings({}: SystemSettingsProps) {
</Button>
</div>
{/* OpenSearch Status Messages */}
{/* Status Messages */}
{openSearchStatus.reindex.message && (
<div className={`text-sm p-3 rounded mb-3 ${
openSearchStatus.reindex.success
@@ -753,73 +484,12 @@ export default function SystemSettings({}: SystemSettingsProps) {
</div>
)}
</div>
</div>
</div>
{/* Legacy Typesense Management */}
<div className="theme-card theme-shadow rounded-lg p-6">
<h2 className="text-xl font-semibold theme-header mb-4">Legacy Typesense Management</h2>
<p className="theme-text mb-6">
Manage Typesense search indexes (for backwards compatibility and during migration). These tools will be removed once migration is complete.
</p>
<div className="space-y-6">
{/* Simplified Operations */}
<div className="border theme-border rounded-lg p-4">
<h3 className="text-lg font-semibold theme-header mb-3">Search Operations</h3>
<p className="text-sm theme-text mb-4">
Perform maintenance operations on all search indexes (stories, authors, collections, etc.).
</p>
<div className="flex flex-col sm:flex-row gap-3 mb-4">
<Button
onClick={handleFullReindex}
disabled={typesenseStatus.reindex.loading || typesenseStatus.recreate.loading}
loading={typesenseStatus.reindex.loading}
variant="ghost"
className="flex-1"
>
{typesenseStatus.reindex.loading ? 'Reindexing All...' : '🔄 Full Reindex'}
</Button>
<Button
onClick={handleRecreateAllCollections}
disabled={typesenseStatus.reindex.loading || typesenseStatus.recreate.loading}
loading={typesenseStatus.recreate.loading}
variant="secondary"
className="flex-1"
>
{typesenseStatus.recreate.loading ? 'Recreating All...' : '🏗️ Recreate All Collections'}
</Button>
</div>
{/* Status Messages */}
{typesenseStatus.reindex.message && (
<div className={`text-sm p-3 rounded mb-3 ${
typesenseStatus.reindex.success
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
}`}>
{typesenseStatus.reindex.message}
</div>
)}
{typesenseStatus.recreate.message && (
<div className={`text-sm p-3 rounded mb-3 ${
typesenseStatus.recreate.success
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
}`}>
{typesenseStatus.recreate.message}
</div>
)}
</div>
<div className="text-sm theme-text bg-blue-50 dark:bg-blue-900/20 p-3 rounded-lg">
<p className="font-medium mb-1">When to use these tools:</p>
<ul className="text-xs space-y-1 ml-4">
<li>• <strong>Full Reindex:</strong> Refresh all search data while keeping existing schemas (fixes data sync issues)</li>
<li>• <strong>Recreate All Collections:</strong> Delete and rebuild all search indexes from scratch (fixes schema and structure issues)</li>
<li>• <strong>Operations run in parallel</strong> across all index types for better performance</li>
<li>• <strong>Reindex All:</strong> Refresh all search data while keeping existing schemas (fixes data sync issues)</li>
<li>• <strong>Recreate Indices:</strong> Delete and rebuild all search indexes from scratch (fixes schema and structure issues)</li>
</ul>
</div>
</div>

46
frontend/src/lib/api.ts
View File

@@ -179,15 +179,6 @@ export const storyApi = {
return response.data;
},
reindexTypesense: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
const response = await api.post('/stories/reindex-typesense');
return response.data;
},
recreateTypesenseCollection: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
const response = await api.post('/stories/recreate-typesense-collection');
return response.data;
},
checkDuplicate: async (title: string, authorName: string): Promise<{
hasDuplicates: boolean;
@@ -305,38 +296,6 @@ export const authorApi = {
await api.delete(`/authors/${id}/avatar`);
},
searchAuthorsTypesense: async (params?: {
q?: string;
page?: number;
size?: number;
sortBy?: string;
sortOrder?: string;
}): Promise<{
results: Author[];
totalHits: number;
page: number;
perPage: number;
query: string;
searchTimeMs: number;
}> => {
const response = await api.get('/authors/search-typesense', { params });
return response.data;
},
reindexTypesense: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
const response = await api.post('/authors/reindex-typesense');
return response.data;
},
recreateTypesenseCollection: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
const response = await api.post('/authors/recreate-typesense-collection');
return response.data;
},
getTypesenseSchema: async (): Promise<{ success: boolean; schema?: any; error?: string }> => {
const response = await api.get('/authors/typesense-schema');
return response.data;
},
};
// Tag endpoints
@@ -617,7 +576,6 @@ export const searchAdminApi = {
getStatus: async (): Promise<{
primaryEngine: string;
dualWrite: boolean;
typesenseAvailable: boolean;
openSearchAvailable: boolean;
}> => {
const response = await api.get('/admin/search/status');
@@ -647,10 +605,6 @@ export const searchAdminApi = {
return response.data;
},
switchToTypesense: async (): Promise<{ message: string }> => {
const response = await api.post('/admin/search/switch/typesense');
return response.data;
},
// Emergency rollback
emergencyRollback: async (): Promise<{ message: string }> => {

2
frontend/tsconfig.tsbuildinfo
View File

File diff suppressed because one or more lines are too long