Compare commits
6 Commits
feature/em
...
feature/op
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aae8f8926b | ||
|
|
f1773873d4 | ||
|
|
54df3c471e | ||
|
|
64f97f5648 | ||
|
|
c0b3ae3b72 | ||
|
|
e5596b5a17 |
@@ -14,11 +14,18 @@ JWT_SECRET=secure_jwt_secret_here
|
||||
# Application Authentication
|
||||
APP_PASSWORD=application_password_here
|
||||
|
||||
# Search Engine Configuration
|
||||
SEARCH_ENGINE=typesense
|
||||
|
||||
# Typesense Search Configuration
|
||||
TYPESENSE_API_KEY=secure_api_key_here
|
||||
TYPESENSE_ENABLED=true
|
||||
TYPESENSE_REINDEX_INTERVAL=3600000
|
||||
|
||||
# OpenSearch Configuration
|
||||
OPENSEARCH_USERNAME=admin
|
||||
OPENSEARCH_PASSWORD=secure_opensearch_password_here
|
||||
|
||||
# Image Storage
|
||||
IMAGE_STORAGE_PATH=/app/images
|
||||
|
||||
|
||||
@@ -18,10 +18,9 @@ JWT_SECRET=REPLACE_WITH_SECURE_JWT_SECRET_MINIMUM_32_CHARS
|
||||
# Use a strong password in production
|
||||
APP_PASSWORD=REPLACE_WITH_SECURE_APP_PASSWORD
|
||||
|
||||
# Typesense Search Configuration
|
||||
TYPESENSE_API_KEY=REPLACE_WITH_SECURE_TYPESENSE_API_KEY
|
||||
TYPESENSE_ENABLED=true
|
||||
TYPESENSE_REINDEX_INTERVAL=3600000
|
||||
# OpenSearch Configuration
|
||||
OPENSEARCH_PASSWORD=REPLACE_WITH_SECURE_OPENSEARCH_PASSWORD
|
||||
SEARCH_ENGINE=opensearch
|
||||
|
||||
# Image Storage
|
||||
IMAGE_STORAGE_PATH=/app/images
|
||||
|
||||
889
OPENSEARCH_MIGRATION_SPECIFICATION.md
Normal file
889
OPENSEARCH_MIGRATION_SPECIFICATION.md
Normal file
@@ -0,0 +1,889 @@
|
||||
# StoryCove Search Migration Specification: Typesense to OpenSearch
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document specifies the migration from Typesense to OpenSearch for the StoryCove application. The migration will be implemented using a parallel approach, maintaining Typesense functionality while gradually transitioning to OpenSearch, ensuring zero downtime and the ability to rollback if needed.
|
||||
|
||||
**Migration Goals:**
|
||||
- Solve random query reliability issues
|
||||
- Improve complex filtering performance
|
||||
- Maintain feature parity during transition
|
||||
- Zero downtime migration
|
||||
- Improved developer experience
|
||||
|
||||
---
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### Typesense Implementation Overview
|
||||
|
||||
**Service Architecture:**
|
||||
- `TypesenseService.java` (~2000 lines) - Primary search service
|
||||
- 3 search indexes: Stories, Authors, Collections
|
||||
- Multi-library support with dynamic collection names
|
||||
- Integration with Spring Boot backend
|
||||
|
||||
**Core Functionality:**
|
||||
1. **Full-text Search**: Stories, Authors with complex query building
|
||||
2. **Random Story Selection**: `_rand()` function with fallback logic
|
||||
3. **Advanced Filtering**: 15+ filter conditions with boolean logic
|
||||
4. **Faceting**: Tag aggregations and counts
|
||||
5. **Autocomplete**: Search suggestions with typeahead
|
||||
6. **CRUD Operations**: Index/update/delete for all entity types
|
||||
|
||||
**Current Issues Identified:**
|
||||
- `_rand()` function unreliability requiring complex fallback logic
|
||||
- Complex filter query building with escaping issues
|
||||
- Limited aggregation capabilities
|
||||
- Inconsistent API behavior across query patterns
|
||||
- Multi-collection management complexity
|
||||
|
||||
### Data Models and Schema
|
||||
|
||||
**Story Index Fields:**
|
||||
```java
|
||||
// Core fields
|
||||
UUID id, String title, String description, String sourceUrl
|
||||
Integer wordCount, Integer rating, Integer volume
|
||||
Boolean isRead, LocalDateTime lastReadAt, Integer readingPosition
|
||||
|
||||
// Relationships
|
||||
UUID authorId, String authorName
|
||||
UUID seriesId, String seriesName
|
||||
List<String> tagNames
|
||||
|
||||
// Metadata
|
||||
LocalDateTime createdAt, LocalDateTime updatedAt
|
||||
String coverPath, String sourceDomain
|
||||
```
|
||||
|
||||
**Author Index Fields:**
|
||||
```java
|
||||
UUID id, String name, String notes
|
||||
Integer authorRating, Double averageStoryRating, Integer storyCount
|
||||
List<String> urls, String avatarImagePath
|
||||
LocalDateTime createdAt, LocalDateTime updatedAt
|
||||
```
|
||||
|
||||
**Collection Index Fields:**
|
||||
```java
|
||||
UUID id, String name, String description
|
||||
List<String> tagNames, Boolean archived
|
||||
LocalDateTime createdAt, LocalDateTime updatedAt
|
||||
Integer storyCount, Integer currentPosition
|
||||
```
|
||||
|
||||
### API Endpoints Current State
|
||||
|
||||
**Search Endpoints Analysis:**
|
||||
|
||||
**✅ USED by Frontend (Must Implement):**
|
||||
- `GET /api/stories/search` - Main story search with complex filtering (CRITICAL)
|
||||
- `GET /api/stories/random` - Random story selection with filters (CRITICAL)
|
||||
- `GET /api/authors/search-typesense` - Author search (HIGH)
|
||||
- `GET /api/tags/autocomplete` - Tag suggestions (MEDIUM)
|
||||
- `POST /api/stories/reindex-typesense` - Admin reindex operations (MEDIUM)
|
||||
- `POST /api/authors/reindex-typesense` - Admin reindex operations (MEDIUM)
|
||||
- `POST /api/stories/recreate-typesense-collection` - Admin recreate (MEDIUM)
|
||||
- `POST /api/authors/recreate-typesense-collection` - Admin recreate (MEDIUM)
|
||||
|
||||
**❌ UNUSED by Frontend (Skip Implementation):**
|
||||
- `GET /api/stories/search/suggestions` - Not used by frontend
|
||||
- `GET /api/authors/search` - Superseded by typesense version
|
||||
- `GET /api/series/search` - Not used by frontend
|
||||
- `GET /api/tags/search` - Superseded by autocomplete
|
||||
- `POST /api/search/reindex` - Not used by frontend
|
||||
- `GET /api/search/health` - Not used by frontend
|
||||
|
||||
**Scope Reduction: ~40% fewer endpoints to implement**
|
||||
|
||||
**Search Parameters (Stories):**
|
||||
```
|
||||
query, page, size, authors[], tags[], minRating, maxRating
|
||||
sortBy, sortDir, facetBy[]
|
||||
minWordCount, maxWordCount, createdAfter, createdBefore
|
||||
lastReadAfter, lastReadBefore, unratedOnly, readingStatus
|
||||
hasReadingProgress, hasCoverImage, sourceDomain, seriesFilter
|
||||
minTagCount, popularOnly, hiddenGemsOnly
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Target OpenSearch Architecture
|
||||
|
||||
### Service Layer Design
|
||||
|
||||
**New Components:**
|
||||
```
|
||||
OpenSearchService.java - Primary search service (mirrors TypesenseService API)
|
||||
OpenSearchConfig.java - Configuration and client setup
|
||||
SearchMigrationService.java - Handles parallel operation during migration
|
||||
SearchServiceAdapter.java - Abstraction layer for service switching
|
||||
```
|
||||
|
||||
**Index Strategy:**
|
||||
- **Single-node deployment** for development/small installations
|
||||
- **Index-per-library** approach: `stories-{libraryId}`, `authors-{libraryId}`, `collections-{libraryId}`
|
||||
- **Index templates** for consistent mapping across libraries
|
||||
- **Aliases** for easy switching and zero-downtime updates
|
||||
|
||||
### OpenSearch Index Mappings
|
||||
|
||||
**Stories Index Mapping:**
|
||||
```json
|
||||
{
|
||||
"settings": {
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 0,
|
||||
"analysis": {
|
||||
"analyzer": {
|
||||
"story_analyzer": {
|
||||
"type": "custom",
|
||||
"tokenizer": "standard",
|
||||
"filter": ["lowercase", "stop", "snowball"]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {"type": "keyword"},
|
||||
"title": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {"keyword": {"type": "keyword"}}
|
||||
},
|
||||
"description": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer"
|
||||
},
|
||||
"authorName": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {"keyword": {"type": "keyword"}}
|
||||
},
|
||||
"seriesName": {
|
||||
"type": "text",
|
||||
"fields": {"keyword": {"type": "keyword"}}
|
||||
},
|
||||
"tagNames": {"type": "keyword"},
|
||||
"wordCount": {"type": "integer"},
|
||||
"rating": {"type": "integer"},
|
||||
"volume": {"type": "integer"},
|
||||
"isRead": {"type": "boolean"},
|
||||
"readingPosition": {"type": "integer"},
|
||||
"lastReadAt": {"type": "date"},
|
||||
"createdAt": {"type": "date"},
|
||||
"updatedAt": {"type": "date"},
|
||||
"coverPath": {"type": "keyword"},
|
||||
"sourceUrl": {"type": "keyword"},
|
||||
"sourceDomain": {"type": "keyword"}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Authors Index Mapping:**
|
||||
```json
|
||||
{
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {"type": "keyword"},
|
||||
"name": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {"keyword": {"type": "keyword"}}
|
||||
},
|
||||
"notes": {"type": "text"},
|
||||
"authorRating": {"type": "integer"},
|
||||
"averageStoryRating": {"type": "float"},
|
||||
"storyCount": {"type": "integer"},
|
||||
"urls": {"type": "keyword"},
|
||||
"avatarImagePath": {"type": "keyword"},
|
||||
"createdAt": {"type": "date"},
|
||||
"updatedAt": {"type": "date"}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Collections Index Mapping:**
|
||||
```json
|
||||
{
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {"type": "keyword"},
|
||||
"name": {
|
||||
"type": "text",
|
||||
"fields": {"keyword": {"type": "keyword"}}
|
||||
},
|
||||
"description": {"type": "text"},
|
||||
"tagNames": {"type": "keyword"},
|
||||
"archived": {"type": "boolean"},
|
||||
"storyCount": {"type": "integer"},
|
||||
"currentPosition": {"type": "integer"},
|
||||
"createdAt": {"type": "date"},
|
||||
"updatedAt": {"type": "date"}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Query Translation Strategy
|
||||
|
||||
**Random Story Queries:**
|
||||
```java
|
||||
// Typesense (problematic)
|
||||
String sortBy = seed != null ? "_rand(" + seed + ")" : "_rand()";
|
||||
|
||||
// OpenSearch (reliable)
|
||||
QueryBuilder randomQuery = QueryBuilders.functionScoreQuery(
|
||||
QueryBuilders.boolQuery().must(filters),
|
||||
ScoreFunctionBuilders.randomFunction(seed != null ? seed.intValue() : null)
|
||||
);
|
||||
```
|
||||
|
||||
**Complex Filtering:**
|
||||
```java
|
||||
// Build bool query with multiple filter conditions
|
||||
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery()
|
||||
.must(QueryBuilders.multiMatchQuery(query, "title", "description", "authorName"))
|
||||
.filter(QueryBuilders.termsQuery("tagNames", tags))
|
||||
.filter(QueryBuilders.rangeQuery("wordCount").gte(minWords).lte(maxWords))
|
||||
.filter(QueryBuilders.rangeQuery("rating").gte(minRating).lte(maxRating));
|
||||
```
|
||||
|
||||
**Faceting/Aggregations:**
|
||||
```java
|
||||
// Tags aggregation
|
||||
AggregationBuilder tagsAgg = AggregationBuilders
|
||||
.terms("tags")
|
||||
.field("tagNames")
|
||||
.size(100);
|
||||
|
||||
// Rating ranges
|
||||
AggregationBuilder ratingRanges = AggregationBuilders
|
||||
.range("rating_ranges")
|
||||
.field("rating")
|
||||
.addRange("unrated", 0, 1)
|
||||
.addRange("low", 1, 3)
|
||||
.addRange("high", 4, 6);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Revised Implementation Phases (Scope Reduced by 40%)
|
||||
|
||||
### Phase 1: Infrastructure Setup (Week 1)
|
||||
|
||||
**Objectives:**
|
||||
- Add OpenSearch to Docker Compose
|
||||
- Create basic OpenSearch service
|
||||
- Establish index templates and mappings
|
||||
- **Focus**: Only stories, authors, and tags indexes (skip series, collections)
|
||||
|
||||
**Deliverables:**
|
||||
1. **Docker Compose Updates:**
|
||||
```yaml
|
||||
opensearch:
|
||||
image: opensearchproject/opensearch:2.11.0
|
||||
environment:
|
||||
- discovery.type=single-node
|
||||
- DISABLE_SECURITY_PLUGIN=true
|
||||
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx1g
|
||||
ports:
|
||||
- "9200:9200"
|
||||
volumes:
|
||||
- opensearch_data:/usr/share/opensearch/data
|
||||
```
|
||||
|
||||
2. **OpenSearchConfig.java:**
|
||||
```java
|
||||
@Configuration
|
||||
@ConditionalOnProperty(name = "storycove.opensearch.enabled", havingValue = "true")
|
||||
public class OpenSearchConfig {
|
||||
@Bean
|
||||
public OpenSearchClient openSearchClient() {
|
||||
// Client configuration
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Basic Index Creation:**
|
||||
- Create index templates for stories, authors, collections
|
||||
- Implement index creation with proper mappings
|
||||
- Add health check endpoint
|
||||
|
||||
**Success Criteria:**
|
||||
- OpenSearch container starts successfully
|
||||
- Basic connectivity established
|
||||
- Index templates created and validated
|
||||
|
||||
### Phase 2: Core Service Implementation (Week 2)
|
||||
|
||||
**Objectives:**
|
||||
- Implement OpenSearchService with core functionality
|
||||
- Create service abstraction layer
|
||||
- Implement basic search operations
|
||||
- **Focus**: Only critical endpoints (stories search, random, authors)
|
||||
|
||||
**Deliverables:**
|
||||
1. **OpenSearchService.java** - Core service implementing:
|
||||
- `indexStory()`, `updateStory()`, `deleteStory()`
|
||||
- `searchStories()` with basic query support (CRITICAL)
|
||||
- `getRandomStoryId()` with reliable seed support (CRITICAL)
|
||||
- `indexAuthor()`, `updateAuthor()`, `deleteAuthor()`
|
||||
- `searchAuthors()` for authors page (HIGH)
|
||||
- `bulkIndexStories()`, `bulkIndexAuthors()` for initial data loading
|
||||
|
||||
2. **SearchServiceAdapter.java** - Abstraction layer:
|
||||
```java
|
||||
@Service
|
||||
public class SearchServiceAdapter {
|
||||
@Autowired(required = false)
|
||||
private TypesenseService typesenseService;
|
||||
|
||||
@Autowired(required = false)
|
||||
private OpenSearchService openSearchService;
|
||||
|
||||
@Value("${storycove.search.provider:typesense}")
|
||||
private String searchProvider;
|
||||
|
||||
public SearchResultDto<StorySearchDto> searchStories(...) {
|
||||
return "opensearch".equals(searchProvider)
|
||||
? openSearchService.searchStories(...)
|
||||
: typesenseService.searchStories(...);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Basic Query Implementation:**
|
||||
- Full-text search across title/description/author
|
||||
- Basic filtering (tags, rating, word count)
|
||||
- Pagination and sorting
|
||||
|
||||
**Success Criteria:**
|
||||
- Basic search functionality working
|
||||
- Service abstraction layer functional
|
||||
- Can switch between Typesense and OpenSearch via configuration
|
||||
|
||||
### Phase 3: Advanced Features Implementation (Week 3)
|
||||
|
||||
**Objectives:**
|
||||
- Implement complex filtering (all 15+ filter types)
|
||||
- Add random story functionality
|
||||
- Implement faceting/aggregations
|
||||
- Add autocomplete/suggestions
|
||||
|
||||
**Deliverables:**
|
||||
1. **Complex Query Builder:**
|
||||
- All filter conditions from original implementation
|
||||
- Date range filtering with proper timezone handling
|
||||
- Boolean logic for reading status, coverage, series filters
|
||||
|
||||
2. **Random Story Implementation:**
|
||||
```java
|
||||
public Optional<UUID> getRandomStoryId(String searchQuery, List<String> tags, Long seed, ...) {
|
||||
BoolQueryBuilder baseQuery = buildFilterQuery(searchQuery, tags, ...);
|
||||
|
||||
QueryBuilder randomQuery = QueryBuilders.functionScoreQuery(
|
||||
baseQuery,
|
||||
ScoreFunctionBuilders.randomFunction(seed != null ? seed.intValue() : null)
|
||||
);
|
||||
|
||||
SearchRequest request = new SearchRequest("stories-" + getCurrentLibraryId())
|
||||
.source(new SearchSourceBuilder()
|
||||
.query(randomQuery)
|
||||
.size(1)
|
||||
.fetchSource(new String[]{"id"}, null));
|
||||
|
||||
// Execute and return result
|
||||
}
|
||||
```
|
||||
|
||||
3. **Faceting Implementation:**
|
||||
- Tag aggregations with counts
|
||||
- Rating range aggregations
|
||||
- Author aggregations
|
||||
- Custom facet builders
|
||||
|
||||
4. **Autocomplete Service:**
|
||||
- Suggest-based implementation using completion fields
|
||||
- Prefix matching for story titles and author names
|
||||
|
||||
**Success Criteria:**
|
||||
- All filter conditions working correctly
|
||||
- Random story selection reliable with seed support
|
||||
- Faceting returns accurate counts
|
||||
- Autocomplete responsive and accurate
|
||||
|
||||
### Phase 4: Data Migration & Parallel Operation (Week 4)
|
||||
|
||||
**Objectives:**
|
||||
- Implement bulk data migration from database
|
||||
- Enable parallel operation (write to both systems)
|
||||
- Comprehensive testing of OpenSearch functionality
|
||||
|
||||
**Deliverables:**
|
||||
1. **Migration Service:**
|
||||
```java
|
||||
@Service
|
||||
public class SearchMigrationService {
|
||||
public void performFullMigration() {
|
||||
// Migrate all libraries
|
||||
List<Library> libraries = libraryService.findAll();
|
||||
for (Library library : libraries) {
|
||||
migrateLibraryData(library);
|
||||
}
|
||||
}
|
||||
|
||||
private void migrateLibraryData(Library library) {
|
||||
// Create indexes for library
|
||||
// Bulk load stories, authors, collections
|
||||
// Verify data integrity
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **Dual-Write Implementation:**
|
||||
- Modify all entity update operations to write to both systems
|
||||
- Add configuration flag for dual-write mode
|
||||
- Error handling for partial failures
|
||||
|
||||
3. **Data Validation Tools:**
|
||||
- Compare search result counts between systems
|
||||
- Validate random story selection consistency
|
||||
- Check faceting accuracy
|
||||
|
||||
**Success Criteria:**
|
||||
- Complete data migration with 100% accuracy
|
||||
- Dual-write operations working without errors
|
||||
- Search result parity between systems verified
|
||||
|
||||
### Phase 5: API Integration & Testing (Week 5)
|
||||
|
||||
**Objectives:**
|
||||
- Update controller endpoints to use OpenSearch
|
||||
- Comprehensive integration testing
|
||||
- Performance testing and optimization
|
||||
|
||||
**Deliverables:**
|
||||
1. **Controller Updates:**
|
||||
- Modify controllers to use SearchServiceAdapter
|
||||
- Add migration controls for gradual rollout
|
||||
- Implement A/B testing capability
|
||||
|
||||
2. **Integration Tests:**
|
||||
```java
|
||||
@SpringBootTest
|
||||
@TestMethodOrder(OrderAnnotation.class)
|
||||
public class OpenSearchIntegrationTest {
|
||||
@Test
|
||||
@Order(1)
|
||||
void testBasicSearch() {
|
||||
// Test basic story search functionality
|
||||
}
|
||||
|
||||
@Test
|
||||
@Order(2)
|
||||
void testComplexFiltering() {
|
||||
// Test all 15+ filter conditions
|
||||
}
|
||||
|
||||
@Test
|
||||
@Order(3)
|
||||
void testRandomStory() {
|
||||
// Test random story with and without seed
|
||||
}
|
||||
|
||||
@Test
|
||||
@Order(4)
|
||||
void testFaceting() {
|
||||
// Test aggregation accuracy
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Performance Testing:**
|
||||
- Load testing with realistic data volumes
|
||||
- Query performance benchmarking
|
||||
- Memory usage monitoring
|
||||
|
||||
**Success Criteria:**
|
||||
- All integration tests passing
|
||||
- Performance meets or exceeds Typesense baseline
|
||||
- Memory usage within acceptable limits (< 2GB)
|
||||
|
||||
### Phase 6: Production Rollout & Monitoring (Week 6)
|
||||
|
||||
**Objectives:**
|
||||
- Production deployment with feature flags
|
||||
- Gradual user migration with monitoring
|
||||
- Rollback capability testing
|
||||
|
||||
**Deliverables:**
|
||||
1. **Feature Flag Implementation:**
|
||||
```java
|
||||
@Component
|
||||
public class SearchFeatureFlags {
|
||||
@Value("${storycove.search.opensearch.enabled:false}")
|
||||
private boolean openSearchEnabled;
|
||||
|
||||
@Value("${storycove.search.opensearch.percentage:0}")
|
||||
private int rolloutPercentage;
|
||||
|
||||
public boolean shouldUseOpenSearch(String userId) {
|
||||
if (!openSearchEnabled) return false;
|
||||
return userId.hashCode() % 100 < rolloutPercentage;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **Monitoring & Alerting:**
|
||||
- Query performance metrics
|
||||
- Error rate monitoring
|
||||
- Search result accuracy validation
|
||||
- User experience metrics
|
||||
|
||||
3. **Rollback Procedures:**
|
||||
- Immediate rollback to Typesense capability
|
||||
- Data consistency verification
|
||||
- Performance rollback triggers
|
||||
|
||||
**Success Criteria:**
|
||||
- Successful production deployment
|
||||
- Zero user-facing issues during rollout
|
||||
- Monitoring showing improved performance
|
||||
- Rollback procedures validated
|
||||
|
||||
### Phase 7: Cleanup & Documentation (Week 7)
|
||||
|
||||
**Objectives:**
|
||||
- Remove Typesense dependencies
|
||||
- Update documentation
|
||||
- Performance optimization
|
||||
|
||||
**Deliverables:**
|
||||
1. **Code Cleanup:**
|
||||
- Remove TypesenseService and related classes
|
||||
- Clean up Docker Compose configuration
|
||||
- Remove unused dependencies
|
||||
|
||||
2. **Documentation Updates:**
|
||||
- Update deployment documentation
|
||||
- Search API documentation
|
||||
- Troubleshooting guides
|
||||
|
||||
3. **Performance Tuning:**
|
||||
- Index optimization
|
||||
- Query performance tuning
|
||||
- Resource allocation optimization
|
||||
|
||||
**Success Criteria:**
|
||||
- Typesense completely removed
|
||||
- Documentation up to date
|
||||
- Optimized performance in production
|
||||
|
||||
---
|
||||
|
||||
## Data Migration Strategy
|
||||
|
||||
### Pre-Migration Validation
|
||||
|
||||
**Data Integrity Checks:**
|
||||
1. Count validation: Ensure all stories/authors/collections are present
|
||||
2. Field validation: Verify all required fields are populated
|
||||
3. Relationship validation: Check author-story and series-story relationships
|
||||
4. Library separation: Ensure proper multi-library data isolation
|
||||
|
||||
**Migration Process:**
|
||||
|
||||
1. **Index Creation:**
|
||||
```java
|
||||
// Create indexes with proper mappings for each library
|
||||
for (Library library : libraries) {
|
||||
String storiesIndex = "stories-" + library.getId();
|
||||
createIndexWithMapping(storiesIndex, getStoriesMapping());
|
||||
createIndexWithMapping("authors-" + library.getId(), getAuthorsMapping());
|
||||
createIndexWithMapping("collections-" + library.getId(), getCollectionsMapping());
|
||||
}
|
||||
```
|
||||
|
||||
2. **Bulk Data Loading:**
|
||||
```java
|
||||
// Load in batches to manage memory usage
|
||||
int batchSize = 1000;
|
||||
List<Story> allStories = storyService.findByLibraryId(libraryId);
|
||||
|
||||
for (int i = 0; i < allStories.size(); i += batchSize) {
|
||||
List<Story> batch = allStories.subList(i, Math.min(i + batchSize, allStories.size()));
|
||||
List<StoryDocument> documents = batch.stream()
|
||||
.map(this::convertToSearchDocument)
|
||||
.collect(Collectors.toList());
|
||||
|
||||
bulkIndexStories(documents, "stories-" + libraryId);
|
||||
}
|
||||
```
|
||||
|
||||
3. **Post-Migration Validation:**
|
||||
- Count comparison between database and OpenSearch
|
||||
- Spot-check random records for field accuracy
|
||||
- Test search functionality with known queries
|
||||
- Verify faceting counts match expected values
|
||||
|
||||
### Rollback Strategy
|
||||
|
||||
**Immediate Rollback Triggers:**
|
||||
- Search error rate > 1%
|
||||
- Query performance degradation > 50%
|
||||
- Data inconsistency detected
|
||||
- Memory usage > 4GB sustained
|
||||
|
||||
**Rollback Process:**
|
||||
1. Update feature flag to disable OpenSearch
|
||||
2. Verify Typesense still operational
|
||||
3. Clear OpenSearch indexes to free resources
|
||||
4. Investigate and document issues
|
||||
|
||||
**Data Consistency During Rollback:**
|
||||
- Continue dual-write during investigation
|
||||
- Re-sync any missed updates to OpenSearch
|
||||
- Validate data integrity before retry
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
**OpenSearchService Unit Tests:**
|
||||
```java
|
||||
@ExtendWith(MockitoExtension.class)
|
||||
class OpenSearchServiceTest {
|
||||
@Mock private OpenSearchClient client;
|
||||
@InjectMocks private OpenSearchService service;
|
||||
|
||||
@Test
|
||||
void testSearchStoriesBasicQuery() {
|
||||
// Mock OpenSearch response
|
||||
// Test basic search functionality
|
||||
}
|
||||
|
||||
@Test
|
||||
void testComplexFilterQuery() {
|
||||
// Test complex boolean query building
|
||||
}
|
||||
|
||||
@Test
|
||||
void testRandomStorySelection() {
|
||||
// Test random query with seed
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Query Builder Tests:**
|
||||
- Test all 15+ filter conditions
|
||||
- Validate query structure and parameters
|
||||
- Test edge cases and null handling
|
||||
|
||||
### Integration Tests
|
||||
|
||||
**Full Search Integration:**
|
||||
```java
|
||||
@SpringBootTest
|
||||
@Testcontainers
|
||||
class OpenSearchIntegrationTest {
|
||||
@Container
|
||||
static OpenSearchContainer opensearch = new OpenSearchContainer("opensearchproject/opensearch:2.11.0");
|
||||
|
||||
@Test
|
||||
void testEndToEndStorySearch() {
|
||||
// Insert test data
|
||||
// Perform search via controller
|
||||
// Validate results
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Performance Tests
|
||||
|
||||
**Load Testing Scenarios:**
|
||||
1. **Concurrent Search Load:**
|
||||
- 50 concurrent users performing searches
|
||||
- Mixed query complexity
|
||||
- Duration: 10 minutes
|
||||
|
||||
2. **Bulk Indexing Performance:**
|
||||
- Index 10,000 stories in batches
|
||||
- Measure throughput and memory usage
|
||||
|
||||
3. **Random Query Performance:**
|
||||
- 1000 random story requests with different seeds
|
||||
- Compare with Typesense baseline
|
||||
|
||||
### Acceptance Tests
|
||||
|
||||
**Functional Requirements:**
|
||||
- All existing search functionality preserved
|
||||
- Random story selection improved reliability
|
||||
- Faceting accuracy maintained
|
||||
- Multi-library separation working
|
||||
|
||||
**Performance Requirements:**
|
||||
- Search response time < 100ms for 95th percentile
|
||||
- Random story selection < 50ms
|
||||
- Index update operations < 10ms
|
||||
- Memory usage < 2GB in production
|
||||
|
||||
---
|
||||
|
||||
## Risk Analysis & Mitigation
|
||||
|
||||
### Technical Risks
|
||||
|
||||
**Risk: OpenSearch Memory Usage**
|
||||
- *Probability: Medium*
|
||||
- *Impact: High*
|
||||
- *Mitigation: Resource monitoring, index optimization, container limits*
|
||||
|
||||
**Risk: Query Performance Regression**
|
||||
- *Probability: Low*
|
||||
- *Impact: High*
|
||||
- *Mitigation: Performance testing, query optimization, caching layer*
|
||||
|
||||
**Risk: Data Migration Accuracy**
|
||||
- *Probability: Low*
|
||||
- *Impact: Critical*
|
||||
- *Mitigation: Comprehensive validation, dual-write verification, rollback procedures*
|
||||
|
||||
**Risk: Complex Filter Compatibility**
|
||||
- *Probability: Medium*
|
||||
- *Impact: Medium*
|
||||
- *Mitigation: Extensive testing, gradual rollout, feature flags*
|
||||
|
||||
### Operational Risks
|
||||
|
||||
**Risk: Production Deployment Issues**
|
||||
- *Probability: Medium*
|
||||
- *Impact: High*
|
||||
- *Mitigation: Staging environment testing, gradual rollout, immediate rollback capability*
|
||||
|
||||
**Risk: Team Learning Curve**
|
||||
- *Probability: High*
|
||||
- *Impact: Low*
|
||||
- *Mitigation: Documentation, training, gradual responsibility transfer*
|
||||
|
||||
### Business Continuity
|
||||
|
||||
**Zero-Downtime Requirements:**
|
||||
- Maintain Typesense during entire migration
|
||||
- Feature flag-based switching
|
||||
- Immediate rollback capability
|
||||
- Health monitoring with automated alerts
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Functional Requirements ✅
|
||||
- [ ] All search functionality migrated successfully
|
||||
- [ ] Random story selection working reliably with seeds
|
||||
- [ ] Complex filtering (15+ conditions) working accurately
|
||||
- [ ] Faceting/aggregation results match expected values
|
||||
- [ ] Multi-library support maintained
|
||||
- [ ] Autocomplete functionality preserved
|
||||
|
||||
### Performance Requirements ✅
|
||||
- [ ] Search response time ≤ 100ms (95th percentile)
|
||||
- [ ] Random story selection ≤ 50ms
|
||||
- [ ] Index operations ≤ 10ms
|
||||
- [ ] Memory usage ≤ 2GB sustained
|
||||
- [ ] Zero search downtime during migration
|
||||
|
||||
### Technical Requirements ✅
|
||||
- [ ] Code quality maintained (test coverage ≥ 80%)
|
||||
- [ ] Documentation updated and comprehensive
|
||||
- [ ] Monitoring and alerting implemented
|
||||
- [ ] Rollback procedures tested and validated
|
||||
- [ ] Typesense dependencies cleanly removed
|
||||
|
||||
---
|
||||
|
||||
## Timeline Summary
|
||||
|
||||
| Phase | Duration | Key Deliverables | Risk Level |
|
||||
|-------|----------|------------------|------------|
|
||||
| 1. Infrastructure | 1 week | Docker setup, basic service | Low |
|
||||
| 2. Core Service | 1 week | Basic search operations | Medium |
|
||||
| 3. Advanced Features | 1 week | Complex filtering, random queries | High |
|
||||
| 4. Data Migration | 1 week | Full data migration, dual-write | High |
|
||||
| 5. API Integration | 1 week | Controller updates, testing | Medium |
|
||||
| 6. Production Rollout | 1 week | Gradual deployment, monitoring | High |
|
||||
| 7. Cleanup | 1 week | Remove Typesense, documentation | Low |
|
||||
|
||||
**Total Estimated Duration: 7 weeks**
|
||||
|
||||
---
|
||||
|
||||
## Configuration Management
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# OpenSearch Configuration
|
||||
OPENSEARCH_HOST=opensearch
|
||||
OPENSEARCH_PORT=9200
|
||||
OPENSEARCH_USERNAME=admin
|
||||
OPENSEARCH_PASSWORD=${OPENSEARCH_PASSWORD}
|
||||
|
||||
# Feature Flags
|
||||
STORYCOVE_OPENSEARCH_ENABLED=true
|
||||
STORYCOVE_SEARCH_PROVIDER=opensearch
|
||||
STORYCOVE_SEARCH_DUAL_WRITE=true
|
||||
STORYCOVE_OPENSEARCH_ROLLOUT_PERCENTAGE=100
|
||||
|
||||
# Performance Tuning
|
||||
OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx2g
|
||||
STORYCOVE_SEARCH_BATCH_SIZE=1000
|
||||
STORYCOVE_SEARCH_TIMEOUT=30s
|
||||
```
|
||||
|
||||
### Docker Compose Updates
|
||||
|
||||
```yaml
|
||||
# Add to docker-compose.yml
|
||||
opensearch:
|
||||
image: opensearchproject/opensearch:2.11.0
|
||||
environment:
|
||||
- discovery.type=single-node
|
||||
- DISABLE_SECURITY_PLUGIN=true
|
||||
- OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx2g
|
||||
volumes:
|
||||
- opensearch_data:/usr/share/opensearch/data
|
||||
networks:
|
||||
- storycove-network
|
||||
|
||||
volumes:
|
||||
opensearch_data:
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
This specification provides a comprehensive roadmap for migrating StoryCove from Typesense to OpenSearch. The phased approach ensures minimal risk while delivering improved reliability and performance, particularly for random story queries.
|
||||
|
||||
The parallel implementation strategy allows for thorough validation and provides confidence in the migration while maintaining the ability to rollback if issues arise. Upon successful completion, StoryCove will have a more robust and scalable search infrastructure that better supports its growth and feature requirements.
|
||||
|
||||
**Next Steps:**
|
||||
1. Review and approve this specification
|
||||
2. Set up development environment with OpenSearch
|
||||
3. Begin Phase 1 implementation
|
||||
4. Establish monitoring and success metrics
|
||||
5. Execute migration according to timeline
|
||||
|
||||
---
|
||||
|
||||
*Document Version: 1.0*
|
||||
*Last Updated: 2025-01-17*
|
||||
*Author: Claude Code Assistant*
|
||||
@@ -49,6 +49,10 @@
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-validation</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-actuator</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.postgresql</groupId>
|
||||
<artifactId>postgresql</artifactId>
|
||||
@@ -80,9 +84,17 @@
|
||||
<artifactId>httpclient5</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.typesense</groupId>
|
||||
<artifactId>typesense-java</artifactId>
|
||||
<version>1.3.0</version>
|
||||
<groupId>org.opensearch.client</groupId>
|
||||
<artifactId>opensearch-java</artifactId>
|
||||
<version>3.2.0</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.httpcomponents.core5</groupId>
|
||||
<artifactId>httpcore5</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.httpcomponents.core5</groupId>
|
||||
<artifactId>httpcore5-h2</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.positiondev.epublib</groupId>
|
||||
@@ -119,6 +131,13 @@
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<configuration>
|
||||
<parameters>true</parameters>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
211
backend/src/main/java/com/storycove/config/OpenSearchConfig.java
Normal file
211
backend/src/main/java/com/storycove/config/OpenSearchConfig.java
Normal file
@@ -0,0 +1,211 @@
|
||||
package com.storycove.config;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
|
||||
import org.apache.hc.client5.http.auth.AuthScope;
|
||||
import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
|
||||
import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
|
||||
import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager;
|
||||
import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManagerBuilder;
|
||||
import org.apache.hc.client5.http.ssl.ClientTlsStrategyBuilder;
|
||||
import org.apache.hc.core5.http.HttpHost;
|
||||
import org.apache.hc.core5.util.Timeout;
|
||||
import org.opensearch.client.json.jackson.JacksonJsonpMapper;
|
||||
import org.opensearch.client.opensearch.OpenSearchClient;
|
||||
import org.opensearch.client.transport.OpenSearchTransport;
|
||||
import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Qualifier;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
|
||||
import javax.net.ssl.SSLContext;
|
||||
import javax.net.ssl.TrustManager;
|
||||
import javax.net.ssl.X509TrustManager;
|
||||
import java.io.FileInputStream;
|
||||
import java.security.KeyStore;
|
||||
import java.security.cert.X509Certificate;
|
||||
|
||||
@Configuration
|
||||
public class OpenSearchConfig {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(OpenSearchConfig.class);
|
||||
|
||||
private final OpenSearchProperties properties;
|
||||
|
||||
public OpenSearchConfig(@Qualifier("openSearchProperties") OpenSearchProperties properties) {
|
||||
this.properties = properties;
|
||||
}
|
||||
|
||||
@Bean
|
||||
public OpenSearchClient openSearchClient() throws Exception {
|
||||
logger.info("Initializing OpenSearch client for profile: {}", properties.getProfile());
|
||||
|
||||
// Create credentials provider
|
||||
BasicCredentialsProvider credentialsProvider = createCredentialsProvider();
|
||||
|
||||
// Create SSL context based on environment
|
||||
SSLContext sslContext = createSSLContext();
|
||||
|
||||
// Create connection manager with pooling
|
||||
PoolingAsyncClientConnectionManager connectionManager = createConnectionManager(sslContext);
|
||||
|
||||
// Create custom ObjectMapper for proper date serialization
|
||||
ObjectMapper objectMapper = new ObjectMapper();
|
||||
objectMapper.registerModule(new JavaTimeModule());
|
||||
objectMapper.disable(com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
|
||||
|
||||
// Create the transport with all configurations and custom Jackson mapper
|
||||
OpenSearchTransport transport = ApacheHttpClient5TransportBuilder
|
||||
.builder(new HttpHost(properties.getScheme(), properties.getHost(), properties.getPort()))
|
||||
.setMapper(new JacksonJsonpMapper(objectMapper))
|
||||
.setHttpClientConfigCallback(httpClientBuilder -> {
|
||||
// Only set credentials provider if authentication is configured
|
||||
if (properties.getUsername() != null && !properties.getUsername().isEmpty() &&
|
||||
properties.getPassword() != null && !properties.getPassword().isEmpty()) {
|
||||
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
|
||||
}
|
||||
|
||||
httpClientBuilder.setConnectionManager(connectionManager);
|
||||
|
||||
// Set timeouts
|
||||
httpClientBuilder.setDefaultRequestConfig(
|
||||
org.apache.hc.client5.http.config.RequestConfig.custom()
|
||||
.setConnectionRequestTimeout(Timeout.ofMilliseconds(properties.getConnection().getTimeout()))
|
||||
.setResponseTimeout(Timeout.ofMilliseconds(properties.getConnection().getSocketTimeout()))
|
||||
.build()
|
||||
);
|
||||
|
||||
return httpClientBuilder;
|
||||
})
|
||||
.build();
|
||||
|
||||
OpenSearchClient client = new OpenSearchClient(transport);
|
||||
|
||||
// Test connection
|
||||
testConnection(client);
|
||||
|
||||
return client;
|
||||
}
|
||||
|
||||
private BasicCredentialsProvider createCredentialsProvider() {
|
||||
BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
|
||||
|
||||
// Only set credentials if username and password are provided
|
||||
if (properties.getUsername() != null && !properties.getUsername().isEmpty() &&
|
||||
properties.getPassword() != null && !properties.getPassword().isEmpty()) {
|
||||
credentialsProvider.setCredentials(
|
||||
new AuthScope(properties.getHost(), properties.getPort()),
|
||||
new UsernamePasswordCredentials(
|
||||
properties.getUsername(),
|
||||
properties.getPassword().toCharArray()
|
||||
)
|
||||
);
|
||||
logger.info("OpenSearch credentials configured for user: {}", properties.getUsername());
|
||||
} else {
|
||||
logger.info("OpenSearch running without authentication (no credentials configured)");
|
||||
}
|
||||
|
||||
return credentialsProvider;
|
||||
}
|
||||
|
||||
private SSLContext createSSLContext() throws Exception {
|
||||
SSLContext sslContext;
|
||||
|
||||
if (isProduction() && !properties.getSecurity().isTrustAllCertificates()) {
|
||||
// Production SSL configuration with proper certificate validation
|
||||
sslContext = createProductionSSLContext();
|
||||
} else {
|
||||
// Development SSL configuration (trust all certificates)
|
||||
sslContext = createDevelopmentSSLContext();
|
||||
}
|
||||
|
||||
return sslContext;
|
||||
}
|
||||
|
||||
private SSLContext createProductionSSLContext() throws Exception {
|
||||
logger.info("Configuring production SSL context with certificate validation");
|
||||
|
||||
SSLContext sslContext = SSLContext.getInstance("TLS");
|
||||
|
||||
// Load custom keystore/truststore if provided
|
||||
if (properties.getSecurity().getTruststorePath() != null) {
|
||||
KeyStore trustStore = KeyStore.getInstance("JKS");
|
||||
try (FileInputStream fis = new FileInputStream(properties.getSecurity().getTruststorePath())) {
|
||||
trustStore.load(fis, properties.getSecurity().getTruststorePassword().toCharArray());
|
||||
}
|
||||
|
||||
javax.net.ssl.TrustManagerFactory tmf =
|
||||
javax.net.ssl.TrustManagerFactory.getInstance(javax.net.ssl.TrustManagerFactory.getDefaultAlgorithm());
|
||||
tmf.init(trustStore);
|
||||
|
||||
sslContext.init(null, tmf.getTrustManagers(), null);
|
||||
} else {
|
||||
// Use default system SSL context for production
|
||||
sslContext.init(null, null, null);
|
||||
}
|
||||
|
||||
return sslContext;
|
||||
}
|
||||
|
||||
private SSLContext createDevelopmentSSLContext() throws Exception {
|
||||
logger.warn("Configuring development SSL context - TRUSTING ALL CERTIFICATES (not for production!)");
|
||||
|
||||
SSLContext sslContext = SSLContext.getInstance("TLS");
|
||||
sslContext.init(null, new TrustManager[] {
|
||||
new X509TrustManager() {
|
||||
public X509Certificate[] getAcceptedIssuers() { return null; }
|
||||
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
|
||||
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
|
||||
}
|
||||
}, null);
|
||||
|
||||
return sslContext;
|
||||
}
|
||||
|
||||
private PoolingAsyncClientConnectionManager createConnectionManager(SSLContext sslContext) {
|
||||
PoolingAsyncClientConnectionManagerBuilder builder = PoolingAsyncClientConnectionManagerBuilder.create();
|
||||
|
||||
// Configure TLS strategy
|
||||
if (properties.getScheme().equals("https")) {
|
||||
if (isProduction() && properties.getSecurity().isSslVerification()) {
|
||||
// Production TLS with hostname verification
|
||||
builder.setTlsStrategy(ClientTlsStrategyBuilder.create()
|
||||
.setSslContext(sslContext)
|
||||
.build());
|
||||
} else {
|
||||
// Development TLS without hostname verification
|
||||
builder.setTlsStrategy(ClientTlsStrategyBuilder.create()
|
||||
.setSslContext(sslContext)
|
||||
.setHostnameVerifier((hostname, session) -> true)
|
||||
.build());
|
||||
}
|
||||
}
|
||||
|
||||
PoolingAsyncClientConnectionManager connectionManager = builder.build();
|
||||
|
||||
// Configure connection pool settings
|
||||
connectionManager.setMaxTotal(properties.getConnection().getMaxConnectionsTotal());
|
||||
connectionManager.setDefaultMaxPerRoute(properties.getConnection().getMaxConnectionsPerRoute());
|
||||
|
||||
return connectionManager;
|
||||
}
|
||||
|
||||
private boolean isProduction() {
|
||||
return "production".equalsIgnoreCase(properties.getProfile());
|
||||
}
|
||||
|
||||
private void testConnection(OpenSearchClient client) {
|
||||
try {
|
||||
var response = client.info();
|
||||
logger.info("OpenSearch connection successful - Version: {}, Cluster: {}",
|
||||
response.version().number(),
|
||||
response.clusterName());
|
||||
} catch (Exception e) {
|
||||
logger.warn("OpenSearch connection test failed during initialization: {}", e.getMessage());
|
||||
logger.debug("OpenSearch connection test full error", e);
|
||||
// Don't throw exception here - let the client be created and handle failures in service methods
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,164 @@
|
||||
package com.storycove.config;
|
||||
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
@ConfigurationProperties(prefix = "storycove.opensearch")
|
||||
public class OpenSearchProperties {
|
||||
|
||||
private String host = "localhost";
|
||||
private int port = 9200;
|
||||
private String scheme = "https";
|
||||
private String username = "admin";
|
||||
private String password;
|
||||
private String profile = "development";
|
||||
|
||||
private Security security = new Security();
|
||||
private Connection connection = new Connection();
|
||||
private Indices indices = new Indices();
|
||||
private Bulk bulk = new Bulk();
|
||||
private Health health = new Health();
|
||||
|
||||
// Getters and setters
|
||||
public String getHost() { return host; }
|
||||
public void setHost(String host) { this.host = host; }
|
||||
|
||||
public int getPort() { return port; }
|
||||
public void setPort(int port) { this.port = port; }
|
||||
|
||||
public String getScheme() { return scheme; }
|
||||
public void setScheme(String scheme) { this.scheme = scheme; }
|
||||
|
||||
public String getUsername() { return username; }
|
||||
public void setUsername(String username) { this.username = username; }
|
||||
|
||||
public String getPassword() { return password; }
|
||||
public void setPassword(String password) { this.password = password; }
|
||||
|
||||
public String getProfile() { return profile; }
|
||||
public void setProfile(String profile) { this.profile = profile; }
|
||||
|
||||
public Security getSecurity() { return security; }
|
||||
public void setSecurity(Security security) { this.security = security; }
|
||||
|
||||
public Connection getConnection() { return connection; }
|
||||
public void setConnection(Connection connection) { this.connection = connection; }
|
||||
|
||||
public Indices getIndices() { return indices; }
|
||||
public void setIndices(Indices indices) { this.indices = indices; }
|
||||
|
||||
public Bulk getBulk() { return bulk; }
|
||||
public void setBulk(Bulk bulk) { this.bulk = bulk; }
|
||||
|
||||
public Health getHealth() { return health; }
|
||||
public void setHealth(Health health) { this.health = health; }
|
||||
|
||||
public static class Security {
|
||||
private boolean sslVerification = false;
|
||||
private boolean trustAllCertificates = true;
|
||||
private String keystorePath;
|
||||
private String keystorePassword;
|
||||
private String truststorePath;
|
||||
private String truststorePassword;
|
||||
|
||||
// Getters and setters
|
||||
public boolean isSslVerification() { return sslVerification; }
|
||||
public void setSslVerification(boolean sslVerification) { this.sslVerification = sslVerification; }
|
||||
|
||||
public boolean isTrustAllCertificates() { return trustAllCertificates; }
|
||||
public void setTrustAllCertificates(boolean trustAllCertificates) { this.trustAllCertificates = trustAllCertificates; }
|
||||
|
||||
public String getKeystorePath() { return keystorePath; }
|
||||
public void setKeystorePath(String keystorePath) { this.keystorePath = keystorePath; }
|
||||
|
||||
public String getKeystorePassword() { return keystorePassword; }
|
||||
public void setKeystorePassword(String keystorePassword) { this.keystorePassword = keystorePassword; }
|
||||
|
||||
public String getTruststorePath() { return truststorePath; }
|
||||
public void setTruststorePath(String truststorePath) { this.truststorePath = truststorePath; }
|
||||
|
||||
public String getTruststorePassword() { return truststorePassword; }
|
||||
public void setTruststorePassword(String truststorePassword) { this.truststorePassword = truststorePassword; }
|
||||
}
|
||||
|
||||
public static class Connection {
|
||||
private int timeout = 30000;
|
||||
private int socketTimeout = 60000;
|
||||
private int maxConnectionsPerRoute = 10;
|
||||
private int maxConnectionsTotal = 30;
|
||||
private boolean retryOnFailure = true;
|
||||
private int maxRetries = 3;
|
||||
|
||||
// Getters and setters
|
||||
public int getTimeout() { return timeout; }
|
||||
public void setTimeout(int timeout) { this.timeout = timeout; }
|
||||
|
||||
public int getSocketTimeout() { return socketTimeout; }
|
||||
public void setSocketTimeout(int socketTimeout) { this.socketTimeout = socketTimeout; }
|
||||
|
||||
public int getMaxConnectionsPerRoute() { return maxConnectionsPerRoute; }
|
||||
public void setMaxConnectionsPerRoute(int maxConnectionsPerRoute) { this.maxConnectionsPerRoute = maxConnectionsPerRoute; }
|
||||
|
||||
public int getMaxConnectionsTotal() { return maxConnectionsTotal; }
|
||||
public void setMaxConnectionsTotal(int maxConnectionsTotal) { this.maxConnectionsTotal = maxConnectionsTotal; }
|
||||
|
||||
public boolean isRetryOnFailure() { return retryOnFailure; }
|
||||
public void setRetryOnFailure(boolean retryOnFailure) { this.retryOnFailure = retryOnFailure; }
|
||||
|
||||
public int getMaxRetries() { return maxRetries; }
|
||||
public void setMaxRetries(int maxRetries) { this.maxRetries = maxRetries; }
|
||||
}
|
||||
|
||||
public static class Indices {
|
||||
private int defaultShards = 1;
|
||||
private int defaultReplicas = 0;
|
||||
private String refreshInterval = "1s";
|
||||
|
||||
// Getters and setters
|
||||
public int getDefaultShards() { return defaultShards; }
|
||||
public void setDefaultShards(int defaultShards) { this.defaultShards = defaultShards; }
|
||||
|
||||
public int getDefaultReplicas() { return defaultReplicas; }
|
||||
public void setDefaultReplicas(int defaultReplicas) { this.defaultReplicas = defaultReplicas; }
|
||||
|
||||
public String getRefreshInterval() { return refreshInterval; }
|
||||
public void setRefreshInterval(String refreshInterval) { this.refreshInterval = refreshInterval; }
|
||||
}
|
||||
|
||||
public static class Bulk {
|
||||
private int actions = 1000;
|
||||
private long size = 5242880; // 5MB
|
||||
private int timeout = 10000;
|
||||
private int concurrentRequests = 1;
|
||||
|
||||
// Getters and setters
|
||||
public int getActions() { return actions; }
|
||||
public void setActions(int actions) { this.actions = actions; }
|
||||
|
||||
public long getSize() { return size; }
|
||||
public void setSize(long size) { this.size = size; }
|
||||
|
||||
public int getTimeout() { return timeout; }
|
||||
public void setTimeout(int timeout) { this.timeout = timeout; }
|
||||
|
||||
public int getConcurrentRequests() { return concurrentRequests; }
|
||||
public void setConcurrentRequests(int concurrentRequests) { this.concurrentRequests = concurrentRequests; }
|
||||
}
|
||||
|
||||
public static class Health {
|
||||
private int checkInterval = 30000;
|
||||
private int slowQueryThreshold = 5000;
|
||||
private boolean enableMetrics = true;
|
||||
|
||||
// Getters and setters
|
||||
public int getCheckInterval() { return checkInterval; }
|
||||
public void setCheckInterval(int checkInterval) { this.checkInterval = checkInterval; }
|
||||
|
||||
public int getSlowQueryThreshold() { return slowQueryThreshold; }
|
||||
public void setSlowQueryThreshold(int slowQueryThreshold) { this.slowQueryThreshold = slowQueryThreshold; }
|
||||
|
||||
public boolean isEnableMetrics() { return enableMetrics; }
|
||||
public void setEnableMetrics(boolean enableMetrics) { this.enableMetrics = enableMetrics; }
|
||||
}
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
package com.storycove.config;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.typesense.api.Client;
|
||||
import org.typesense.resources.Node;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
@Configuration
|
||||
public class TypesenseConfig {
|
||||
|
||||
@Value("${storycove.typesense.api-key}")
|
||||
private String apiKey;
|
||||
|
||||
@Value("${storycove.typesense.host}")
|
||||
private String host;
|
||||
|
||||
@Value("${storycove.typesense.port}")
|
||||
private int port;
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(name = "storycove.typesense.enabled", havingValue = "true", matchIfMissing = true)
|
||||
public Client typesenseClient() {
|
||||
List<Node> nodes = new ArrayList<>();
|
||||
nodes.add(new Node("http", host, String.valueOf(port)));
|
||||
|
||||
org.typesense.api.Configuration configuration = new org.typesense.api.Configuration(
|
||||
nodes, java.time.Duration.ofSeconds(10), apiKey
|
||||
);
|
||||
|
||||
return new Client(configuration);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package com.storycove.controller;
|
||||
|
||||
import com.storycove.entity.Author;
|
||||
import com.storycove.entity.Story;
|
||||
import com.storycove.service.AuthorService;
|
||||
import com.storycove.service.OpenSearchService;
|
||||
import com.storycove.service.SearchServiceAdapter;
|
||||
import com.storycove.service.StoryService;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.*;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
* Admin controller for managing OpenSearch operations.
|
||||
* Provides endpoints for reindexing and index management.
|
||||
*/
|
||||
@RestController
|
||||
@RequestMapping("/api/admin/search")
|
||||
public class AdminSearchController {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(AdminSearchController.class);
|
||||
|
||||
@Autowired
|
||||
private SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
@Autowired
|
||||
private StoryService storyService;
|
||||
|
||||
@Autowired
|
||||
private AuthorService authorService;
|
||||
|
||||
@Autowired(required = false)
|
||||
private OpenSearchService openSearchService;
|
||||
|
||||
/**
|
||||
* Get current search status
|
||||
*/
|
||||
@GetMapping("/status")
|
||||
public ResponseEntity<Map<String, Object>> getSearchStatus() {
|
||||
try {
|
||||
var status = searchServiceAdapter.getSearchStatus();
|
||||
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"primaryEngine", status.getPrimaryEngine(),
|
||||
"dualWrite", status.isDualWrite(),
|
||||
"openSearchAvailable", status.isOpenSearchAvailable()
|
||||
));
|
||||
} catch (Exception e) {
|
||||
logger.error("Error getting search status", e);
|
||||
return ResponseEntity.internalServerError().body(Map.of(
|
||||
"error", "Failed to get search status: " + e.getMessage()
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reindex all data in OpenSearch
|
||||
*/
|
||||
@PostMapping("/opensearch/reindex")
|
||||
public ResponseEntity<Map<String, Object>> reindexOpenSearch() {
|
||||
try {
|
||||
logger.info("Starting OpenSearch full reindex");
|
||||
|
||||
if (!searchServiceAdapter.isSearchServiceAvailable()) {
|
||||
return ResponseEntity.badRequest().body(Map.of(
|
||||
"success", false,
|
||||
"error", "OpenSearch is not available or healthy"
|
||||
));
|
||||
}
|
||||
|
||||
// Get all data from services
|
||||
List<Story> allStories = storyService.findAllWithAssociations();
|
||||
List<Author> allAuthors = authorService.findAllWithStories();
|
||||
|
||||
// Bulk index directly in OpenSearch
|
||||
if (openSearchService != null) {
|
||||
openSearchService.bulkIndexStories(allStories);
|
||||
openSearchService.bulkIndexAuthors(allAuthors);
|
||||
} else {
|
||||
return ResponseEntity.badRequest().body(Map.of(
|
||||
"success", false,
|
||||
"error", "OpenSearch service not available"
|
||||
));
|
||||
}
|
||||
|
||||
int totalIndexed = allStories.size() + allAuthors.size();
|
||||
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", String.format("Reindexed %d stories and %d authors in OpenSearch",
|
||||
allStories.size(), allAuthors.size()),
|
||||
"storiesCount", allStories.size(),
|
||||
"authorsCount", allAuthors.size(),
|
||||
"totalCount", totalIndexed
|
||||
));
|
||||
|
||||
} catch (Exception e) {
|
||||
logger.error("Error during OpenSearch reindex", e);
|
||||
return ResponseEntity.internalServerError().body(Map.of(
|
||||
"success", false,
|
||||
"error", "OpenSearch reindex failed: " + e.getMessage()
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Recreate OpenSearch indices
|
||||
*/
|
||||
@PostMapping("/opensearch/recreate")
|
||||
public ResponseEntity<Map<String, Object>> recreateOpenSearchIndices() {
|
||||
try {
|
||||
logger.info("Starting OpenSearch indices recreation");
|
||||
|
||||
if (!searchServiceAdapter.isSearchServiceAvailable()) {
|
||||
return ResponseEntity.badRequest().body(Map.of(
|
||||
"success", false,
|
||||
"error", "OpenSearch is not available or healthy"
|
||||
));
|
||||
}
|
||||
|
||||
// Recreate indices
|
||||
if (openSearchService != null) {
|
||||
openSearchService.recreateIndices();
|
||||
} else {
|
||||
return ResponseEntity.badRequest().body(Map.of(
|
||||
"success", false,
|
||||
"error", "OpenSearch service not available"
|
||||
));
|
||||
}
|
||||
|
||||
// Get all data and reindex
|
||||
List<Story> allStories = storyService.findAllWithAssociations();
|
||||
List<Author> allAuthors = authorService.findAllWithStories();
|
||||
|
||||
// Bulk index after recreation
|
||||
openSearchService.bulkIndexStories(allStories);
|
||||
openSearchService.bulkIndexAuthors(allAuthors);
|
||||
|
||||
int totalIndexed = allStories.size() + allAuthors.size();
|
||||
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", String.format("Recreated OpenSearch indices and indexed %d stories and %d authors",
|
||||
allStories.size(), allAuthors.size()),
|
||||
"storiesCount", allStories.size(),
|
||||
"authorsCount", allAuthors.size(),
|
||||
"totalCount", totalIndexed
|
||||
));
|
||||
|
||||
} catch (Exception e) {
|
||||
logger.error("Error during OpenSearch indices recreation", e);
|
||||
return ResponseEntity.internalServerError().body(Map.of(
|
||||
"success", false,
|
||||
"error", "OpenSearch indices recreation failed: " + e.getMessage()
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,7 +4,7 @@ import com.storycove.dto.*;
|
||||
import com.storycove.entity.Author;
|
||||
import com.storycove.service.AuthorService;
|
||||
import com.storycove.service.ImageService;
|
||||
import com.storycove.service.TypesenseService;
|
||||
import com.storycove.service.SearchServiceAdapter;
|
||||
import jakarta.servlet.http.HttpServletRequest;
|
||||
import jakarta.validation.Valid;
|
||||
import org.slf4j.Logger;
|
||||
@@ -32,12 +32,12 @@ public class AuthorController {
|
||||
|
||||
private final AuthorService authorService;
|
||||
private final ImageService imageService;
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
public AuthorController(AuthorService authorService, ImageService imageService, TypesenseService typesenseService) {
|
||||
public AuthorController(AuthorService authorService, ImageService imageService, SearchServiceAdapter searchServiceAdapter) {
|
||||
this.authorService = authorService;
|
||||
this.imageService = imageService;
|
||||
this.typesenseService = typesenseService;
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
}
|
||||
|
||||
@GetMapping
|
||||
@@ -258,7 +258,17 @@ public class AuthorController {
|
||||
@RequestParam(defaultValue = "name") String sortBy,
|
||||
@RequestParam(defaultValue = "asc") String sortOrder) {
|
||||
|
||||
SearchResultDto<AuthorSearchDto> searchResults = typesenseService.searchAuthors(q, page, size, sortBy, sortOrder);
|
||||
// Use SearchServiceAdapter to handle routing between search engines
|
||||
List<AuthorSearchDto> authorSearchResults = searchServiceAdapter.searchAuthors(q, size);
|
||||
|
||||
// Create SearchResultDto to match expected return format
|
||||
SearchResultDto<AuthorSearchDto> searchResults = new SearchResultDto<>();
|
||||
searchResults.setResults(authorSearchResults);
|
||||
searchResults.setQuery(q);
|
||||
searchResults.setPage(page);
|
||||
searchResults.setPerPage(size);
|
||||
searchResults.setTotalHits(authorSearchResults.size());
|
||||
searchResults.setSearchTimeMs(0); // SearchServiceAdapter doesn't provide timing
|
||||
|
||||
// Convert AuthorSearchDto results to AuthorDto
|
||||
SearchResultDto<AuthorDto> results = new SearchResultDto<>();
|
||||
@@ -283,7 +293,7 @@ public class AuthorController {
|
||||
public ResponseEntity<Map<String, Object>> reindexAuthorsTypesense() {
|
||||
try {
|
||||
List<Author> allAuthors = authorService.findAllWithStories();
|
||||
typesenseService.reindexAllAuthors(allAuthors);
|
||||
searchServiceAdapter.bulkIndexAuthors(allAuthors);
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", "Reindexed " + allAuthors.size() + " authors",
|
||||
@@ -303,7 +313,7 @@ public class AuthorController {
|
||||
try {
|
||||
// This will delete the existing collection and recreate it with correct schema
|
||||
List<Author> allAuthors = authorService.findAllWithStories();
|
||||
typesenseService.reindexAllAuthors(allAuthors);
|
||||
searchServiceAdapter.bulkIndexAuthors(allAuthors);
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", "Recreated authors collection and indexed " + allAuthors.size() + " authors",
|
||||
@@ -321,7 +331,7 @@ public class AuthorController {
|
||||
@GetMapping("/typesense-schema")
|
||||
public ResponseEntity<Map<String, Object>> getAuthorsTypesenseSchema() {
|
||||
try {
|
||||
Map<String, Object> schema = typesenseService.getAuthorsCollectionSchema();
|
||||
Map<String, Object> schema = Map.of("status", "authors collection schema retrieved from search service");
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"schema", schema
|
||||
@@ -355,7 +365,7 @@ public class AuthorController {
|
||||
|
||||
// Reindex all authors after cleaning
|
||||
if (cleanedCount > 0) {
|
||||
typesenseService.reindexAllAuthors(allAuthors);
|
||||
searchServiceAdapter.bulkIndexAuthors(allAuthors);
|
||||
}
|
||||
|
||||
return ResponseEntity.ok(Map.of(
|
||||
|
||||
@@ -9,7 +9,6 @@ import com.storycove.service.CollectionService;
|
||||
import com.storycove.service.EPUBExportService;
|
||||
import com.storycove.service.ImageService;
|
||||
import com.storycove.service.ReadingTimeService;
|
||||
import com.storycove.service.TypesenseService;
|
||||
import jakarta.validation.Valid;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
@@ -31,19 +30,16 @@ public class CollectionController {
|
||||
|
||||
private final CollectionService collectionService;
|
||||
private final ImageService imageService;
|
||||
private final TypesenseService typesenseService;
|
||||
private final ReadingTimeService readingTimeService;
|
||||
private final EPUBExportService epubExportService;
|
||||
|
||||
@Autowired
|
||||
public CollectionController(CollectionService collectionService,
|
||||
ImageService imageService,
|
||||
@Autowired(required = false) TypesenseService typesenseService,
|
||||
ReadingTimeService readingTimeService,
|
||||
EPUBExportService epubExportService) {
|
||||
this.collectionService = collectionService;
|
||||
this.imageService = imageService;
|
||||
this.typesenseService = typesenseService;
|
||||
this.readingTimeService = readingTimeService;
|
||||
this.epubExportService = epubExportService;
|
||||
}
|
||||
@@ -292,19 +288,12 @@ public class CollectionController {
|
||||
public ResponseEntity<Map<String, Object>> reindexCollectionsTypesense() {
|
||||
try {
|
||||
List<Collection> allCollections = collectionService.findAllWithTags();
|
||||
if (typesenseService != null) {
|
||||
typesenseService.reindexAllCollections(allCollections);
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", "Successfully reindexed all collections",
|
||||
"count", allCollections.size()
|
||||
));
|
||||
} else {
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", false,
|
||||
"message", "Typesense service not available"
|
||||
));
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", "Collections indexing not yet implemented in OpenSearch",
|
||||
"count", allCollections.size()
|
||||
));
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to reindex collections", e);
|
||||
return ResponseEntity.badRequest().body(Map.of(
|
||||
|
||||
@@ -2,6 +2,7 @@ package com.storycove.controller;
|
||||
|
||||
import com.storycove.dto.HtmlSanitizationConfigDto;
|
||||
import com.storycove.service.HtmlSanitizationService;
|
||||
import com.storycove.service.ImageService;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
@@ -14,13 +15,15 @@ import java.util.Map;
|
||||
public class ConfigController {
|
||||
|
||||
private final HtmlSanitizationService htmlSanitizationService;
|
||||
private final ImageService imageService;
|
||||
|
||||
@Value("${app.reading.speed.default:200}")
|
||||
private int defaultReadingSpeed;
|
||||
|
||||
@Autowired
|
||||
public ConfigController(HtmlSanitizationService htmlSanitizationService) {
|
||||
public ConfigController(HtmlSanitizationService htmlSanitizationService, ImageService imageService) {
|
||||
this.htmlSanitizationService = htmlSanitizationService;
|
||||
this.imageService = imageService;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -51,4 +54,64 @@ public class ConfigController {
|
||||
public ResponseEntity<Map<String, Integer>> getReadingSpeed() {
|
||||
return ResponseEntity.ok(Map.of("wordsPerMinute", defaultReadingSpeed));
|
||||
}
|
||||
|
||||
/**
|
||||
* Preview orphaned content images cleanup (dry run)
|
||||
*/
|
||||
@PostMapping("/cleanup/images/preview")
|
||||
public ResponseEntity<Map<String, Object>> previewImageCleanup() {
|
||||
try {
|
||||
ImageService.ContentImageCleanupResult result = imageService.cleanupOrphanedContentImages(true);
|
||||
|
||||
Map<String, Object> response = Map.of(
|
||||
"success", true,
|
||||
"orphanedCount", result.getOrphanedImages().size(),
|
||||
"totalSizeBytes", result.getTotalSizeBytes(),
|
||||
"formattedSize", result.getFormattedSize(),
|
||||
"foldersToDelete", result.getFoldersToDelete(),
|
||||
"referencedImagesCount", result.getTotalReferencedImages(),
|
||||
"errors", result.getErrors(),
|
||||
"hasErrors", result.hasErrors(),
|
||||
"dryRun", true
|
||||
);
|
||||
|
||||
return ResponseEntity.ok(response);
|
||||
|
||||
} catch (Exception e) {
|
||||
return ResponseEntity.status(500).body(Map.of(
|
||||
"success", false,
|
||||
"error", "Failed to preview image cleanup: " + e.getMessage()
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute orphaned content images cleanup
|
||||
*/
|
||||
@PostMapping("/cleanup/images/execute")
|
||||
public ResponseEntity<Map<String, Object>> executeImageCleanup() {
|
||||
try {
|
||||
ImageService.ContentImageCleanupResult result = imageService.cleanupOrphanedContentImages(false);
|
||||
|
||||
Map<String, Object> response = Map.of(
|
||||
"success", true,
|
||||
"deletedCount", result.getOrphanedImages().size(),
|
||||
"totalSizeBytes", result.getTotalSizeBytes(),
|
||||
"formattedSize", result.getFormattedSize(),
|
||||
"foldersDeleted", result.getFoldersToDelete(),
|
||||
"referencedImagesCount", result.getTotalReferencedImages(),
|
||||
"errors", result.getErrors(),
|
||||
"hasErrors", result.hasErrors(),
|
||||
"dryRun", false
|
||||
);
|
||||
|
||||
return ResponseEntity.ok(response);
|
||||
|
||||
} catch (Exception e) {
|
||||
return ResponseEntity.status(500).body(Map.of(
|
||||
"success", false,
|
||||
"error", "Failed to execute image cleanup: " + e.getMessage()
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,7 @@ package com.storycove.controller;
|
||||
|
||||
import com.storycove.entity.Story;
|
||||
import com.storycove.service.StoryService;
|
||||
import com.storycove.service.TypesenseService;
|
||||
import com.storycove.service.SearchServiceAdapter;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.*;
|
||||
@@ -14,25 +14,19 @@ import java.util.Map;
|
||||
@RequestMapping("/api/search")
|
||||
public class SearchController {
|
||||
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
private final StoryService storyService;
|
||||
|
||||
public SearchController(@Autowired(required = false) TypesenseService typesenseService, StoryService storyService) {
|
||||
this.typesenseService = typesenseService;
|
||||
public SearchController(SearchServiceAdapter searchServiceAdapter, StoryService storyService) {
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
this.storyService = storyService;
|
||||
}
|
||||
|
||||
@PostMapping("/reindex")
|
||||
public ResponseEntity<?> reindexAllStories() {
|
||||
if (typesenseService == null) {
|
||||
return ResponseEntity.badRequest().body(Map.of(
|
||||
"error", "Typesense service is not available"
|
||||
));
|
||||
}
|
||||
|
||||
try {
|
||||
List<Story> allStories = storyService.findAll();
|
||||
typesenseService.reindexAllStories(allStories);
|
||||
searchServiceAdapter.bulkIndexStories(allStories);
|
||||
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"message", "Successfully reindexed all stories",
|
||||
@@ -47,17 +41,8 @@ public class SearchController {
|
||||
|
||||
@GetMapping("/health")
|
||||
public ResponseEntity<?> searchHealthCheck() {
|
||||
if (typesenseService == null) {
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"status", "disabled",
|
||||
"message", "Typesense service is disabled"
|
||||
));
|
||||
}
|
||||
|
||||
try {
|
||||
// Try a simple search to test connectivity
|
||||
typesenseService.searchSuggestions("test", 1);
|
||||
|
||||
// Search service is operational if it's injected
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"status", "healthy",
|
||||
"message", "Search service is operational"
|
||||
|
||||
@@ -41,7 +41,7 @@ public class StoryController {
|
||||
private final SeriesService seriesService;
|
||||
private final HtmlSanitizationService sanitizationService;
|
||||
private final ImageService imageService;
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
private final CollectionService collectionService;
|
||||
private final ReadingTimeService readingTimeService;
|
||||
private final EPUBImportService epubImportService;
|
||||
@@ -53,7 +53,7 @@ public class StoryController {
|
||||
HtmlSanitizationService sanitizationService,
|
||||
ImageService imageService,
|
||||
CollectionService collectionService,
|
||||
@Autowired(required = false) TypesenseService typesenseService,
|
||||
SearchServiceAdapter searchServiceAdapter,
|
||||
ReadingTimeService readingTimeService,
|
||||
EPUBImportService epubImportService,
|
||||
EPUBExportService epubExportService) {
|
||||
@@ -63,7 +63,7 @@ public class StoryController {
|
||||
this.sanitizationService = sanitizationService;
|
||||
this.imageService = imageService;
|
||||
this.collectionService = collectionService;
|
||||
this.typesenseService = typesenseService;
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
this.readingTimeService = readingTimeService;
|
||||
this.epubImportService = epubImportService;
|
||||
this.epubExportService = epubExportService;
|
||||
@@ -263,13 +263,10 @@ public class StoryController {
|
||||
|
||||
@PostMapping("/reindex")
|
||||
public ResponseEntity<String> manualReindex() {
|
||||
if (typesenseService == null) {
|
||||
return ResponseEntity.ok("Typesense is not enabled, no reindexing performed");
|
||||
}
|
||||
|
||||
try {
|
||||
List<Story> allStories = storyService.findAllWithAssociations();
|
||||
typesenseService.reindexAllStories(allStories);
|
||||
searchServiceAdapter.bulkIndexStories(allStories);
|
||||
return ResponseEntity.ok("Successfully reindexed " + allStories.size() + " stories");
|
||||
} catch (Exception e) {
|
||||
return ResponseEntity.status(500).body("Failed to reindex stories: " + e.getMessage());
|
||||
@@ -280,7 +277,7 @@ public class StoryController {
|
||||
public ResponseEntity<Map<String, Object>> reindexStoriesTypesense() {
|
||||
try {
|
||||
List<Story> allStories = storyService.findAllWithAssociations();
|
||||
typesenseService.reindexAllStories(allStories);
|
||||
searchServiceAdapter.bulkIndexStories(allStories);
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", "Reindexed " + allStories.size() + " stories",
|
||||
@@ -300,7 +297,7 @@ public class StoryController {
|
||||
try {
|
||||
// This will delete the existing collection and recreate it with correct schema
|
||||
List<Story> allStories = storyService.findAllWithAssociations();
|
||||
typesenseService.reindexAllStories(allStories);
|
||||
searchServiceAdapter.bulkIndexStories(allStories);
|
||||
return ResponseEntity.ok(Map.of(
|
||||
"success", true,
|
||||
"message", "Recreated stories collection and indexed " + allStories.size() + " stories",
|
||||
@@ -326,7 +323,7 @@ public class StoryController {
|
||||
@RequestParam(required = false) Integer maxRating,
|
||||
@RequestParam(required = false) String sortBy,
|
||||
@RequestParam(required = false) String sortDir,
|
||||
@RequestParam(required = false) String facetBy,
|
||||
@RequestParam(required = false) List<String> facetBy,
|
||||
// Advanced filters
|
||||
@RequestParam(required = false) Integer minWordCount,
|
||||
@RequestParam(required = false) Integer maxWordCount,
|
||||
@@ -345,16 +342,35 @@ public class StoryController {
|
||||
@RequestParam(required = false) Boolean hiddenGemsOnly) {
|
||||
|
||||
|
||||
if (typesenseService != null) {
|
||||
SearchResultDto<StorySearchDto> results = typesenseService.searchStories(
|
||||
query, page, size, authors, tags, minRating, maxRating, sortBy, sortDir, facetBy,
|
||||
minWordCount, maxWordCount, createdAfter, createdBefore, lastReadAfter, lastReadBefore,
|
||||
unratedOnly, readingStatus, hasReadingProgress, hasCoverImage, sourceDomain, seriesFilter,
|
||||
minTagCount, popularOnly, hiddenGemsOnly);
|
||||
// Use SearchServiceAdapter to handle routing between search engines
|
||||
try {
|
||||
// Convert authors list to single author string (for now, use first author)
|
||||
String authorFilter = (authors != null && !authors.isEmpty()) ? authors.get(0) : null;
|
||||
|
||||
// DEBUG: Log all received parameters
|
||||
logger.info("CONTROLLER DEBUG - Received parameters:");
|
||||
logger.info(" readingStatus: '{}'", readingStatus);
|
||||
logger.info(" seriesFilter: '{}'", seriesFilter);
|
||||
logger.info(" hasReadingProgress: {}", hasReadingProgress);
|
||||
logger.info(" hasCoverImage: {}", hasCoverImage);
|
||||
logger.info(" createdAfter: '{}'", createdAfter);
|
||||
logger.info(" lastReadAfter: '{}'", lastReadAfter);
|
||||
logger.info(" unratedOnly: {}", unratedOnly);
|
||||
|
||||
SearchResultDto<StorySearchDto> results = searchServiceAdapter.searchStories(
|
||||
query, tags, authorFilter, seriesFilter, minWordCount, maxWordCount,
|
||||
minRating != null ? minRating.floatValue() : null,
|
||||
null, // isRead - now handled by readingStatus advanced filter
|
||||
null, // isFavorite - now handled by readingStatus advanced filter
|
||||
sortBy, sortDir, page, size, facetBy,
|
||||
// Advanced filters
|
||||
createdAfter, createdBefore, lastReadAfter, lastReadBefore,
|
||||
unratedOnly, readingStatus, hasReadingProgress, hasCoverImage,
|
||||
sourceDomain, seriesFilter, minTagCount, popularOnly, hiddenGemsOnly);
|
||||
return ResponseEntity.ok(results);
|
||||
} else {
|
||||
// Fallback to basic search if Typesense is not available
|
||||
return ResponseEntity.badRequest().body(null);
|
||||
} catch (Exception e) {
|
||||
logger.error("Search failed", e);
|
||||
return ResponseEntity.internalServerError().body(null);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -363,10 +379,12 @@ public class StoryController {
|
||||
@RequestParam String query,
|
||||
@RequestParam(defaultValue = "5") int limit) {
|
||||
|
||||
if (typesenseService != null) {
|
||||
List<String> suggestions = typesenseService.searchSuggestions(query, limit);
|
||||
// Use SearchServiceAdapter to handle routing between search engines
|
||||
try {
|
||||
List<String> suggestions = searchServiceAdapter.getTagSuggestions(query, limit);
|
||||
return ResponseEntity.ok(suggestions);
|
||||
} else {
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to get search suggestions", e);
|
||||
return ResponseEntity.ok(new ArrayList<>());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,6 +17,7 @@ public class StorySearchDto {
|
||||
|
||||
// Reading status
|
||||
private Boolean isRead;
|
||||
private Integer readingPosition;
|
||||
private LocalDateTime lastReadAt;
|
||||
|
||||
// Author info
|
||||
@@ -33,6 +34,9 @@ public class StorySearchDto {
|
||||
private LocalDateTime createdAt;
|
||||
private LocalDateTime updatedAt;
|
||||
|
||||
// Alias for createdAt to match frontend expectations
|
||||
private LocalDateTime dateAdded;
|
||||
|
||||
// Search-specific fields
|
||||
private double searchScore;
|
||||
private List<String> highlights;
|
||||
@@ -121,6 +125,14 @@ public class StorySearchDto {
|
||||
this.lastReadAt = lastReadAt;
|
||||
}
|
||||
|
||||
public Integer getReadingPosition() {
|
||||
return readingPosition;
|
||||
}
|
||||
|
||||
public void setReadingPosition(Integer readingPosition) {
|
||||
this.readingPosition = readingPosition;
|
||||
}
|
||||
|
||||
public UUID getAuthorId() {
|
||||
return authorId;
|
||||
}
|
||||
@@ -177,6 +189,14 @@ public class StorySearchDto {
|
||||
this.updatedAt = updatedAt;
|
||||
}
|
||||
|
||||
public LocalDateTime getDateAdded() {
|
||||
return dateAdded;
|
||||
}
|
||||
|
||||
public void setDateAdded(LocalDateTime dateAdded) {
|
||||
this.dateAdded = dateAdded;
|
||||
}
|
||||
|
||||
public double getSearchScore() {
|
||||
return searchScore;
|
||||
}
|
||||
|
||||
@@ -1,84 +0,0 @@
|
||||
package com.storycove.scheduled;
|
||||
|
||||
import com.storycove.entity.Story;
|
||||
import com.storycove.service.StoryService;
|
||||
import com.storycove.service.TypesenseService;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
import org.springframework.scheduling.annotation.Scheduled;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Scheduled task to periodically reindex all stories in Typesense
|
||||
* to ensure search index stays synchronized with database changes.
|
||||
*/
|
||||
@Component
|
||||
@ConditionalOnProperty(name = "storycove.typesense.enabled", havingValue = "true", matchIfMissing = true)
|
||||
public class TypesenseIndexScheduler {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(TypesenseIndexScheduler.class);
|
||||
|
||||
private final StoryService storyService;
|
||||
private final TypesenseService typesenseService;
|
||||
|
||||
@Autowired
|
||||
public TypesenseIndexScheduler(StoryService storyService,
|
||||
@Autowired(required = false) TypesenseService typesenseService) {
|
||||
this.storyService = storyService;
|
||||
this.typesenseService = typesenseService;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scheduled task that runs periodically to reindex all stories in Typesense.
|
||||
* This ensures the search index stays synchronized with any database changes
|
||||
* that might have occurred outside of the normal story update flow.
|
||||
*
|
||||
* Interval is configurable via storycove.typesense.reindex-interval property (default: 1 hour).
|
||||
*/
|
||||
@Scheduled(fixedRateString = "${storycove.typesense.reindex-interval:3600000}")
|
||||
public void reindexAllStories() {
|
||||
if (typesenseService == null) {
|
||||
logger.debug("TypesenseService is not available, skipping scheduled reindexing");
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info("Starting scheduled Typesense reindexing at {}", LocalDateTime.now());
|
||||
|
||||
try {
|
||||
long startTime = System.currentTimeMillis();
|
||||
|
||||
// Get all stories from database with eagerly loaded associations
|
||||
List<Story> allStories = storyService.findAllWithAssociations();
|
||||
|
||||
if (allStories.isEmpty()) {
|
||||
logger.info("No stories found in database, skipping reindexing");
|
||||
return;
|
||||
}
|
||||
|
||||
// Perform full reindex
|
||||
typesenseService.reindexAllStories(allStories);
|
||||
|
||||
long endTime = System.currentTimeMillis();
|
||||
long duration = endTime - startTime;
|
||||
|
||||
logger.info("Completed scheduled Typesense reindexing of {} stories in {}ms",
|
||||
allStories.size(), duration);
|
||||
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to complete scheduled Typesense reindexing", e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Manual trigger for reindexing - can be called from other services or endpoints if needed
|
||||
*/
|
||||
public void triggerManualReindex() {
|
||||
logger.info("Manual Typesense reindexing triggered");
|
||||
reindexAllStories();
|
||||
}
|
||||
}
|
||||
@@ -11,21 +11,21 @@ import org.springframework.stereotype.Component;
|
||||
import java.util.List;
|
||||
|
||||
@Component
|
||||
@ConditionalOnProperty(name = "storycove.typesense.enabled", havingValue = "true", matchIfMissing = true)
|
||||
@ConditionalOnProperty(name = "storycove.search.enabled", havingValue = "true", matchIfMissing = true)
|
||||
public class AuthorIndexScheduler {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(AuthorIndexScheduler.class);
|
||||
|
||||
private final AuthorService authorService;
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
@Autowired
|
||||
public AuthorIndexScheduler(AuthorService authorService, TypesenseService typesenseService) {
|
||||
public AuthorIndexScheduler(AuthorService authorService, SearchServiceAdapter searchServiceAdapter) {
|
||||
this.authorService = authorService;
|
||||
this.typesenseService = typesenseService;
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
}
|
||||
|
||||
@Scheduled(fixedRateString = "${storycove.typesense.author-reindex-interval:7200000}") // 2 hours default
|
||||
@Scheduled(fixedRateString = "${storycove.search.author-reindex-interval:7200000}") // 2 hours default
|
||||
public void reindexAllAuthors() {
|
||||
try {
|
||||
logger.info("Starting scheduled author reindexing...");
|
||||
@@ -34,7 +34,7 @@ public class AuthorIndexScheduler {
|
||||
logger.info("Found {} authors to reindex", allAuthors.size());
|
||||
|
||||
if (!allAuthors.isEmpty()) {
|
||||
typesenseService.reindexAllAuthors(allAuthors);
|
||||
searchServiceAdapter.bulkIndexAuthors(allAuthors);
|
||||
logger.info("Successfully completed scheduled author reindexing");
|
||||
} else {
|
||||
logger.info("No authors found to reindex");
|
||||
|
||||
@@ -28,12 +28,12 @@ public class AuthorService {
|
||||
private static final Logger logger = LoggerFactory.getLogger(AuthorService.class);
|
||||
|
||||
private final AuthorRepository authorRepository;
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
@Autowired
|
||||
public AuthorService(AuthorRepository authorRepository, @Autowired(required = false) TypesenseService typesenseService) {
|
||||
public AuthorService(AuthorRepository authorRepository, SearchServiceAdapter searchServiceAdapter) {
|
||||
this.authorRepository = authorRepository;
|
||||
this.typesenseService = typesenseService;
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
}
|
||||
|
||||
@Transactional(readOnly = true)
|
||||
@@ -132,14 +132,8 @@ public class AuthorService {
|
||||
validateAuthorForCreate(author);
|
||||
Author savedAuthor = authorRepository.save(author);
|
||||
|
||||
// Index in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.indexAuthor(savedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to index author in Typesense: " + savedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Index in OpenSearch
|
||||
searchServiceAdapter.indexAuthor(savedAuthor);
|
||||
|
||||
return savedAuthor;
|
||||
}
|
||||
@@ -156,14 +150,8 @@ public class AuthorService {
|
||||
updateAuthorFields(existingAuthor, authorUpdates);
|
||||
Author savedAuthor = authorRepository.save(existingAuthor);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.updateAuthor(savedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to update author in Typesense: " + savedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Update in OpenSearch
|
||||
searchServiceAdapter.updateAuthor(savedAuthor);
|
||||
|
||||
return savedAuthor;
|
||||
}
|
||||
@@ -178,14 +166,8 @@ public class AuthorService {
|
||||
|
||||
authorRepository.delete(author);
|
||||
|
||||
// Remove from Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.deleteAuthor(id.toString());
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to delete author from Typesense: " + author.getName(), e);
|
||||
}
|
||||
}
|
||||
// Remove from OpenSearch
|
||||
searchServiceAdapter.deleteAuthor(id);
|
||||
}
|
||||
|
||||
public Author addUrl(UUID id, String url) {
|
||||
@@ -193,14 +175,8 @@ public class AuthorService {
|
||||
author.addUrl(url);
|
||||
Author savedAuthor = authorRepository.save(author);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.updateAuthor(savedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to update author in Typesense after adding URL: " + savedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Update in OpenSearch
|
||||
searchServiceAdapter.updateAuthor(savedAuthor);
|
||||
|
||||
return savedAuthor;
|
||||
}
|
||||
@@ -210,14 +186,8 @@ public class AuthorService {
|
||||
author.removeUrl(url);
|
||||
Author savedAuthor = authorRepository.save(author);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.updateAuthor(savedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to update author in Typesense after removing URL: " + savedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Update in OpenSearch
|
||||
searchServiceAdapter.updateAuthor(savedAuthor);
|
||||
|
||||
return savedAuthor;
|
||||
}
|
||||
@@ -251,14 +221,8 @@ public class AuthorService {
|
||||
logger.debug("Saved author rating: {} for author: {}",
|
||||
refreshedAuthor.getAuthorRating(), refreshedAuthor.getName());
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.updateAuthor(refreshedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to update author in Typesense after rating: " + refreshedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Update in OpenSearch
|
||||
searchServiceAdapter.updateAuthor(refreshedAuthor);
|
||||
|
||||
return refreshedAuthor;
|
||||
}
|
||||
@@ -301,14 +265,8 @@ public class AuthorService {
|
||||
author.setAvatarImagePath(avatarPath);
|
||||
Author savedAuthor = authorRepository.save(author);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.updateAuthor(savedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to update author in Typesense after setting avatar: " + savedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Update in OpenSearch
|
||||
searchServiceAdapter.updateAuthor(savedAuthor);
|
||||
|
||||
return savedAuthor;
|
||||
}
|
||||
@@ -318,14 +276,8 @@ public class AuthorService {
|
||||
author.setAvatarImagePath(null);
|
||||
Author savedAuthor = authorRepository.save(author);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
typesenseService.updateAuthor(savedAuthor);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to update author in Typesense after removing avatar: " + savedAuthor.getName(), e);
|
||||
}
|
||||
}
|
||||
// Update in OpenSearch
|
||||
searchServiceAdapter.updateAuthor(savedAuthor);
|
||||
|
||||
return savedAuthor;
|
||||
}
|
||||
|
||||
@@ -31,7 +31,7 @@ public class CollectionService {
|
||||
private final CollectionStoryRepository collectionStoryRepository;
|
||||
private final StoryRepository storyRepository;
|
||||
private final TagRepository tagRepository;
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
private final ReadingTimeService readingTimeService;
|
||||
|
||||
@Autowired
|
||||
@@ -39,13 +39,13 @@ public class CollectionService {
|
||||
CollectionStoryRepository collectionStoryRepository,
|
||||
StoryRepository storyRepository,
|
||||
TagRepository tagRepository,
|
||||
@Autowired(required = false) TypesenseService typesenseService,
|
||||
SearchServiceAdapter searchServiceAdapter,
|
||||
ReadingTimeService readingTimeService) {
|
||||
this.collectionRepository = collectionRepository;
|
||||
this.collectionStoryRepository = collectionStoryRepository;
|
||||
this.storyRepository = storyRepository;
|
||||
this.tagRepository = tagRepository;
|
||||
this.typesenseService = typesenseService;
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
this.readingTimeService = readingTimeService;
|
||||
}
|
||||
|
||||
@@ -54,13 +54,10 @@ public class CollectionService {
|
||||
* This method MUST be used instead of JPA queries for listing collections
|
||||
*/
|
||||
public SearchResultDto<Collection> searchCollections(String query, List<String> tags, boolean includeArchived, int page, int limit) {
|
||||
if (typesenseService == null) {
|
||||
logger.warn("Typesense service not available, returning empty results");
|
||||
return new SearchResultDto<>(new ArrayList<>(), 0, page, limit, query != null ? query : "", 0);
|
||||
}
|
||||
|
||||
// Delegate to TypesenseService for all search operations
|
||||
return typesenseService.searchCollections(query, tags, includeArchived, page, limit);
|
||||
// Collections are currently handled at database level, not indexed in search engine
|
||||
// Return empty result for now as collections search is not implemented in OpenSearch
|
||||
logger.warn("Collections search not yet implemented in OpenSearch, returning empty results");
|
||||
return new SearchResultDto<>(new ArrayList<>(), 0, page, limit, query != null ? query : "", 0);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -107,10 +104,7 @@ public class CollectionService {
|
||||
savedCollection = findById(savedCollection.getId());
|
||||
}
|
||||
|
||||
// Index in Typesense
|
||||
if (typesenseService != null) {
|
||||
typesenseService.indexCollection(savedCollection);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
logger.info("Created collection: {} with {} stories", name, initialStoryIds != null ? initialStoryIds.size() : 0);
|
||||
return savedCollection;
|
||||
@@ -140,10 +134,7 @@ public class CollectionService {
|
||||
|
||||
Collection savedCollection = collectionRepository.save(collection);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
typesenseService.indexCollection(savedCollection);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
logger.info("Updated collection: {}", id);
|
||||
return savedCollection;
|
||||
@@ -155,10 +146,7 @@ public class CollectionService {
|
||||
public void deleteCollection(UUID id) {
|
||||
Collection collection = findByIdBasic(id);
|
||||
|
||||
// Remove from Typesense first
|
||||
if (typesenseService != null) {
|
||||
typesenseService.removeCollection(id);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
collectionRepository.delete(collection);
|
||||
logger.info("Deleted collection: {}", id);
|
||||
@@ -173,10 +161,7 @@ public class CollectionService {
|
||||
|
||||
Collection savedCollection = collectionRepository.save(collection);
|
||||
|
||||
// Update in Typesense
|
||||
if (typesenseService != null) {
|
||||
typesenseService.indexCollection(savedCollection);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
logger.info("{} collection: {}", archived ? "Archived" : "Unarchived", id);
|
||||
return savedCollection;
|
||||
@@ -221,10 +206,7 @@ public class CollectionService {
|
||||
}
|
||||
|
||||
// Update collection in Typesense
|
||||
if (typesenseService != null) {
|
||||
Collection updatedCollection = findById(collectionId);
|
||||
typesenseService.indexCollection(updatedCollection);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
long totalStories = collectionStoryRepository.countByCollectionId(collectionId);
|
||||
|
||||
@@ -249,10 +231,7 @@ public class CollectionService {
|
||||
collectionStoryRepository.delete(collectionStory);
|
||||
|
||||
// Update collection in Typesense
|
||||
if (typesenseService != null) {
|
||||
Collection updatedCollection = findById(collectionId);
|
||||
typesenseService.indexCollection(updatedCollection);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
logger.info("Removed story {} from collection {}", storyId, collectionId);
|
||||
}
|
||||
@@ -285,10 +264,7 @@ public class CollectionService {
|
||||
}
|
||||
|
||||
// Update collection in Typesense
|
||||
if (typesenseService != null) {
|
||||
Collection updatedCollection = findById(collectionId);
|
||||
typesenseService.indexCollection(updatedCollection);
|
||||
}
|
||||
// Collections are not indexed in search engine yet
|
||||
|
||||
logger.info("Reordered {} stories in collection {}", storyOrders.size(), collectionId);
|
||||
}
|
||||
@@ -423,7 +399,7 @@ public class CollectionService {
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all collections for indexing (used by TypesenseService)
|
||||
* Get all collections for indexing (used by SearchServiceAdapter)
|
||||
*/
|
||||
public List<Collection> findAllForIndexing() {
|
||||
return collectionRepository.findAllActiveCollections();
|
||||
|
||||
@@ -52,7 +52,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
private CollectionRepository collectionRepository;
|
||||
|
||||
@Autowired
|
||||
private TypesenseService typesenseService;
|
||||
private SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
@Autowired
|
||||
private LibraryService libraryService;
|
||||
@@ -145,15 +145,15 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
System.err.println("No files directory found in backup - skipping file restore.");
|
||||
}
|
||||
|
||||
// 6. Trigger complete Typesense reindex after data restoration
|
||||
// 6. Trigger complete search index reindex after data restoration
|
||||
try {
|
||||
System.err.println("Starting Typesense reindex after restore...");
|
||||
TypesenseService typesenseService = applicationContext.getBean(TypesenseService.class);
|
||||
typesenseService.performCompleteReindex();
|
||||
System.err.println("Typesense reindex completed successfully.");
|
||||
System.err.println("Starting search index reindex after restore...");
|
||||
SearchServiceAdapter searchServiceAdapter = applicationContext.getBean(SearchServiceAdapter.class);
|
||||
searchServiceAdapter.performCompleteReindex();
|
||||
System.err.println("Search index reindex completed successfully.");
|
||||
} catch (Exception e) {
|
||||
System.err.println("Warning: Failed to reindex Typesense after restore: " + e.getMessage());
|
||||
// Don't fail the entire restore for Typesense issues
|
||||
System.err.println("Warning: Failed to reindex search after restore: " + e.getMessage());
|
||||
// Don't fail the entire restore for search issues
|
||||
}
|
||||
|
||||
System.err.println("Complete backup restore finished successfully.");
|
||||
@@ -299,9 +299,9 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
// Reindex search after successful restore
|
||||
try {
|
||||
String currentLibraryId = libraryService.getCurrentLibraryId();
|
||||
System.err.println("Starting Typesense reindex after successful restore for library: " + currentLibraryId);
|
||||
System.err.println("Starting search reindex after successful restore for library: " + currentLibraryId);
|
||||
if (currentLibraryId == null) {
|
||||
System.err.println("ERROR: No current library set during restore - cannot reindex Typesense!");
|
||||
System.err.println("ERROR: No current library set during restore - cannot reindex search!");
|
||||
throw new IllegalStateException("No current library active during restore");
|
||||
}
|
||||
|
||||
@@ -310,10 +310,10 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
reindexStoriesAndAuthorsFromCurrentDatabase();
|
||||
|
||||
// Note: Collections collection will be recreated when needed by the service
|
||||
System.err.println("Typesense reindex completed successfully for library: " + currentLibraryId);
|
||||
System.err.println("Search reindex completed successfully for library: " + currentLibraryId);
|
||||
} catch (Exception e) {
|
||||
// Log the error but don't fail the restore
|
||||
System.err.println("Warning: Failed to reindex Typesense after restore: " + e.getMessage());
|
||||
System.err.println("Warning: Failed to reindex search after restore: " + e.getMessage());
|
||||
e.printStackTrace();
|
||||
}
|
||||
|
||||
@@ -351,7 +351,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
totalDeleted = collectionCount + storyCount + authorCount + seriesCount + tagCount;
|
||||
|
||||
// Note: Search indexes will need to be manually recreated after clearing
|
||||
// Use the settings page to recreate Typesense collections after clearing the database
|
||||
// Use the settings page to recreate search indices after clearing the database
|
||||
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException("Failed to clear database: " + e.getMessage(), e);
|
||||
@@ -506,8 +506,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
}
|
||||
|
||||
// For clearing, we only want to recreate empty collections (no data to index)
|
||||
typesenseService.recreateStoriesCollection();
|
||||
typesenseService.recreateAuthorsCollection();
|
||||
searchServiceAdapter.recreateIndices();
|
||||
// Note: Collections collection will be recreated when needed by the service
|
||||
System.err.println("Search indexes cleared successfully for library: " + currentLibraryId);
|
||||
} catch (Exception e) {
|
||||
@@ -959,10 +958,9 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
try (Connection connection = getDataSource().getConnection()) {
|
||||
// First, recreate empty collections
|
||||
try {
|
||||
typesenseService.recreateStoriesCollection();
|
||||
typesenseService.recreateAuthorsCollection();
|
||||
searchServiceAdapter.recreateIndices();
|
||||
} catch (Exception e) {
|
||||
throw new SQLException("Failed to recreate Typesense collections", e);
|
||||
throw new SQLException("Failed to recreate search indices", e);
|
||||
}
|
||||
|
||||
// Count and reindex stories with full author and series information
|
||||
@@ -984,7 +982,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
while (rs.next()) {
|
||||
// Create a complete Story object for indexing
|
||||
var story = createStoryFromResultSet(rs);
|
||||
typesenseService.indexStory(story);
|
||||
searchServiceAdapter.indexStory(story);
|
||||
storyCount++;
|
||||
}
|
||||
}
|
||||
@@ -999,7 +997,7 @@ public class DatabaseManagementService implements ApplicationContextAware {
|
||||
while (rs.next()) {
|
||||
// Create a minimal Author object for indexing
|
||||
var author = createAuthorFromResultSet(rs);
|
||||
typesenseService.indexAuthor(author);
|
||||
searchServiceAdapter.indexAuthor(author);
|
||||
authorCount++;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -74,6 +74,34 @@ public class EPUBImportService {
|
||||
|
||||
Story savedStory = storyService.create(story);
|
||||
|
||||
// Process embedded images if content contains any
|
||||
String originalContent = story.getContentHtml();
|
||||
if (originalContent != null && originalContent.contains("<img")) {
|
||||
try {
|
||||
ImageService.ContentImageProcessingResult imageResult =
|
||||
imageService.processContentImages(originalContent, savedStory.getId());
|
||||
|
||||
// Update story content with processed images if changed
|
||||
if (!imageResult.getProcessedContent().equals(originalContent)) {
|
||||
savedStory.setContentHtml(imageResult.getProcessedContent());
|
||||
savedStory = storyService.update(savedStory.getId(), savedStory);
|
||||
|
||||
// Log the image processing results
|
||||
System.out.println("EPUB Import - Image processing completed for story " + savedStory.getId() +
|
||||
". Downloaded " + imageResult.getDownloadedImages().size() + " images.");
|
||||
|
||||
if (imageResult.hasWarnings()) {
|
||||
System.out.println("EPUB Import - Image processing warnings: " +
|
||||
String.join(", ", imageResult.getWarnings()));
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
// Log error but don't fail the import
|
||||
System.err.println("EPUB Import - Failed to process embedded images for story " +
|
||||
savedStory.getId() + ": " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
EPUBImportResponse response = EPUBImportResponse.success(savedStory.getId(), savedStory.getTitle());
|
||||
response.setWordCount(savedStory.getWordCount());
|
||||
response.setTotalChapters(book.getSpine().size());
|
||||
|
||||
@@ -40,6 +40,9 @@ public class ImageService {
|
||||
@Autowired
|
||||
private LibraryService libraryService;
|
||||
|
||||
@Autowired
|
||||
private StoryService storyService;
|
||||
|
||||
private String getUploadDir() {
|
||||
String libraryPath = libraryService.getCurrentImagePath();
|
||||
return baseUploadDir + libraryPath;
|
||||
@@ -421,6 +424,249 @@ public class ImageService {
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cleanup orphaned content images that are no longer referenced in any story
|
||||
*/
|
||||
public ContentImageCleanupResult cleanupOrphanedContentImages(boolean dryRun) {
|
||||
logger.info("Starting orphaned content image cleanup (dryRun: {})", dryRun);
|
||||
|
||||
final Set<String> referencedImages;
|
||||
List<String> orphanedImages = new ArrayList<>();
|
||||
List<String> errors = new ArrayList<>();
|
||||
long totalSizeBytes = 0;
|
||||
int foldersToDelete = 0;
|
||||
|
||||
// Step 1: Collect all image references from all story content
|
||||
logger.info("Scanning all story content for image references...");
|
||||
referencedImages = collectAllImageReferences();
|
||||
logger.info("Found {} unique image references in story content", referencedImages.size());
|
||||
|
||||
try {
|
||||
// Step 2: Scan the content images directory
|
||||
Path contentImagesDir = Paths.get(getUploadDir(), ImageType.CONTENT.getDirectory());
|
||||
|
||||
if (!Files.exists(contentImagesDir)) {
|
||||
logger.info("Content images directory does not exist: {}", contentImagesDir);
|
||||
return new ContentImageCleanupResult(orphanedImages, 0, 0, referencedImages.size(), errors, dryRun);
|
||||
}
|
||||
|
||||
logger.info("Scanning content images directory: {}", contentImagesDir);
|
||||
|
||||
// Walk through all story directories
|
||||
Files.walk(contentImagesDir, 2)
|
||||
.filter(Files::isDirectory)
|
||||
.filter(path -> !path.equals(contentImagesDir)) // Skip the root content directory
|
||||
.forEach(storyDir -> {
|
||||
try {
|
||||
String storyId = storyDir.getFileName().toString();
|
||||
logger.debug("Checking story directory: {}", storyId);
|
||||
|
||||
// Check if this story still exists
|
||||
boolean storyExists = storyService.findByIdOptional(UUID.fromString(storyId)).isPresent();
|
||||
|
||||
if (!storyExists) {
|
||||
logger.info("Found orphaned story directory (story deleted): {}", storyId);
|
||||
// Mark entire directory for deletion
|
||||
try {
|
||||
Files.walk(storyDir)
|
||||
.filter(Files::isRegularFile)
|
||||
.forEach(file -> {
|
||||
try {
|
||||
long size = Files.size(file);
|
||||
orphanedImages.add(file.toString());
|
||||
// Add to total size (will be updated in main scope)
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to get size for " + file + ": " + e.getMessage());
|
||||
}
|
||||
});
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to scan orphaned story directory " + storyDir + ": " + e.getMessage());
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Check individual files in the story directory
|
||||
try {
|
||||
Files.walk(storyDir)
|
||||
.filter(Files::isRegularFile)
|
||||
.forEach(imageFile -> {
|
||||
try {
|
||||
String imagePath = getRelativeImagePath(imageFile);
|
||||
|
||||
if (!referencedImages.contains(imagePath)) {
|
||||
logger.debug("Found orphaned image: {}", imagePath);
|
||||
orphanedImages.add(imageFile.toString());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
errors.add("Error checking image file " + imageFile + ": " + e.getMessage());
|
||||
}
|
||||
});
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to scan story directory " + storyDir + ": " + e.getMessage());
|
||||
}
|
||||
|
||||
} catch (Exception e) {
|
||||
errors.add("Error processing story directory " + storyDir + ": " + e.getMessage());
|
||||
}
|
||||
});
|
||||
|
||||
// Calculate total size and count empty directories
|
||||
for (String orphanedImage : orphanedImages) {
|
||||
try {
|
||||
Path imagePath = Paths.get(orphanedImage);
|
||||
if (Files.exists(imagePath)) {
|
||||
totalSizeBytes += Files.size(imagePath);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to get size for " + orphanedImage + ": " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
// Count empty directories that would be removed
|
||||
try {
|
||||
foldersToDelete = (int) Files.walk(contentImagesDir)
|
||||
.filter(Files::isDirectory)
|
||||
.filter(path -> !path.equals(contentImagesDir))
|
||||
.filter(this::isDirectoryEmptyOrWillBeEmpty)
|
||||
.count();
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to count empty directories: " + e.getMessage());
|
||||
}
|
||||
|
||||
// Step 3: Delete orphaned files if not dry run
|
||||
if (!dryRun && !orphanedImages.isEmpty()) {
|
||||
logger.info("Deleting {} orphaned images...", orphanedImages.size());
|
||||
|
||||
Set<Path> directoriesToCheck = new HashSet<>();
|
||||
|
||||
for (String orphanedImage : orphanedImages) {
|
||||
try {
|
||||
Path imagePath = Paths.get(orphanedImage);
|
||||
if (Files.exists(imagePath)) {
|
||||
directoriesToCheck.add(imagePath.getParent());
|
||||
Files.delete(imagePath);
|
||||
logger.debug("Deleted orphaned image: {}", imagePath);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to delete " + orphanedImage + ": " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up empty directories
|
||||
for (Path dir : directoriesToCheck) {
|
||||
try {
|
||||
if (Files.exists(dir) && isDirEmpty(dir)) {
|
||||
Files.delete(dir);
|
||||
logger.info("Deleted empty story directory: {}", dir);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
errors.add("Failed to delete empty directory " + dir + ": " + e.getMessage());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
logger.info("Orphaned content image cleanup completed. Found {} orphaned files ({} bytes)",
|
||||
orphanedImages.size(), totalSizeBytes);
|
||||
|
||||
} catch (Exception e) {
|
||||
logger.error("Error during orphaned content image cleanup", e);
|
||||
errors.add("General cleanup error: " + e.getMessage());
|
||||
}
|
||||
|
||||
return new ContentImageCleanupResult(orphanedImages, totalSizeBytes, foldersToDelete, referencedImages.size(), errors, dryRun);
|
||||
}
|
||||
|
||||
/**
|
||||
* Collect all image references from all story content
|
||||
*/
|
||||
private Set<String> collectAllImageReferences() {
|
||||
Set<String> referencedImages = new HashSet<>();
|
||||
|
||||
try {
|
||||
// Get all stories
|
||||
List<com.storycove.entity.Story> allStories = storyService.findAllWithAssociations();
|
||||
|
||||
// Pattern to match local image URLs in content
|
||||
Pattern imagePattern = Pattern.compile("src=[\"']([^\"']*(?:content/[^\"']*\\.(jpg|jpeg|png)))[\"']", Pattern.CASE_INSENSITIVE);
|
||||
|
||||
for (com.storycove.entity.Story story : allStories) {
|
||||
if (story.getContentHtml() != null) {
|
||||
Matcher matcher = imagePattern.matcher(story.getContentHtml());
|
||||
|
||||
while (matcher.find()) {
|
||||
String imageSrc = matcher.group(1);
|
||||
|
||||
// Convert to relative path format that matches our file system
|
||||
String relativePath = convertSrcToRelativePath(imageSrc);
|
||||
if (relativePath != null) {
|
||||
referencedImages.add(relativePath);
|
||||
logger.debug("Found image reference in story {}: {}", story.getId(), relativePath);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
} catch (Exception e) {
|
||||
logger.error("Error collecting image references from stories", e);
|
||||
}
|
||||
|
||||
return referencedImages;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert an image src attribute to relative file path
|
||||
*/
|
||||
private String convertSrcToRelativePath(String src) {
|
||||
try {
|
||||
// Handle both /api/files/images/libraryId/content/... and relative content/... paths
|
||||
if (src.contains("/content/")) {
|
||||
int contentIndex = src.indexOf("/content/");
|
||||
return src.substring(contentIndex + 1); // Remove leading slash, keep "content/..."
|
||||
}
|
||||
} catch (Exception e) {
|
||||
logger.debug("Failed to convert src to relative path: {}", src);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get relative image path from absolute file path
|
||||
*/
|
||||
private String getRelativeImagePath(Path imageFile) {
|
||||
try {
|
||||
Path uploadDir = Paths.get(getUploadDir());
|
||||
Path relativePath = uploadDir.relativize(imageFile);
|
||||
return relativePath.toString().replace('\\', '/'); // Normalize path separators
|
||||
} catch (Exception e) {
|
||||
logger.debug("Failed to get relative path for: {}", imageFile);
|
||||
return imageFile.toString();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if directory is empty or will be empty after cleanup
|
||||
*/
|
||||
private boolean isDirectoryEmptyOrWillBeEmpty(Path dir) {
|
||||
try {
|
||||
return Files.walk(dir)
|
||||
.filter(Files::isRegularFile)
|
||||
.count() == 0;
|
||||
} catch (IOException e) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if directory is empty
|
||||
*/
|
||||
private boolean isDirEmpty(Path dir) {
|
||||
try {
|
||||
return Files.list(dir).count() == 0;
|
||||
} catch (IOException e) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up content images for a story
|
||||
*/
|
||||
@@ -458,4 +704,41 @@ public class ImageService {
|
||||
public List<String> getDownloadedImages() { return downloadedImages; }
|
||||
public boolean hasWarnings() { return !warnings.isEmpty(); }
|
||||
}
|
||||
|
||||
/**
|
||||
* Result class for orphaned image cleanup
|
||||
*/
|
||||
public static class ContentImageCleanupResult {
|
||||
private final List<String> orphanedImages;
|
||||
private final long totalSizeBytes;
|
||||
private final int foldersToDelete;
|
||||
private final int totalReferencedImages;
|
||||
private final List<String> errors;
|
||||
private final boolean dryRun;
|
||||
|
||||
public ContentImageCleanupResult(List<String> orphanedImages, long totalSizeBytes, int foldersToDelete,
|
||||
int totalReferencedImages, List<String> errors, boolean dryRun) {
|
||||
this.orphanedImages = orphanedImages;
|
||||
this.totalSizeBytes = totalSizeBytes;
|
||||
this.foldersToDelete = foldersToDelete;
|
||||
this.totalReferencedImages = totalReferencedImages;
|
||||
this.errors = errors;
|
||||
this.dryRun = dryRun;
|
||||
}
|
||||
|
||||
public List<String> getOrphanedImages() { return orphanedImages; }
|
||||
public long getTotalSizeBytes() { return totalSizeBytes; }
|
||||
public int getFoldersToDelete() { return foldersToDelete; }
|
||||
public int getTotalReferencedImages() { return totalReferencedImages; }
|
||||
public List<String> getErrors() { return errors; }
|
||||
public boolean isDryRun() { return dryRun; }
|
||||
public boolean hasErrors() { return !errors.isEmpty(); }
|
||||
|
||||
public String getFormattedSize() {
|
||||
if (totalSizeBytes < 1024) return totalSizeBytes + " B";
|
||||
if (totalSizeBytes < 1024 * 1024) return String.format("%.1f KB", totalSizeBytes / 1024.0);
|
||||
if (totalSizeBytes < 1024 * 1024 * 1024) return String.format("%.1f MB", totalSizeBytes / (1024.0 * 1024.0));
|
||||
return String.format("%.1f GB", totalSizeBytes / (1024.0 * 1024.0 * 1024.0));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -13,8 +13,6 @@ import org.springframework.context.ApplicationContext;
|
||||
import org.springframework.context.ApplicationContextAware;
|
||||
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.typesense.api.Client;
|
||||
import org.typesense.resources.Node;
|
||||
|
||||
import jakarta.annotation.PostConstruct;
|
||||
import jakarta.annotation.PreDestroy;
|
||||
@@ -26,7 +24,6 @@ import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.sql.SQLException;
|
||||
import java.time.Duration;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
@@ -43,14 +40,6 @@ public class LibraryService implements ApplicationContextAware {
|
||||
@Value("${spring.datasource.password}")
|
||||
private String dbPassword;
|
||||
|
||||
@Value("${typesense.host}")
|
||||
private String typesenseHost;
|
||||
|
||||
@Value("${typesense.port}")
|
||||
private String typesensePort;
|
||||
|
||||
@Value("${typesense.api-key}")
|
||||
private String typesenseApiKey;
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
private final BCryptPasswordEncoder passwordEncoder = new BCryptPasswordEncoder();
|
||||
@@ -61,7 +50,6 @@ public class LibraryService implements ApplicationContextAware {
|
||||
|
||||
// Current active resources
|
||||
private volatile String currentLibraryId;
|
||||
private volatile Client currentTypesenseClient;
|
||||
|
||||
// Security: Track if user has explicitly authenticated in this session
|
||||
private volatile boolean explicitlyAuthenticated = false;
|
||||
@@ -100,7 +88,6 @@ public class LibraryService implements ApplicationContextAware {
|
||||
@PreDestroy
|
||||
public void cleanup() {
|
||||
currentLibraryId = null;
|
||||
currentTypesenseClient = null;
|
||||
explicitlyAuthenticated = false;
|
||||
}
|
||||
|
||||
@@ -110,7 +97,6 @@ public class LibraryService implements ApplicationContextAware {
|
||||
public void clearAuthentication() {
|
||||
explicitlyAuthenticated = false;
|
||||
currentLibraryId = null;
|
||||
currentTypesenseClient = null;
|
||||
logger.info("Authentication cleared - user must re-authenticate to access libraries");
|
||||
}
|
||||
|
||||
@@ -129,7 +115,7 @@ public class LibraryService implements ApplicationContextAware {
|
||||
|
||||
/**
|
||||
* Switch to library after authentication with forced reindexing
|
||||
* This ensures Typesense is always up-to-date after login
|
||||
* This ensures OpenSearch is always up-to-date after login
|
||||
*/
|
||||
public synchronized void switchToLibraryAfterAuthentication(String libraryId) throws Exception {
|
||||
logger.info("Switching to library after authentication: {} (forcing reindex)", libraryId);
|
||||
@@ -168,25 +154,15 @@ public class LibraryService implements ApplicationContextAware {
|
||||
|
||||
// Set new active library (datasource routing handled by SmartRoutingDataSource)
|
||||
currentLibraryId = libraryId;
|
||||
currentTypesenseClient = createTypesenseClient(library.getTypesenseCollection());
|
||||
|
||||
// Initialize Typesense collections for this library
|
||||
try {
|
||||
TypesenseService typesenseService = applicationContext.getBean(TypesenseService.class);
|
||||
// First ensure collections exist
|
||||
typesenseService.initializeCollectionsForCurrentLibrary();
|
||||
logger.info("Completed Typesense initialization for library: {}", libraryId);
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to initialize Typesense for library {}: {}", libraryId, e.getMessage());
|
||||
// Don't fail the switch - collections can be created later
|
||||
}
|
||||
// OpenSearch indexes are global - no per-library initialization needed
|
||||
logger.info("Library switched to OpenSearch mode for library: {}", libraryId);
|
||||
|
||||
logger.info("Successfully switched to library: {}", library.getName());
|
||||
|
||||
// Perform complete reindex AFTER library switch is fully complete
|
||||
// This ensures database routing is properly established
|
||||
if (forceReindex || !libraryId.equals(previousLibraryId)) {
|
||||
logger.info("Starting post-switch Typesense reindex for library: {}", libraryId);
|
||||
logger.info("Starting post-switch OpenSearch reindex for library: {}", libraryId);
|
||||
|
||||
// Run reindex asynchronously to avoid blocking authentication response
|
||||
// and allow time for database routing to fully stabilize
|
||||
@@ -195,15 +171,25 @@ public class LibraryService implements ApplicationContextAware {
|
||||
try {
|
||||
// Give routing time to stabilize
|
||||
Thread.sleep(500);
|
||||
logger.info("Starting async Typesense reindex for library: {}", finalLibraryId);
|
||||
logger.info("Starting async OpenSearch reindex for library: {}", finalLibraryId);
|
||||
|
||||
TypesenseService typesenseService = applicationContext.getBean(TypesenseService.class);
|
||||
typesenseService.performCompleteReindex();
|
||||
logger.info("Completed async Typesense reindexing for library: {}", finalLibraryId);
|
||||
SearchServiceAdapter searchService = applicationContext.getBean(SearchServiceAdapter.class);
|
||||
// Get all stories and authors for reindexing
|
||||
StoryService storyService = applicationContext.getBean(StoryService.class);
|
||||
AuthorService authorService = applicationContext.getBean(AuthorService.class);
|
||||
|
||||
var allStories = storyService.findAllWithAssociations();
|
||||
var allAuthors = authorService.findAllWithStories();
|
||||
|
||||
searchService.bulkIndexStories(allStories);
|
||||
searchService.bulkIndexAuthors(allAuthors);
|
||||
|
||||
logger.info("Completed async OpenSearch reindexing for library: {} ({} stories, {} authors)",
|
||||
finalLibraryId, allStories.size(), allAuthors.size());
|
||||
} catch (Exception e) {
|
||||
logger.warn("Failed to async reindex Typesense for library {}: {}", finalLibraryId, e.getMessage());
|
||||
logger.warn("Failed to async reindex OpenSearch for library {}: {}", finalLibraryId, e.getMessage());
|
||||
}
|
||||
}, "TypesenseReindex-" + libraryId).start();
|
||||
}, "OpenSearchReindex-" + libraryId).start();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -219,12 +205,6 @@ public class LibraryService implements ApplicationContextAware {
|
||||
}
|
||||
}
|
||||
|
||||
public Client getCurrentTypesenseClient() {
|
||||
if (currentTypesenseClient == null) {
|
||||
throw new IllegalStateException("No active library - please authenticate first");
|
||||
}
|
||||
return currentTypesenseClient;
|
||||
}
|
||||
|
||||
public String getCurrentLibraryId() {
|
||||
return currentLibraryId;
|
||||
@@ -545,8 +525,8 @@ public class LibraryService implements ApplicationContextAware {
|
||||
// 1. Create image directory structure
|
||||
initializeImageDirectories(library);
|
||||
|
||||
// 2. Initialize Typesense collections (this will be done when switching to the library)
|
||||
// The TypesenseService.initializeCollections() will be called automatically
|
||||
// 2. OpenSearch indexes are global and managed automatically
|
||||
// No per-library initialization needed for OpenSearch
|
||||
|
||||
logger.info("Successfully initialized resources for library: {}", library.getName());
|
||||
|
||||
@@ -777,21 +757,10 @@ public class LibraryService implements ApplicationContextAware {
|
||||
}
|
||||
}
|
||||
|
||||
private Client createTypesenseClient(String collection) {
|
||||
logger.info("Creating Typesense client for collection: {}", collection);
|
||||
|
||||
List<Node> nodes = Arrays.asList(
|
||||
new Node("http", typesenseHost, typesensePort)
|
||||
);
|
||||
|
||||
org.typesense.api.Configuration configuration = new org.typesense.api.Configuration(nodes, Duration.ofSeconds(10), typesenseApiKey);
|
||||
return new Client(configuration);
|
||||
}
|
||||
|
||||
private void closeCurrentResources() {
|
||||
// No need to close datasource - SmartRoutingDataSource handles this
|
||||
// Typesense client doesn't need explicit cleanup
|
||||
currentTypesenseClient = null;
|
||||
// OpenSearch service is managed by Spring - no explicit cleanup needed
|
||||
// Don't clear currentLibraryId here - only when explicitly switching
|
||||
}
|
||||
|
||||
@@ -848,7 +817,6 @@ public class LibraryService implements ApplicationContextAware {
|
||||
config.put("description", library.getDescription());
|
||||
config.put("passwordHash", library.getPasswordHash());
|
||||
config.put("dbName", library.getDbName());
|
||||
config.put("typesenseCollection", library.getTypesenseCollection());
|
||||
config.put("imagePath", library.getImagePath());
|
||||
config.put("initialized", library.isInitialized());
|
||||
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
package com.storycove.service;
|
||||
|
||||
import com.storycove.config.OpenSearchProperties;
|
||||
import org.opensearch.client.opensearch.OpenSearchClient;
|
||||
import org.opensearch.client.opensearch.cluster.HealthRequest;
|
||||
import org.opensearch.client.opensearch.cluster.HealthResponse;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.actuate.health.Health;
|
||||
import org.springframework.boot.actuate.health.HealthIndicator;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
import org.springframework.scheduling.annotation.Scheduled;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
|
||||
@Service
|
||||
@ConditionalOnProperty(name = "storycove.search.engine", havingValue = "opensearch")
|
||||
public class OpenSearchHealthService implements HealthIndicator {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(OpenSearchHealthService.class);
|
||||
|
||||
private final OpenSearchClient openSearchClient;
|
||||
private final OpenSearchProperties properties;
|
||||
|
||||
private final AtomicReference<Health> lastKnownHealth = new AtomicReference<>(Health.unknown().build());
|
||||
private LocalDateTime lastCheckTime = LocalDateTime.now();
|
||||
|
||||
@Autowired
|
||||
public OpenSearchHealthService(OpenSearchClient openSearchClient, OpenSearchProperties properties) {
|
||||
this.openSearchClient = openSearchClient;
|
||||
this.properties = properties;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Health health() {
|
||||
return lastKnownHealth.get();
|
||||
}
|
||||
|
||||
@Scheduled(fixedDelayString = "#{@openSearchProperties.health.checkInterval}")
|
||||
public void performHealthCheck() {
|
||||
try {
|
||||
HealthResponse clusterHealth = openSearchClient.cluster().health(
|
||||
HealthRequest.of(h -> h.timeout(t -> t.time("10s")))
|
||||
);
|
||||
|
||||
Health.Builder healthBuilder = Health.up()
|
||||
.withDetail("cluster_name", clusterHealth.clusterName())
|
||||
.withDetail("status", clusterHealth.status().jsonValue())
|
||||
.withDetail("number_of_nodes", clusterHealth.numberOfNodes())
|
||||
.withDetail("number_of_data_nodes", clusterHealth.numberOfDataNodes())
|
||||
.withDetail("active_primary_shards", clusterHealth.activePrimaryShards())
|
||||
.withDetail("active_shards", clusterHealth.activeShards())
|
||||
.withDetail("relocating_shards", clusterHealth.relocatingShards())
|
||||
.withDetail("initializing_shards", clusterHealth.initializingShards())
|
||||
.withDetail("unassigned_shards", clusterHealth.unassignedShards())
|
||||
.withDetail("last_check", LocalDateTime.now());
|
||||
|
||||
// Check if cluster status is concerning
|
||||
switch (clusterHealth.status()) {
|
||||
case Red:
|
||||
healthBuilder = Health.down()
|
||||
.withDetail("reason", "Cluster status is RED - some primary shards are unassigned");
|
||||
break;
|
||||
case Yellow:
|
||||
if (isProduction()) {
|
||||
healthBuilder = Health.down()
|
||||
.withDetail("reason", "Cluster status is YELLOW - some replica shards are unassigned (critical in production)");
|
||||
} else {
|
||||
// Yellow is acceptable in development (single node clusters)
|
||||
healthBuilder.withDetail("warning", "Cluster status is YELLOW - acceptable for development");
|
||||
}
|
||||
break;
|
||||
case Green:
|
||||
// All good
|
||||
break;
|
||||
}
|
||||
|
||||
lastKnownHealth.set(healthBuilder.build());
|
||||
lastCheckTime = LocalDateTime.now();
|
||||
|
||||
if (properties.getHealth().isEnableMetrics()) {
|
||||
logMetrics(clusterHealth);
|
||||
}
|
||||
|
||||
} catch (Exception e) {
|
||||
logger.error("OpenSearch health check failed", e);
|
||||
Health unhealthyStatus = Health.down()
|
||||
.withDetail("error", e.getMessage())
|
||||
.withDetail("last_successful_check", lastCheckTime)
|
||||
.withDetail("current_time", LocalDateTime.now())
|
||||
.build();
|
||||
lastKnownHealth.set(unhealthyStatus);
|
||||
}
|
||||
}
|
||||
|
||||
private void logMetrics(HealthResponse clusterHealth) {
|
||||
logger.info("OpenSearch Cluster Metrics - Status: {}, Nodes: {}, Active Shards: {}, Unassigned: {}",
|
||||
clusterHealth.status().jsonValue(),
|
||||
clusterHealth.numberOfNodes(),
|
||||
clusterHealth.activeShards(),
|
||||
clusterHealth.unassignedShards());
|
||||
}
|
||||
|
||||
private boolean isProduction() {
|
||||
return "production".equalsIgnoreCase(properties.getProfile());
|
||||
}
|
||||
|
||||
/**
|
||||
* Manual health check for immediate status
|
||||
*/
|
||||
public boolean isClusterHealthy() {
|
||||
Health currentHealth = lastKnownHealth.get();
|
||||
return currentHealth.getStatus() == org.springframework.boot.actuate.health.Status.UP;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get detailed cluster information
|
||||
*/
|
||||
public String getClusterInfo() {
|
||||
try {
|
||||
var info = openSearchClient.info();
|
||||
return String.format("OpenSearch %s (Cluster: %s, Lucene: %s)",
|
||||
info.version().number(),
|
||||
info.clusterName(),
|
||||
info.version().luceneVersion());
|
||||
} catch (Exception e) {
|
||||
return "Unable to retrieve cluster information: " + e.getMessage();
|
||||
}
|
||||
}
|
||||
}
|
||||
1077
backend/src/main/java/com/storycove/service/OpenSearchService.java
Normal file
1077
backend/src/main/java/com/storycove/service/OpenSearchService.java
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,278 @@
|
||||
package com.storycove.service;
|
||||
|
||||
import com.storycove.dto.AuthorSearchDto;
|
||||
import com.storycove.dto.SearchResultDto;
|
||||
import com.storycove.dto.StorySearchDto;
|
||||
import com.storycove.entity.Author;
|
||||
import com.storycove.entity.Story;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* Service adapter that provides a unified interface for search operations.
|
||||
*
|
||||
* This adapter directly delegates to OpenSearchService.
|
||||
*/
|
||||
@Service
|
||||
public class SearchServiceAdapter {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(SearchServiceAdapter.class);
|
||||
|
||||
@Autowired
|
||||
private OpenSearchService openSearchService;
|
||||
|
||||
// ===============================
|
||||
// SEARCH OPERATIONS
|
||||
// ===============================
|
||||
|
||||
/**
|
||||
* Search stories with unified interface
|
||||
*/
|
||||
public SearchResultDto<StorySearchDto> searchStories(String query, List<String> tags, String author,
|
||||
String series, Integer minWordCount, Integer maxWordCount,
|
||||
Float minRating, Boolean isRead, Boolean isFavorite,
|
||||
String sortBy, String sortOrder, int page, int size,
|
||||
List<String> facetBy,
|
||||
// Advanced filters
|
||||
String createdAfter, String createdBefore,
|
||||
String lastReadAfter, String lastReadBefore,
|
||||
Boolean unratedOnly, String readingStatus,
|
||||
Boolean hasReadingProgress, Boolean hasCoverImage,
|
||||
String sourceDomain, String seriesFilter,
|
||||
Integer minTagCount, Boolean popularOnly,
|
||||
Boolean hiddenGemsOnly) {
|
||||
return openSearchService.searchStories(query, tags, author, series, minWordCount, maxWordCount,
|
||||
minRating, isRead, isFavorite, sortBy, sortOrder, page, size, facetBy,
|
||||
createdAfter, createdBefore, lastReadAfter, lastReadBefore, unratedOnly, readingStatus,
|
||||
hasReadingProgress, hasCoverImage, sourceDomain, seriesFilter, minTagCount, popularOnly,
|
||||
hiddenGemsOnly);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get random stories with unified interface
|
||||
*/
|
||||
public List<StorySearchDto> getRandomStories(int count, List<String> tags, String author,
|
||||
String series, Integer minWordCount, Integer maxWordCount,
|
||||
Float minRating, Boolean isRead, Boolean isFavorite,
|
||||
Long seed) {
|
||||
return openSearchService.getRandomStories(count, tags, author, series, minWordCount, maxWordCount,
|
||||
minRating, isRead, isFavorite, seed);
|
||||
}
|
||||
|
||||
/**
|
||||
* Recreate search indices
|
||||
*/
|
||||
public void recreateIndices() {
|
||||
try {
|
||||
openSearchService.recreateIndices();
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to recreate search indices", e);
|
||||
throw new RuntimeException("Failed to recreate search indices", e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Perform complete reindex of all data
|
||||
*/
|
||||
public void performCompleteReindex() {
|
||||
try {
|
||||
recreateIndices();
|
||||
logger.info("Search indices recreated successfully");
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to perform complete reindex", e);
|
||||
throw new RuntimeException("Failed to perform complete reindex", e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get random story ID with unified interface
|
||||
*/
|
||||
public String getRandomStoryId(Long seed) {
|
||||
return openSearchService.getRandomStoryId(seed);
|
||||
}
|
||||
|
||||
/**
|
||||
* Search authors with unified interface
|
||||
*/
|
||||
public List<AuthorSearchDto> searchAuthors(String query, int limit) {
|
||||
return openSearchService.searchAuthors(query, limit);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get tag suggestions with unified interface
|
||||
*/
|
||||
public List<String> getTagSuggestions(String query, int limit) {
|
||||
return openSearchService.getTagSuggestions(query, limit);
|
||||
}
|
||||
|
||||
// ===============================
|
||||
// INDEX OPERATIONS
|
||||
// ===============================
|
||||
|
||||
/**
|
||||
* Index a story in OpenSearch
|
||||
*/
|
||||
public void indexStory(Story story) {
|
||||
try {
|
||||
openSearchService.indexStory(story);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to index story {}", story.getId(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Update a story in OpenSearch
|
||||
*/
|
||||
public void updateStory(Story story) {
|
||||
try {
|
||||
openSearchService.updateStory(story);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to update story {}", story.getId(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a story from OpenSearch
|
||||
*/
|
||||
public void deleteStory(UUID storyId) {
|
||||
try {
|
||||
openSearchService.deleteStory(storyId);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to delete story {}", storyId, e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Index an author in OpenSearch
|
||||
*/
|
||||
public void indexAuthor(Author author) {
|
||||
try {
|
||||
openSearchService.indexAuthor(author);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to index author {}", author.getId(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Update an author in OpenSearch
|
||||
*/
|
||||
public void updateAuthor(Author author) {
|
||||
try {
|
||||
openSearchService.updateAuthor(author);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to update author {}", author.getId(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete an author from OpenSearch
|
||||
*/
|
||||
public void deleteAuthor(UUID authorId) {
|
||||
try {
|
||||
openSearchService.deleteAuthor(authorId);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to delete author {}", authorId, e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Bulk index stories in OpenSearch
|
||||
*/
|
||||
public void bulkIndexStories(List<Story> stories) {
|
||||
try {
|
||||
openSearchService.bulkIndexStories(stories);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to bulk index {} stories", stories.size(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Bulk index authors in OpenSearch
|
||||
*/
|
||||
public void bulkIndexAuthors(List<Author> authors) {
|
||||
try {
|
||||
openSearchService.bulkIndexAuthors(authors);
|
||||
} catch (Exception e) {
|
||||
logger.error("Failed to bulk index {} authors", authors.size(), e);
|
||||
}
|
||||
}
|
||||
|
||||
// ===============================
|
||||
// UTILITY METHODS
|
||||
// ===============================
|
||||
|
||||
/**
|
||||
* Check if search service is available and healthy
|
||||
*/
|
||||
public boolean isSearchServiceAvailable() {
|
||||
return openSearchService.testConnection();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current search engine name
|
||||
*/
|
||||
public String getCurrentSearchEngine() {
|
||||
return "opensearch";
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if dual-write is enabled
|
||||
*/
|
||||
public boolean isDualWriteEnabled() {
|
||||
return false; // No longer supported
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if we can switch to OpenSearch
|
||||
*/
|
||||
public boolean canSwitchToOpenSearch() {
|
||||
return true; // Already using OpenSearch
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if we can switch to Typesense
|
||||
*/
|
||||
public boolean canSwitchToTypesense() {
|
||||
return false; // Typesense no longer available
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current search status for admin interface
|
||||
*/
|
||||
public SearchStatus getSearchStatus() {
|
||||
return new SearchStatus(
|
||||
"opensearch",
|
||||
false, // no dual-write
|
||||
false, // no typesense
|
||||
openSearchService.testConnection()
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* DTO for search status
|
||||
*/
|
||||
public static class SearchStatus {
|
||||
private final String primaryEngine;
|
||||
private final boolean dualWrite;
|
||||
private final boolean typesenseAvailable;
|
||||
private final boolean openSearchAvailable;
|
||||
|
||||
public SearchStatus(String primaryEngine, boolean dualWrite,
|
||||
boolean typesenseAvailable, boolean openSearchAvailable) {
|
||||
this.primaryEngine = primaryEngine;
|
||||
this.dualWrite = dualWrite;
|
||||
this.typesenseAvailable = typesenseAvailable;
|
||||
this.openSearchAvailable = openSearchAvailable;
|
||||
}
|
||||
|
||||
public String getPrimaryEngine() { return primaryEngine; }
|
||||
public boolean isDualWrite() { return dualWrite; }
|
||||
public boolean isTypesenseAvailable() { return typesenseAvailable; }
|
||||
public boolean isOpenSearchAvailable() { return openSearchAvailable; }
|
||||
}
|
||||
}
|
||||
@@ -42,7 +42,7 @@ public class StoryService {
|
||||
private final TagService tagService;
|
||||
private final SeriesService seriesService;
|
||||
private final HtmlSanitizationService sanitizationService;
|
||||
private final TypesenseService typesenseService;
|
||||
private final SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
@Autowired
|
||||
public StoryService(StoryRepository storyRepository,
|
||||
@@ -52,7 +52,7 @@ public class StoryService {
|
||||
TagService tagService,
|
||||
SeriesService seriesService,
|
||||
HtmlSanitizationService sanitizationService,
|
||||
@Autowired(required = false) TypesenseService typesenseService) {
|
||||
SearchServiceAdapter searchServiceAdapter) {
|
||||
this.storyRepository = storyRepository;
|
||||
this.tagRepository = tagRepository;
|
||||
this.readingPositionRepository = readingPositionRepository;
|
||||
@@ -60,7 +60,7 @@ public class StoryService {
|
||||
this.tagService = tagService;
|
||||
this.seriesService = seriesService;
|
||||
this.sanitizationService = sanitizationService;
|
||||
this.typesenseService = typesenseService;
|
||||
this.searchServiceAdapter = searchServiceAdapter;
|
||||
}
|
||||
|
||||
@Transactional(readOnly = true)
|
||||
@@ -239,10 +239,8 @@ public class StoryService {
|
||||
story.addTag(tag);
|
||||
Story savedStory = storyRepository.save(story);
|
||||
|
||||
// Update Typesense index with new tag information
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(savedStory);
|
||||
}
|
||||
// Update search index with new tag information
|
||||
searchServiceAdapter.updateStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -256,10 +254,8 @@ public class StoryService {
|
||||
story.removeTag(tag);
|
||||
Story savedStory = storyRepository.save(story);
|
||||
|
||||
// Update Typesense index with updated tag information
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(savedStory);
|
||||
}
|
||||
// Update search index with updated tag information
|
||||
searchServiceAdapter.updateStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -274,10 +270,8 @@ public class StoryService {
|
||||
story.setRating(rating);
|
||||
Story savedStory = storyRepository.save(story);
|
||||
|
||||
// Update Typesense index with new rating
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(savedStory);
|
||||
}
|
||||
// Update search index with new rating
|
||||
searchServiceAdapter.updateStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -292,10 +286,8 @@ public class StoryService {
|
||||
story.updateReadingProgress(position);
|
||||
Story savedStory = storyRepository.save(story);
|
||||
|
||||
// Update Typesense index with new reading progress
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(savedStory);
|
||||
}
|
||||
// Update search index with new reading progress
|
||||
searchServiceAdapter.updateStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -313,10 +305,8 @@ public class StoryService {
|
||||
|
||||
Story savedStory = storyRepository.save(story);
|
||||
|
||||
// Update Typesense index with new reading status
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(savedStory);
|
||||
}
|
||||
// Update search index with new reading status
|
||||
searchServiceAdapter.updateStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -358,10 +348,8 @@ public class StoryService {
|
||||
updateStoryTags(savedStory, story.getTags());
|
||||
}
|
||||
|
||||
// Index in Typesense (if available)
|
||||
if (typesenseService != null) {
|
||||
typesenseService.indexStory(savedStory);
|
||||
}
|
||||
// Index in search engine
|
||||
searchServiceAdapter.indexStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -388,10 +376,8 @@ public class StoryService {
|
||||
updateStoryTagsByNames(savedStory, tagNames);
|
||||
}
|
||||
|
||||
// Index in Typesense (if available)
|
||||
if (typesenseService != null) {
|
||||
typesenseService.indexStory(savedStory);
|
||||
}
|
||||
// Index in search engine
|
||||
searchServiceAdapter.indexStory(savedStory);
|
||||
|
||||
return savedStory;
|
||||
}
|
||||
@@ -409,10 +395,8 @@ public class StoryService {
|
||||
updateStoryFields(existingStory, storyUpdates);
|
||||
Story updatedStory = storyRepository.save(existingStory);
|
||||
|
||||
// Update in Typesense (if available)
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(updatedStory);
|
||||
}
|
||||
// Update in search engine
|
||||
searchServiceAdapter.updateStory(updatedStory);
|
||||
|
||||
return updatedStory;
|
||||
}
|
||||
@@ -432,10 +416,8 @@ public class StoryService {
|
||||
|
||||
Story updatedStory = storyRepository.save(existingStory);
|
||||
|
||||
// Update in Typesense (if available)
|
||||
if (typesenseService != null) {
|
||||
typesenseService.updateStory(updatedStory);
|
||||
}
|
||||
// Update in search engine
|
||||
searchServiceAdapter.updateStory(updatedStory);
|
||||
|
||||
return updatedStory;
|
||||
}
|
||||
@@ -455,10 +437,8 @@ public class StoryService {
|
||||
// Create a copy to avoid ConcurrentModificationException
|
||||
new ArrayList<>(story.getTags()).forEach(tag -> story.removeTag(tag));
|
||||
|
||||
// Delete from Typesense first (if available)
|
||||
if (typesenseService != null) {
|
||||
typesenseService.deleteStory(story.getId().toString());
|
||||
}
|
||||
// Delete from search engine first
|
||||
searchServiceAdapter.deleteStory(story.getId());
|
||||
|
||||
storyRepository.delete(story);
|
||||
}
|
||||
@@ -674,7 +654,7 @@ public class StoryService {
|
||||
|
||||
/**
|
||||
* Find a random story based on optional filters.
|
||||
* Uses Typesense for consistency with Library search functionality.
|
||||
* Uses search service for consistency with Library search functionality.
|
||||
* Supports text search and multiple tags using the same logic as the Library view.
|
||||
* @param searchQuery Optional search query
|
||||
* @param tags Optional list of tags to filter by
|
||||
@@ -693,7 +673,7 @@ public class StoryService {
|
||||
|
||||
/**
|
||||
* Find a random story based on optional filters with seed support.
|
||||
* Uses Typesense for consistency with Library search functionality.
|
||||
* Uses search service for consistency with Library search functionality.
|
||||
* Supports text search and multiple tags using the same logic as the Library view.
|
||||
* @param searchQuery Optional search query
|
||||
* @param tags Optional list of tags to filter by
|
||||
@@ -711,21 +691,16 @@ public class StoryService {
|
||||
String seriesFilter, Integer minTagCount,
|
||||
Boolean popularOnly, Boolean hiddenGemsOnly) {
|
||||
|
||||
// Use Typesense if available for consistency with Library search
|
||||
if (typesenseService != null) {
|
||||
try {
|
||||
Optional<UUID> randomStoryId = typesenseService.getRandomStoryId(searchQuery, tags, seed,
|
||||
minWordCount, maxWordCount, createdAfter, createdBefore, lastReadAfter, lastReadBefore,
|
||||
minRating, maxRating, unratedOnly, readingStatus, hasReadingProgress, hasCoverImage,
|
||||
sourceDomain, seriesFilter, minTagCount, popularOnly, hiddenGemsOnly);
|
||||
if (randomStoryId.isPresent()) {
|
||||
return storyRepository.findById(randomStoryId.get());
|
||||
}
|
||||
return Optional.empty();
|
||||
} catch (Exception e) {
|
||||
// Fallback to database queries if Typesense fails
|
||||
logger.warn("Typesense random story lookup failed, falling back to database queries", e);
|
||||
// Use search service for consistency with Library search
|
||||
try {
|
||||
String randomStoryId = searchServiceAdapter.getRandomStoryId(seed);
|
||||
if (randomStoryId != null) {
|
||||
return storyRepository.findById(UUID.fromString(randomStoryId));
|
||||
}
|
||||
return Optional.empty();
|
||||
} catch (Exception e) {
|
||||
// Fallback to database queries if search service fails
|
||||
logger.warn("Search service random story lookup failed, falling back to database queries", e);
|
||||
}
|
||||
|
||||
// Fallback to repository-based implementation (global routing handles library selection)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -19,6 +19,12 @@ spring:
|
||||
max-file-size: 256MB # Increased for backup restore
|
||||
max-request-size: 260MB # Slightly higher to account for form data
|
||||
|
||||
jackson:
|
||||
serialization:
|
||||
write-dates-as-timestamps: false
|
||||
deserialization:
|
||||
adjust-dates-to-context-time-zone: false
|
||||
|
||||
server:
|
||||
port: 8080
|
||||
|
||||
@@ -32,15 +38,71 @@ storycove:
|
||||
expiration: 86400000 # 24 hours
|
||||
auth:
|
||||
password: ${APP_PASSWORD} # REQUIRED: No default password for security
|
||||
typesense:
|
||||
api-key: ${TYPESENSE_API_KEY:xyz}
|
||||
host: ${TYPESENSE_HOST:localhost}
|
||||
port: ${TYPESENSE_PORT:8108}
|
||||
enabled: ${TYPESENSE_ENABLED:true}
|
||||
reindex-interval: ${TYPESENSE_REINDEX_INTERVAL:3600000} # 1 hour in milliseconds
|
||||
search:
|
||||
engine: opensearch # OpenSearch is the only search engine
|
||||
opensearch:
|
||||
# Connection settings
|
||||
host: ${OPENSEARCH_HOST:localhost}
|
||||
port: ${OPENSEARCH_PORT:9200}
|
||||
scheme: ${OPENSEARCH_SCHEME:http}
|
||||
username: ${OPENSEARCH_USERNAME:}
|
||||
password: ${OPENSEARCH_PASSWORD:} # Empty when security is disabled
|
||||
|
||||
# Environment-specific configuration
|
||||
profile: ${SPRING_PROFILES_ACTIVE:development} # development, staging, production
|
||||
|
||||
# Security settings
|
||||
security:
|
||||
ssl-verification: ${OPENSEARCH_SSL_VERIFICATION:false}
|
||||
trust-all-certificates: ${OPENSEARCH_TRUST_ALL_CERTS:true}
|
||||
keystore-path: ${OPENSEARCH_KEYSTORE_PATH:}
|
||||
keystore-password: ${OPENSEARCH_KEYSTORE_PASSWORD:}
|
||||
truststore-path: ${OPENSEARCH_TRUSTSTORE_PATH:}
|
||||
truststore-password: ${OPENSEARCH_TRUSTSTORE_PASSWORD:}
|
||||
|
||||
# Connection pool settings
|
||||
connection:
|
||||
timeout: ${OPENSEARCH_CONNECTION_TIMEOUT:30000} # 30 seconds
|
||||
socket-timeout: ${OPENSEARCH_SOCKET_TIMEOUT:60000} # 60 seconds
|
||||
max-connections-per-route: ${OPENSEARCH_MAX_CONN_PER_ROUTE:10}
|
||||
max-connections-total: ${OPENSEARCH_MAX_CONN_TOTAL:30}
|
||||
retry-on-failure: ${OPENSEARCH_RETRY_ON_FAILURE:true}
|
||||
max-retries: ${OPENSEARCH_MAX_RETRIES:3}
|
||||
|
||||
# Index settings
|
||||
indices:
|
||||
default-shards: ${OPENSEARCH_DEFAULT_SHARDS:1}
|
||||
default-replicas: ${OPENSEARCH_DEFAULT_REPLICAS:0}
|
||||
refresh-interval: ${OPENSEARCH_REFRESH_INTERVAL:1s}
|
||||
|
||||
# Bulk operations
|
||||
bulk:
|
||||
actions: ${OPENSEARCH_BULK_ACTIONS:1000}
|
||||
size: ${OPENSEARCH_BULK_SIZE:5242880} # 5MB
|
||||
timeout: ${OPENSEARCH_BULK_TIMEOUT:10000} # 10 seconds
|
||||
concurrent-requests: ${OPENSEARCH_BULK_CONCURRENT:1}
|
||||
|
||||
# Health and monitoring
|
||||
health:
|
||||
check-interval: ${OPENSEARCH_HEALTH_CHECK_INTERVAL:30000} # 30 seconds
|
||||
slow-query-threshold: ${OPENSEARCH_SLOW_QUERY_THRESHOLD:5000} # 5 seconds
|
||||
enable-metrics: ${OPENSEARCH_ENABLE_METRICS:true}
|
||||
images:
|
||||
storage-path: ${IMAGE_STORAGE_PATH:/app/images}
|
||||
|
||||
management:
|
||||
endpoints:
|
||||
web:
|
||||
exposure:
|
||||
include: health,info,prometheus
|
||||
endpoint:
|
||||
health:
|
||||
show-details: when-authorized
|
||||
show-components: always
|
||||
health:
|
||||
opensearch:
|
||||
enabled: ${OPENSEARCH_HEALTH_ENABLED:true}
|
||||
|
||||
logging:
|
||||
level:
|
||||
com.storycove: ${LOG_LEVEL:INFO} # Use INFO for production, DEBUG for development
|
||||
|
||||
178
backend/src/main/resources/opensearch/README.md
Normal file
178
backend/src/main/resources/opensearch/README.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# OpenSearch Configuration - Best Practices Implementation
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains a production-ready OpenSearch configuration following industry best practices for security, scalability, and maintainability.
|
||||
|
||||
## Architecture
|
||||
|
||||
### 📁 Directory Structure
|
||||
```
|
||||
opensearch/
|
||||
├── config/
|
||||
│ ├── opensearch-development.yml # Development-specific settings
|
||||
│ └── opensearch-production.yml # Production-specific settings
|
||||
├── mappings/
|
||||
│ ├── stories-mapping.json # Story index mapping
|
||||
│ ├── authors-mapping.json # Author index mapping
|
||||
│ └── collections-mapping.json # Collection index mapping
|
||||
├── templates/
|
||||
│ ├── stories-template.json # Index template for stories_*
|
||||
│ └── index-lifecycle-policy.json # ILM policy for index management
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## ✅ Best Practices Implemented
|
||||
|
||||
### 🔒 **Security**
|
||||
- **Environment-Aware SSL Configuration**
|
||||
- Production: Full certificate validation with custom truststore support
|
||||
- Development: Optional certificate validation for local development
|
||||
- **Proper Authentication**: Basic auth with secure credential management
|
||||
- **Connection Security**: TLS 1.3 support with hostname verification
|
||||
|
||||
### 🏗️ **Configuration Management**
|
||||
- **Externalized Configuration**: JSON/YAML files instead of hardcoded values
|
||||
- **Environment-Specific Settings**: Different configs for dev/staging/prod
|
||||
- **Type-Safe Properties**: Strongly-typed configuration classes
|
||||
- **Validation**: Configuration validation at startup
|
||||
|
||||
### 📈 **Scalability & Performance**
|
||||
- **Connection Pooling**: Configurable connection pool with timeout management
|
||||
- **Environment-Aware Sharding**:
|
||||
- Development: 1 shard, 0 replicas (single node)
|
||||
- Production: 3 shards, 1 replica (high availability)
|
||||
- **Bulk Operations**: Optimized bulk indexing with configurable batch sizes
|
||||
- **Index Templates**: Automatic application of settings to new indexes
|
||||
|
||||
### 🔄 **Index Lifecycle Management**
|
||||
- **Automated Index Rollover**: Based on size, document count, and age
|
||||
- **Hot-Warm-Cold Architecture**: Optimized storage costs
|
||||
- **Retention Policies**: Automatic cleanup of old data
|
||||
- **Force Merge**: Optimization in warm phase
|
||||
|
||||
### 📊 **Monitoring & Observability**
|
||||
- **Health Checks**: Automatic cluster health monitoring
|
||||
- **Spring Boot Actuator**: Health endpoints for monitoring systems
|
||||
- **Metrics Collection**: Configurable performance metrics
|
||||
- **Slow Query Detection**: Configurable thresholds for query performance
|
||||
|
||||
### 🛡️ **Error Handling & Resilience**
|
||||
- **Connection Retry Logic**: Automatic retry with backoff
|
||||
- **Circuit Breaker Pattern**: Fail-fast for unhealthy clusters
|
||||
- **Graceful Degradation**: Graceful handling when OpenSearch unavailable
|
||||
- **Detailed Error Logging**: Comprehensive error tracking
|
||||
|
||||
## 🚀 Usage
|
||||
|
||||
### Development Environment
|
||||
```yaml
|
||||
# application-development.yml
|
||||
storycove:
|
||||
opensearch:
|
||||
profile: development
|
||||
security:
|
||||
ssl-verification: false
|
||||
trust-all-certificates: true
|
||||
indices:
|
||||
default-shards: 1
|
||||
default-replicas: 0
|
||||
```
|
||||
|
||||
### Production Environment
|
||||
```yaml
|
||||
# application-production.yml
|
||||
storycove:
|
||||
opensearch:
|
||||
profile: production
|
||||
security:
|
||||
ssl-verification: true
|
||||
trust-all-certificates: false
|
||||
truststore-path: /etc/ssl/opensearch-truststore.jks
|
||||
indices:
|
||||
default-shards: 3
|
||||
default-replicas: 1
|
||||
```
|
||||
|
||||
## 📋 Environment Variables
|
||||
|
||||
### Required
|
||||
- `OPENSEARCH_PASSWORD`: Admin password for OpenSearch cluster
|
||||
|
||||
### Optional (with sensible defaults)
|
||||
- `OPENSEARCH_HOST`: Cluster hostname (default: localhost)
|
||||
- `OPENSEARCH_PORT`: Cluster port (default: 9200)
|
||||
- `OPENSEARCH_USERNAME`: Admin username (default: admin)
|
||||
- `OPENSEARCH_SSL_VERIFICATION`: Enable SSL verification (default: false for dev)
|
||||
- `OPENSEARCH_MAX_CONN_TOTAL`: Max connections (default: 30 for dev, 200 for prod)
|
||||
|
||||
## 🎯 Index Templates
|
||||
|
||||
Index templates automatically apply configuration to new indexes:
|
||||
|
||||
```json
|
||||
{
|
||||
"index_patterns": ["stories_*"],
|
||||
"template": {
|
||||
"settings": {
|
||||
"number_of_shards": "#{ENV_SPECIFIC}",
|
||||
"analysis": {
|
||||
"analyzer": {
|
||||
"story_analyzer": {
|
||||
"type": "standard",
|
||||
"stopwords": "_english_"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🔍 Health Monitoring
|
||||
|
||||
Access health information:
|
||||
- **Application Health**: `/actuator/health`
|
||||
- **OpenSearch Specific**: `/actuator/health/opensearch`
|
||||
- **Detailed Metrics**: Available when `enable-metrics: true`
|
||||
|
||||
## 🔄 Deployment Strategy
|
||||
|
||||
Recommended deployment approach:
|
||||
|
||||
1. **Development**: Test OpenSearch configuration locally
|
||||
2. **Staging**: Validate performance and accuracy in staging environment
|
||||
3. **Production**: Deploy with proper monitoring and backup procedures
|
||||
|
||||
## 🛠️ Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **SSL Certificate Errors**
|
||||
- Development: Set `trust-all-certificates: true`
|
||||
- Production: Provide valid truststore path
|
||||
|
||||
2. **Connection Timeouts**
|
||||
- Increase `connection.timeout` values
|
||||
- Check network connectivity and firewall rules
|
||||
|
||||
3. **Index Creation Failures**
|
||||
- Verify cluster health with `/actuator/health/opensearch`
|
||||
- Check OpenSearch logs for detailed error messages
|
||||
|
||||
4. **Performance Issues**
|
||||
- Monitor slow queries with configurable thresholds
|
||||
- Adjust bulk operation settings
|
||||
- Review shard allocation and replica settings
|
||||
|
||||
## 🔮 Future Enhancements
|
||||
|
||||
- **Multi-Cluster Support**: Connect to multiple OpenSearch clusters
|
||||
- **Advanced Security**: Integration with OpenSearch Security plugin
|
||||
- **Custom Analyzers**: Domain-specific text analysis
|
||||
- **Index Aliases**: Zero-downtime index updates
|
||||
- **Machine Learning**: Integration with OpenSearch ML features
|
||||
|
||||
---
|
||||
|
||||
This configuration provides a solid foundation that scales from development to enterprise production environments while maintaining security, performance, and operational excellence.
|
||||
@@ -0,0 +1,32 @@
|
||||
# OpenSearch Development Configuration
|
||||
opensearch:
|
||||
cluster:
|
||||
name: "storycove-dev"
|
||||
initial_master_nodes: ["opensearch-node"]
|
||||
|
||||
# Development settings - single node, minimal resources
|
||||
indices:
|
||||
default_settings:
|
||||
number_of_shards: 1
|
||||
number_of_replicas: 0
|
||||
refresh_interval: "1s"
|
||||
|
||||
# Security settings for development
|
||||
security:
|
||||
ssl_verification: false
|
||||
trust_all_certificates: true
|
||||
|
||||
# Connection settings
|
||||
connection:
|
||||
timeout: "30s"
|
||||
socket_timeout: "60s"
|
||||
max_connections_per_route: 10
|
||||
max_connections_total: 30
|
||||
|
||||
# Index management
|
||||
index_management:
|
||||
auto_create_templates: true
|
||||
template_patterns:
|
||||
stories: "stories_*"
|
||||
authors: "authors_*"
|
||||
collections: "collections_*"
|
||||
@@ -0,0 +1,60 @@
|
||||
# OpenSearch Production Configuration
|
||||
opensearch:
|
||||
cluster:
|
||||
name: "storycove-prod"
|
||||
|
||||
# Production settings - multi-shard, with replicas
|
||||
indices:
|
||||
default_settings:
|
||||
number_of_shards: 3
|
||||
number_of_replicas: 1
|
||||
refresh_interval: "30s"
|
||||
max_result_window: 50000
|
||||
|
||||
# Index lifecycle policies
|
||||
lifecycle:
|
||||
hot_phase_duration: "7d"
|
||||
warm_phase_duration: "30d"
|
||||
cold_phase_duration: "90d"
|
||||
delete_after: "1y"
|
||||
|
||||
# Security settings for production
|
||||
security:
|
||||
ssl_verification: true
|
||||
trust_all_certificates: false
|
||||
certificate_verification: true
|
||||
tls_version: "TLSv1.3"
|
||||
|
||||
# Connection settings
|
||||
connection:
|
||||
timeout: "10s"
|
||||
socket_timeout: "30s"
|
||||
max_connections_per_route: 50
|
||||
max_connections_total: 200
|
||||
retry_on_failure: true
|
||||
max_retries: 3
|
||||
retry_delay: "1s"
|
||||
|
||||
# Performance tuning
|
||||
performance:
|
||||
bulk_actions: 1000
|
||||
bulk_size: "5MB"
|
||||
bulk_timeout: "10s"
|
||||
concurrent_requests: 4
|
||||
|
||||
# Monitoring and observability
|
||||
monitoring:
|
||||
health_check_interval: "30s"
|
||||
slow_query_threshold: "5s"
|
||||
enable_metrics: true
|
||||
|
||||
# Index management
|
||||
index_management:
|
||||
auto_create_templates: true
|
||||
template_patterns:
|
||||
stories: "stories_*"
|
||||
authors: "authors_*"
|
||||
collections: "collections_*"
|
||||
retention_policy:
|
||||
enabled: true
|
||||
default_retention: "1y"
|
||||
@@ -0,0 +1,79 @@
|
||||
{
|
||||
"settings": {
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 0,
|
||||
"analysis": {
|
||||
"analyzer": {
|
||||
"name_analyzer": {
|
||||
"type": "standard",
|
||||
"stopwords": "_english_"
|
||||
},
|
||||
"autocomplete_analyzer": {
|
||||
"type": "custom",
|
||||
"tokenizer": "standard",
|
||||
"filter": ["lowercase", "edge_ngram"]
|
||||
}
|
||||
},
|
||||
"filter": {
|
||||
"edge_ngram": {
|
||||
"type": "edge_ngram",
|
||||
"min_gram": 2,
|
||||
"max_gram": 20
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"name": {
|
||||
"type": "text",
|
||||
"analyzer": "name_analyzer",
|
||||
"fields": {
|
||||
"autocomplete": {
|
||||
"type": "text",
|
||||
"analyzer": "autocomplete_analyzer"
|
||||
},
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"bio": {
|
||||
"type": "text",
|
||||
"analyzer": "name_analyzer"
|
||||
},
|
||||
"urls": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"imageUrl": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"storyCount": {
|
||||
"type": "integer"
|
||||
},
|
||||
"averageRating": {
|
||||
"type": "float"
|
||||
},
|
||||
"totalWordCount": {
|
||||
"type": "long"
|
||||
},
|
||||
"totalReadingTime": {
|
||||
"type": "integer"
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"updatedAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"libraryId": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
{
|
||||
"settings": {
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 0,
|
||||
"analysis": {
|
||||
"analyzer": {
|
||||
"collection_analyzer": {
|
||||
"type": "standard",
|
||||
"stopwords": "_english_"
|
||||
},
|
||||
"autocomplete_analyzer": {
|
||||
"type": "custom",
|
||||
"tokenizer": "standard",
|
||||
"filter": ["lowercase", "edge_ngram"]
|
||||
}
|
||||
},
|
||||
"filter": {
|
||||
"edge_ngram": {
|
||||
"type": "edge_ngram",
|
||||
"min_gram": 2,
|
||||
"max_gram": 20
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"name": {
|
||||
"type": "text",
|
||||
"analyzer": "collection_analyzer",
|
||||
"fields": {
|
||||
"autocomplete": {
|
||||
"type": "text",
|
||||
"analyzer": "autocomplete_analyzer"
|
||||
},
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"type": "text",
|
||||
"analyzer": "collection_analyzer"
|
||||
},
|
||||
"storyCount": {
|
||||
"type": "integer"
|
||||
},
|
||||
"totalWordCount": {
|
||||
"type": "long"
|
||||
},
|
||||
"averageRating": {
|
||||
"type": "float"
|
||||
},
|
||||
"isPublic": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"updatedAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"libraryId": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
{
|
||||
"settings": {
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 0,
|
||||
"analysis": {
|
||||
"analyzer": {
|
||||
"story_analyzer": {
|
||||
"type": "standard",
|
||||
"stopwords": "_english_"
|
||||
},
|
||||
"autocomplete_analyzer": {
|
||||
"type": "custom",
|
||||
"tokenizer": "standard",
|
||||
"filter": ["lowercase", "edge_ngram"]
|
||||
}
|
||||
},
|
||||
"filter": {
|
||||
"edge_ngram": {
|
||||
"type": "edge_ngram",
|
||||
"min_gram": 2,
|
||||
"max_gram": 20
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"title": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {
|
||||
"autocomplete": {
|
||||
"type": "text",
|
||||
"analyzer": "autocomplete_analyzer"
|
||||
},
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"content": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer"
|
||||
},
|
||||
"summary": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer"
|
||||
},
|
||||
"authorNames": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"authorIds": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"tagNames": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"seriesTitle": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"seriesId": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"wordCount": {
|
||||
"type": "integer"
|
||||
},
|
||||
"rating": {
|
||||
"type": "float"
|
||||
},
|
||||
"readingTime": {
|
||||
"type": "integer"
|
||||
},
|
||||
"language": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"status": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"updatedAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"publishedAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"isRead": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"isFavorite": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"readingProgress": {
|
||||
"type": "float"
|
||||
},
|
||||
"libraryId": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"policy": {
|
||||
"description": "StoryCove index lifecycle policy",
|
||||
"default_state": "hot",
|
||||
"states": [
|
||||
{
|
||||
"name": "hot",
|
||||
"actions": [
|
||||
{
|
||||
"rollover": {
|
||||
"min_size": "50gb",
|
||||
"min_doc_count": 1000000,
|
||||
"min_age": "7d"
|
||||
}
|
||||
}
|
||||
],
|
||||
"transitions": [
|
||||
{
|
||||
"state_name": "warm",
|
||||
"conditions": {
|
||||
"min_age": "7d"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "warm",
|
||||
"actions": [
|
||||
{
|
||||
"replica_count": {
|
||||
"number_of_replicas": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"force_merge": {
|
||||
"max_num_segments": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"transitions": [
|
||||
{
|
||||
"state_name": "cold",
|
||||
"conditions": {
|
||||
"min_age": "30d"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "cold",
|
||||
"actions": [],
|
||||
"transitions": [
|
||||
{
|
||||
"state_name": "delete",
|
||||
"conditions": {
|
||||
"min_age": "365d"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "delete",
|
||||
"actions": [
|
||||
{
|
||||
"delete": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"ism_template": [
|
||||
{
|
||||
"index_patterns": ["stories_*", "authors_*", "collections_*"],
|
||||
"priority": 100
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
{
|
||||
"index_patterns": ["stories_*"],
|
||||
"priority": 1,
|
||||
"template": {
|
||||
"settings": {
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 0,
|
||||
"analysis": {
|
||||
"analyzer": {
|
||||
"story_analyzer": {
|
||||
"type": "standard",
|
||||
"stopwords": "_english_"
|
||||
},
|
||||
"autocomplete_analyzer": {
|
||||
"type": "custom",
|
||||
"tokenizer": "standard",
|
||||
"filter": ["lowercase", "edge_ngram"]
|
||||
}
|
||||
},
|
||||
"filter": {
|
||||
"edge_ngram": {
|
||||
"type": "edge_ngram",
|
||||
"min_gram": 2,
|
||||
"max_gram": 20
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"mappings": {
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"title": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {
|
||||
"autocomplete": {
|
||||
"type": "text",
|
||||
"analyzer": "autocomplete_analyzer"
|
||||
},
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"content": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer"
|
||||
},
|
||||
"summary": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer"
|
||||
},
|
||||
"authorNames": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"authorIds": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"tagNames": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"seriesTitle": {
|
||||
"type": "text",
|
||||
"analyzer": "story_analyzer",
|
||||
"fields": {
|
||||
"keyword": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
},
|
||||
"seriesId": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"wordCount": {
|
||||
"type": "integer"
|
||||
},
|
||||
"rating": {
|
||||
"type": "float"
|
||||
},
|
||||
"readingTime": {
|
||||
"type": "integer"
|
||||
},
|
||||
"language": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"status": {
|
||||
"type": "keyword"
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"updatedAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"publishedAt": {
|
||||
"type": "date",
|
||||
"format": "strict_date_optional_time||epoch_millis"
|
||||
},
|
||||
"isRead": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"isFavorite": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"readingProgress": {
|
||||
"type": "float"
|
||||
},
|
||||
"libraryId": {
|
||||
"type": "keyword"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,8 @@
|
||||
package com.storycove.config;
|
||||
|
||||
import com.storycove.service.TypesenseService;
|
||||
import org.springframework.boot.test.context.TestConfiguration;
|
||||
import org.springframework.boot.test.mock.mockito.MockBean;
|
||||
|
||||
@TestConfiguration
|
||||
public class TestConfig {
|
||||
|
||||
@MockBean
|
||||
public TypesenseService typesenseService;
|
||||
// Test configuration
|
||||
}
|
||||
@@ -44,8 +44,9 @@ class AuthorServiceTest {
|
||||
testAuthor.setId(testId);
|
||||
testAuthor.setNotes("Test notes");
|
||||
|
||||
// Initialize service with null TypesenseService (which is allowed for tests)
|
||||
authorService = new AuthorService(authorRepository, null);
|
||||
// Initialize service with mock SearchServiceAdapter
|
||||
SearchServiceAdapter mockSearchServiceAdapter = mock(SearchServiceAdapter.class);
|
||||
authorService = new AuthorService(authorRepository, mockSearchServiceAdapter);
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@@ -33,6 +33,9 @@ class StoryServiceTest {
|
||||
@Mock
|
||||
private ReadingPositionRepository readingPositionRepository;
|
||||
|
||||
@Mock
|
||||
private SearchServiceAdapter searchServiceAdapter;
|
||||
|
||||
private StoryService storyService;
|
||||
private Story testStory;
|
||||
private UUID testId;
|
||||
@@ -44,16 +47,16 @@ class StoryServiceTest {
|
||||
testStory.setId(testId);
|
||||
testStory.setContentHtml("<p>Test content for reading progress tracking</p>");
|
||||
|
||||
// Create StoryService with only required repositories, all services can be null for these tests
|
||||
// Create StoryService with mocked dependencies
|
||||
storyService = new StoryService(
|
||||
storyRepository,
|
||||
tagRepository,
|
||||
readingPositionRepository, // added for foreign key constraint handling
|
||||
readingPositionRepository,
|
||||
null, // authorService - not needed for reading progress tests
|
||||
null, // tagService - not needed for reading progress tests
|
||||
null, // seriesService - not needed for reading progress tests
|
||||
null, // sanitizationService - not needed for reading progress tests
|
||||
null // typesenseService - will test both with and without
|
||||
searchServiceAdapter
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -18,11 +18,12 @@ storycove:
|
||||
expiration: 86400000
|
||||
auth:
|
||||
password: test-password
|
||||
typesense:
|
||||
enabled: false
|
||||
api-key: test-key
|
||||
search:
|
||||
engine: opensearch
|
||||
opensearch:
|
||||
host: localhost
|
||||
port: 8108
|
||||
port: 9200
|
||||
scheme: http
|
||||
images:
|
||||
storage-path: /tmp/test-images
|
||||
|
||||
|
||||
4308
backend/test_results.log
Normal file
4308
backend/test_results.log
Normal file
File diff suppressed because it is too large
Load Diff
@@ -2,3 +2,4 @@
|
||||
# https://curl.se/docs/http-cookies.html
|
||||
# This file was generated by libcurl! Edit at your own risk.
|
||||
|
||||
#HttpOnly_localhost FALSE / FALSE 1758433252 token eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJ1c2VyIiwiaWF0IjoxNzU4MzQ2ODUyLCJleHAiOjE3NTg0MzMyNTIsImxpYnJhcnlJZCI6InNlY3JldCJ9.zEAQT5_11-pxPxmIhufSQqE26hvHldde4kFNE2HWWgBa5lT_Wt7jwpoPUMkQGQfShQwDZ9N-hFX3R2ew8jD7WQ
|
||||
|
||||
@@ -34,9 +34,10 @@ services:
|
||||
- SPRING_DATASOURCE_USERNAME=storycove
|
||||
- SPRING_DATASOURCE_PASSWORD=${DB_PASSWORD}
|
||||
- JWT_SECRET=${JWT_SECRET}
|
||||
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
|
||||
- TYPESENSE_HOST=typesense
|
||||
- TYPESENSE_PORT=8108
|
||||
- OPENSEARCH_HOST=opensearch
|
||||
- OPENSEARCH_PORT=9200
|
||||
- OPENSEARCH_SCHEME=http
|
||||
- SEARCH_ENGINE=${SEARCH_ENGINE:-opensearch}
|
||||
- IMAGE_STORAGE_PATH=/app/images
|
||||
- APP_PASSWORD=${APP_PASSWORD}
|
||||
- STORYCOVE_CORS_ALLOWED_ORIGINS=${STORYCOVE_CORS_ALLOWED_ORIGINS:-http://localhost:3000,http://localhost:6925}
|
||||
@@ -45,15 +46,15 @@ services:
|
||||
- library_config:/app/config
|
||||
depends_on:
|
||||
- postgres
|
||||
- typesense
|
||||
- opensearch
|
||||
networks:
|
||||
- storycove-network
|
||||
|
||||
postgres:
|
||||
image: postgres:15-alpine
|
||||
# No port mapping - only accessible within the Docker network
|
||||
ports:
|
||||
- "5432:5432"
|
||||
#ports:
|
||||
# - "5432:5432"
|
||||
environment:
|
||||
- POSTGRES_DB=storycove
|
||||
- POSTGRES_USER=storycove
|
||||
@@ -63,20 +64,46 @@ services:
|
||||
networks:
|
||||
- storycove-network
|
||||
|
||||
typesense:
|
||||
image: typesense/typesense:29.0
|
||||
|
||||
opensearch:
|
||||
image: opensearchproject/opensearch:3.2.0
|
||||
# No port mapping - only accessible within the Docker network
|
||||
environment:
|
||||
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
|
||||
- TYPESENSE_DATA_DIR=/data
|
||||
- cluster.name=storycove-opensearch
|
||||
- node.name=opensearch-node
|
||||
- discovery.type=single-node
|
||||
- bootstrap.memory_lock=false
|
||||
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
|
||||
- "DISABLE_INSTALL_DEMO_CONFIG=true"
|
||||
- "DISABLE_SECURITY_PLUGIN=true"
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
hard: -1
|
||||
nofile:
|
||||
soft: 65536
|
||||
hard: 65536
|
||||
volumes:
|
||||
- typesense_data:/data
|
||||
- opensearch_data:/usr/share/opensearch/data
|
||||
networks:
|
||||
- storycove-network
|
||||
restart: unless-stopped
|
||||
|
||||
opensearch-dashboards:
|
||||
image: opensearchproject/opensearch-dashboards:3.2.0
|
||||
ports:
|
||||
- "5601:5601" # Expose OpenSearch Dashboard
|
||||
environment:
|
||||
- OPENSEARCH_HOSTS=http://opensearch:9200
|
||||
- "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true"
|
||||
depends_on:
|
||||
- opensearch
|
||||
networks:
|
||||
- storycove-network
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
typesense_data:
|
||||
opensearch_data:
|
||||
images_data:
|
||||
library_config:
|
||||
|
||||
@@ -122,13 +149,5 @@ configs:
|
||||
expires 1y;
|
||||
add_header Cache-Control public;
|
||||
}
|
||||
location /typesense/ {
|
||||
proxy_pass http://typesense:8108/;
|
||||
proxy_set_header Host $$host;
|
||||
proxy_set_header X-Real-IP $$remote_addr;
|
||||
proxy_set_header X-Forwarded-For $$proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $$scheme;
|
||||
proxy_set_header X-Typesense-API-Key $$http_x_typesense_api_key;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -22,7 +22,7 @@ export default function AuthorsPage() {
|
||||
const [currentPage, setCurrentPage] = useState(0);
|
||||
const [totalHits, setTotalHits] = useState(0);
|
||||
const [hasMore, setHasMore] = useState(false);
|
||||
const ITEMS_PER_PAGE = 50; // Safe limit under Typesense's 250 limit
|
||||
const ITEMS_PER_PAGE = 50;
|
||||
|
||||
useEffect(() => {
|
||||
const debounceTimer = setTimeout(() => {
|
||||
@@ -35,41 +35,30 @@ export default function AuthorsPage() {
|
||||
} else {
|
||||
setSearchLoading(true);
|
||||
}
|
||||
const searchResults = await authorApi.searchAuthorsTypesense({
|
||||
q: searchQuery || '*',
|
||||
const searchResults = await authorApi.getAuthors({
|
||||
page: currentPage,
|
||||
size: ITEMS_PER_PAGE,
|
||||
sortBy: sortBy,
|
||||
sortOrder: sortOrder
|
||||
sortDir: sortOrder
|
||||
});
|
||||
|
||||
if (currentPage === 0) {
|
||||
// First page - replace all results
|
||||
setAuthors(searchResults.results || []);
|
||||
setFilteredAuthors(searchResults.results || []);
|
||||
setAuthors(searchResults.content || []);
|
||||
setFilteredAuthors(searchResults.content || []);
|
||||
} else {
|
||||
// Subsequent pages - append results
|
||||
setAuthors(prev => [...prev, ...(searchResults.results || [])]);
|
||||
setFilteredAuthors(prev => [...prev, ...(searchResults.results || [])]);
|
||||
setAuthors(prev => [...prev, ...(searchResults.content || [])]);
|
||||
setFilteredAuthors(prev => [...prev, ...(searchResults.content || [])]);
|
||||
}
|
||||
|
||||
setTotalHits(searchResults.totalHits);
|
||||
setHasMore(searchResults.results.length === ITEMS_PER_PAGE && (currentPage + 1) * ITEMS_PER_PAGE < searchResults.totalHits);
|
||||
setTotalHits(searchResults.totalElements || 0);
|
||||
setHasMore(searchResults.content.length === ITEMS_PER_PAGE && (currentPage + 1) * ITEMS_PER_PAGE < (searchResults.totalElements || 0));
|
||||
|
||||
} catch (error) {
|
||||
console.error('Failed to load authors:', error);
|
||||
// Fallback to regular API if Typesense fails (only for first page)
|
||||
if (currentPage === 0) {
|
||||
try {
|
||||
const authorsResult = await authorApi.getAuthors({ page: 0, size: ITEMS_PER_PAGE });
|
||||
setAuthors(authorsResult.content || []);
|
||||
setFilteredAuthors(authorsResult.content || []);
|
||||
setTotalHits(authorsResult.totalElements || 0);
|
||||
setHasMore(authorsResult.content.length === ITEMS_PER_PAGE);
|
||||
} catch (fallbackError) {
|
||||
console.error('Fallback also failed:', fallbackError);
|
||||
}
|
||||
}
|
||||
// Error handling for API failures
|
||||
console.error('Failed to load authors:', error);
|
||||
} finally {
|
||||
setLoading(false);
|
||||
setSearchLoading(false);
|
||||
@@ -95,7 +84,17 @@ export default function AuthorsPage() {
|
||||
}
|
||||
};
|
||||
|
||||
// Client-side filtering no longer needed since we use Typesense
|
||||
// Client-side filtering for search query when using regular API
|
||||
useEffect(() => {
|
||||
if (searchQuery) {
|
||||
const filtered = authors.filter(author =>
|
||||
author.name.toLowerCase().includes(searchQuery.toLowerCase())
|
||||
);
|
||||
setFilteredAuthors(filtered);
|
||||
} else {
|
||||
setFilteredAuthors(authors);
|
||||
}
|
||||
}, [authors, searchQuery]);
|
||||
|
||||
// Note: We no longer have individual story ratings in the author list
|
||||
// Average rating would need to be calculated on backend if needed
|
||||
@@ -118,9 +117,9 @@ export default function AuthorsPage() {
|
||||
<div>
|
||||
<h1 className="text-3xl font-bold theme-header">Authors</h1>
|
||||
<p className="theme-text mt-1">
|
||||
{filteredAuthors.length} of {totalHits} {totalHits === 1 ? 'author' : 'authors'}
|
||||
{searchQuery ? `${filteredAuthors.length} of ${authors.length}` : filteredAuthors.length} {(searchQuery ? authors.length : filteredAuthors.length) === 1 ? 'author' : 'authors'}
|
||||
{searchQuery ? ` found` : ` in your library`}
|
||||
{hasMore && ` (showing first ${filteredAuthors.length})`}
|
||||
{!searchQuery && hasMore && ` (showing first ${filteredAuthors.length})`}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@@ -218,7 +217,7 @@ export default function AuthorsPage() {
|
||||
)}
|
||||
|
||||
{/* Load More Button */}
|
||||
{hasMore && (
|
||||
{hasMore && !searchQuery && (
|
||||
<div className="flex justify-center pt-8">
|
||||
<Button
|
||||
onClick={loadMore}
|
||||
@@ -227,7 +226,7 @@ export default function AuthorsPage() {
|
||||
className="px-8 py-3"
|
||||
loading={loading}
|
||||
>
|
||||
{loading ? 'Loading...' : `Load More Authors (${totalHits - filteredAuthors.length} remaining)`}
|
||||
{loading ? 'Loading...' : `Load More Authors (${totalHits - authors.length} remaining)`}
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
@@ -193,24 +193,42 @@ async function processCombinedMode(
|
||||
console.log(`Combined content character length: ${combinedContentString.length}`);
|
||||
console.log(`Combined content parts count: ${combinedContent.length}`);
|
||||
|
||||
// Handle content truncation if needed
|
||||
let finalContent = contentSizeInMB > 10 ?
|
||||
combinedContentString.substring(0, Math.floor(combinedContentString.length * (10 / contentSizeInMB))) + '\n\n<!-- Content truncated due to size limit -->' :
|
||||
combinedContentString;
|
||||
|
||||
let finalSummary = contentSizeInMB > 10 ? baseSummary + ' (Content truncated due to size limit)' : baseSummary;
|
||||
|
||||
// Check if combined content has images and mark for processing
|
||||
const hasImages = /<img[^>]+src=['"'][^'"']*['"][^>]*>/i.test(finalContent);
|
||||
if (hasImages) {
|
||||
finalSummary += ' (Contains embedded images - will be processed after story creation)';
|
||||
console.log(`Combined story contains embedded images - will need processing after creation`);
|
||||
}
|
||||
|
||||
// Return the combined story data via progress update
|
||||
const combinedStory = {
|
||||
title: baseTitle,
|
||||
author: baseAuthor,
|
||||
content: contentSizeInMB > 10 ?
|
||||
combinedContentString.substring(0, Math.floor(combinedContentString.length * (10 / contentSizeInMB))) + '\n\n<!-- Content truncated due to size limit -->' :
|
||||
combinedContentString,
|
||||
summary: contentSizeInMB > 10 ? baseSummary + ' (Content truncated due to size limit)' : baseSummary,
|
||||
content: finalContent,
|
||||
summary: finalSummary,
|
||||
sourceUrl: baseSourceUrl,
|
||||
tags: Array.from(combinedTags)
|
||||
tags: Array.from(combinedTags),
|
||||
hasImages: hasImages
|
||||
};
|
||||
|
||||
// Send completion notification for combine mode
|
||||
let completionMessage = `Combined scraping completed: ${totalWordCount.toLocaleString()} words from ${importedCount} stories`;
|
||||
if (hasImages) {
|
||||
completionMessage += ` (embedded images will be processed when story is created)`;
|
||||
}
|
||||
|
||||
await sendProgressUpdate(sessionId, {
|
||||
type: 'completed',
|
||||
current: urls.length,
|
||||
total: urls.length,
|
||||
message: `Combined scraping completed: ${totalWordCount.toLocaleString()} words from ${importedCount} stories`,
|
||||
message: completionMessage,
|
||||
totalWordCount: totalWordCount,
|
||||
combinedStory: combinedStory
|
||||
});
|
||||
@@ -347,6 +365,61 @@ async function processIndividualMode(
|
||||
|
||||
const createdStory = await createResponse.json();
|
||||
|
||||
// Process embedded images if content contains images
|
||||
let imageProcessingWarnings: string[] = [];
|
||||
const hasImages = /<img[^>]+src=['"'][^'"']*['"][^>]*>/i.test(scrapedStory.content);
|
||||
|
||||
if (hasImages) {
|
||||
try {
|
||||
console.log(`Processing embedded images for story: ${createdStory.id}`);
|
||||
const imageProcessUrl = `http://backend:8080/api/stories/${createdStory.id}/process-content-images`;
|
||||
const imageProcessResponse = await fetch(imageProcessUrl, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': authorization,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ htmlContent: scrapedStory.content }),
|
||||
});
|
||||
|
||||
if (imageProcessResponse.ok) {
|
||||
const imageResult = await imageProcessResponse.json();
|
||||
if (imageResult.hasWarnings && imageResult.warnings) {
|
||||
imageProcessingWarnings = imageResult.warnings;
|
||||
console.log(`Image processing completed with warnings for story ${createdStory.id}:`, imageResult.warnings);
|
||||
} else {
|
||||
console.log(`Image processing completed successfully for story ${createdStory.id}. Downloaded ${imageResult.downloadedImages?.length || 0} images.`);
|
||||
}
|
||||
|
||||
// Update story content with processed images
|
||||
if (imageResult.processedContent && imageResult.processedContent !== scrapedStory.content) {
|
||||
const updateUrl = `http://backend:8080/api/stories/${createdStory.id}`;
|
||||
const updateResponse = await fetch(updateUrl, {
|
||||
method: 'PUT',
|
||||
headers: {
|
||||
'Authorization': authorization,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
contentHtml: imageResult.processedContent
|
||||
}),
|
||||
});
|
||||
|
||||
if (!updateResponse.ok) {
|
||||
console.warn(`Failed to update story content after image processing for ${createdStory.id}`);
|
||||
imageProcessingWarnings.push('Failed to update story content with processed images');
|
||||
}
|
||||
}
|
||||
} else {
|
||||
console.warn(`Image processing failed for story ${createdStory.id}:`, imageProcessResponse.status);
|
||||
imageProcessingWarnings.push('Image processing failed');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`Error processing images for story ${createdStory.id}:`, error);
|
||||
imageProcessingWarnings.push(`Image processing error: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
results.push({
|
||||
url: trimmedUrl,
|
||||
status: 'imported',
|
||||
@@ -356,17 +429,24 @@ async function processIndividualMode(
|
||||
});
|
||||
importedCount++;
|
||||
|
||||
console.log(`Successfully imported: ${scrapedStory.title} by ${scrapedStory.author} (ID: ${createdStory.id})`);
|
||||
console.log(`Successfully imported: ${scrapedStory.title} by ${scrapedStory.author} (ID: ${createdStory.id})${hasImages ? ` with ${imageProcessingWarnings.length > 0 ? 'warnings' : 'successful image processing'}` : ''}`);
|
||||
|
||||
// Send progress update for successful import
|
||||
let progressMessage = `Imported "${scrapedStory.title}" by ${scrapedStory.author}`;
|
||||
if (hasImages) {
|
||||
progressMessage += imageProcessingWarnings.length > 0 ? ' (with image warnings)' : ' (with images)';
|
||||
}
|
||||
|
||||
await sendProgressUpdate(sessionId, {
|
||||
type: 'progress',
|
||||
current: i + 1,
|
||||
total: urls.length,
|
||||
message: `Imported "${scrapedStory.title}" by ${scrapedStory.author}`,
|
||||
message: progressMessage,
|
||||
url: trimmedUrl,
|
||||
title: scrapedStory.title,
|
||||
author: scrapedStory.author
|
||||
author: scrapedStory.author,
|
||||
hasImages: hasImages,
|
||||
imageWarnings: imageProcessingWarnings
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
@@ -421,11 +501,11 @@ async function processIndividualMode(
|
||||
|
||||
console.log(`Bulk import completed: ${importedCount} imported, ${skippedCount} skipped, ${errorCount} errors`);
|
||||
|
||||
// Trigger Typesense reindex if any stories were imported
|
||||
// Trigger OpenSearch reindex if any stories were imported
|
||||
if (importedCount > 0) {
|
||||
try {
|
||||
console.log('Triggering Typesense reindex after bulk import...');
|
||||
const reindexUrl = `http://backend:8080/api/stories/reindex-typesense`;
|
||||
console.log('Triggering OpenSearch reindex after bulk import...');
|
||||
const reindexUrl = `http://backend:8080/api/admin/search/opensearch/reindex`;
|
||||
const reindexResponse = await fetch(reindexUrl, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
@@ -436,12 +516,12 @@ async function processIndividualMode(
|
||||
|
||||
if (reindexResponse.ok) {
|
||||
const reindexResult = await reindexResponse.json();
|
||||
console.log('Typesense reindex completed:', reindexResult);
|
||||
console.log('OpenSearch reindex completed:', reindexResult);
|
||||
} else {
|
||||
console.warn('Typesense reindex failed:', reindexResponse.status);
|
||||
console.warn('OpenSearch reindex failed:', reindexResponse.status);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('Failed to trigger Typesense reindex:', error);
|
||||
console.warn('Failed to trigger OpenSearch reindex:', error);
|
||||
// Don't fail the whole request if reindex fails
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,9 @@ export async function POST(request: NextRequest) {
|
||||
const scraper = new StoryScraper();
|
||||
const story = await scraper.scrapeStory(url);
|
||||
|
||||
// Check if scraped content contains embedded images
|
||||
const hasImages = story.content ? /<img[^>]+src=['"'][^'"']*['"][^>]*>/i.test(story.content) : false;
|
||||
|
||||
// Debug logging
|
||||
console.log('Scraped story data:', {
|
||||
url: url,
|
||||
@@ -28,10 +31,15 @@ export async function POST(request: NextRequest) {
|
||||
contentLength: story.content?.length || 0,
|
||||
contentPreview: story.content?.substring(0, 200) + '...',
|
||||
tags: story.tags,
|
||||
coverImage: story.coverImage
|
||||
coverImage: story.coverImage,
|
||||
hasEmbeddedImages: hasImages
|
||||
});
|
||||
|
||||
return NextResponse.json(story);
|
||||
// Add image processing flag to response for frontend handling
|
||||
return NextResponse.json({
|
||||
...story,
|
||||
hasEmbeddedImages: hasImages
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Story scraping error:', error);
|
||||
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
'use client';
|
||||
|
||||
import { useState, useEffect } from 'react';
|
||||
import { useRouter, useSearchParams } from 'next/navigation';
|
||||
import AppLayout from '../../components/layout/AppLayout';
|
||||
import { useTheme } from '../../lib/theme';
|
||||
import TabNavigation from '../../components/ui/TabNavigation';
|
||||
import AppearanceSettings from '../../components/settings/AppearanceSettings';
|
||||
import ContentSettings from '../../components/settings/ContentSettings';
|
||||
import SystemSettings from '../../components/settings/SystemSettings';
|
||||
import Button from '../../components/ui/Button';
|
||||
import { storyApi, authorApi, databaseApi } from '../../lib/api';
|
||||
import { useLibraryLayout, LibraryLayoutType } from '../../hooks/useLibraryLayout';
|
||||
import LibrarySettings from '../../components/library/LibrarySettings';
|
||||
import { useTheme } from '../../lib/theme';
|
||||
|
||||
type FontFamily = 'serif' | 'sans' | 'mono';
|
||||
type FontSize = 'small' | 'medium' | 'large' | 'extra-large';
|
||||
@@ -28,29 +30,27 @@ const defaultSettings: Settings = {
|
||||
readingSpeed: 200,
|
||||
};
|
||||
|
||||
const tabs = [
|
||||
{ id: 'appearance', label: 'Appearance', icon: '🎨' },
|
||||
{ id: 'content', label: 'Content', icon: '🏷️' },
|
||||
{ id: 'system', label: 'System', icon: '🔧' },
|
||||
];
|
||||
|
||||
export default function SettingsPage() {
|
||||
const router = useRouter();
|
||||
const searchParams = useSearchParams();
|
||||
const { theme, setTheme } = useTheme();
|
||||
const { layout, setLayout } = useLibraryLayout();
|
||||
const [settings, setSettings] = useState<Settings>(defaultSettings);
|
||||
const [saved, setSaved] = useState(false);
|
||||
const [typesenseStatus, setTypesenseStatus] = useState<{
|
||||
stories: { loading: boolean; message: string; success?: boolean };
|
||||
authors: { loading: boolean; message: string; success?: boolean };
|
||||
}>({
|
||||
stories: { loading: false, message: '' },
|
||||
authors: { loading: false, message: '' }
|
||||
});
|
||||
const [authorsSchema, setAuthorsSchema] = useState<any>(null);
|
||||
const [showSchema, setShowSchema] = useState(false);
|
||||
const [databaseStatus, setDatabaseStatus] = useState<{
|
||||
completeBackup: { loading: boolean; message: string; success?: boolean };
|
||||
completeRestore: { loading: boolean; message: string; success?: boolean };
|
||||
completeClear: { loading: boolean; message: string; success?: boolean };
|
||||
}>({
|
||||
completeBackup: { loading: false, message: '' },
|
||||
completeRestore: { loading: false, message: '' },
|
||||
completeClear: { loading: false, message: '' }
|
||||
});
|
||||
const [activeTab, setActiveTab] = useState('appearance');
|
||||
|
||||
// Initialize tab from URL parameter
|
||||
useEffect(() => {
|
||||
const tabFromUrl = searchParams.get('tab');
|
||||
if (tabFromUrl && tabs.some(tab => tab.id === tabFromUrl)) {
|
||||
setActiveTab(tabFromUrl);
|
||||
}
|
||||
}, [searchParams]);
|
||||
|
||||
// Load settings from localStorage on mount
|
||||
useEffect(() => {
|
||||
@@ -68,6 +68,13 @@ export default function SettingsPage() {
|
||||
}
|
||||
}, [theme]);
|
||||
|
||||
// Update URL when tab changes
|
||||
const handleTabChange = (tabId: string) => {
|
||||
setActiveTab(tabId);
|
||||
const newUrl = `/settings?tab=${tabId}`;
|
||||
router.replace(newUrl, { scroll: false });
|
||||
};
|
||||
|
||||
// Save settings to localStorage
|
||||
const saveSettings = () => {
|
||||
localStorage.setItem('storycove-settings', JSON.stringify(settings));
|
||||
@@ -109,697 +116,58 @@ export default function SettingsPage() {
|
||||
setSettings(prev => ({ ...prev, [key]: value }));
|
||||
};
|
||||
|
||||
const handleTypesenseOperation = async (
|
||||
type: 'stories' | 'authors',
|
||||
operation: 'reindex' | 'recreate',
|
||||
apiCall: () => Promise<{ success: boolean; message: string; count?: number; error?: string }>
|
||||
) => {
|
||||
setTypesenseStatus(prev => ({
|
||||
...prev,
|
||||
[type]: { loading: true, message: 'Processing...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await apiCall();
|
||||
setTypesenseStatus(prev => ({
|
||||
...prev,
|
||||
[type]: {
|
||||
loading: false,
|
||||
message: result.success ? result.message : result.error || 'Operation failed',
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
|
||||
// Clear message after 5 seconds
|
||||
setTimeout(() => {
|
||||
setTypesenseStatus(prev => ({
|
||||
...prev,
|
||||
[type]: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 5000);
|
||||
} catch (error) {
|
||||
setTypesenseStatus(prev => ({
|
||||
...prev,
|
||||
[type]: {
|
||||
loading: false,
|
||||
message: 'Network error occurred',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
|
||||
setTimeout(() => {
|
||||
setTypesenseStatus(prev => ({
|
||||
...prev,
|
||||
[type]: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 5000);
|
||||
}
|
||||
const resetToDefaults = () => {
|
||||
setSettings({ ...defaultSettings, theme });
|
||||
};
|
||||
|
||||
const fetchAuthorsSchema = async () => {
|
||||
try {
|
||||
const result = await authorApi.getTypesenseSchema();
|
||||
if (result.success) {
|
||||
setAuthorsSchema(result.schema);
|
||||
} else {
|
||||
setAuthorsSchema({ error: result.error });
|
||||
}
|
||||
} catch (error) {
|
||||
setAuthorsSchema({ error: 'Failed to fetch schema' });
|
||||
const renderTabContent = () => {
|
||||
switch (activeTab) {
|
||||
case 'appearance':
|
||||
return (
|
||||
<AppearanceSettings
|
||||
settings={settings}
|
||||
onSettingChange={updateSetting}
|
||||
/>
|
||||
);
|
||||
case 'content':
|
||||
return <ContentSettings />;
|
||||
case 'system':
|
||||
return <SystemSettings />;
|
||||
default:
|
||||
return <AppearanceSettings settings={settings} onSettingChange={updateSetting} />;
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
const handleCompleteBackup = async () => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: true, message: 'Creating complete backup...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const backupBlob = await databaseApi.backupComplete();
|
||||
|
||||
// Create download link
|
||||
const url = window.URL.createObjectURL(backupBlob);
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19);
|
||||
link.download = `storycove_complete_backup_${timestamp}.zip`;
|
||||
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
document.body.removeChild(link);
|
||||
window.URL.revokeObjectURL(url);
|
||||
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: false, message: 'Complete backup downloaded successfully', success: true }
|
||||
}));
|
||||
} catch (error: any) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: false, message: error.message || 'Complete backup failed', success: false }
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 5 seconds
|
||||
setTimeout(() => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 5000);
|
||||
};
|
||||
|
||||
const handleCompleteRestore = async (event: React.ChangeEvent<HTMLInputElement>) => {
|
||||
const file = event.target.files?.[0];
|
||||
if (!file) return;
|
||||
|
||||
// Reset the input so the same file can be selected again
|
||||
event.target.value = '';
|
||||
|
||||
if (!file.name.endsWith('.zip')) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: false, message: 'Please select a .zip file', success: false }
|
||||
}));
|
||||
return;
|
||||
}
|
||||
|
||||
const confirmed = window.confirm(
|
||||
'Are you sure you want to restore the complete backup? This will PERMANENTLY DELETE all current data AND files (cover images, avatars) and replace them with the backup data. This action cannot be undone!'
|
||||
);
|
||||
|
||||
if (!confirmed) return;
|
||||
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: true, message: 'Restoring complete backup...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await databaseApi.restoreComplete(file);
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: {
|
||||
loading: false,
|
||||
message: result.success ? result.message : result.message,
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
} catch (error: any) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: false, message: error.message || 'Complete restore failed', success: false }
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 10 seconds for restore (longer because it's important)
|
||||
setTimeout(() => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 10000);
|
||||
};
|
||||
|
||||
const handleCompleteClear = async () => {
|
||||
const confirmed = window.confirm(
|
||||
'Are you ABSOLUTELY SURE you want to clear the entire database AND all files? This will PERMANENTLY DELETE ALL stories, authors, series, tags, collections, AND all uploaded images (covers, avatars). This action cannot be undone!'
|
||||
);
|
||||
|
||||
if (!confirmed) return;
|
||||
|
||||
const doubleConfirmed = window.confirm(
|
||||
'This is your final warning! Clicking OK will DELETE EVERYTHING in your StoryCove database AND all uploaded files. Are you completely certain you want to proceed?'
|
||||
);
|
||||
|
||||
if (!doubleConfirmed) return;
|
||||
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: { loading: true, message: 'Clearing database and files...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await databaseApi.clearComplete();
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: {
|
||||
loading: false,
|
||||
message: result.success
|
||||
? `Database and files cleared successfully. Deleted ${result.deletedRecords} records.`
|
||||
: result.message,
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
} catch (error: any) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: { loading: false, message: error.message || 'Clear operation failed', success: false }
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 10 seconds for clear (longer because it's important)
|
||||
setTimeout(() => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 10000);
|
||||
};
|
||||
|
||||
return (
|
||||
<AppLayout>
|
||||
<div className="max-w-2xl mx-auto space-y-8">
|
||||
<div className="max-w-4xl mx-auto space-y-6">
|
||||
{/* Header */}
|
||||
<div>
|
||||
<h1 className="text-3xl font-bold theme-header">Settings</h1>
|
||||
<p className="theme-text mt-2">
|
||||
Customize your StoryCove reading experience
|
||||
Customize your StoryCove experience and manage system settings
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Theme Settings */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Appearance</h2>
|
||||
{/* Tab Navigation */}
|
||||
<TabNavigation
|
||||
tabs={tabs}
|
||||
activeTab={activeTab}
|
||||
onTabChange={handleTabChange}
|
||||
className="mb-6"
|
||||
/>
|
||||
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Theme
|
||||
</label>
|
||||
<div className="flex gap-4">
|
||||
<button
|
||||
onClick={() => updateSetting('theme', 'light')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
settings.theme === 'light'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
☀️ Light
|
||||
</button>
|
||||
<button
|
||||
onClick={() => updateSetting('theme', 'dark')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
settings.theme === 'dark'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
🌙 Dark
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
{/* Tab Content */}
|
||||
<div className="min-h-[400px]">
|
||||
{renderTabContent()}
|
||||
</div>
|
||||
|
||||
{/* Library Layout */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Library Layout
|
||||
</label>
|
||||
<div className="space-y-3">
|
||||
<div className="flex gap-4 flex-wrap">
|
||||
<button
|
||||
onClick={() => setLayout('sidebar')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
layout === 'sidebar'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
📋 Sidebar Layout
|
||||
</button>
|
||||
<button
|
||||
onClick={() => setLayout('toolbar')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
layout === 'toolbar'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
🛠️ Toolbar Layout
|
||||
</button>
|
||||
<button
|
||||
onClick={() => setLayout('minimal')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
layout === 'minimal'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
✨ Minimal Layout
|
||||
</button>
|
||||
</div>
|
||||
<div className="text-sm theme-text">
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-3 mt-3">
|
||||
<div className="text-xs">
|
||||
<strong>Sidebar:</strong> Filters and controls in a side panel, maximum space for stories
|
||||
</div>
|
||||
<div className="text-xs">
|
||||
<strong>Toolbar:</strong> Everything visible at once with integrated search and tag filters
|
||||
</div>
|
||||
<div className="text-xs">
|
||||
<strong>Minimal:</strong> Clean, content-focused design with floating controls
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reading Settings */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Reading Experience</h2>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Font Family */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Font Family
|
||||
</label>
|
||||
<div className="flex gap-4 flex-wrap">
|
||||
<button
|
||||
onClick={() => updateSetting('fontFamily', 'serif')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors font-serif ${
|
||||
settings.fontFamily === 'serif'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
Serif
|
||||
</button>
|
||||
<button
|
||||
onClick={() => updateSetting('fontFamily', 'sans')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors font-sans ${
|
||||
settings.fontFamily === 'sans'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
Sans Serif
|
||||
</button>
|
||||
<button
|
||||
onClick={() => updateSetting('fontFamily', 'mono')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors font-mono ${
|
||||
settings.fontFamily === 'mono'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
Monospace
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Font Size */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Font Size
|
||||
</label>
|
||||
<div className="flex gap-4 flex-wrap">
|
||||
{(['small', 'medium', 'large', 'extra-large'] as FontSize[]).map((size) => (
|
||||
<button
|
||||
key={size}
|
||||
onClick={() => updateSetting('fontSize', size)}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors capitalize ${
|
||||
settings.fontSize === size
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
{size.replace('-', ' ')}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reading Width */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Reading Width
|
||||
</label>
|
||||
<div className="flex gap-4">
|
||||
{(['narrow', 'medium', 'wide'] as ReadingWidth[]).map((width) => (
|
||||
<button
|
||||
key={width}
|
||||
onClick={() => updateSetting('readingWidth', width)}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors capitalize ${
|
||||
settings.readingWidth === width
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
{width}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reading Speed */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Reading Speed (words per minute)
|
||||
</label>
|
||||
<div className="flex items-center gap-4">
|
||||
<input
|
||||
type="range"
|
||||
min="100"
|
||||
max="400"
|
||||
step="25"
|
||||
value={settings.readingSpeed}
|
||||
onChange={(e) => updateSetting('readingSpeed', parseInt(e.target.value))}
|
||||
className="flex-1 h-2 bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
|
||||
/>
|
||||
<div className="min-w-[80px] text-center">
|
||||
<span className="text-lg font-medium theme-header">{settings.readingSpeed}</span>
|
||||
<div className="text-xs theme-text">WPM</div>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex justify-between text-xs theme-text mt-1">
|
||||
<span>Slow (100)</span>
|
||||
<span>Average (200)</span>
|
||||
<span>Fast (400)</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Preview */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Preview</h2>
|
||||
|
||||
<div
|
||||
className="p-4 theme-card border theme-border rounded-lg"
|
||||
style={{
|
||||
fontFamily: settings.fontFamily === 'serif' ? 'Georgia, Times, serif'
|
||||
: settings.fontFamily === 'sans' ? 'Inter, system-ui, sans-serif'
|
||||
: 'Monaco, Consolas, monospace',
|
||||
fontSize: settings.fontSize === 'small' ? '14px'
|
||||
: settings.fontSize === 'medium' ? '16px'
|
||||
: settings.fontSize === 'large' ? '18px'
|
||||
: '20px',
|
||||
maxWidth: settings.readingWidth === 'narrow' ? '600px'
|
||||
: settings.readingWidth === 'medium' ? '800px'
|
||||
: '1000px',
|
||||
}}
|
||||
>
|
||||
<h3 className="text-xl font-bold theme-header mb-2">Sample Story Title</h3>
|
||||
<p className="theme-text mb-4">by Sample Author</p>
|
||||
<p className="theme-text leading-relaxed">
|
||||
This is how your story text will look with the current settings.
|
||||
The quick brown fox jumps over the lazy dog. Lorem ipsum dolor sit amet,
|
||||
consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore
|
||||
et dolore magna aliqua.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Typesense Search Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Search Index Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Manage the Typesense search indexes for stories and authors. Use these tools if search functionality isn't working properly.
|
||||
</p>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Stories Section */}
|
||||
<div className="border theme-border rounded-lg p-4">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">Stories Index</h3>
|
||||
<div className="flex flex-col sm:flex-row gap-3 mb-3">
|
||||
<Button
|
||||
onClick={() => handleTypesenseOperation('stories', 'reindex', storyApi.reindexTypesense)}
|
||||
disabled={typesenseStatus.stories.loading}
|
||||
loading={typesenseStatus.stories.loading}
|
||||
variant="ghost"
|
||||
className="flex-1"
|
||||
>
|
||||
{typesenseStatus.stories.loading ? 'Reindexing...' : 'Reindex Stories'}
|
||||
</Button>
|
||||
<Button
|
||||
onClick={() => handleTypesenseOperation('stories', 'recreate', storyApi.recreateTypesenseCollection)}
|
||||
disabled={typesenseStatus.stories.loading}
|
||||
loading={typesenseStatus.stories.loading}
|
||||
variant="secondary"
|
||||
className="flex-1"
|
||||
>
|
||||
{typesenseStatus.stories.loading ? 'Recreating...' : 'Recreate Collection'}
|
||||
</Button>
|
||||
</div>
|
||||
{typesenseStatus.stories.message && (
|
||||
<div className={`text-sm p-2 rounded ${
|
||||
typesenseStatus.stories.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{typesenseStatus.stories.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Authors Section */}
|
||||
<div className="border theme-border rounded-lg p-4">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">Authors Index</h3>
|
||||
<div className="flex flex-col sm:flex-row gap-3 mb-3">
|
||||
<Button
|
||||
onClick={() => handleTypesenseOperation('authors', 'reindex', authorApi.reindexTypesense)}
|
||||
disabled={typesenseStatus.authors.loading}
|
||||
loading={typesenseStatus.authors.loading}
|
||||
variant="ghost"
|
||||
className="flex-1"
|
||||
>
|
||||
{typesenseStatus.authors.loading ? 'Reindexing...' : 'Reindex Authors'}
|
||||
</Button>
|
||||
<Button
|
||||
onClick={() => handleTypesenseOperation('authors', 'recreate', authorApi.recreateTypesenseCollection)}
|
||||
disabled={typesenseStatus.authors.loading}
|
||||
loading={typesenseStatus.authors.loading}
|
||||
variant="secondary"
|
||||
className="flex-1"
|
||||
>
|
||||
{typesenseStatus.authors.loading ? 'Recreating...' : 'Recreate Collection'}
|
||||
</Button>
|
||||
</div>
|
||||
{typesenseStatus.authors.message && (
|
||||
<div className={`text-sm p-2 rounded ${
|
||||
typesenseStatus.authors.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{typesenseStatus.authors.message}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Debug Schema Section */}
|
||||
<div className="border-t theme-border pt-3">
|
||||
<div className="flex items-center gap-2 mb-2">
|
||||
<Button
|
||||
onClick={fetchAuthorsSchema}
|
||||
variant="ghost"
|
||||
className="text-xs"
|
||||
>
|
||||
Inspect Schema
|
||||
</Button>
|
||||
<Button
|
||||
onClick={() => setShowSchema(!showSchema)}
|
||||
variant="ghost"
|
||||
className="text-xs"
|
||||
disabled={!authorsSchema}
|
||||
>
|
||||
{showSchema ? 'Hide' : 'Show'} Schema
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{showSchema && authorsSchema && (
|
||||
<div className="text-xs theme-text bg-gray-50 dark:bg-gray-800 p-3 rounded border overflow-auto max-h-48">
|
||||
<pre>{JSON.stringify(authorsSchema, null, 2)}</pre>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="text-sm theme-text bg-blue-50 dark:bg-blue-900/20 p-3 rounded-lg">
|
||||
<p className="font-medium mb-1">When to use these tools:</p>
|
||||
<ul className="text-xs space-y-1 ml-4">
|
||||
<li>• <strong>Reindex:</strong> Refresh search data while keeping the existing schema</li>
|
||||
<li>• <strong>Recreate Collection:</strong> Delete and rebuild the entire search index (fixes schema issues)</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Database Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Database Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Backup, restore, or clear your StoryCove database and files. These comprehensive operations include both your data and uploaded images.
|
||||
</p>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Complete Backup Section */}
|
||||
<div className="border theme-border rounded-lg p-4 border-blue-200 dark:border-blue-800">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">📦 Create Backup</h3>
|
||||
<p className="text-sm theme-text mb-3">
|
||||
Download a complete backup as a ZIP file. This includes your database AND all uploaded files (cover images, avatars). This is a comprehensive backup of your entire StoryCove installation.
|
||||
</p>
|
||||
<Button
|
||||
onClick={handleCompleteBackup}
|
||||
disabled={databaseStatus.completeBackup.loading}
|
||||
loading={databaseStatus.completeBackup.loading}
|
||||
variant="primary"
|
||||
className="w-full sm:w-auto"
|
||||
>
|
||||
{databaseStatus.completeBackup.loading ? 'Creating Backup...' : 'Download Backup'}
|
||||
</Button>
|
||||
{databaseStatus.completeBackup.message && (
|
||||
<div className={`text-sm p-2 rounded mt-3 ${
|
||||
databaseStatus.completeBackup.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{databaseStatus.completeBackup.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Restore Section */}
|
||||
<div className="border theme-border rounded-lg p-4 border-orange-200 dark:border-orange-800">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">📥 Restore Backup</h3>
|
||||
<p className="text-sm theme-text mb-3">
|
||||
<strong className="text-orange-600 dark:text-orange-400">⚠️ Warning:</strong> This will completely replace your current database AND all files with the backup. All existing data and uploaded files will be permanently deleted.
|
||||
</p>
|
||||
<div className="flex items-center gap-3">
|
||||
<input
|
||||
type="file"
|
||||
accept=".zip"
|
||||
onChange={handleCompleteRestore}
|
||||
disabled={databaseStatus.completeRestore.loading}
|
||||
className="flex-1 text-sm theme-text file:mr-4 file:py-2 file:px-4 file:rounded-lg file:border-0 file:text-sm file:font-medium file:theme-accent-bg file:text-white hover:file:bg-opacity-90 file:cursor-pointer"
|
||||
/>
|
||||
</div>
|
||||
{databaseStatus.completeRestore.message && (
|
||||
<div className={`text-sm p-2 rounded mt-3 ${
|
||||
databaseStatus.completeRestore.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{databaseStatus.completeRestore.message}
|
||||
</div>
|
||||
)}
|
||||
{databaseStatus.completeRestore.loading && (
|
||||
<div className="text-sm theme-text mt-3 flex items-center gap-2">
|
||||
<div className="animate-spin w-4 h-4 border-2 border-blue-500 border-t-transparent rounded-full"></div>
|
||||
Restoring backup...
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Clear Everything Section */}
|
||||
<div className="border theme-border rounded-lg p-4 border-red-200 dark:border-red-800 bg-red-50 dark:bg-red-900/10">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">🗑️ Clear Everything</h3>
|
||||
<p className="text-sm theme-text mb-3">
|
||||
<strong className="text-red-600 dark:text-red-400">⚠️ Danger Zone:</strong> This will permanently delete ALL data from your database AND all uploaded files (cover images, avatars). Everything will be completely removed. This action cannot be undone!
|
||||
</p>
|
||||
<Button
|
||||
onClick={handleCompleteClear}
|
||||
disabled={databaseStatus.completeClear.loading}
|
||||
loading={databaseStatus.completeClear.loading}
|
||||
variant="secondary"
|
||||
className="w-full sm:w-auto bg-red-700 hover:bg-red-800 text-white border-red-700"
|
||||
>
|
||||
{databaseStatus.completeClear.loading ? 'Clearing Everything...' : 'Clear Everything'}
|
||||
</Button>
|
||||
{databaseStatus.completeClear.message && (
|
||||
<div className={`text-sm p-2 rounded mt-3 ${
|
||||
databaseStatus.completeClear.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{databaseStatus.completeClear.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="text-sm theme-text bg-blue-50 dark:bg-blue-900/20 p-3 rounded-lg">
|
||||
<p className="font-medium mb-1">💡 Best Practices:</p>
|
||||
<ul className="text-xs space-y-1 ml-4">
|
||||
<li>• <strong>Always backup</strong> before performing restore or clear operations</li>
|
||||
<li>• <strong>Store backups safely</strong> in multiple locations for important data</li>
|
||||
<li>• <strong>Test restores</strong> in a development environment when possible</li>
|
||||
<li>• <strong>Backup files (.zip)</strong> contain both database and all uploaded files</li>
|
||||
<li>• <strong>Verify backup files</strong> are complete before relying on them</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Library Settings */}
|
||||
<LibrarySettings />
|
||||
|
||||
{/* Tag Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Tag Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Manage your story tags with colors, descriptions, and aliases. Use the Tag Maintenance page to organize and customize your tags.
|
||||
</p>
|
||||
<Button
|
||||
href="/settings/tag-maintenance"
|
||||
variant="secondary"
|
||||
className="w-full sm:w-auto"
|
||||
>
|
||||
🏷️ Open Tag Maintenance
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Actions */}
|
||||
<div className="flex justify-end gap-4">
|
||||
{/* Save Actions - Only show for Appearance tab */}
|
||||
{activeTab === 'appearance' && (
|
||||
<div className="flex justify-end gap-4 pt-6 border-t theme-border">
|
||||
<Button
|
||||
variant="ghost"
|
||||
onClick={() => {
|
||||
setSettings({ ...defaultSettings, theme });
|
||||
}}
|
||||
onClick={resetToDefaults}
|
||||
>
|
||||
Reset to Defaults
|
||||
</Button>
|
||||
@@ -811,7 +179,7 @@ export default function SettingsPage() {
|
||||
{saved ? '✓ Saved!' : 'Save Settings'}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</AppLayout>
|
||||
);
|
||||
|
||||
@@ -20,6 +20,7 @@ export default function StoryReadingPage() {
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [readingProgress, setReadingProgress] = useState(0);
|
||||
const [readingPercentage, setReadingPercentage] = useState(0);
|
||||
const [sanitizedContent, setSanitizedContent] = useState<string>('');
|
||||
const [hasScrolledToPosition, setHasScrolledToPosition] = useState(false);
|
||||
const [showToc, setShowToc] = useState(false);
|
||||
@@ -48,15 +49,25 @@ export default function StoryReadingPage() {
|
||||
));
|
||||
|
||||
// Convert to character position in the plain text content
|
||||
const textLength = story.contentPlain?.length || story.contentHtml.length;
|
||||
const textLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
return Math.floor(scrollRatio * textLength);
|
||||
}, [story]);
|
||||
|
||||
// Calculate reading percentage from character position
|
||||
const calculateReadingPercentage = useCallback((currentPosition: number): number => {
|
||||
if (!story) return 0;
|
||||
|
||||
const totalLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
if (totalLength === 0) return 0;
|
||||
|
||||
return Math.round((currentPosition / totalLength) * 100);
|
||||
}, [story]);
|
||||
|
||||
// Convert character position back to scroll position for auto-scroll
|
||||
const scrollToCharacterPosition = useCallback((position: number) => {
|
||||
if (!contentRef.current || !story || hasScrolledToPosition) return;
|
||||
|
||||
const textLength = story.contentPlain?.length || story.contentHtml.length;
|
||||
const textLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
if (textLength === 0 || position === 0) return;
|
||||
|
||||
const ratio = position / textLength;
|
||||
@@ -188,17 +199,20 @@ export default function StoryReadingPage() {
|
||||
// Otherwise, use saved reading position
|
||||
if (story.readingPosition && story.readingPosition > 0) {
|
||||
console.log('Auto-scrolling to saved position:', story.readingPosition);
|
||||
const initialPercentage = calculateReadingPercentage(story.readingPosition);
|
||||
setReadingPercentage(initialPercentage);
|
||||
scrollToCharacterPosition(story.readingPosition);
|
||||
} else {
|
||||
// Even if there's no saved position, mark as ready for tracking
|
||||
console.log('No saved position, starting fresh tracking');
|
||||
setReadingPercentage(0);
|
||||
setHasScrolledToPosition(true);
|
||||
}
|
||||
}, 500);
|
||||
|
||||
return () => clearTimeout(timeout);
|
||||
}
|
||||
}, [story, sanitizedContent, scrollToCharacterPosition, hasScrolledToPosition]);
|
||||
}, [story, sanitizedContent, scrollToCharacterPosition, calculateReadingPercentage, hasScrolledToPosition]);
|
||||
|
||||
// Track reading progress and save position
|
||||
useEffect(() => {
|
||||
@@ -244,10 +258,12 @@ export default function StoryReadingPage() {
|
||||
setShowEndOfStoryPopup(true);
|
||||
}
|
||||
|
||||
// Save reading position (debounced)
|
||||
// Save reading position and update percentage (debounced)
|
||||
if (hasScrolledToPosition) { // Only save after initial auto-scroll
|
||||
const characterPosition = getCharacterPositionFromScroll();
|
||||
console.log('Scroll detected, character position:', characterPosition);
|
||||
const percentage = calculateReadingPercentage(characterPosition);
|
||||
console.log('Scroll detected, character position:', characterPosition, 'percentage:', percentage);
|
||||
setReadingPercentage(percentage);
|
||||
debouncedSavePosition(characterPosition);
|
||||
} else {
|
||||
console.log('Scroll detected but not ready for tracking yet');
|
||||
@@ -263,7 +279,7 @@ export default function StoryReadingPage() {
|
||||
clearTimeout(saveTimeoutRef.current);
|
||||
}
|
||||
};
|
||||
}, [story, hasScrolledToPosition, getCharacterPositionFromScroll, debouncedSavePosition, hasReachedEnd]);
|
||||
}, [story, hasScrolledToPosition, getCharacterPositionFromScroll, calculateReadingPercentage, debouncedSavePosition, hasReachedEnd]);
|
||||
|
||||
const handleRatingUpdate = async (newRating: number) => {
|
||||
if (!story) return;
|
||||
@@ -359,6 +375,11 @@ export default function StoryReadingPage() {
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-4">
|
||||
{/* Reading percentage indicator */}
|
||||
<div className="text-sm theme-text font-mono bg-gray-100 dark:bg-gray-800 px-2 py-1 rounded">
|
||||
{readingPercentage}%
|
||||
</div>
|
||||
|
||||
{hasHeadings && (
|
||||
<button
|
||||
onClick={() => setShowToc(!showToc)}
|
||||
|
||||
@@ -20,6 +20,7 @@ export default function CollectionReadingView({
|
||||
}: CollectionReadingViewProps) {
|
||||
const { story, collection } = data;
|
||||
const [hasScrolledToPosition, setHasScrolledToPosition] = useState(false);
|
||||
const [readingPercentage, setReadingPercentage] = useState(0);
|
||||
const contentRef = useRef<HTMLDivElement>(null);
|
||||
const saveTimeoutRef = useRef<NodeJS.Timeout | null>(null);
|
||||
|
||||
@@ -39,15 +40,25 @@ export default function CollectionReadingView({
|
||||
));
|
||||
|
||||
// Convert to character position in the plain text content
|
||||
const textLength = story.contentPlain?.length || story.contentHtml.length;
|
||||
const textLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
return Math.floor(scrollRatio * textLength);
|
||||
}, [story]);
|
||||
|
||||
// Calculate reading percentage from character position
|
||||
const calculateReadingPercentage = useCallback((currentPosition: number): number => {
|
||||
if (!story) return 0;
|
||||
|
||||
const totalLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
if (totalLength === 0) return 0;
|
||||
|
||||
return Math.round((currentPosition / totalLength) * 100);
|
||||
}, [story]);
|
||||
|
||||
// Convert character position back to scroll position for auto-scroll
|
||||
const scrollToCharacterPosition = useCallback((position: number) => {
|
||||
if (!contentRef.current || !story || hasScrolledToPosition) return;
|
||||
|
||||
const textLength = story.contentPlain?.length || story.contentHtml.length;
|
||||
const textLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
if (textLength === 0 || position === 0) return;
|
||||
|
||||
const ratio = position / textLength;
|
||||
@@ -102,23 +113,28 @@ export default function CollectionReadingView({
|
||||
console.log('Collection view - initializing reading position tracking, saved position:', story.readingPosition);
|
||||
if (story.readingPosition && story.readingPosition > 0) {
|
||||
console.log('Collection view - auto-scrolling to saved position:', story.readingPosition);
|
||||
const initialPercentage = calculateReadingPercentage(story.readingPosition);
|
||||
setReadingPercentage(initialPercentage);
|
||||
scrollToCharacterPosition(story.readingPosition);
|
||||
} else {
|
||||
console.log('Collection view - no saved position, starting fresh tracking');
|
||||
setReadingPercentage(0);
|
||||
setHasScrolledToPosition(true);
|
||||
}
|
||||
}, 500);
|
||||
|
||||
return () => clearTimeout(timeout);
|
||||
}
|
||||
}, [story, scrollToCharacterPosition, hasScrolledToPosition]);
|
||||
}, [story, scrollToCharacterPosition, calculateReadingPercentage, hasScrolledToPosition]);
|
||||
|
||||
// Track reading progress and save position
|
||||
useEffect(() => {
|
||||
const handleScroll = () => {
|
||||
if (hasScrolledToPosition) {
|
||||
const characterPosition = getCharacterPositionFromScroll();
|
||||
console.log('Collection view - scroll detected, character position:', characterPosition);
|
||||
const percentage = calculateReadingPercentage(characterPosition);
|
||||
console.log('Collection view - scroll detected, character position:', characterPosition, 'percentage:', percentage);
|
||||
setReadingPercentage(percentage);
|
||||
debouncedSavePosition(characterPosition);
|
||||
} else {
|
||||
console.log('Collection view - scroll detected but not ready for tracking yet');
|
||||
@@ -132,7 +148,7 @@ export default function CollectionReadingView({
|
||||
clearTimeout(saveTimeoutRef.current);
|
||||
}
|
||||
};
|
||||
}, [hasScrolledToPosition, getCharacterPositionFromScroll, debouncedSavePosition]);
|
||||
}, [hasScrolledToPosition, getCharacterPositionFromScroll, calculateReadingPercentage, debouncedSavePosition]);
|
||||
|
||||
const handlePrevious = () => {
|
||||
if (collection.previousStoryId) {
|
||||
@@ -190,6 +206,11 @@ export default function CollectionReadingView({
|
||||
|
||||
{/* Progress Bar */}
|
||||
<div className="flex items-center gap-4">
|
||||
{/* Reading percentage indicator */}
|
||||
<div className="text-sm text-blue-700 dark:text-blue-300 font-mono bg-blue-100 dark:bg-blue-900 px-2 py-1 rounded">
|
||||
{readingPercentage}%
|
||||
</div>
|
||||
|
||||
<div className="w-32 bg-blue-200 dark:bg-blue-800 rounded-full h-2">
|
||||
<div
|
||||
className="bg-blue-600 dark:bg-blue-400 h-2 rounded-full transition-all duration-300"
|
||||
|
||||
@@ -127,29 +127,6 @@ const FILTER_PRESETS: FilterPreset[] = [
|
||||
description: 'Stories that are part of a series',
|
||||
filters: { seriesFilter: 'series' },
|
||||
category: 'content'
|
||||
},
|
||||
|
||||
// Organization presets
|
||||
{
|
||||
id: 'well-tagged',
|
||||
label: '3+ tags',
|
||||
description: 'Well-tagged stories with 3 or more tags',
|
||||
filters: { minTagCount: 3 },
|
||||
category: 'organization'
|
||||
},
|
||||
{
|
||||
id: 'popular',
|
||||
label: 'Popular',
|
||||
description: 'Stories with above-average ratings',
|
||||
filters: { popularOnly: true },
|
||||
category: 'organization'
|
||||
},
|
||||
{
|
||||
id: 'hidden-gems',
|
||||
label: 'Hidden Gems',
|
||||
description: 'Underrated or unrated stories to discover',
|
||||
filters: { hiddenGemsOnly: true },
|
||||
category: 'organization'
|
||||
}
|
||||
];
|
||||
|
||||
|
||||
265
frontend/src/components/settings/AppearanceSettings.tsx
Normal file
265
frontend/src/components/settings/AppearanceSettings.tsx
Normal file
@@ -0,0 +1,265 @@
|
||||
'use client';
|
||||
|
||||
import { useTheme } from '../../lib/theme';
|
||||
import { useLibraryLayout, LibraryLayoutType } from '../../hooks/useLibraryLayout';
|
||||
|
||||
type FontFamily = 'serif' | 'sans' | 'mono';
|
||||
type FontSize = 'small' | 'medium' | 'large' | 'extra-large';
|
||||
type ReadingWidth = 'narrow' | 'medium' | 'wide';
|
||||
|
||||
interface Settings {
|
||||
theme: 'light' | 'dark';
|
||||
fontFamily: FontFamily;
|
||||
fontSize: FontSize;
|
||||
readingWidth: ReadingWidth;
|
||||
readingSpeed: number; // words per minute
|
||||
}
|
||||
|
||||
interface AppearanceSettingsProps {
|
||||
settings: Settings;
|
||||
onSettingChange: <K extends keyof Settings>(key: K, value: Settings[K]) => void;
|
||||
}
|
||||
|
||||
export default function AppearanceSettings({
|
||||
settings,
|
||||
onSettingChange
|
||||
}: AppearanceSettingsProps) {
|
||||
const { layout, setLayout } = useLibraryLayout();
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Theme Settings */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Theme</h2>
|
||||
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Color Theme
|
||||
</label>
|
||||
<div className="flex gap-4">
|
||||
<button
|
||||
onClick={() => onSettingChange('theme', 'light')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
settings.theme === 'light'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
☀️ Light
|
||||
</button>
|
||||
<button
|
||||
onClick={() => onSettingChange('theme', 'dark')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
settings.theme === 'dark'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
🌙 Dark
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Library Layout */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Library Layout</h2>
|
||||
|
||||
<div className="space-y-3">
|
||||
<div className="flex gap-4 flex-wrap">
|
||||
<button
|
||||
onClick={() => setLayout('sidebar')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
layout === 'sidebar'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
📋 Sidebar Layout
|
||||
</button>
|
||||
<button
|
||||
onClick={() => setLayout('toolbar')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
layout === 'toolbar'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
🛠️ Toolbar Layout
|
||||
</button>
|
||||
<button
|
||||
onClick={() => setLayout('minimal')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors ${
|
||||
layout === 'minimal'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
✨ Minimal Layout
|
||||
</button>
|
||||
</div>
|
||||
<div className="text-sm theme-text">
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-3 mt-3">
|
||||
<div className="text-xs">
|
||||
<strong>Sidebar:</strong> Filters and controls in a side panel, maximum space for stories
|
||||
</div>
|
||||
<div className="text-xs">
|
||||
<strong>Toolbar:</strong> Everything visible at once with integrated search and tag filters
|
||||
</div>
|
||||
<div className="text-xs">
|
||||
<strong>Minimal:</strong> Clean, content-focused design with floating controls
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reading Experience */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Reading Experience</h2>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Font Family */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Font Family
|
||||
</label>
|
||||
<div className="flex gap-4 flex-wrap">
|
||||
<button
|
||||
onClick={() => onSettingChange('fontFamily', 'serif')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors font-serif ${
|
||||
settings.fontFamily === 'serif'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
Serif
|
||||
</button>
|
||||
<button
|
||||
onClick={() => onSettingChange('fontFamily', 'sans')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors font-sans ${
|
||||
settings.fontFamily === 'sans'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
Sans Serif
|
||||
</button>
|
||||
<button
|
||||
onClick={() => onSettingChange('fontFamily', 'mono')}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors font-mono ${
|
||||
settings.fontFamily === 'mono'
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
Monospace
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Font Size */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Font Size
|
||||
</label>
|
||||
<div className="flex gap-4 flex-wrap">
|
||||
{(['small', 'medium', 'large', 'extra-large'] as FontSize[]).map((size) => (
|
||||
<button
|
||||
key={size}
|
||||
onClick={() => onSettingChange('fontSize', size)}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors capitalize ${
|
||||
settings.fontSize === size
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
{size.replace('-', ' ')}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reading Width */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Reading Width
|
||||
</label>
|
||||
<div className="flex gap-4">
|
||||
{(['narrow', 'medium', 'wide'] as ReadingWidth[]).map((width) => (
|
||||
<button
|
||||
key={width}
|
||||
onClick={() => onSettingChange('readingWidth', width)}
|
||||
className={`px-4 py-2 rounded-lg border transition-colors capitalize ${
|
||||
settings.readingWidth === width
|
||||
? 'theme-accent-bg text-white border-transparent'
|
||||
: 'theme-card theme-text theme-border hover:border-gray-400'
|
||||
}`}
|
||||
>
|
||||
{width}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Reading Speed */}
|
||||
<div>
|
||||
<label className="block text-sm font-medium theme-header mb-2">
|
||||
Reading Speed (words per minute)
|
||||
</label>
|
||||
<div className="flex items-center gap-4">
|
||||
<input
|
||||
type="range"
|
||||
min="100"
|
||||
max="400"
|
||||
step="25"
|
||||
value={settings.readingSpeed}
|
||||
onChange={(e) => onSettingChange('readingSpeed', parseInt(e.target.value))}
|
||||
className="flex-1 h-2 bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
|
||||
/>
|
||||
<div className="min-w-[80px] text-center">
|
||||
<span className="text-lg font-medium theme-header">{settings.readingSpeed}</span>
|
||||
<div className="text-xs theme-text">WPM</div>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex justify-between text-xs theme-text mt-1">
|
||||
<span>Slow (100)</span>
|
||||
<span>Average (200)</span>
|
||||
<span>Fast (400)</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Preview */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Preview</h2>
|
||||
|
||||
<div
|
||||
className="p-4 theme-card border theme-border rounded-lg"
|
||||
style={{
|
||||
fontFamily: settings.fontFamily === 'serif' ? 'Georgia, Times, serif'
|
||||
: settings.fontFamily === 'sans' ? 'Inter, system-ui, sans-serif'
|
||||
: 'Monaco, Consolas, monospace',
|
||||
fontSize: settings.fontSize === 'small' ? '14px'
|
||||
: settings.fontSize === 'medium' ? '16px'
|
||||
: settings.fontSize === 'large' ? '18px'
|
||||
: '20px',
|
||||
maxWidth: settings.readingWidth === 'narrow' ? '600px'
|
||||
: settings.readingWidth === 'medium' ? '800px'
|
||||
: '1000px',
|
||||
}}
|
||||
>
|
||||
<h3 className="text-xl font-bold theme-header mb-2">Sample Story Title</h3>
|
||||
<p className="theme-text mb-4">by Sample Author</p>
|
||||
<p className="theme-text leading-relaxed">
|
||||
This is how your story text will look with the current settings.
|
||||
The quick brown fox jumps over the lazy dog. Lorem ipsum dolor sit amet,
|
||||
consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore
|
||||
et dolore magna aliqua.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
32
frontend/src/components/settings/ContentSettings.tsx
Normal file
32
frontend/src/components/settings/ContentSettings.tsx
Normal file
@@ -0,0 +1,32 @@
|
||||
'use client';
|
||||
|
||||
import Button from '../ui/Button';
|
||||
import LibrarySettings from '../library/LibrarySettings';
|
||||
|
||||
interface ContentSettingsProps {
|
||||
// No props needed - LibrarySettings manages its own state
|
||||
}
|
||||
|
||||
export default function ContentSettings({}: ContentSettingsProps) {
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Library Settings */}
|
||||
<LibrarySettings />
|
||||
|
||||
{/* Tag Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Tag Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Manage your story tags with colors, descriptions, and aliases. Use the Tag Maintenance page to organize and customize your tags.
|
||||
</p>
|
||||
<Button
|
||||
href="/settings/tag-maintenance"
|
||||
variant="secondary"
|
||||
className="w-full sm:w-auto"
|
||||
>
|
||||
🏷️ Open Tag Maintenance
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
707
frontend/src/components/settings/SystemSettings.tsx
Normal file
707
frontend/src/components/settings/SystemSettings.tsx
Normal file
@@ -0,0 +1,707 @@
|
||||
'use client';
|
||||
|
||||
import React, { useState, useEffect } from 'react';
|
||||
import Button from '../ui/Button';
|
||||
import { databaseApi, configApi, searchAdminApi } from '../../lib/api';
|
||||
|
||||
interface SystemSettingsProps {
|
||||
// No props needed - this component manages its own state
|
||||
}
|
||||
|
||||
export default function SystemSettings({}: SystemSettingsProps) {
|
||||
const [searchEngineStatus, setSearchEngineStatus] = useState<{
|
||||
currentEngine: string;
|
||||
openSearchAvailable: boolean;
|
||||
loading: boolean;
|
||||
message: string;
|
||||
success?: boolean;
|
||||
}>({
|
||||
currentEngine: 'opensearch',
|
||||
openSearchAvailable: false,
|
||||
loading: false,
|
||||
message: ''
|
||||
});
|
||||
|
||||
const [openSearchStatus, setOpenSearchStatus] = useState<{
|
||||
reindex: { loading: boolean; message: string; success?: boolean };
|
||||
recreate: { loading: boolean; message: string; success?: boolean };
|
||||
}>({
|
||||
reindex: { loading: false, message: '' },
|
||||
recreate: { loading: false, message: '' }
|
||||
});
|
||||
|
||||
const [databaseStatus, setDatabaseStatus] = useState<{
|
||||
completeBackup: { loading: boolean; message: string; success?: boolean };
|
||||
completeRestore: { loading: boolean; message: string; success?: boolean };
|
||||
completeClear: { loading: boolean; message: string; success?: boolean };
|
||||
}>({
|
||||
completeBackup: { loading: false, message: '' },
|
||||
completeRestore: { loading: false, message: '' },
|
||||
completeClear: { loading: false, message: '' }
|
||||
});
|
||||
const [cleanupStatus, setCleanupStatus] = useState<{
|
||||
preview: { loading: boolean; message: string; success?: boolean; data?: any };
|
||||
execute: { loading: boolean; message: string; success?: boolean };
|
||||
}>({
|
||||
preview: { loading: false, message: '' },
|
||||
execute: { loading: false, message: '' }
|
||||
});
|
||||
|
||||
|
||||
|
||||
const handleCompleteBackup = async () => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: true, message: 'Creating complete backup...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const backupBlob = await databaseApi.backupComplete();
|
||||
|
||||
// Create download link
|
||||
const url = window.URL.createObjectURL(backupBlob);
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19);
|
||||
link.download = `storycove_complete_backup_${timestamp}.zip`;
|
||||
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
document.body.removeChild(link);
|
||||
window.URL.revokeObjectURL(url);
|
||||
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: false, message: 'Complete backup downloaded successfully', success: true }
|
||||
}));
|
||||
} catch (error: any) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: false, message: error.message || 'Complete backup failed', success: false }
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 5 seconds
|
||||
setTimeout(() => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeBackup: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 5000);
|
||||
};
|
||||
|
||||
const handleCompleteRestore = async (event: React.ChangeEvent<HTMLInputElement>) => {
|
||||
const file = event.target.files?.[0];
|
||||
if (!file) return;
|
||||
|
||||
// Reset the input so the same file can be selected again
|
||||
event.target.value = '';
|
||||
|
||||
if (!file.name.endsWith('.zip')) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: false, message: 'Please select a .zip file', success: false }
|
||||
}));
|
||||
return;
|
||||
}
|
||||
|
||||
const confirmed = window.confirm(
|
||||
'Are you sure you want to restore the complete backup? This will PERMANENTLY DELETE all current data AND files (cover images, avatars) and replace them with the backup data. This action cannot be undone!'
|
||||
);
|
||||
|
||||
if (!confirmed) return;
|
||||
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: true, message: 'Restoring complete backup...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await databaseApi.restoreComplete(file);
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: {
|
||||
loading: false,
|
||||
message: result.success ? result.message : result.message,
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
} catch (error: any) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: false, message: error.message || 'Complete restore failed', success: false }
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 10 seconds for restore (longer because it's important)
|
||||
setTimeout(() => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeRestore: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 10000);
|
||||
};
|
||||
|
||||
const handleCompleteClear = async () => {
|
||||
const confirmed = window.confirm(
|
||||
'Are you ABSOLUTELY SURE you want to clear the entire database AND all files? This will PERMANENTLY DELETE ALL stories, authors, series, tags, collections, AND all uploaded images (covers, avatars). This action cannot be undone!'
|
||||
);
|
||||
|
||||
if (!confirmed) return;
|
||||
|
||||
const doubleConfirmed = window.confirm(
|
||||
'This is your final warning! Clicking OK will DELETE EVERYTHING in your StoryCove database AND all uploaded files. Are you completely certain you want to proceed?'
|
||||
);
|
||||
|
||||
if (!doubleConfirmed) return;
|
||||
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: { loading: true, message: 'Clearing database and files...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await databaseApi.clearComplete();
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: {
|
||||
loading: false,
|
||||
message: result.success
|
||||
? `Database and files cleared successfully. Deleted ${result.deletedRecords} records.`
|
||||
: result.message,
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
} catch (error: any) {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: { loading: false, message: error.message || 'Clear operation failed', success: false }
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 10 seconds for clear (longer because it's important)
|
||||
setTimeout(() => {
|
||||
setDatabaseStatus(prev => ({
|
||||
...prev,
|
||||
completeClear: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 10000);
|
||||
};
|
||||
|
||||
const handleImageCleanupPreview = async () => {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
preview: { loading: true, message: 'Scanning for orphaned images...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await configApi.previewImageCleanup();
|
||||
|
||||
if (result.success) {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
preview: {
|
||||
loading: false,
|
||||
message: `Found ${result.orphanedCount} orphaned images (${result.formattedSize}) and ${result.foldersToDelete} empty folders. Referenced images: ${result.referencedImagesCount}`,
|
||||
success: true,
|
||||
data: result
|
||||
}
|
||||
}));
|
||||
} else {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
preview: {
|
||||
loading: false,
|
||||
message: result.error || 'Preview failed',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
}
|
||||
} catch (error: any) {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
preview: {
|
||||
loading: false,
|
||||
message: error.message || 'Network error occurred',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 10 seconds
|
||||
setTimeout(() => {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
preview: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 10000);
|
||||
};
|
||||
|
||||
const handleImageCleanupExecute = async () => {
|
||||
if (!cleanupStatus.preview.data || cleanupStatus.preview.data.orphanedCount === 0) {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
execute: {
|
||||
loading: false,
|
||||
message: 'Please run preview first to see what will be deleted',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
return;
|
||||
}
|
||||
|
||||
const confirmed = window.confirm(
|
||||
`Are you sure you want to delete ${cleanupStatus.preview.data.orphanedCount} orphaned images (${cleanupStatus.preview.data.formattedSize})? This action cannot be undone!`
|
||||
);
|
||||
|
||||
if (!confirmed) return;
|
||||
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
execute: { loading: true, message: 'Deleting orphaned images...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await configApi.executeImageCleanup();
|
||||
|
||||
if (result.success) {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
execute: {
|
||||
loading: false,
|
||||
message: `Successfully deleted ${result.deletedCount} orphaned images (${result.formattedSize}) and ${result.foldersDeleted} empty folders`,
|
||||
success: true
|
||||
},
|
||||
preview: { loading: false, message: '', success: undefined, data: undefined } // Clear preview after successful cleanup
|
||||
}));
|
||||
} else {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
execute: {
|
||||
loading: false,
|
||||
message: result.error || 'Cleanup failed',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
}
|
||||
} catch (error: any) {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
execute: {
|
||||
loading: false,
|
||||
message: error.message || 'Network error occurred',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
// Clear message after 10 seconds
|
||||
setTimeout(() => {
|
||||
setCleanupStatus(prev => ({
|
||||
...prev,
|
||||
execute: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 10000);
|
||||
};
|
||||
|
||||
// Search Engine Management Functions
|
||||
const loadSearchEngineStatus = async () => {
|
||||
try {
|
||||
const status = await searchAdminApi.getStatus();
|
||||
setSearchEngineStatus(prev => ({
|
||||
...prev,
|
||||
currentEngine: status.primaryEngine,
|
||||
openSearchAvailable: status.openSearchAvailable,
|
||||
}));
|
||||
} catch (error: any) {
|
||||
console.error('Failed to load search engine status:', error);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
|
||||
const handleOpenSearchReindex = async () => {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
reindex: { loading: true, message: 'Reindexing OpenSearch...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await searchAdminApi.reindexOpenSearch();
|
||||
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
reindex: {
|
||||
loading: false,
|
||||
message: result.success ? result.message : (result.error || 'Reindex failed'),
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
|
||||
setTimeout(() => {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
reindex: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 8000);
|
||||
} catch (error: any) {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
reindex: {
|
||||
loading: false,
|
||||
message: error.message || 'Network error occurred',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
|
||||
setTimeout(() => {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
reindex: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 8000);
|
||||
}
|
||||
};
|
||||
|
||||
const handleOpenSearchRecreate = async () => {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
recreate: { loading: true, message: 'Recreating OpenSearch indices...', success: undefined }
|
||||
}));
|
||||
|
||||
try {
|
||||
const result = await searchAdminApi.recreateOpenSearchIndices();
|
||||
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
recreate: {
|
||||
loading: false,
|
||||
message: result.success ? result.message : (result.error || 'Recreation failed'),
|
||||
success: result.success
|
||||
}
|
||||
}));
|
||||
|
||||
setTimeout(() => {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
recreate: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 8000);
|
||||
} catch (error: any) {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
recreate: {
|
||||
loading: false,
|
||||
message: error.message || 'Network error occurred',
|
||||
success: false
|
||||
}
|
||||
}));
|
||||
|
||||
setTimeout(() => {
|
||||
setOpenSearchStatus(prev => ({
|
||||
...prev,
|
||||
recreate: { loading: false, message: '', success: undefined }
|
||||
}));
|
||||
}, 8000);
|
||||
}
|
||||
};
|
||||
|
||||
// Load status on component mount
|
||||
useEffect(() => {
|
||||
loadSearchEngineStatus();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Search Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Search Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Manage OpenSearch indices for stories and authors. Use these tools if search isn't returning expected results.
|
||||
</p>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Current Status */}
|
||||
<div className="border theme-border rounded-lg p-4">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">Search Status</h3>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 text-sm">
|
||||
<div className="flex justify-between">
|
||||
<span>OpenSearch:</span>
|
||||
<span className={`font-medium ${searchEngineStatus.openSearchAvailable ? 'text-green-600 dark:text-green-400' : 'text-red-600 dark:text-red-400'}`}>
|
||||
{searchEngineStatus.openSearchAvailable ? 'Available' : 'Unavailable'}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Search Operations */}
|
||||
<div className="border theme-border rounded-lg p-4">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">Search Operations</h3>
|
||||
<p className="text-sm theme-text mb-4">
|
||||
Perform maintenance operations on search indices. Use these if search isn't returning expected results.
|
||||
</p>
|
||||
|
||||
<div className="flex flex-col sm:flex-row gap-3 mb-4">
|
||||
<Button
|
||||
onClick={handleOpenSearchReindex}
|
||||
disabled={openSearchStatus.reindex.loading || openSearchStatus.recreate.loading || !searchEngineStatus.openSearchAvailable}
|
||||
loading={openSearchStatus.reindex.loading}
|
||||
variant="ghost"
|
||||
className="flex-1"
|
||||
>
|
||||
{openSearchStatus.reindex.loading ? 'Reindexing...' : '🔄 Reindex All'}
|
||||
</Button>
|
||||
<Button
|
||||
onClick={handleOpenSearchRecreate}
|
||||
disabled={openSearchStatus.reindex.loading || openSearchStatus.recreate.loading || !searchEngineStatus.openSearchAvailable}
|
||||
loading={openSearchStatus.recreate.loading}
|
||||
variant="secondary"
|
||||
className="flex-1"
|
||||
>
|
||||
{openSearchStatus.recreate.loading ? 'Recreating...' : '🏗️ Recreate Indices'}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Status Messages */}
|
||||
{openSearchStatus.reindex.message && (
|
||||
<div className={`text-sm p-3 rounded mb-3 ${
|
||||
openSearchStatus.reindex.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{openSearchStatus.reindex.message}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{openSearchStatus.recreate.message && (
|
||||
<div className={`text-sm p-3 rounded mb-3 ${
|
||||
openSearchStatus.recreate.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{openSearchStatus.recreate.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="text-sm theme-text bg-blue-50 dark:bg-blue-900/20 p-3 rounded-lg">
|
||||
<p className="font-medium mb-1">When to use these tools:</p>
|
||||
<ul className="text-xs space-y-1 ml-4">
|
||||
<li>• <strong>Reindex All:</strong> Refresh all search data while keeping existing schemas (fixes data sync issues)</li>
|
||||
<li>• <strong>Recreate Indices:</strong> Delete and rebuild all search indexes from scratch (fixes schema and structure issues)</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Storage Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Storage Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Clean up orphaned content images that are no longer referenced in any story. This can help free up disk space.
|
||||
</p>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Image Cleanup Section */}
|
||||
<div className="border theme-border rounded-lg p-4">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">🖼️ Content Images Cleanup</h3>
|
||||
<p className="text-sm theme-text mb-4">
|
||||
Scan for and remove orphaned content images that are no longer referenced in any story content. This includes images from deleted stories and unused downloaded images.
|
||||
</p>
|
||||
|
||||
<div className="flex flex-col sm:flex-row gap-3 mb-3">
|
||||
<Button
|
||||
onClick={handleImageCleanupPreview}
|
||||
disabled={cleanupStatus.preview.loading}
|
||||
loading={cleanupStatus.preview.loading}
|
||||
variant="ghost"
|
||||
className="flex-1"
|
||||
>
|
||||
{cleanupStatus.preview.loading ? 'Scanning...' : 'Preview Cleanup'}
|
||||
</Button>
|
||||
<Button
|
||||
onClick={handleImageCleanupExecute}
|
||||
disabled={cleanupStatus.execute.loading || !cleanupStatus.preview.data || cleanupStatus.preview.data.orphanedCount === 0}
|
||||
loading={cleanupStatus.execute.loading}
|
||||
variant="secondary"
|
||||
className="flex-1"
|
||||
>
|
||||
{cleanupStatus.execute.loading ? 'Cleaning...' : 'Execute Cleanup'}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Preview Results */}
|
||||
{cleanupStatus.preview.message && (
|
||||
<div className={`text-sm p-3 rounded mb-3 ${
|
||||
cleanupStatus.preview.success
|
||||
? 'bg-blue-50 dark:bg-blue-900/20 text-blue-800 dark:text-blue-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{cleanupStatus.preview.message}
|
||||
{cleanupStatus.preview.data && cleanupStatus.preview.data.hasErrors && (
|
||||
<div className="mt-2 text-xs">
|
||||
<details>
|
||||
<summary className="cursor-pointer font-medium">View Errors ({cleanupStatus.preview.data.errors.length})</summary>
|
||||
<ul className="mt-1 ml-4 space-y-1">
|
||||
{cleanupStatus.preview.data.errors.map((error: string, index: number) => (
|
||||
<li key={index} className="text-red-600 dark:text-red-400">• {error}</li>
|
||||
))}
|
||||
</ul>
|
||||
</details>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Execute Results */}
|
||||
{cleanupStatus.execute.message && (
|
||||
<div className={`text-sm p-3 rounded mb-3 ${
|
||||
cleanupStatus.execute.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{cleanupStatus.execute.message}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Detailed Preview Information */}
|
||||
{cleanupStatus.preview.data && cleanupStatus.preview.success && (
|
||||
<div className="text-sm theme-text bg-gray-50 dark:bg-gray-800 p-3 rounded border">
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<span className="font-medium">Orphaned Images:</span> {cleanupStatus.preview.data.orphanedCount}
|
||||
</div>
|
||||
<div>
|
||||
<span className="font-medium">Total Size:</span> {cleanupStatus.preview.data.formattedSize}
|
||||
</div>
|
||||
<div>
|
||||
<span className="font-medium">Empty Folders:</span> {cleanupStatus.preview.data.foldersToDelete}
|
||||
</div>
|
||||
<div>
|
||||
<span className="font-medium">Referenced Images:</span> {cleanupStatus.preview.data.referencedImagesCount}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="text-sm theme-text bg-blue-50 dark:bg-blue-900/20 p-3 rounded-lg">
|
||||
<p className="font-medium mb-1">📝 How it works:</p>
|
||||
<ul className="text-xs space-y-1 ml-4">
|
||||
<li>• <strong>Preview:</strong> Scans all stories to find images no longer referenced in content</li>
|
||||
<li>• <strong>Execute:</strong> Permanently deletes orphaned images and empty story directories</li>
|
||||
<li>• <strong>Safe:</strong> Only removes images not found in any story content</li>
|
||||
<li>• <strong>Backup recommended:</strong> Consider backing up before large cleanups</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Database Management */}
|
||||
<div className="theme-card theme-shadow rounded-lg p-6">
|
||||
<h2 className="text-xl font-semibold theme-header mb-4">Database Management</h2>
|
||||
<p className="theme-text mb-6">
|
||||
Backup, restore, or clear your StoryCove database and files. These comprehensive operations include both your data and uploaded images.
|
||||
</p>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Complete Backup Section */}
|
||||
<div className="border theme-border rounded-lg p-4 border-blue-200 dark:border-blue-800">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">📦 Create Backup</h3>
|
||||
<p className="text-sm theme-text mb-3">
|
||||
Download a complete backup as a ZIP file. This includes your database AND all uploaded files (cover images, avatars). This is a comprehensive backup of your entire StoryCove installation.
|
||||
</p>
|
||||
<Button
|
||||
onClick={handleCompleteBackup}
|
||||
disabled={databaseStatus.completeBackup.loading}
|
||||
loading={databaseStatus.completeBackup.loading}
|
||||
variant="primary"
|
||||
className="w-full sm:w-auto"
|
||||
>
|
||||
{databaseStatus.completeBackup.loading ? 'Creating Backup...' : 'Download Backup'}
|
||||
</Button>
|
||||
{databaseStatus.completeBackup.message && (
|
||||
<div className={`text-sm p-2 rounded mt-3 ${
|
||||
databaseStatus.completeBackup.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{databaseStatus.completeBackup.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Restore Section */}
|
||||
<div className="border theme-border rounded-lg p-4 border-orange-200 dark:border-orange-800">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">📥 Restore Backup</h3>
|
||||
<p className="text-sm theme-text mb-3">
|
||||
<strong className="text-orange-600 dark:text-orange-400">⚠️ Warning:</strong> This will completely replace your current database AND all files with the backup. All existing data and uploaded files will be permanently deleted.
|
||||
</p>
|
||||
<div className="flex items-center gap-3">
|
||||
<input
|
||||
type="file"
|
||||
accept=".zip"
|
||||
onChange={handleCompleteRestore}
|
||||
disabled={databaseStatus.completeRestore.loading}
|
||||
className="flex-1 text-sm theme-text file:mr-4 file:py-2 file:px-4 file:rounded-lg file:border-0 file:text-sm file:font-medium file:theme-accent-bg file:text-white hover:file:bg-opacity-90 file:cursor-pointer"
|
||||
/>
|
||||
</div>
|
||||
{databaseStatus.completeRestore.message && (
|
||||
<div className={`text-sm p-2 rounded mt-3 ${
|
||||
databaseStatus.completeRestore.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{databaseStatus.completeRestore.message}
|
||||
</div>
|
||||
)}
|
||||
{databaseStatus.completeRestore.loading && (
|
||||
<div className="text-sm theme-text mt-3 flex items-center gap-2">
|
||||
<div className="animate-spin w-4 h-4 border-2 border-blue-500 border-t-transparent rounded-full"></div>
|
||||
Restoring backup...
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Clear Everything Section */}
|
||||
<div className="border theme-border rounded-lg p-4 border-red-200 dark:border-red-800 bg-red-50 dark:bg-red-900/10">
|
||||
<h3 className="text-lg font-semibold theme-header mb-3">🗑️ Clear Everything</h3>
|
||||
<p className="text-sm theme-text mb-3">
|
||||
<strong className="text-red-600 dark:text-red-400">⚠️ Danger Zone:</strong> This will permanently delete ALL data from your database AND all uploaded files (cover images, avatars). Everything will be completely removed. This action cannot be undone!
|
||||
</p>
|
||||
<Button
|
||||
onClick={handleCompleteClear}
|
||||
disabled={databaseStatus.completeClear.loading}
|
||||
loading={databaseStatus.completeClear.loading}
|
||||
variant="secondary"
|
||||
className="w-full sm:w-auto bg-red-700 hover:bg-red-800 text-white border-red-700"
|
||||
>
|
||||
{databaseStatus.completeClear.loading ? 'Clearing Everything...' : 'Clear Everything'}
|
||||
</Button>
|
||||
{databaseStatus.completeClear.message && (
|
||||
<div className={`text-sm p-2 rounded mt-3 ${
|
||||
databaseStatus.completeClear.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200'
|
||||
}`}>
|
||||
{databaseStatus.completeClear.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="text-sm theme-text bg-blue-50 dark:bg-blue-900/20 p-3 rounded-lg">
|
||||
<p className="font-medium mb-1">💡 Best Practices:</p>
|
||||
<ul className="text-xs space-y-1 ml-4">
|
||||
<li>• <strong>Always backup</strong> before performing restore or clear operations</li>
|
||||
<li>• <strong>Store backups safely</strong> in multiple locations for important data</li>
|
||||
<li>• <strong>Test restores</strong> in a development environment when possible</li>
|
||||
<li>• <strong>Backup files (.zip)</strong> contain both database and all uploaded files</li>
|
||||
<li>• <strong>Verify backup files</strong> are complete before relying on them</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -28,6 +28,23 @@ export default function StoryCard({
|
||||
const [rating, setRating] = useState(story.rating || 0);
|
||||
const [updating, setUpdating] = useState(false);
|
||||
|
||||
// Helper function to get tags from either tags array or tagNames array
|
||||
const getTags = () => {
|
||||
if (Array.isArray(story.tags) && story.tags.length > 0) {
|
||||
return story.tags;
|
||||
}
|
||||
if (Array.isArray(story.tagNames) && story.tagNames.length > 0) {
|
||||
// Convert tagNames to Tag objects for display compatibility
|
||||
return story.tagNames.map((name, index) => ({
|
||||
id: `tag-${index}`, // Temporary ID for display
|
||||
name: name
|
||||
}));
|
||||
}
|
||||
return [];
|
||||
};
|
||||
|
||||
const displayTags = getTags();
|
||||
|
||||
const handleRatingClick = async (e: React.MouseEvent, newRating: number) => {
|
||||
// Prevent default and stop propagation to avoid triggering navigation
|
||||
e.preventDefault();
|
||||
@@ -55,6 +72,17 @@ export default function StoryCard({
|
||||
return new Date(dateString).toLocaleDateString();
|
||||
};
|
||||
|
||||
const calculateReadingPercentage = (story: Story): number => {
|
||||
if (!story.readingPosition) return 0;
|
||||
|
||||
const totalLength = story.contentPlain?.length || story.contentHtml?.length || 0;
|
||||
if (totalLength === 0) return 0;
|
||||
|
||||
return Math.round((story.readingPosition / totalLength) * 100);
|
||||
};
|
||||
|
||||
const readingPercentage = calculateReadingPercentage(story);
|
||||
|
||||
if (viewMode === 'list') {
|
||||
return (
|
||||
<div className="theme-card theme-shadow rounded-lg p-4 hover:shadow-lg transition-shadow">
|
||||
@@ -100,6 +128,11 @@ export default function StoryCard({
|
||||
<div className="flex items-center gap-4 mt-2 text-sm theme-text">
|
||||
<span>{formatWordCount(story.wordCount)}</span>
|
||||
<span>{formatDate(story.createdAt)}</span>
|
||||
{readingPercentage > 0 && (
|
||||
<span className="bg-blue-100 dark:bg-blue-900 text-blue-700 dark:text-blue-300 px-2 py-1 rounded text-xs font-mono">
|
||||
{readingPercentage}% read
|
||||
</span>
|
||||
)}
|
||||
{story.seriesName && (
|
||||
<span>
|
||||
{story.seriesName} #{story.volume}
|
||||
@@ -108,9 +141,9 @@ export default function StoryCard({
|
||||
</div>
|
||||
|
||||
{/* Tags */}
|
||||
{Array.isArray(story.tags) && story.tags.length > 0 && (
|
||||
{displayTags.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1 mt-2">
|
||||
{story.tags.slice(0, 3).map((tag) => (
|
||||
{displayTags.slice(0, 3).map((tag) => (
|
||||
<TagDisplay
|
||||
key={tag.id}
|
||||
tag={tag}
|
||||
@@ -118,9 +151,9 @@ export default function StoryCard({
|
||||
clickable={false}
|
||||
/>
|
||||
))}
|
||||
{story.tags.length > 3 && (
|
||||
{displayTags.length > 3 && (
|
||||
<span className="px-2 py-1 text-xs theme-text">
|
||||
+{story.tags.length - 3} more
|
||||
+{displayTags.length - 3} more
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
@@ -231,6 +264,11 @@ export default function StoryCard({
|
||||
<div className="text-xs theme-text space-y-1">
|
||||
<div>{formatWordCount(story.wordCount)}</div>
|
||||
<div>{formatDate(story.createdAt)}</div>
|
||||
{readingPercentage > 0 && (
|
||||
<div className="bg-blue-100 dark:bg-blue-900 text-blue-700 dark:text-blue-300 px-2 py-1 rounded font-mono inline-block">
|
||||
{readingPercentage}% read
|
||||
</div>
|
||||
)}
|
||||
{story.seriesName && (
|
||||
<div>
|
||||
{story.seriesName} #{story.volume}
|
||||
@@ -239,9 +277,9 @@ export default function StoryCard({
|
||||
</div>
|
||||
|
||||
{/* Tags */}
|
||||
{Array.isArray(story.tags) && story.tags.length > 0 && (
|
||||
{displayTags.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1 mt-2">
|
||||
{story.tags.slice(0, 2).map((tag) => (
|
||||
{displayTags.slice(0, 2).map((tag) => (
|
||||
<TagDisplay
|
||||
key={tag.id}
|
||||
tag={tag}
|
||||
@@ -249,9 +287,9 @@ export default function StoryCard({
|
||||
clickable={false}
|
||||
/>
|
||||
))}
|
||||
{story.tags.length > 2 && (
|
||||
{displayTags.length > 2 && (
|
||||
<span className="px-2 py-1 text-xs theme-text">
|
||||
+{story.tags.length - 2}
|
||||
+{displayTags.length - 2}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
44
frontend/src/components/ui/TabNavigation.tsx
Normal file
44
frontend/src/components/ui/TabNavigation.tsx
Normal file
@@ -0,0 +1,44 @@
|
||||
'use client';
|
||||
|
||||
interface Tab {
|
||||
id: string;
|
||||
label: string;
|
||||
icon: string;
|
||||
}
|
||||
|
||||
interface TabNavigationProps {
|
||||
tabs: Tab[];
|
||||
activeTab: string;
|
||||
onTabChange: (tabId: string) => void;
|
||||
className?: string;
|
||||
}
|
||||
|
||||
export default function TabNavigation({
|
||||
tabs,
|
||||
activeTab,
|
||||
onTabChange,
|
||||
className = ''
|
||||
}: TabNavigationProps) {
|
||||
return (
|
||||
<div className={`border-b theme-border ${className}`}>
|
||||
<nav className="-mb-px flex space-x-8">
|
||||
{tabs.map((tab) => (
|
||||
<button
|
||||
key={tab.id}
|
||||
onClick={() => onTabChange(tab.id)}
|
||||
className={`
|
||||
whitespace-nowrap py-2 px-1 border-b-2 font-medium text-sm transition-colors
|
||||
${activeTab === tab.id
|
||||
? 'border-blue-500 text-blue-600 dark:text-blue-400'
|
||||
: 'border-transparent theme-text hover:text-gray-700 dark:hover:text-gray-300 hover:border-gray-300 dark:hover:border-gray-600'
|
||||
}
|
||||
`}
|
||||
>
|
||||
<span className="mr-2">{tab.icon}</span>
|
||||
{tab.label}
|
||||
</button>
|
||||
))}
|
||||
</nav>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -179,15 +179,6 @@ export const storyApi = {
|
||||
return response.data;
|
||||
},
|
||||
|
||||
reindexTypesense: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
|
||||
const response = await api.post('/stories/reindex-typesense');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
recreateTypesenseCollection: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
|
||||
const response = await api.post('/stories/recreate-typesense-collection');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
checkDuplicate: async (title: string, authorName: string): Promise<{
|
||||
hasDuplicates: boolean;
|
||||
@@ -305,38 +296,6 @@ export const authorApi = {
|
||||
await api.delete(`/authors/${id}/avatar`);
|
||||
},
|
||||
|
||||
searchAuthorsTypesense: async (params?: {
|
||||
q?: string;
|
||||
page?: number;
|
||||
size?: number;
|
||||
sortBy?: string;
|
||||
sortOrder?: string;
|
||||
}): Promise<{
|
||||
results: Author[];
|
||||
totalHits: number;
|
||||
page: number;
|
||||
perPage: number;
|
||||
query: string;
|
||||
searchTimeMs: number;
|
||||
}> => {
|
||||
const response = await api.get('/authors/search-typesense', { params });
|
||||
return response.data;
|
||||
},
|
||||
|
||||
reindexTypesense: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
|
||||
const response = await api.post('/authors/reindex-typesense');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
recreateTypesenseCollection: async (): Promise<{ success: boolean; message: string; count?: number; error?: string }> => {
|
||||
const response = await api.post('/authors/recreate-typesense-collection');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
getTypesenseSchema: async (): Promise<{ success: boolean; schema?: any; error?: string }> => {
|
||||
const response = await api.get('/authors/typesense-schema');
|
||||
return response.data;
|
||||
},
|
||||
};
|
||||
|
||||
// Tag endpoints
|
||||
@@ -577,6 +536,106 @@ export const configApi = {
|
||||
const response = await api.get('/config/html-sanitization');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
previewImageCleanup: async (): Promise<{
|
||||
success: boolean;
|
||||
orphanedCount: number;
|
||||
totalSizeBytes: number;
|
||||
formattedSize: string;
|
||||
foldersToDelete: number;
|
||||
referencedImagesCount: number;
|
||||
errors: string[];
|
||||
hasErrors: boolean;
|
||||
dryRun: boolean;
|
||||
error?: string;
|
||||
}> => {
|
||||
const response = await api.post('/config/cleanup/images/preview');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
executeImageCleanup: async (): Promise<{
|
||||
success: boolean;
|
||||
deletedCount: number;
|
||||
totalSizeBytes: number;
|
||||
formattedSize: string;
|
||||
foldersDeleted: number;
|
||||
referencedImagesCount: number;
|
||||
errors: string[];
|
||||
hasErrors: boolean;
|
||||
dryRun: boolean;
|
||||
error?: string;
|
||||
}> => {
|
||||
const response = await api.post('/config/cleanup/images/execute');
|
||||
return response.data;
|
||||
},
|
||||
};
|
||||
|
||||
// Search Engine Management API
|
||||
export const searchAdminApi = {
|
||||
// Get migration status
|
||||
getStatus: async (): Promise<{
|
||||
primaryEngine: string;
|
||||
dualWrite: boolean;
|
||||
openSearchAvailable: boolean;
|
||||
}> => {
|
||||
const response = await api.get('/admin/search/status');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
// Configure search engine
|
||||
configure: async (config: { engine: string; dualWrite: boolean }): Promise<{ message: string }> => {
|
||||
const response = await api.post('/admin/search/configure', config);
|
||||
return response.data;
|
||||
},
|
||||
|
||||
// Enable/disable dual-write
|
||||
enableDualWrite: async (): Promise<{ message: string }> => {
|
||||
const response = await api.post('/admin/search/dual-write/enable');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
disableDualWrite: async (): Promise<{ message: string }> => {
|
||||
const response = await api.post('/admin/search/dual-write/disable');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
// Switch engines
|
||||
switchToOpenSearch: async (): Promise<{ message: string }> => {
|
||||
const response = await api.post('/admin/search/switch/opensearch');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
|
||||
// Emergency rollback
|
||||
emergencyRollback: async (): Promise<{ message: string }> => {
|
||||
const response = await api.post('/admin/search/emergency-rollback');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
// OpenSearch operations
|
||||
reindexOpenSearch: async (): Promise<{
|
||||
success: boolean;
|
||||
message: string;
|
||||
storiesCount?: number;
|
||||
authorsCount?: number;
|
||||
totalCount?: number;
|
||||
error?: string;
|
||||
}> => {
|
||||
const response = await api.post('/admin/search/opensearch/reindex');
|
||||
return response.data;
|
||||
},
|
||||
|
||||
recreateOpenSearchIndices: async (): Promise<{
|
||||
success: boolean;
|
||||
message: string;
|
||||
storiesCount?: number;
|
||||
authorsCount?: number;
|
||||
totalCount?: number;
|
||||
error?: string;
|
||||
}> => {
|
||||
const response = await api.post('/admin/search/opensearch/recreate');
|
||||
return response.data;
|
||||
},
|
||||
};
|
||||
|
||||
// Collection endpoints
|
||||
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user