storycove/storycove-collections-spec.md

# StoryCove - Story Collections Feature Specification

> **✅ Implementation Status: COMPLETED**
> This feature has been fully implemented and is available in the system.
> Last updated: January 2025

## 1. Feature Overview

Story Collections allow users to organize stories into ordered lists for better content management and reading workflows. Collections support custom ordering, metadata, and provide an enhanced reading experience for grouped content.

### 1.1 Core Capabilities
- Create and manage ordered lists of stories
- Stories can belong to multiple collections
- Drag-and-drop reordering
- Collection-level metadata and ratings
- Dedicated collection reading flow
- Batch operations on collection contents

## 2. Data Model Updates

### 2.1 New Database Tables

#### Collections Table
```sql
CREATE TABLE collections (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(500) NOT NULL,
    description TEXT,
    rating INTEGER CHECK (rating >= 1 AND rating <= 5),
    cover_image_path VARCHAR(500),
    is_archived BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_collections_archived ON collections(is_archived);
```

#### Collection Stories Junction Table
```sql
CREATE TABLE collection_stories (
    collection_id UUID NOT NULL,
    story_id UUID NOT NULL,
    position INTEGER NOT NULL,
    added_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (collection_id, story_id),
    FOREIGN KEY (collection_id) REFERENCES collections(id) ON DELETE CASCADE,
    FOREIGN KEY (story_id) REFERENCES stories(id) ON DELETE CASCADE,
    UNIQUE(collection_id, position)
);

CREATE INDEX idx_collection_stories_position ON collection_stories(collection_id, position);
```

#### Collection Tags Junction Table
```sql
CREATE TABLE collection_tags (
    collection_id UUID NOT NULL,
    tag_id UUID NOT NULL,
    PRIMARY KEY (collection_id, tag_id),
    FOREIGN KEY (collection_id) REFERENCES collections(id) ON DELETE CASCADE,
    FOREIGN KEY (tag_id) REFERENCES tags(id) ON DELETE CASCADE
);
```

### 2.2 Typesense Schema Update

Add new collection schema:
```json
{
  "name": "collections",
  "fields": [
    {"name": "id", "type": "string"},
    {"name": "name", "type": "string"},
    {"name": "description", "type": "string", "optional": true},
    {"name": "tags", "type": "string[]", "optional": true},
    {"name": "story_count", "type": "int32"},
    {"name": "total_word_count", "type": "int32"},
    {"name": "rating", "type": "int32", "optional": true},
    {"name": "is_archived", "type": "bool"},
    {"name": "created_at", "type": "int64"},
    {"name": "updated_at", "type": "int64"}
  ],
  "default_sorting_field": "updated_at"
}
```

## 3. API Specification

### 3.1 Collection Endpoints

#### GET /api/collections
**IMPORTANT**: This endpoint MUST use Typesense for all search and filtering operations.
Do NOT implement search/filter logic using JPA/SQL queries.

Query parameters:
- `page` (integer): Page number
- `limit` (integer): Items per page
- `search` (string): Search in name and description (via Typesense)
- `tags` (string[]): Filter by tags (via Typesense)
- `archived` (boolean): Include archived collections (via Typesense filter)

Implementation note:
```java
// CORRECT: Use Typesense
return typesenseService.searchCollections(search, tags, archived, page, limit);

// INCORRECT: Do not use repository queries like this
// return collectionRepository.findByNameContainingAndTags(...);
```

Response includes:
- Collection metadata
- Story count
- Average story rating
- Total word count
- Estimated reading time

#### POST /api/collections
```json
Request (multipart/form-data):
{
  "name": "string",
  "description": "string",
  "tags": ["string"],
  "storyIds": ["uuid"], // Optional initial stories
  "coverImage": "file (optional)"
}

Response:
{
  "id": "uuid",
  "name": "string",
  "description": "string",
  "tags": ["string"],
  "storyCount": 0,
  "averageStoryRating": null,
  "rating": null,
  "createdAt": "timestamp"
}
```

#### GET /api/collections/{id}
Returns full collection details with ordered story list

Response:
```json
{
  "id": "uuid",
  "name": "string",
  "description": "string",
  "tags": ["string"],
  "rating": 1-5,
  "coverImagePath": "string",
  "storyCount": "integer",
  "totalWordCount": "integer",
  "estimatedReadingTime": "integer (minutes)",
  "averageStoryRating": "float",
  "stories": [
    {
      "id": "uuid",
      "title": "string",
      "author": "string",
      "wordCount": "integer",
      "rating": 1-5,
      "coverImagePath": "string",
      "position": "integer"
    }
  ],
  "createdAt": "timestamp",
  "updatedAt": "timestamp"
}
```

#### PUT /api/collections/{id}
Update collection metadata (same structure as POST without storyIds)

#### DELETE /api/collections/{id}
Delete a collection (stories remain in the system)

#### PUT /api/collections/{id}/archive
Archive/unarchive a collection
```json
Request:
{
  "archived": boolean
}
```

### 3.2 Collection Story Management

#### POST /api/collections/{id}/stories
Add stories to collection
```json
Request:
{
  "storyIds": ["uuid"],
  "position": "integer" // Optional, defaults to end
}

Response:
{
  "added": 3,
  "skipped": 1, // Already in collection
  "totalStories": 15
}
```

#### PUT /api/collections/{id}/stories/order
Reorder stories in collection
```json
Request:
{
  "storyOrders": [
    {"storyId": "uuid", "position": 1},
    {"storyId": "uuid", "position": 2}
  ]
}
```

#### DELETE /api/collections/{id}/stories/{storyId}
Remove a story from collection

#### GET /api/collections/{id}/read/{storyId}
Get story content with collection navigation context
```json
Response:
{
  "story": { /* full story data */ },
  "collection": {
    "id": "uuid",
    "name": "string",
    "currentPosition": 3,
    "totalStories": 10,
    "previousStoryId": "uuid",
    "nextStoryId": "uuid"
  }
}
```

### 3.3 Collection EPUB Export

#### GET /api/collections/{id}/epub
Export collection as EPUB file with default settings
- Includes all stories in collection order
- Includes collection metadata as book metadata
- Includes cover image if available
- Generates table of contents

#### POST /api/collections/{id}/epub
Export collection as EPUB with custom options
```json
Request:
{
  "includeCoverImage": true,
  "includeMetadata": true,
  "includeTableOfContents": true
}

Response: EPUB file download with filename format:
{collection-name}-{export-date}.epub
```

### 3.4 Collection Statistics

#### GET /api/collections/{id}/stats
Get detailed collection statistics
```json
Response:
{
  "totalStories": 15,
  "totalWordCount": 125000,
  "estimatedReadingTime": 625,
  "averageStoryRating": 4.2,
  "tagFrequency": {
    "fantasy": 12,
    "adventure": 8
  },
  "authorDistribution": [
    {"authorName": "string", "storyCount": 5}
  ],
  "readingProgress": {
    "storiesRead": 8,
    "percentComplete": 53.3
  }
}
```

### 3.5 Batch Operations

#### POST /api/stories/batch/add-to-collection
Add multiple stories to a collection
```json
Request:
{
  "storyIds": ["uuid"],
  "collectionId": "uuid" // null to create new
  "newCollectionName": "string" // if creating new
}
```

### 3.4 Collection Statistics

#### GET /api/collections/{id}/stats
Detailed statistics for a collection
```json
Response:
{
  "totalStories": 15,
  "totalWordCount": 125000,
  "estimatedReadingTime": 625, // minutes
  "averageStoryRating": 4.2,
  "averageWordCount": 8333,
  "tagFrequency": {
    "fantasy": 12,
    "adventure": 8
  },
  "authorDistribution": [
    {"authorName": "string", "storyCount": 5}
  ]
}
```

## 4. UI/UX Specifications

### 4.1 Navigation Updates
Add "Collections" to the main navigation menu, same level as "Stories" and "Authors"

### 4.2 Collections Overview Page
- Grid/List view toggle
- Collection cards showing:
  - Cover image (or first 4 story covers as mosaic)
  - Name and description preview
  - Story count and total reading time
  - Rating stars
  - Tag badges
- **Pagination controls** (page size selector, page navigation)
- Filter by tags
- Search collections
- "Create New Collection" button
- Archive toggle

**IMPORTANT**: This view MUST use pagination via Typesense. Do NOT load all collections at once.
Default page size: 20 collections per page (configurable: 10, 20, 50)

### 4.3 Collection Creation/Edit Modal
- Name input (required)
- Description textarea
- Tag input with autocomplete
- Cover image upload
- Initial story selection (optional):
  - Search and filter stories
  - Checkbox selection
  - Selected stories preview

### 4.4 Collection Detail View
- Header section:
  - Cover image or story mosaic
  - Collection name (editable inline)
  - Description (editable inline)
  - Statistics bar: X stories • Y hours reading time • Average rating
  - Action buttons: Read Collection, Edit, Export (Phase 2), Archive, Delete
- Story list section:
  - Drag-and-drop reordering (drag handle on each row)
  - Story cards with position number
  - Remove from collection button
  - Add stories button
- Rating section:
  - Collection rating (manual)
  - Average story rating (calculated)

### 4.5 Story List View Updates
- Multi-select mode:
  - Checkbox appears on hover/in mobile
  - Selection toolbar with "Add to Collection" button
  - Create new or add to existing collection

### 4.6 Story Detail View Updates
- "Add to Collection" button in the action bar
- "Part of Collections" section showing which collections include this story

### 4.7 Reading View Updates
When reading from a collection:
- Collection name and progress (Story 3 of 15) in header
- Previous/Next navigation uses collection order
- "Back to Collection" button
- Progress bar showing position in collection

### 4.8 Responsive Design Considerations
- Mobile: Single column layout, bottom sheet for actions
- Tablet: Two-column layout for collection detail
- Desktop: Full drag-and-drop, hover states

## 5. Technical Implementation Details

### 5.1 Frontend Updates

#### State Management
```typescript
// Collection context for managing active collection during reading
interface CollectionReadingContext {
  collectionId: string;
  currentPosition: number;
  totalStories: number;
  stories: StoryPreview[];
}

// Drag and drop using @dnd-kit/sortable
interface DragEndEvent {
  active: { id: string };
  over: { id: string };
}
```

#### Components Structure
```
components/
  collections/
    CollectionCard.tsx
    CollectionGrid.tsx
    CollectionForm.tsx
    CollectionDetailView.tsx
    StoryReorderList.tsx
    AddToCollectionModal.tsx
  stories/
    StoryMultiSelect.tsx
    StorySelectionToolbar.tsx
```

#### Pagination Implementation
```typescript
// Collections overview MUST use pagination
interface CollectionsPageState {
  page: number;
  pageSize: number; // 10, 20, or 50
  totalPages: number;
  totalCollections: number;
}

// Fetch collections with pagination via Typesense
const fetchCollections = async (page: number, pageSize: number, filters: any) => {
  // MUST use Typesense API with pagination params
  const response = await api.get('/api/collections', {
    params: {
      page,
      limit: pageSize,
      ...filters
    }
  });
  return response.data;
};

// Component must handle:
// - Page navigation (previous/next, direct page input)
// - Page size selection
// - Maintaining filters/search across page changes
// - URL state sync for shareable/bookmarkable pages
```

### 5.2 Backend Updates

#### Service Layer
```java
@Service
public class CollectionService {
    @Autowired
    private TypesenseService typesenseService;
    
    // IMPORTANT: All search and filtering MUST use Typesense, not JPA queries
    public Page<Collection> searchCollections(String query, List<String> tags, boolean includeArchived) {
        // Use typesenseService.searchCollections() - NOT repository queries
        return typesenseService.searchCollections(query, tags, includeArchived);
    }
    
    // Calculate statistics dynamically
    public CollectionStats calculateStats(UUID collectionId);
    
    // Validate no duplicate stories
    public void validateStoryAddition(UUID collectionId, List<UUID> storyIds);
    
    // Reorder with gap-based positioning
    public void reorderStories(UUID collectionId, List<StoryOrder> newOrder);
    
    // Cascade position updates on removal
    public void removeStoryAndReorder(UUID collectionId, UUID storyId);
}
```

#### Search Implementation Requirements
```java
// CRITICAL: All collection search operations MUST go through Typesense
// DO NOT implement search/filter logic in JPA repositories

@Service
public class TypesenseService {
    // Existing methods for stories...
    
    // New methods for collections
    public SearchResult<Collection> searchCollections(
        String query, 
        List<String> tags,
        boolean includeArchived,
        int page, 
        int limit
    ) {
        // Build Typesense query
        // Handle text search, tag filtering, archive status
        // Return paginated results
    }
    
    public void indexCollection(Collection collection) {
        // Index/update collection in Typesense
        // Include calculated fields like story_count, total_word_count
    }
    
    public void removeCollection(UUID collectionId) {
        // Remove from Typesense index
    }
}

// Repository should ONLY be used for:
// - CRUD operations by ID
// - Relationship management
// - Position updates
// NOT for search, filtering, or listing with criteria
```

#### Position Management Strategy
Use gap-based positioning (multiples of 1000) to minimize reorder updates:
- Initial positions: 1000, 2000, 3000...
- Insert between: (prev + next) / 2
- Rebalance when gaps get too small

### 5.3 Performance Optimizations

1. **Lazy Loading**: Load collection stories in batches
2. **Caching**: Cache collection statistics for 5 minutes
3. **Batch Operations**: Multi-story operations in single transaction
4. **Optimistic Updates**: Immediate UI updates for reordering

### 5.4 Critical Implementation Guidelines

#### Search and Filtering Architecture
**MANDATORY**: All search, filtering, and listing operations MUST use Typesense as the primary data source.

```java
// Controller pattern for ALL list/search endpoints
@GetMapping("/api/collections")
public ResponseEntity<Page<CollectionDTO>> getCollections(
    @RequestParam(required = false) String search,
    @RequestParam(required = false) List<String> tags,
    @RequestParam(defaultValue = "false") boolean archived,
    Pageable pageable
) {
    // MUST delegate to Typesense service
    return ResponseEntity.ok(
        typesenseService.searchCollections(search, tags, archived, pageable)
    );
}

// Service layer MUST use Typesense
// NEVER implement search logic in repositories
// Repository pattern should ONLY be used for:
// 1. Direct ID lookups
// 2. Saving/updating entities
// 3. Managing relationships
// 4. NOT for searching, filtering, or conditional queries
```

#### Synchronization Strategy
1. On every collection create/update/delete → immediately sync with Typesense
2. Include denormalized data in Typesense documents (story count, word count, etc.)
3. Use database only as source of truth for relationships and detailed data
4. Use Typesense for all discovery operations (search, filter, list)

## 6. Migration and Upgrade Path

### 6.1 Database Migration
```sql
-- Run migrations in order
-- 1. Create new tables
-- 2. Add indexes
-- 3. No data migration needed (new feature)
```

### 6.2 Search Index Update
- Deploy new Typesense schema
- No reindexing needed for existing stories

## 7. Testing Requirements

### 7.1 Unit Tests
- Collection CRUD operations
- Position management logic
- Statistics calculation
- Duplicate prevention

### 7.2 Integration Tests
- Multi-story batch operations
- Drag-and-drop reordering
- Collection reading flow
- Search and filtering

### 7.3 E2E Tests
- Create collection from multiple entry points
- Complete reading flow through collection
- Reorder and verify persistence

## 8. Future Enhancements (Phase 2+)

1. **Collection Templates**: Pre-configured collection types
2. **Smart Collections**: Auto-populate based on criteria
3. **Collection Sharing**: Generate shareable links
4. **Reading Progress**: Track progress through collections
5. **Export Collections**: PDF/EPUB with proper ordering
6. **Collection Recommendations**: Based on reading patterns
7. **Nested Collections**: Collections within collections
8. **Collection Permissions**: For multi-user scenarios

## 9. Implementation Checklist

### Backend Tasks
- [ ] Create database migrations
- [ ] Implement entity models
- [ ] Create repository interfaces
- [ ] Implement service layer with business logic
- [ ] Create REST controllers
- [ ] Add validation and error handling
- [ ] Update Typesense sync logic
- [ ] Write unit and integration tests

### Frontend Tasks
- [ ] Update navigation structure
- [ ] Create collection components
- [ ] Implement drag-and-drop reordering
- [ ] Add multi-select to story list
- [ ] Update story detail view
- [ ] Implement collection reading flow
- [ ] Add loading and error states
- [ ] Write component tests

### Documentation Tasks
- [ ] Update API documentation
- [ ] Create user guide for collections
- [ ] Document position management strategy
- [ ] Add collection examples to README

## 10. Acceptance Criteria

1. Users can create, edit, and delete collections
2. Stories can be added/removed from collections without duplication
3. Collection order persists across sessions
4. Drag-and-drop reordering works smoothly
5. Collection statistics update in real-time
6. Reading flow respects collection order
7. **Search and filtering work for collections using Typesense (NOT database queries)**
8. All actions are validated and provide clear feedback
9. Performance remains smooth with large collections (100+ stories)
10. Mobile experience is fully functional

## 11. Implementation Anti-Patterns to Avoid

### CRITICAL: Search Implementation
The following patterns MUST NOT be used for search/filter operations:

```java
// ❌ WRONG - Do not use JPA/Repository for search
@Repository
public interface CollectionRepository extends JpaRepository<Collection, UUID> {
    // DO NOT ADD THESE METHODS:
    List<Collection> findByNameContaining(String name);
    List<Collection> findByTagsIn(List<String> tags);
    Page<Collection> findByNameContainingAndArchived(String name, boolean archived, Pageable pageable);
}

// ❌ WRONG - Do not implement search in service using repositories
public Page<Collection> searchCollections(String query) {
    return collectionRepository.findByNameContaining(query);
}

// ✅ CORRECT - Always use Typesense for search/filter
public Page<Collection> searchCollections(String query, List<String> tags) {
    SearchResult result = typesenseClient.collections("collections")
        .documents()
        .search(searchParameters);
    return convertToPage(result);
}
```

### Remember:
1. **Typesense** = Search, filter, list, discover (with pagination)
2. **Database** = Store, retrieve by ID, manage relationships
3. **Never** mix search logic between the two systems
4. **Always** paginate list views - never load all items at once