Rework data model and add migrations

2026-03-15 00:39:23 +01:00
parent 9232ac6133
commit c30ad57051
14 changed files with 744 additions and 30 deletions
--- a/docs/development-plan.md
+++ b/docs/development-plan.md
@@ -0,0 +1,213 @@
+# HXBooks Development Plan & Progress
+
+## User's Priorities (March 2026)
+1. ✅ **Fix the domain and data model** 
+2. ✅ **Make sure everything related to the database is good** 
+3. 🚧 **Make a CLI so I can test things manually** (In Progress)
+4. **Make sure search and other basic functionality is good and can be accessed through CLI**
+5. **Set up automated tests**
+6. **Fully rework the GUI**
+
+*Everything else will come later.*
+
+---
+
+## ✅ COMPLETED: Domain Model & Database (Phase 1-2)
+
+### Domain Model Decisions Made
+- **No book/instance separation**: Keep it simple, treat duplicate editions as separate books
+- **Author/Genre relationships**: Proper many-to-many instead of JSON fields
+- **Location hierarchy**: `location_place` + `location_bookshelf` + `location_shelf` (numeric)
+- **Auto-complete approach**: Authors/genres created on-demand with nice UI later
+- **Multiple readings**: Separate records per reading session
+- **Simple loaning**: `loaned_to` string + `loaned_date` for tracking
+
+### Database Infrastructure ✅ DONE
+- ✅ Flask-Migrate + Alembic set up
+- ✅ Initial migration created and applied
+- ✅ Fixed instance folder location (project root instead of src/instance)
+- ✅ Database in correct location: `/hxbooks.sqlite`
+- ✅ All tables created: author, genre, book, book_author, book_genre, reading, wishlist
+
+### New Data Model ✅ IMPLEMENTED
+```sql
+-- Core entities
+Author(id, name)
+Genre(id, name)  
+Book(id, title, description, isbn, edition, publisher, notes, 
+     added_date, bought_date,
+     location_place, location_bookshelf, location_shelf,
+     loaned_to, loaned_date, owner_id)
+
+-- Many-to-many relationships  
+BookAuthor(book_id, author_id)
+BookGenre(book_id, genre_id)
+
+-- User activity
+Reading(id, user_id, book_id, start_date, end_date, 
+        finished, dropped, rating, comments)
+Wishlist(id, user_id, book_id, wishlisted_date)
+```
+
+---
+
+## 🚧 IN PROGRESS: CLI Development (Phase 3)
+
+### CLI Requirements for Manual Testing
+- [ ] Book CRUD operations (add, edit, delete, list)
+- [ ] Author/Genre management (auto-create, list)
+- [ ] Location management (place, bookshelf, shelf)
+- [ ] Reading tracking (start, finish, rate)
+- [ ] Search functionality testing
+- [ ] Data import from old format
+- [ ] Loaning operations
+
+### CLI Commands Planned
+```bash
+hx book add "Title" --authors "Author1,Author2" --genres "Fiction"
+hx book list --location "my house" --shelf 2
+hx book search "keyword"
+hx reading start <book_id>
+hx reading finish <book_id> --rating 4
+hx loan <book_id> --to "Alice" 
+```
+
+---
+
+## 📋 TODO: Remaining Phases
+
+### Phase 4: Search & Core Features
+- [ ] Implement proper FTS with new schema
+- [ ] Add faceted search (by author, genre, location)
+- [ ] Create search result serializers
+- [ ] Add pagination
+- [ ] Optimize query performance with proper indexes
+
+### Phase 5: Testing Framework
+- [ ] Set up pytest with database fixtures
+- [ ] API endpoint tests
+- [ ] Search functionality tests  
+- [ ] CLI command tests
+- [ ] Migration tests
+
+### Phase 6: GUI Rework
+- [ ] Update templates for new data model
+- [ ] Mobile-first responsive design
+- [ ] Author/Genre autocomplete interfaces
+- [ ] Location hierarchy picker
+- [ ] Touch-optimized interactions
+
+---
+
+## 🗄️ Original Critique Archive
+
+### Critical Issues RESOLVED ✅
+- ❌ **Book ownership model**: Fixed - no artificial scarcity
+- ❌ **JSON denormalization**: Fixed - proper Author/Genre relationships  
+- ❌ **Mixed properties**: Fixed - structured location hierarchy
+- ❌ **No migrations**: Fixed - Alembic set up and working
+- ❌ **Poor folder structure**: Fixed - database in project root
+
+### Issues for Later Phases
+- **Authentication**: Username-only insufficient (Phase 6+)
+- **Configuration management**: No environment handling (Phase 6+) 
+- **Mobile UX**: Tables don't work on mobile (Phase 6)
+- **Testing infrastructure**: No framework yet (Phase 5)
+- **Error handling**: No proper boundaries (Phase 6+)
+- **Performance**: No indexing strategy yet (Phase 4)
+
+---
+
+*Last updated: March 14, 2026*
+*Status: Phase 1-2 Complete ✅ | Phase 3 In Progress 🚧*
+
+### Medium Priority Issues (Priority 3-4: CLI & Search)
+
+#### Search & Discovery
+- **Limited FTS capabilities**: Current implementation incomplete
+- **No faceted search**: Missing filters by author, genre, year, etc.
+- **Performance concerns**: JSON contains operations will be slow
+- **Missing recommendations**: No "similar books" functionality
+
+#### CLI Requirements for Testing
+- Book CRUD operations
+- Search functionality testing
+- User management
+- Reading tracking
+- Data import/export capabilities
+
+### Lower Priority Issues (Priority 5-6: Testing & GUI)
+
+#### Testing Infrastructure Missing
+- No testing framework configured
+- No test database setup
+- No fixtures or mock data
+- No CI/CD pipeline
+
+#### GUI/UX Issues
+- Mobile responsiveness needs work
+- No offline capabilities  
+- Tables don't work well on mobile
+- Missing accessibility features
+
+### Security & DevOps (Future)
+- **Authentication**: Username-only is insufficient
+- **Configuration management**: No environment handling
+- **Deployment**: Dockerfile/requirements.txt mismatch
+- **Secret management**: Hardcoded dev secrets
+
+### Technical Debt
+- **Python 3.14 requirement**: Too aggressive (doesn't exist yet)
+- **Error handling**: No proper error boundaries
+- **Logging**: No production logging configuration
+- **Code quality**: Missing linting, formatting tools
+
+---
+
+## Implementation Notes
+
+### Phase 1: Domain Model Rework
+- [ ] Design new schema with proper relationships
+- [ ] Create migration system
+- [ ] Implement Author and Genre entities
+- [ ] Separate Book from BookInstance
+- [ ] Update all models with proper typing
+
+### Phase 2: Database Infrastructure  
+- [ ] Set up Alembic migrations
+- [ ] Add proper indexing
+- [ ] Implement FTS correctly
+- [ ] Add database constraints
+- [ ] Create seed data
+
+### Phase 3: CLI Development
+- [ ] Create Click-based CLI
+- [ ] Book management commands
+- [ ] Search functionality
+- [ ] User operations
+- [ ] Import/export tools
+
+### Phase 4: Search & Core Features
+- [ ] Implement proper FTS
+- [ ] Add faceted search
+- [ ] Create search result serializers
+- [ ] Add pagination
+- [ ] Optimize query performance
+
+### Phase 5: Testing Framework
+- [ ] Set up pytest  
+- [ ] Database testing fixtures
+- [ ] API endpoint tests
+- [ ] Search functionality tests
+- [ ] CLI command tests
+
+### Phase 6: GUI Rework
+- [ ] Mobile-first responsive design
+- [ ] Progressive enhancement
+- [ ] Accessibility improvements
+- [ ] Modern HTMX patterns
+- [ ] Touch-optimized interactions
+
+---
+
+*Last updated: March 14, 2026*