Compare commits

..

185 Commits

Author SHA1 Message Date
e26219989f feat: add job runs chart and scrollable reading lists on dashboard
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m5s
- Add multi-line chart showing job runs over time by type (scan,
  rebuild, thumbnails, other) with the same day/week/month toggle
- Limit currently reading and recently read lists to 3 visible items
  with a scrollbar for overflow
- Fix NUMERIC→BIGINT cast for SUM/COALESCE in jobs SQL queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 10:43:45 +01:00
5d33a35407 chore: bump version to 1.27.0 2026-03-22 10:43:25 +01:00
d53572dc33 chore: bump version to 1.26.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m49s
2026-03-22 10:27:59 +01:00
cf1953d11f feat: add day/week/month period toggle for dashboard line charts
Add a period selector (day, week, month) to the reading activity and
books added charts. The API now accepts a ?period= query param and
returns gap-filled data using generate_series so all time slots appear
even with zero values. Labels are locale-aware (short month, weekday).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 10:27:24 +01:00
6f663eaee7 docs: add MIT license
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 10:08:15 +01:00
ee65c6263a perf: add ETag and server-side caching for thumbnail proxy
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 49s
Add ETag header to API thumbnail responses for 304 Not Modified support.
Forward If-None-Match/ETag through the Next.js proxy route handler and
add next.revalidate for 24h server-side fetch caching to reduce
SSR-to-API round trips on the libraries page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 06:52:47 +01:00
691b6b22ab chore: bump version to 1.25.0 2026-03-22 06:52:02 +01:00
11c80a16a3 docs: add Telegram notifications and updated dashboard to README and FEATURES
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 46s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 06:40:34 +01:00
c366b44c54 chore: bump version to 1.24.1 2026-03-22 06:39:23 +01:00
92f80542e6 perf: skip Next.js image re-optimization and stream proxy responses
Thumbnails are already optimized (WebP) by the API, so disable Next.js
image optimization to avoid redundant CPU work. Switch route handlers
from buffering (arrayBuffer) to streaming (response.body) to reduce
memory usage and latency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 06:38:46 +01:00
3a25e42a20 chore: bump version to 1.24.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m7s
2026-03-22 06:31:56 +01:00
24763bf5a7 fix: show absolute date/time in jobs "created" column
Replace relative time formatting (which incorrectly showed "just now"
for many jobs due to negative time diffs from server/client timezone
mismatch) with absolute locale-formatted date/time.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 06:31:37 +01:00
08f0397029 feat: add reading stats and replace dashboard charts with recharts
Add currently reading, recently read, and reading activity sections to
the dashboard. Replace all custom SVG/CSS charts with recharts library
(donut, area, stacked bar, horizontal bar). Reorganize layout: libraries
and popular series side by side, books added chart full width below.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 06:26:45 +01:00
766e3a01b2 chore: bump version to 1.23.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 45s
2026-03-21 17:43:11 +01:00
626e2e035d feat: send book thumbnails in Telegram notifications
Use Telegram sendPhoto API for conversion and metadata-approved events
when a book thumbnail is available on disk. Falls back to text message
if photo upload fails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 17:43:01 +01:00
cfd2321db2 chore: bump version to 1.22.0 2026-03-21 17:40:22 +01:00
1b715033ce fix: add missing Next.js route handler for Telegram test endpoint
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 17:39:46 +01:00
81d1586501 feat: add Telegram notification system with granular event toggles
Add notifications crate shared between API and indexer to send Telegram
messages on scan/thumbnail/conversion completion/failure, metadata linking,
batch and refresh events. Configurable via a new Notifications tab in the
backoffice settings with per-event toggle switches grouped by category.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 17:24:43 +01:00
bd74c9e3e3 docs: add comprehensive features list to README and docs/FEATURES.md
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m1s
Replace the minimal README features section with a concise categorized
summary and link to a detailed docs/FEATURES.md covering all features,
business rules, API endpoints, and integrations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 14:34:36 +01:00
41228430cf chore: bump version to 1.21.2 2026-03-21 14:34:32 +01:00
6a4ba06fac fix: include series_metadata authors in authors listing and detail pages
Authors were only sourced from books.authors/books.author fields which are
often empty. Now also aggregates authors from series_metadata.authors
(populated by metadata providers like bedetheque). Adds author filter to
/series endpoint and updates the author detail page to use it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 14:34:11 +01:00
e5c3542d3f refactor: split books.rs into books+series, reorganize OpenAPI tags and fix access control
- Extract series code from books.rs into dedicated series.rs module
- Reorganize OpenAPI tags: split overloaded "books" tag into books, series, search, stats
- Add missing endpoints to OpenAPI: metadata_batch, metadata_refresh, komga, update_metadata_provider
- Add missing schemas: MissingVolumeInput, Komga/Batch/Refresh DTOs
- Fix access control: move GET /libraries and POST /libraries/:id/scan to read routes
  so non-admin tokens can list libraries and trigger scans

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 14:23:19 +01:00
24516f1069 chore: bump version to 1.21.1
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 41s
2026-03-21 13:42:17 +01:00
5383cdef60 feat: allow batch metadata and refresh metadata on all libraries
When no specific library is selected, iterate over all libraries and
trigger a job for each one, skipping libraries with metadata disabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:42:08 +01:00
be5c3f7a34 fix: pass explicit locale to date formatting to prevent hydration mismatch
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 41s
Server and client could use different default locales for
toLocaleDateString/toLocaleString, causing React hydration errors.
Pass the user locale explicitly in JobsList and SettingsPage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:36:35 +01:00
caa9922ff9 chore: bump version to 1.21.0 2026-03-21 13:34:47 +01:00
135f000c71 refactor: switch JobsIndicator from polling to SSE and fix stream endpoint
Replace fetch polling in JobsIndicator with EventSource connected to
/api/jobs/stream. Fix the SSE route to return all jobs (via
/index/status) instead of only active ones, since JobsList also
consumes this stream for the full job history. JobsIndicator now
filters active jobs client-side. SSE server-side uses adaptive
interval (2s active, 15s idle) and only sends when data changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:33:58 +01:00
d9e50a4235 chore: bump version to 1.20.1
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m13s
2026-03-21 13:13:39 +01:00
5f6eb5a5cb perf: add selective fetch caching for stable API endpoints
Make apiFetch support Next.js revalidate option instead of
hardcoding cache: no-store on every request. Stable endpoints
(libraries, settings, stats, series statuses) now use time-based
revalidation while dynamic data (books, search, jobs) stays uncached.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:13:28 +01:00
41c77fca2e chore: bump version to 1.20.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m15s
2026-03-21 13:06:28 +01:00
49621f3fb1 perf: wrap BookCard and BookImage with React.memo
Prevent unnecessary re-renders of book grid items when parent
components update without changing book data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:03:24 +01:00
6df743b2e6 perf: lazy-load heavy modal components with next/dynamic
Dynamic import EditBookForm, EditSeriesForm, MetadataSearchModal, and
ProwlarrSearchModal so their code is split into separate chunks and
only fetched when the user interacts with them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 13:02:10 +01:00
edfefc0128 perf: optimize JobsIndicator polling with visibility API and adaptive interval
Pause polling when the tab is hidden, refetch immediately when it
becomes visible again, and use a 30s interval when no jobs are active
instead of polling every 2s unconditionally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 12:59:06 +01:00
b0185abefe perf: enable Next.js image optimization across backoffice
Remove `unoptimized` flag from all thumbnail/cover Image components
and add proper responsive `sizes` props. Convert raw `<img>` tags on
the libraries page to next/image. Add 24h minimumCacheTTL for
optimized images. BookPreview keeps `unoptimized` since the API
already returns optimized WebP.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 12:57:10 +01:00
b9e54cbfd8 chore: bump version to 1.19.1
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 54s
2026-03-21 12:47:31 +01:00
3f0bd783cd feat: include series_count and thumbnail_book_ids in libraries API response
Eliminates N+1 sequential fetchSeries calls on the libraries page by
returning series count and up to 5 thumbnail book IDs (one per series)
directly from GET /libraries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 12:47:10 +01:00
fc8856c83f chore: bump version to 1.19.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m19s
2026-03-21 08:12:19 +01:00
bd09f3d943 feat: persist filter state in localStorage across pages
Save/restore filter values in LiveSearchForm using localStorage keyed
by basePath (e.g. filters:/books, filters:/series). Filters are restored
on mount when the URL has no active filters, and cleared when the user
clicks the Clear button.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 08:12:10 +01:00
1f434c3d67 feat: add format and metadata filters to books page
Add two new filters to the books listing page:
- Format filter (CBZ/CBR/PDF/EPUB) using existing API support
- Metadata linked/unlinked filter with new API support via
  LEFT JOIN on external_metadata_links (using DISTINCT ON CTE
  matching the series endpoint pattern)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 08:09:37 +01:00
4972a403df chore: bump version to 1.18.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m7s
2026-03-21 07:47:52 +01:00
629708cdd0 feat: redesign libraries page UI with fan thumbnails and modal settings
- Replace thumbnail mosaic with fan/arc layout using series covers as background
- Move library settings from dropdown to full-page portal modal with sections
- Move FolderPicker modal to portal for proper z-index stacking
- Add descriptions to each setting for better clarity
- Move delete button to card header, compact config tags
- Add i18n keys for new labels and descriptions (en/fr)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 07:47:36 +01:00
560087a897 chore: bump version to 1.17.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m12s
2026-03-21 07:23:52 +01:00
27f553b005 feat: add rescan job type and improve full rebuild UX
Add "Deep rescan" job type that clears directory mtimes to force
re-walking all directories, discovering newly supported formats (e.g.
EPUB) without deleting existing data or metadata.

Also improve full rebuild button: red destructive styling instead of
warning, and FR description explicitly mentions metadata/reading status
loss. Rename FR rebuild label to "Mise à jour".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 07:23:38 +01:00
ed7665248e chore: bump version to 1.16.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 1m5s
2026-03-21 07:06:28 +01:00
736b8aedc0 feat: add EPUB format support with spine-aware image extraction
Parse EPUB structure (container.xml → OPF → spine → XHTML) to extract
images in reading order. Zero new dependencies — reuses zip + regex
crates with pre-compiled regexes and per-file index cache for
performance. Falls back to CBZ-style image listing when spine contains
no images. Includes DB migration, API/indexer/backoffice updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 07:05:47 +01:00
3daa49ae6c feat: add live refresh to job detail page via SSE
The job detail page was only server-rendered with no live updates,
unlike the jobs list page. Add a lightweight JobDetailLive client
component that subscribes to the existing SSE endpoint and calls
router.refresh() on each update, keeping the page in sync while
a job is running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 06:52:57 +01:00
5fb24188e1 chore: bump version to 1.15.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 44s
2026-03-20 13:35:36 +01:00
54f972db17 chore: bump version to 1.14.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 45s
2026-03-20 12:48:14 +01:00
acd8b62382 chore: bump version to 1.13.0 2026-03-20 12:44:54 +01:00
cc65e3d1ad feat: highlight missing volumes in Prowlarr search results
API extracts volume numbers from release titles and matches them against
missing volumes sent by the frontend. Matched results are highlighted in
green with badges indicating which missing volumes were found.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 12:44:35 +01:00
70889ca955 chore: bump version to 1.12.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 43s
2026-03-20 11:43:34 +01:00
4ad6d57271 feat: add authors page to backoffice with dedicated API endpoint
Add a new GET /authors endpoint that aggregates unique authors from books
with book/series counts, pagination and search. Add author filter to
GET /books. Backoffice gets a list page with search/sort and a detail
page showing the author's series and books.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 11:43:22 +01:00
fe5de3d5c1 feat: add scheduled metadata refresh for libraries
Add metadata_refresh_mode (manual/hourly/daily/weekly) to libraries,
with automatic scheduling via the indexer. Includes API support,
backoffice UI controls, i18n translations, and DB migration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 10:51:52 +01:00
5a224c48c0 chore: bump version to 1.11.1
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 44s
2026-03-20 10:46:34 +01:00
d08fe31b1b fix: pass metadata_refresh_mode through backoffice proxy to API
The Next.js monitoring route was dropping metadata_refresh_mode from the
request body, so the value was never forwarded to the Rust API and
reverted on reload.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 10:46:22 +01:00
4d69ed91c5 chore: bump version to 1.11.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 56s
2026-03-20 09:46:29 +01:00
c6ddd3e6c7 chore: bump version to 1.10.1
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 49s
2026-03-19 22:33:52 +01:00
504185f31f feat: add editable search input to Prowlarr modal with scrollable badges
- Add text input for custom search queries in Prowlarr modal
- Quick search badges pre-fill the input and trigger search
- Default query uses quoted series name for exact match
- Add custom_query support to backend API
- Limit badge area height with vertical scroll
- Add debug logging for Prowlarr API responses

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 22:33:40 +01:00
acd0cce3f8 fix: reorder Prowlarr button, add collection progress bar, remove redundant missing badge
- Move Prowlarr search button before Metadata button
- Add amber collection progress bar showing owned/expected books ratio
- Remove yellow missing count badge from MetadataSearchModal (now shown in progress bar)
- Fix i18n plural parameter for series read count

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 22:17:49 +01:00
e14da4fc8d chore: bump version to 1.10.0
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 51s
2026-03-19 21:51:45 +01:00
c04d4fb618 feat: add qBittorrent download client integration
Send Prowlarr search results directly to qBittorrent from the modal.
Backend authenticates via SID cookie (login + add torrent endpoints).

- Backend: qbittorrent module with add and test endpoints
- Migration: add qbittorrent settings (url, username, password)
- Settings UI: qBittorrent config card with test connection
- ProwlarrSearchModal: send-to-qBittorrent button per result row
  with spinner/checkmark state progression
- Button only shown when qBittorrent is configured

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 21:51:28 +01:00
57bc82703d feat: add Prowlarr integration for manual release search
Add Prowlarr indexer integration (step 1: config + manual search).
Allows searching for comics/ebooks releases on Prowlarr indexers
directly from the series detail page, with download links and
per-volume search for missing books.

- Backend: new prowlarr module with search and test endpoints
- Migration: add prowlarr settings (url, api_key, categories)
- Settings UI: Prowlarr config card with test connection button
- ProwlarrSearchModal: auto-search on open, missing volumes shortcuts
- Fix series.readCount i18n plural parameter on series pages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 21:43:34 +01:00
e6aa7ebed0 chore: bump version to 1.9.2
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 44s
2026-03-19 13:22:41 +01:00
c44b51d6ef fix: unmap status mappings instead of deleting, store unmapped provider statuses
- Make mapped_status nullable so unmapping (X button) sets NULL instead of
  deleting the row — provider statuses never disappear from the UI
- normalize_series_status now returns the raw provider status (lowercased)
  when no mapping exists, so all statuses are stored in series_metadata
- Fix series_statuses query crash caused by NULL mapped_status values
- Fix metadata batch/refresh server actions crashing page on 400 errors
- StatusMappingDto.mapped_status is now string | null in the backoffice

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 13:22:31 +01:00
d4c48de780 chore: bump version to 1.9.1 2026-03-19 12:59:31 +01:00
8948f75d62 fix: ignore unknown provider statuses instead of storing them
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 5s
normalize_series_status now returns None when no mapping exists,
so unknown provider statuses won't pollute series_metadata.status.
Users can see unmapped statuses in Settings and assign them before
they get stored.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:58:55 +01:00
d304877a83 fix: re-normalize series statuses with UI-added mappings
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 6s
Migration 0041 re-applies status normalization using all current
status_mappings entries, including those added via the UI after the
initial migration 0039 (e.g. "one shot" → "ended").

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:57:14 +01:00
9cec32ba3e fix: normalize series status casing to avoid duplicates
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 6s
- LOWER() all series_metadata.status values in the statuses endpoint
  to prevent "One shot" / "one shot" appearing as separate targets
- Migration 0040: lowercase all existing status values in DB
- Use LOWER() in series status filter queries for consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:56:02 +01:00
e8768dfad7 chore: bump version to 1.9.0 2026-03-19 12:44:30 +01:00
cfc98819ab feat: add configurable status mappings for metadata providers
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 6s
Add a status_mappings table to replace hardcoded provider status
normalization. Users can now configure how provider statuses (e.g.
"releasing", "finie") map to target statuses (e.g. "ongoing", "ended")
via the Settings > Integrations page.

- Migration 0038: status_mappings table with pre-seeded mappings
- Migration 0039: re-normalize existing series_metadata.status values
- API: CRUD endpoints for status mappings, DB-based normalize function
- API: new GET /series/provider-statuses endpoint
- Backoffice: StatusMappingsCard component with create target, assign,
  and delete capabilities
- Fix all clippy warnings across the API crate
- Fix missing OpenAPI schema refs (MetadataStats, ProviderCount)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:44:22 +01:00
bfc1c76fe2 chore: repair deploy
All checks were successful
Deploy with Docker Compose / deploy (push) Successful in 32s
2026-03-19 11:19:50 +01:00
39e9f35acb chore: push deploy stack local with dockerhub images
Some checks failed
Deploy with Docker Compose / deploy (push) Failing after 8s
2026-03-19 11:16:29 +01:00
36987f59b9 chore: bump version to 1.8.1 2026-03-19 11:12:06 +01:00
931d0e06f4 feat: redesign search bars with prominent search input and compact filters
Restructure LiveSearchForm: full-width search input with magnifying glass
icon, filters in a compact row below with contextual icons per field
(library, status, sort, etc.) and inline labels. Remove per-field
className overrides from series and books pages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 11:12:00 +01:00
741a4da878 feat: redesign jobs page action bar with grouped layout
Replace flat button row + separate reference card with a single card
organized in 3 visual groups (Indexation, Thumbnails, Metadata).
Each action is a card-like button with inline description.
Destructive actions have distinct warning styling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 11:03:08 +01:00
e28b78d0e6 chore: bump version to 1.8.0 2026-03-19 09:09:27 +01:00
163dc3698c feat: add metadata refresh job to re-download metadata for linked series
Adds a new job type that refreshes metadata from external providers for
all series already linked via approved external_metadata_links. Tracks
and displays per-field diffs (series and book level), respects locked
fields, and provides a detailed change report in the job detail page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 09:09:10 +01:00
818bd82e0f chore: bump version to 1.7.0 2026-03-18 22:26:15 +01:00
76c8bcbf2c chore: bump version to 1.6.5 2026-03-18 22:20:02 +01:00
00094b22c6 feat: add metadata statistics to dashboard
Add a new metadata row to the dashboard with three cards:
- Series metadata coverage (linked vs unlinked donut)
- Provider breakdown (donut by provider)
- Book metadata quality (summary and ISBN fill rates)

Includes API changes (stats.rs), frontend types, and FR/EN translations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 22:19:53 +01:00
1e4d9acebe fix: normalize French articles in Bedetheque confidence scoring
Bedetheque uses "Légendaires (Les) - Résistance" while local series
names are "Les légendaires - Résistance". Add normalize_title() that
strips leading articles and articles in parentheses before comparing,
so these forms correctly produce a 100% confidence match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 22:04:58 +01:00
b226aa3a35 chore: bump version to 1.6.4 2026-03-18 22:04:40 +01:00
d913be9d2a chore: bump version to 1.6.3 2026-03-18 21:44:37 +01:00
e9bb951d97 feat: auto-match metadata when 100% confidence and matching book count
When multiple provider results exist but the best has 100% confidence,
compare local book count with external total_volumes. If they match,
treat it as an auto-match and link+sync series and book metadata
automatically instead of requiring manual review.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 21:44:31 +01:00
037ede2750 chore: bump version to 1.6.2 2026-03-18 21:36:44 +01:00
06a245d90a feat: add metadata provider filter to series page
- Add `metadata_provider` query param to series API endpoints (linked/unlinked/specific provider)
- Return `metadata_provider` field in series response
- Add metadata filter dropdown on series page with all provider options
- Show small provider icon badge on linked series cards
- LiveSearchForm now wraps filters on two rows when needed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 21:35:38 +01:00
63d5fcaa13 chore: bump version to 1.6.1 2026-03-18 21:19:38 +01:00
020cb6baae feat: make series names clickable in batch metadata report
Links series names to their detail page where the metadata search
modal can be triggered for quick provider lookup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 21:07:21 +01:00
6db8042ffe chore: bump version to 1.6.0 2026-03-18 19:39:10 +01:00
d4f87c4044 feat: add i18n support (FR/EN) to backoffice with English as default
Implement full internationalization for the Next.js backoffice:
- i18n infrastructure: type-safe dictionaries (fr.ts/en.ts), cookie-based locale detection, React Context for client components, server-side translation helper
- Language selector in Settings page (General tab) with cookie + DB persistence
- All ~35 pages and components translated via t() / useTranslation()
- Default locale set to English, French available via settings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 19:39:01 +01:00
055c376222 fix: complete French translation for settings page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 18:26:59 +01:00
1cc5d049ea chore: bump version to 1.5.6 2026-03-18 18:26:50 +01:00
b955c2697c feat: add batch metadata jobs, series filters, and translate backoffice to French
- Add metadata_batch job type with background processing via tokio::spawn
- Auto-apply metadata only when single result at 100% confidence
- Support primary + fallback provider per library, "none" to opt out
- Add batch report/results API endpoints and job detail UI
- Add series_status and has_missing filters to both series listing pages
- Add GET /series/statuses endpoint for dynamic filter options
- Normalize series_metadata status values (migration 0036)
- Hide ComicVine provider tab when no API key configured
- Translate entire backoffice UI from English to French

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 18:26:44 +01:00
9a8c1577af chore: bump version to 1.5.5 2026-03-18 16:11:08 +01:00
52b9b0e00e feat: add series status, improve providers & e2e tests
- Add series status concept (ongoing/ended/hiatus/cancelled/upcoming)
  with normalization across all providers
- Add status field to series_metadata table (migration 0033)
- AniList: use chapters as fallback for volume count on ongoing series,
  add books_message when both volumes and chapters are null
- Bedetheque: extract description from meta tag, genres, parution status,
  origin/language; rewrite book parsing with itemprop microdata for
  clean ISBN, dates, page counts, covers; filter placeholder authors
- Add comprehensive e2e provider tests with field coverage reporting
- Wire status into EditSeriesForm, MetadataSearchModal, and series page

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 16:10:45 +01:00
51ef2fa725 chore: bump version to 1.5.4 2026-03-18 15:27:29 +01:00
7d53babc84 chore: bump version to 1.5.3 2026-03-18 15:23:54 +01:00
00f4445924 fix: use sort-order position as fallback volume for book matching
When books have no volume number, use their 1-based position in the
backoffice sort order (volume ASC NULLS LAST, natural title sort) as
effective volume for matching against external provider books.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 15:21:32 +01:00
1a91c051b5 chore: bump version to 1.5.2 2026-03-18 15:16:21 +01:00
48ca9d0a8b chore: bump version to 1.5.1 2026-03-18 15:12:44 +01:00
f75d795215 fix: improve book matching in metadata sync with bidirectional title search
Pre-fetch all local books in one query instead of N queries per external
book. Match by volume number first, then bidirectional title containment
(external in local OR local in external). Track matched IDs to prevent
double-matching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 15:12:36 +01:00
ac13f53124 chore: bump version to 1.5.0 2026-03-18 15:03:49 +01:00
c9ccf5cd90 feat: add external metadata sync system with multiple providers
Add a complete metadata synchronization system allowing users to search
and sync series/book metadata from external providers (Google Books,
Open Library, ComicVine, AniList, Bédéthèque). Each library can use a
different provider. Matching requires manual approval with detailed sync
reports showing what was updated or skipped (locked fields protection).

Key changes:
- DB migrations: external_metadata_links, external_book_metadata tables,
  library metadata_provider column, locked_fields, total_volumes, book
  metadata fields (summary, isbn, publish_date)
- Rust API: MetadataProvider trait + 5 provider implementations,
  7 metadata endpoints (search, match, approve, reject, links, missing,
  delete), sync report system, provider language preference support
- Backoffice: MetadataSearchModal, ProviderIcon, SafeHtml components,
  settings UI for provider/language config, enriched book detail page,
  edit forms with locked fields support, API proxy routes
- OpenAPI/Swagger documentation for all new endpoints and schemas

Closes #3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 14:59:24 +01:00
a99bfb5a91 perf(docker): optimize Rust build times with dependency layer caching
Replace sccache with a two-stage build strategy: dummy source files cache
dependency compilation in a separate layer, and --mount=type=cache for
Cargo registry/git/target directories. Source-only changes now skip
full dependency recompilation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 11:06:11 +01:00
389d71b42f refactor: replace Meilisearch with PostgreSQL full-text search
Remove Meilisearch dependency entirely. Search is now handled by
PostgreSQL ILIKE with pg_trgm indexes, joining series_metadata for
series-level authors. No external search engine needed.

- Replace search.rs Meilisearch HTTP calls with PostgreSQL queries
- Remove meili.rs from indexer, sync_meili call from job pipeline
- Remove MEILI_URL/MEILI_MASTER_KEY from config, state, env files
- Remove meilisearch service from docker-compose.yml
- Add migration 0027: drop sync_metadata, enable pg_trgm, add indexes
- Remove search resync button/endpoint (no longer needed)
- Update all documentation (CLAUDE.md, README.md, AGENTS.md, PLAN.md)

API contract unchanged — same SearchResponse shape returned.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 10:59:25 +01:00
2985ef5561 chore: bump version to 1.4.0 2026-03-18 10:59:12 +01:00
4be8177683 feat: fix author search, add edit modals, settings tabs & search resync
- Fix Meilisearch indexing to use authors[] array instead of scalar author field
- Join series_metadata to include series-level authors in search documents
- Configure searchable attributes (title, authors, series) in Meilisearch
- Convert EditSeriesForm and EditBookForm from inline forms to modals
- Add tabbed navigation (General / Integrations) to Settings page
- Add Force Search Resync button (POST /settings/search/resync)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 10:45:36 +01:00
a675dcd2a4 chore: bump version to 1.3.0 2026-03-18 10:45:16 +01:00
127cd8a42c feat(komga): add Komga read-status sync with reports and history
Adds Komga sync feature to import read status from a Komga server.
Books are matched by title (case-insensitive) with series+title primary
match and title-only fallback. Sync reports are persisted with matched,
newly marked, and unmatched book lists. UI shows check icon for newly
marked books, sorted to top. Credentials (URL+username) are saved
between sessions. Uses HashSet for O(1) lookups to handle large libraries.

Closes #2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 22:04:19 +01:00
1b9f2d3915 chore: bump version to 1.2.2 2026-03-16 22:04:04 +01:00
f095bf050b chore: bump version to 1.2.1 2026-03-16 21:12:25 +01:00
b17718df9b fix(stats): count authors from both author and authors fields
The total_authors stat now combines distinct values from the legacy
author column and the new authors array column.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 21:11:58 +01:00
5c3ddf7819 chore: bump version to 1.2.0 2026-03-16 19:29:43 +01:00
c56d02a895 chore: bump version to 1.1.0 2026-03-16 19:29:24 +01:00
bc98067871 feat(books): édition des métadonnées livres et séries + champ authors multi-valeurs
- Nouveaux endpoints PATCH /books/:id et PATCH /libraries/:id/series/:name pour éditer les métadonnées
- GET /libraries/:id/series/:name/metadata pour récupérer les métadonnées de série
- Ajout du champ `authors` (Vec<String>) sur les structs Book/BookDetails
- 3 migrations : table series_metadata, colonne authors sur series_metadata et books
- Composants EditBookForm et EditSeriesForm dans le backoffice
- Routes API Next.js correspondantes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 17:21:55 +01:00
a085924f8a fix(books): correction du tri naturel des titres avec sous-titres variables
Remplace REGEXP_REPLACE(title, '[0-9]+', '', 'g') par
REGEXP_REPLACE(title, '[0-9].*$', '') pour n'extraire que le préfixe
avant le premier chiffre.

L'ancienne regex supprimait TOUS les chiffres du titre entier, y compris
dans les sous-titres. Quand chaque volume a un sous-titre différent
(ex: "Deadpool marvel deluxe 3 - Je suis ton homme"), la partie texte
restante variait → l'ordre alphabétique des sous-titres prenait le dessus.

Avec le nouveau préfixe "deadpool marvel deluxe " identique pour tous,
le numéro de volume (2ème clé) dicte correctement l'ordre.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 12:16:00 +01:00
9fbdf793d0 chore: bump version to 1.0.1 2026-03-16 12:08:15 +01:00
b14accbbe0 fix(books): tri des séries par volume + suppression de l'ancienne extract_page
- Ajout de `b.volume NULLS LAST` comme première clé de tri dans list_books
  et dans tous les ROW_NUMBER() OVER (...) des CTEs series, pour corriger
  l'ordre des volumes dont les titres varient en format (ex: "Round" vs "R")
- Suppression de l'ancienne extract_page publique et de ses 4 helpers
  (extract_cbz_page_n, extract_cbz_page_n_streaming, extract_cbr_page_n,
  extract_pdf_page_n) remplacés par la nouvelle implémentation avec cache
- Suppression de archive_index_cache dans AppState (remplacé par le cache
  statique CBZ_INDEX_CACHE dans parsers), import StdMutex nettoyé

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 12:08:03 +01:00
330239d2c3 feat(api): log info par requête HTTP (méthode, path, status, durée)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 08:09:32 +01:00
bf5a20882b perf(pages): cache de l'index d'archive en mémoire (-73% CBZ, -76% CBR cold)
Chaque cold render ré-énumérait toutes les entrées ZIP/RAR pour construire
la liste triée des images. Maintenant la liste est mise en cache dans l'AppState
(LruCache<String, Arc<Vec<String>>>, std::sync::Mutex pour accès spawn_blocking).

Nouvelles fonctions dans parsers :
- list_archive_images(path, format) -> Vec<String>
- extract_image_by_name(path, format, name) -> Vec<u8>

Mesures avant/après (cache disque froid, n=20) :
- CBZ cold : 43ms → 11.9ms (-73%)
- CBR cold : 46ms → 11.0ms (-76%)
- Warm/concurrent : identique

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 08:09:32 +01:00
44c6dd626a feat(backoffice): afficher le format (cbz/cbr/pdf) au lieu du kind sur les cards
- Ajoute `format: string | null` dans BookDto
- BookCard et page détail utilisent `book.format ?? book.kind` avec les couleurs
  success=CBZ, warning=CBR, destructive=PDF

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 08:09:26 +01:00
9153b0c750 refactor(pages): déléguer l'extraction de pages au crate parsers
- Expose `extract_page(path, format, page_number, render_width)` dans parsers
- Rend `is_image_name` publique, ajoute gif/bmp/tif/tiff
- Supprime ~250 lignes dupliquées dans pages.rs (CBZ/CBR/PDF extract)
- Retire zip/unrar/pdfium-render/natord de api, remplacé par parsers

Perf avant/après : stable (±5%, dans le bruit de mesure).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 08:09:26 +01:00
e18bbba4ce feat: add sort parameter (title/latest) to books and series endpoints
Add sort=latest option to GET /books and GET /series API endpoints,
and expose a Sort select in the backoffice books and series pages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 21:46:37 +01:00
2870dd9dbc chore: bump version to 1.0.0 2026-03-15 18:38:01 +01:00
cf2e7a0be7 feat(backoffice): add dashboard statistics with charts
Add GET /stats API endpoint with collection overview, reading status,
format/library breakdowns, top series, and monthly additions.
Replace static home page with interactive dashboard featuring donut
charts, bar charts, and progress bars. Use distinct colors for series
(warning/yellow) across nav, page titles, and quick links.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 18:37:53 +01:00
82444cda02 chore: bump version to 0.3.0 2026-03-15 18:17:27 +01:00
1d25c8869f feat(backoffice): add reading progress management, series page, and live search
- API: add POST /series/mark-read to batch mark all books in a series
- API: add GET /series cross-library endpoint with search, library and status filters
- API: add library_id to SeriesItem response
- Backoffice: mark book as read/unread button on book detail page
- Backoffice: mark series as read/unread button on series cards
- Backoffice: new /series top-level page with search and filters
- Backoffice: new /libraries/[id]/series/[name] series detail page
- Backoffice: opacity on fully read books and series cards
- Backoffice: live search with debounce on books and series pages
- Backoffice: reading status filter on books and series pages
- Fix $2 -> $1 parameter binding in mark-series-read SQL

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 18:17:16 +01:00
fd277602c9 feat(api): add GET /series/ongoing and GET /books/ongoing endpoints
Two new read routes for the home screen:
- /series/ongoing: partially read series sorted by last activity
- /books/ongoing: next unread book per ongoing series

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:24:05 +01:00
673777bc8d chore: bump version to 0.2.0 2026-03-15 16:23:09 +01:00
03af82d065 feat(tokens): allow permanent deletion of revoked tokens
Add POST /admin/tokens/{id}/delete endpoint that permanently removes
a token from the database (only if already revoked). Add delete button
in backoffice UI for revoked tokens.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 15:19:44 +01:00
78e28a269d chore: bump version to 0.1.5 2026-03-15 15:17:16 +01:00
ee05df26c4 fix(indexer): corriger OOM lors du full rebuild (batching + limite threads)
- Extraction par batches de 200 livres (libère mémoire entre chaque batch)
- Limiter tokio spawn_blocking à 8 threads (défaut 512, chaque thread ~8MB stack)
- Réduire concurrence extraction de 8 à 2 max
- Supprimer raw_bytes.clone() inutile (passage par ownership)
- Ajouter log RSS entre chaque batch pour diagnostic mémoire

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 13:34:14 +01:00
96d9efdeed chore: bump version to 0.1.4 2026-03-15 13:20:41 +01:00
9f5183848b chore: bump version to 0.1.3 2026-03-15 13:09:53 +01:00
6f9dd108ef chore: bump version to 0.1.2 2026-03-15 13:06:36 +01:00
61bc307715 perf(parsers): optimiser listing CBZ avec file_names(), ajouter magic bytes check RAR
- Remplacer by_index() par file_names() pour lister les pages ZIP (zero I/O)
- Ajouter vérification magic bytes avant fallback RAR
- Ajouter tracing debug logs dans parsers
- Script docker-push avec version bump interactif

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 13:01:04 +01:00
c7f3ad981d chore: bump version to 0.1.1 2026-03-15 12:51:54 +01:00
0d60d46cae feat(indexer,backoffice): logs par domaine, réduction fd, UI mobile
- Ajout de targets de log par domaine (scan, extraction, thumbnail, watcher)
  contrôlables via RUST_LOG pour activer/désactiver les logs granulaires
- Ajout de logs détaillés dans extracting_pages (per-book timing en debug,
  progression toutes les 25 books en info)
- Réduction de la consommation de fd: walkdir max_open(20/10), comptage
  séquentiel au lieu de par_iter parallèle, suppression de rayon
- Détection ENFILE dans le scanner: abort après 10 erreurs IO consécutives
- Backoffice: settings dans le burger mobile, masquer "backoffice" et
  icône settings en mobile

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 11:57:49 +01:00
6947af10fe perf(api,indexer): optimiser pages, thumbnails, watcher et robustesse fd
- Pages: mode Original (zero-transcoding), ETag/304, cache index CBZ,
  préfetch next 2 pages, filtre Triangle par défaut
- Thumbnails: DCT scaling JPEG via jpeg-decoder (decode 7x plus rapide),
  img.thumbnail() pour resize, support format Original, fix JPEG RGBA8
- API fallback thumbnail: OutputFormat::Original + DCT scaling au lieu
  de WebP full-decode, retour (bytes, content_type) dynamique
- Watcher: remplacement notify par poll léger sans inotify/fd,
  skip poll quand job actif, snapshots en mémoire
- Jobs: mutex exclusif corrigé (tous statuts actifs, tous types exclusifs)
- Robustesse: suppression fs::canonicalize (problèmes fd Docker),
  list_folders avec erreurs explicites, has_children default true
- Backoffice: FormRow items-start pour alignement inputs avec helper text,
  labels settings clarifiés

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 23:07:42 +01:00
fe54f55f47 feat(indexer,backoffice): ajouter warnings dans les stats de job, skip fichiers inaccessibles
- Indexer: ajout du champ `warnings` dans JobStats pour les erreurs
  non-fatales (fichiers inaccessibles, permissions)
- Indexer: skip les fichiers dont le stat échoue au lieu de faire
  crasher tout le scan de la library
- Backoffice: affichage des warnings dans le détail job (summary,
  timeline, Index Statistics) et dans la popin jobs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:44:48 +01:00
f71ca92e85 chore: corriger whitespace et paths dans .env.example
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:26:42 +01:00
7cca7e40c2 fix(parsers,api,indexer,backoffice): corriger CBZ Unicode extra fields, centraliser extraction, nettoyer Meili, fixer header
- Parsers: raw ZIP reader (flate2) contournant la validation CRC32 des
  Unicode extra fields (0x7075) qui bloquait certains CBZ
- Parsers: nouvelle API publique extract_page() pour extraire une page
  par index depuis CBZ/CBR/PDF avec fallbacks automatiques
- API: suppression du code d'extraction dupliqué, délégation à parsers::extract_page()
- API: retrait des dépendances directes zip/unrar/pdfium-render/natord
- Indexer: nettoyage Meili systématique à chaque sync (au lieu de ~10%)
  avec pagination pour supporter les grosses collections — corrige les
  doublons dans la recherche
- Indexer: retrait de la dépendance rand (plus utilisée)
- Backoffice: popin jobs rendue via createPortal avec positionnement
  dynamique — corrige le débordement desktop et le header cassé en mobile

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:26:14 +01:00
5db2a7501b feat(books): ajouter le champ format en base et l'exposer dans l'API
- Migration 0020 : colonne format sur books, backfill depuis book_files
- batch.rs / scanner.rs : l'indexer écrit le format dans books
- books.rs : format dans BookItem + filtre ?format= dans list_books
- perf_pages.sh : benchmarks par format CBZ/CBR/PDF

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 08:55:18 +01:00
85e0945c9d fix(parsers,api): skipper les entrées ZIP corrompues au lieu d'échouer
Une seule entrée illisible dans le central directory ne doit pas bloquer
l'analyse de tout le livre. Le count et la première page lisible sont
retournés même si certaines entrées sont endommagées.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 08:38:38 +01:00
efc2773199 chore(deps): mettre à jour zip 2.4→8.2, notify 6.1→8.2, lopdf 0.35→0.39
- zip 8.x résout nativement les extra fields NTFS (source du bug EOCD)
- notify 8.x améliore le support inotify Linux
- lopdf 0.39 contient des correctifs de parsing PDF

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:30:14 +01:00
1d9a1c76d2 fix(parsers,api): fallback streaming ZIP pour archives avec extra fields NTFS
Les ZIP créés par des outils Windows (version 6.3) contiennent des extra
fields NTFS (tag 0x000A) qui font échouer ZipArchive::new() avec "Could
not find EOCD". Ajout d'un fallback via read_zipfile_from_stream qui lit
les local file headers sans dépendre du central directory.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:24:36 +01:00
3e3e0154fa fix(parsers): corriger récursion infinie CBZ↔CBR causant un stack overflow
analyze_cbz et analyze_cbr se rappelaient mutuellement sans garde quand
un fichier échouait les deux formats → stack overflow à l'analyse.
Ajout d'un paramètre allow_fallback=false pour briser la boucle.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:15:35 +01:00
e73498cc60 fix(docker): retirer sysctls inotify non supportés par ce kernel
Les sysctls fs.inotify.* ne sont pas namespacés sur ce kernel.
La configuration doit se faire sur l'hôte via /etc/sysctl.conf.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:10:14 +01:00
0f4025369c fix(docker): retirer fs.inotify.max_user_instances non namespacé
Ce sysctl n'est pas dans un namespace kernel séparé et provoque une
erreur OCI au démarrage du container. Seul max_user_watches est conservé.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:09:31 +01:00
7d3670e951 fix(api/pages): fallback CBR→ZIP et CBZ→RAR pour archives mal extensionnées
Même correctif que dans le parsers/indexer : un .cbr qui est en réalité
un ZIP (et vice-versa) retourne maintenant la bonne page au lieu d'un 500.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:06:40 +01:00
09682f5836 fix(docker): augmenter les limites inotify pour éviter "Too many open files"
Ajoute les sysctls fs.inotify.max_user_watches=524288 et
fs.inotify.max_user_instances=512 sur le service indexer pour
prévenir l'erreur watcher sur les grosses bibliothèques.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 23:04:27 +01:00
db11c62d2f fix(analyzer): timeout sur analyze_book pour éviter les blocages indefinis
Un fichier corrompu (RAR/ZIP/PDF qui ne répond plus) occupait un slot
de concurrence indéfiniment, bloquant le pipeline à ex. 1517/1521.

- Ajoute tokio::time::timeout autour de spawn_blocking(analyze_book)
- Timeout lu depuis limits.timeout_seconds en DB (défaut 120s)
- Le livre est marqué parse_status='error' en cas de timeout

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 22:44:48 +01:00
7346f1d5b7 fix(parsers): fallback CBR pour les .cbz qui sont en réalité des archives RAR
Symétrique au fallback CBZ→RAR déjà existant dans analyze_cbr.
Détecte les fichiers .cbz avec magic bytes RAR et les traite via le parser unrar.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 22:29:47 +01:00
358896c7d5 perf(indexer): éliminer le pre-count WalkDir en mode incrémental + concurrence adaptative
- Incremental rebuild: remplace le WalkDir de comptage par un COUNT(*) SQL
  → incrémental 67s → 25s (-62%) sur disque externe
- Full rebuild: conserve le WalkDir (DB vidée avant le comptage)
- Concurrence par défaut: num_cpus/2 clampé [2,8] au lieu de 2 fixe
- Ajoute num_cpus comme dépendance workspace
- Backoffice jobs: un seul formulaire avec formAction par bouton (icônes rétablies)
- infra/perf.sh: corrige l'endpoint /index/jobs/:id (pas /details), exporte BASE_API/TOKEN

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 22:15:41 +01:00
1d10044d46 fix: plusieurs correctifs jobs et analyzer
- cancel_job: ajouter 'extracting_pages' aux statuts annulables
- cleanup_stale_jobs: couvrir 'extracting_pages' et 'generating_thumbnails' au redémarrage
- analyzer: ne pas régénérer le thumbnail si déjà existant (skip sub-phase B)
- analyzer: supprimer les dotfiles macOS (._*) encore en DB
- SSE backoffice: réduire le spam de logs en cas d'API injoignable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 21:41:52 +01:00
8d98056375 fix: fallback for fake cbr 2026-03-12 14:17:21 +01:00
4aafed3d31 docs(readme): documenter toutes les variables d'env avec valeurs par défaut
- Réorganise le tableau des variables par service (partagées, API, Indexer, Backoffice)
- Ajoute les variables thumbnail manquantes (THUMBNAIL_*)
- Met à jour l'exemple docker-compose : env inline, optionnelles commentées avec valeur par défaut
- Supprime env_file en faveur de variables explicites
- Corrige le port backoffice dev (3000 → 7082)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 21:53:04 +01:00
3bd2fb7c1f feat(jobs): introduce extracting_pages status and update job progress handling
- Added a new job status 'extracting_pages' to represent the first sub-phase of thumbnail generation.
- Updated the database schema to include a timestamp for when thumbnail generation starts.
- Enhanced job progress components to handle the new status, including UI updates for displaying progress and status labels.
- Refactored job-related logic to accommodate the two-phase process: extracting pages and generating thumbnails.
- Adjusted SQL queries and job detail responses to include the new fields and statuses.

This change improves the clarity of job processing states and enhances user feedback during the thumbnail generation process.
2026-03-11 17:50:48 +01:00
3b6cc2903d perf(api): remplacer unar/pdftoppm par unrar crate et pdfium-render
CBR: extract_cbr_page extrayait TOUT le CBR sur disque pour lire une
seule page. Reécrit avec le crate unrar : listing en mémoire + extraction
ciblée de la page demandée uniquement. Zéro subprocess, zéro temp dir.

PDF: render_pdf_page utilisait pdftoppm subprocess + temp dir. Reécrit
avec pdfium-render in-process. Zéro subprocess, zéro temp dir.

CBZ: sort naturel (natord) pour l'ordre des pages.

Dockerfile API: retire unar et poppler-utils, ajoute libpdfium.so.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 16:52:15 +01:00
6abaa96fba perf(parsers): remplacer tous les subprocesses par des libs in-process
CBR: remplace unrar/unar CLI par le crate `unrar` (bindings libunrar
vendorisé, zéro dépendance système). Supprime XADRegexException, les
forks de processus et les dossiers temporaires.

PDF: remplace pdfinfo + pdftoppm par pdfium-render. Le PDF est ouvert
une seule fois pour obtenir le nombre de pages ET rasteriser la première
page. lopdf reste pour parse_metadata (page count seul).

convert_cbr_to_cbz: reécrit sans subprocess ni dossier temporaire —
les images sont lues en mémoire via unrar puis packées directement en ZIP.

Dockerfile indexer: retire unrar-free, unar, poppler-utils. Télécharge
libpdfium.so depuis bblanchon/pdfium-binaries au build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 16:46:43 +01:00
f2d9bedcc7 fix(parsers): corriger la génération de thumbnails CBR/CBZ/PDF
- CBR: contourner le bug XADRegexException de unar en appelant unar
  avec un symlink à nom neutre (archive.cbr) au lieu du chemin réel,
  qui peut contenir des caractères regex spéciaux comme [ ] ( )
- CBR/CBZ: remplacer le tri lexicographique par natord (tri naturel)
  pour que page2.jpg soit trié avant page10.jpg
- PDF: brancher pdftoppm -scale-to sur config.width.max(config.height)
  au lieu d'une valeur hardcodée (800px → 400px par défaut)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 16:17:20 +01:00
1c106a4ff2 fix(db): ajouter 'cancelled' à la contrainte CHECK de index_jobs.status
La contrainte index_jobs_status_check ne listait pas 'cancelled', ce qui
causait une erreur 500 à chaque tentative d'annulation de job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 15:58:03 +01:00
3ab5b223a8 fix(indexer): détecter l'annulation de job pendant la phase 2 (analyzer)
L'analyzer ne vérifiait jamais le statut cancelled en DB, ce qui faisait
continuer le traitement des thumbnails jusqu'au bout, puis écraser le
statut 'cancelled' avec 'success'. Ajout d'un poller background toutes
les 2s avec AtomicBool partagé pour stopper proprement le stream concurrent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 15:50:11 +01:00
7cfb6cf001 feat(docker): migrations sqlx intégrées dans le démarrage de l'API
- Déplace les migrations du service `migrate` séparé vers un entrypoint.sh
- L'API exécute `sqlx migrate run` au démarrage avant de lancer le binaire
- Gestion de la rétrocompatibilité : détecte un schéma pre-sqlx et crée
  une baseline `_sqlx_migrations` pour éviter les conflits sur les instances existantes
- Installe sqlx-cli dans le builder, copie le binaire et les migrations dans l'image finale
- Supprime le service `migrate` du docker-compose.yml ; l'indexer dépend maintenant
  du healthcheck de l'API (qui garantit que les migrations sont appliquées)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 15:46:28 +01:00
d2fe7f12ab Add Docker push script and registry documentation
- Create scripts/docker-push.sh for building and pushing images
- Add Docker Registry section to README with usage instructions
- Configure for Docker Hub (julienfroidefond32)
2026-03-11 13:23:16 +01:00
64347edabc fix: thumbnails manquants dans les résultats de recherche
- meili.rs: corrige la désérialisation de la réponse paginée de
  Meilisearch (attendait Vec<Value>, l'API retourne {results:[...]}) —
  la suppression des documents obsolètes ne s'exécutait jamais, laissant
  d'anciens UUIDs qui généraient des 404 sur les thumbnails
- books.rs: fallback sur render_book_page_1 si le fichier thumbnail
  n'est plus accessible sur le disque (au lieu de 500)
- pages.rs: retourne 404 au lieu de 500 quand le fichier CBZ est absent
- search.rs + api.ts + BookCard: ajout série hits, statut lecture,
  pagination OFFSET, filtre reading_status, et placeholder onError

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:45:03 +01:00
8261050943 feat(api+backoffice): pagination par page/offset + filtres séries
API:
- Remplace cursor par page (1-indexé) + OFFSET sur GET /books et GET /libraries/:id/series
- BooksPage et SeriesPage retournent total, page, limit
- GET /libraries/:id/series supporte ?q pour filtrer par nom (ILIKE)

Backoffice:
- Remplace CursorPagination par OffsetPagination sur les 3 pages de liste
- Adapte fetchBooks et fetchSeries (cursor → page)
- Met à jour les types BooksPageDto, SeriesPageDto, SeriesDto, BookDto

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 11:06:34 +01:00
a2da5081ea feat(api): enrichir GET /books et series avec filtres et pagination
- fix(auth): parse_prefix supporte les préfixes de token contenant '_'
- feat: GET /books expose reading_status, reading_current_page, reading_last_read_at
- feat: GET /books accepte ?reading_status=unread,reading (CSV multi-valeur)
- feat: SeriesItem expose books_read_count pour dériver le statut de lecture
- feat: GET /libraries/:id/series accepte ?reading_status=unread,reading
- feat: BooksPage et SeriesPage exposent total (count matchant les filtres)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 09:25:31 +01:00
648d86970f feat: suivi de la progression de lecture par livre
- API : nouvelle table book_reading_progress (migration 0016) et module
  reading_progress.rs avec GET/PATCH /books/:id/progress (token read)
- API : GET /books/:id enrichi avec reading_status, reading_current_page,
  reading_last_read_at via LEFT JOIN
- Backoffice : badge de statut (Non lu / En cours · p.N / Lu) sur la page
  de détail et overlay sur les BookCards
- OpenSpec : change reading-progress avec proposal/design/specs/tasks

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 21:53:52 +01:00
278f422206 feat(backoffice): améliorer les détails de job avec historique des phases
- Ajoute migration 0015 : colonne phase2_started_at sur index_jobs
- Indexer : renseigne phase2_started_at lors du passage à generating_thumbnails
- API : expose phase2_started_at et book_id dans IndexJobDetailResponse
- Page détail : timeline avec durée de chaque phase (Discovery / Thumbnails)
- Page détail : banners contextuels (success/failed/cancelled) avec résumé en une ligne
- Page détail : description textuelle du type de job, durée dans l'overview
- Page détail : stats normalisées selon le type (index vs thumbnail-only)
- JobRow : affiche le type via JobTypeBadge (cohérence visuelle)
- Badge : labels lisibles pour tous les types de jobs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 16:40:01 +01:00
ff59ac1eff fix(indexer): full_rebuild par library ne supprime plus les thumbnails des autres libraries
cleanup_orphaned_thumbnails chargeait uniquement les book IDs de la library
en cours de rebuild, considérant les thumbnails des autres libraries comme
orphelins et les supprimant. La fonction charge désormais tous les book IDs
toutes libraries confondues.

Ajout d'un test de régression dans infra/smoke.sh qui vérifie que le
full_rebuild d'une library ne réduit pas le nombre de thumbnails des autres.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 15:52:00 +01:00
7eb9e2dcad fix: bad ignore no settings update 2026-03-09 23:48:08 +01:00
c81f7ce1b7 feat(api): relier les settings DB au comportement runtime
- Ajout de DynamicSettings dans AppState (Arc<RwLock>) chargé depuis la DB
- rate_limit_per_second, timeout_seconds : plus hardcodés, lus depuis settings
- image_processing (format, quality, filter, max_width) : appliqués comme
  valeurs par défaut sur les requêtes de pages (overridables via query params)
- cache.directory : lu depuis settings au lieu de la variable d'env
- update_setting recharge immédiatement le DynamicSettings en mémoire
  pour les clés limits, image_processing et cache (sans redémarrage)
- parse_filter() : mapping lanczos3/triangle/nearest → FilterType

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 23:27:09 +01:00
137e8ce11c fix: slow thumbnail and analyser test 2026-03-09 23:16:21 +01:00
e0b80cae38 feat: conversion CBR → CBZ via job asynchrone
Ajoute la possibilité de convertir un livre CBR en CBZ depuis le backoffice.
La conversion est sécurisée : le CBR original n'est supprimé qu'après vérification
du CBZ généré et mise à jour de la base de données.

- parsers: nouvelle fn `convert_cbr_to_cbz` (unar extract → zip pack → vérification → rename atomique)
- api: `POST /books/:id/convert` crée un job `cbr_to_cbz` (vérifie format CBR, détecte collision)
- indexer: nouveau `converter.rs` dispatché depuis `job.rs`
- backoffice: bouton "Convert to CBZ" sur la page détail (visible si CBR), label dans JobRow
- migrations: colonne `book_id` sur `index_jobs` + type `cbr_to_cbz` dans le check constraint

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 23:02:08 +01:00
e8bb014874 feat(backoffice): amélioration navigation mobile et tablette
- Ajout d'un menu hamburger mobile (MobileNav) avec drawer animé via React Portal (évite le piège du backdrop-filter du header)
- Popin JobsIndicator adaptée mobile : positionnement fixed plein-écran sur petit écran, backdrop semi-transparent
- Navigation tablette (md→lg) : icônes seules avec tooltip natif, labels visibles uniquement sur lg+

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:44:33 +01:00
4c75e08056 fix(api): resolve all OpenAPI schema reference errors
- Add #[schema(value_type = Option<String>)] on chrono::DateTime fields
- Register SeriesPage in openapi.rs components
- Fix module-prefixed ref (index_jobs::IndexJobResponse -> IndexJobResponse)
- Strengthen test: assert all $ref targets exist in components/schemas

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:27:52 +01:00
f1b3aec94a docs(api): complete OpenAPI coverage for all routes
Add missing utoipa annotations:
- GET /books/{id}/thumbnail
- GET/POST /settings, /settings/{key}
- POST /settings/cache/clear
- GET /settings/cache/stats, /settings/thumbnail/stats
Add 'settings' tag and register all new schemas.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:23:28 +01:00
473e849dfa feat(backoffice): add page preview carousel on book detail page
Shows 5 pages at a time in a full-width grid with prev/next navigation.
Pages are fetched via the existing proxy route with webp format.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:18:47 +01:00
cfc896e92f feat: two-phase indexation with direct thumbnail generation in indexer
Phase 1 (discovery): walkdir + filename-only metadata, zero archive I/O.
Books are visible immediately in the UI while Phase 2 runs in background.

Phase 2 (analysis): open each archive once via analyze_book() to extract
page_count and first page bytes, then generate WebP thumbnail directly in
the indexer — removing the HTTP roundtrip to the API checkup endpoint.

- Add parse_metadata_fast() (infallible, no archive I/O)
- Add analyze_book() returning (page_count, first_page_bytes) in one pass
- Add looks_like_image() magic bytes check for unrar p stdout validation
- Add lsar fallback in list_cbr_images() for UTF-16BE encoded filenames
- Add directory_mtimes table to skip unchanged dirs on incremental scans
- Add analyzer.rs: generate_thumbnail, analyze_library_books, regenerate_thumbnails
- Remove run_checkup() from API; indexer handles thumbnail jobs directly
- Remove api_base_url/api_bootstrap_token from IndexerConfig and AppState
- Add unar + poppler-utils to indexer Dockerfile
- Fix smoke.sh: wait for job completion, check thumbnail_url field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:13:05 +01:00
36af34443e refactor: improve API error handling and response structure
- Refactor error handling across various API endpoints to ensure consistent response formats.
- Enhance response structure to include more informative error messages and status codes.
- Update relevant tests to reflect changes in error handling and response formats.
2026-03-09 21:24:22 +01:00
85cad1a7e7 refactor: streamline API calls and enhance configuration management
- Refactor multiple API routes to utilize a centralized configuration function for base URL and token management, improving code consistency and maintainability.
- Replace direct environment variable access with a unified config function in the `lib/api.ts` file.
- Remove redundant error handling and streamline response handling in various API endpoints.
- Delete unused job-related API routes and settings, simplifying the overall API structure.
2026-03-09 14:16:01 +01:00
0f5094575a docs: add AGENTS.md per module and unify ports to 70XX
- Add CLAUDE.md at root and AGENTS.md in apps/api, apps/indexer,
  apps/backoffice, crates/parsers with module-specific guidelines
- Unify all service ports to 70XX (no more internal/external split):
  API 7080, Indexer 7081, Backoffice 7082
- Update docker-compose.yml, Dockerfiles, config.rs defaults,
  .env.example, backoffice routes, bench.sh, smoke.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 13:57:39 +01:00
131c50b1a1 chore: remove docker-compose configuration
- Delete the docker-compose.yml file, which contained service definitions for postgres, meilisearch, migrate, api, indexer, backoffice, and associated volumes.
- This change may indicate a shift in deployment strategy or service management.
2026-03-08 21:34:28 +01:00
6d4c400017 refactor: update AppState references to use state module
- Change all instances of AppState to reference the new state module across multiple files for consistency.
- Clean up imports in auth, books, index_jobs, libraries, pages, search, settings, thumbnails, and tokens modules.
- Simplify main.rs by removing unused code and organizing middleware and route handlers under the new handlers module.
2026-03-08 21:19:22 +01:00
260 changed files with 34062 additions and 4428 deletions

View File

@@ -0,0 +1,152 @@
---
name: "OPSX: Apply"
description: Implement tasks from an OpenSpec change (Experimental)
category: Workflow
tags: [workflow, artifacts, experimental]
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue`
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! You can archive this change with `/opsx:archive`.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,157 @@
---
name: "OPSX: Archive"
description: Archive a completed change in the experimental workflow
category: Workflow
tags: [workflow, archive, experimental]
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Prompt user for confirmation to continue
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Prompt user for confirmation to continue
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Spec sync status (synced / sync skipped / no delta specs)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs
All artifacts complete. All tasks complete.
```
**Output On Success (No Delta Specs)**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** No delta specs
All artifacts complete. All tasks complete.
```
**Output On Success With Warnings**
```
## Archive Complete (with warnings)
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** Sync skipped (user chose to skip)
**Warnings:**
- Archived with 2 incomplete artifacts
- Archived with 3 incomplete tasks
- Delta spec sync was skipped (user chose to skip)
Review the archive if this was not intentional.
```
**Output On Error (Archive Exists)**
```
## Archive Failed
**Change:** <change-name>
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
Target archive directory already exists.
**Options:**
1. Rename the existing archive
2. Delete the existing archive if it's a duplicate
3. Wait until a different date to archive
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,173 @@
---
name: "OPSX: Explore"
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
category: Workflow
tags: [workflow, explore, experimental, thinking]
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
- A vague idea: "real-time collaboration"
- A specific problem: "the auth system is getting unwieldy"
- A change name: "add-dark-mode" (to explore in context of that change)
- A comparison: "postgres vs sqlite for this"
- Nothing (just enter explore mode)
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
If the user mentioned a specific change name, read its artifacts for context.
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,106 @@
---
name: "OPSX: Propose"
description: Propose a new change - create it and generate all artifacts in one step
category: Workflow
tags: [workflow, artifacts, experimental]
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
**Steps**
1. **If no input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` to start implementing."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,152 @@
---
name: /opsx-apply
id: opsx-apply
category: Workflow
description: Implement tasks from an OpenSpec change (Experimental)
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue`
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! You can archive this change with `/opsx:archive`.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,157 @@
---
name: /opsx-archive
id: opsx-archive
category: Workflow
description: Archive a completed change in the experimental workflow
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Prompt user for confirmation to continue
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Prompt user for confirmation to continue
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Spec sync status (synced / sync skipped / no delta specs)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs
All artifacts complete. All tasks complete.
```
**Output On Success (No Delta Specs)**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** No delta specs
All artifacts complete. All tasks complete.
```
**Output On Success With Warnings**
```
## Archive Complete (with warnings)
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** Sync skipped (user chose to skip)
**Warnings:**
- Archived with 2 incomplete artifacts
- Archived with 3 incomplete tasks
- Delta spec sync was skipped (user chose to skip)
Review the archive if this was not intentional.
```
**Output On Error (Archive Exists)**
```
## Archive Failed
**Change:** <change-name>
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
Target archive directory already exists.
**Options:**
1. Rename the existing archive
2. Delete the existing archive if it's a duplicate
3. Wait until a different date to archive
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,173 @@
---
name: /opsx-explore
id: opsx-explore
category: Workflow
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
- A vague idea: "real-time collaboration"
- A specific problem: "the auth system is getting unwieldy"
- A change name: "add-dark-mode" (to explore in context of that change)
- A comparison: "postgres vs sqlite for this"
- Nothing (just enter explore mode)
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
If the user mentioned a specific change name, read its artifacts for context.
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,106 @@
---
name: /opsx-propose
id: opsx-propose
category: Workflow
description: Propose a new change - create it and generate all artifacts in one step
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
**Steps**
1. **If no input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` to start implementing."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -9,9 +9,6 @@
# REQUIRED - Change these values in production! # REQUIRED - Change these values in production!
# ============================================================================= # =============================================================================
# Master key for Meilisearch authentication (required)
MEILI_MASTER_KEY=change-me-in-production
# Bootstrap token for initial API admin access (required) # Bootstrap token for initial API admin access (required)
# Use this token for the first API calls before creating proper API tokens # Use this token for the first API calls before creating proper API tokens
API_BOOTSTRAP_TOKEN=change-me-in-production API_BOOTSTRAP_TOKEN=change-me-in-production
@@ -21,19 +18,34 @@ API_BOOTSTRAP_TOKEN=change-me-in-production
# ============================================================================= # =============================================================================
# API Service # API Service
API_LISTEN_ADDR=0.0.0.0:8080 API_LISTEN_ADDR=0.0.0.0:7080
API_BASE_URL=http://api:8080 API_BASE_URL=http://api:7080
# Indexer Service # Indexer Service
INDEXER_LISTEN_ADDR=0.0.0.0:8081 INDEXER_LISTEN_ADDR=0.0.0.0:7081
INDEXER_SCAN_INTERVAL_SECONDS=5 INDEXER_SCAN_INTERVAL_SECONDS=5
# Meilisearch Search Engine
MEILI_URL=http://meilisearch:7700
# PostgreSQL Database # PostgreSQL Database
DATABASE_URL=postgres://stripstream:stripstream@postgres:5432/stripstream DATABASE_URL=postgres://stripstream:stripstream@postgres:5432/stripstream
# =============================================================================
# Logging
# =============================================================================
# Log levels per domain. Default: indexer=info,scan=info,extraction=info,thumbnail=warn,watcher=info
# Domains:
# scan — filesystem scan (discovery phase)
# extraction — page extraction from archives (extracting_pages phase)
# thumbnail — thumbnail generation (resize/encode)
# watcher — file watcher polling
# indexer — general indexer logs
# Levels: error, warn, info, debug, trace
# Examples:
# RUST_LOG=indexer=info # default, quiet thumbnails
# RUST_LOG=indexer=info,thumbnail=debug # enable thumbnail timing logs
# RUST_LOG=indexer=info,extraction=debug # per-book extraction details
# RUST_LOG=indexer=debug,scan=debug,extraction=debug,thumbnail=debug,watcher=debug # tout voir
# RUST_LOG=indexer=info,scan=info,extraction=info,thumbnail=warn,watcher=info
# ============================================================================= # =============================================================================
# Storage Configuration # Storage Configuration
# ============================================================================= # =============================================================================
@@ -46,18 +58,17 @@ LIBRARIES_ROOT_PATH=/libraries
# Path to libraries directory on host machine (for Docker volume mount) # Path to libraries directory on host machine (for Docker volume mount)
# Default: ../libraries (relative to infra/docker-compose.yml) # Default: ../libraries (relative to infra/docker-compose.yml)
# You can change this to an absolute path on your machine # You can change this to an absolute path on your machine
LIBRARIES_HOST_PATH=../libraries LIBRARIES_HOST_PATH=./libraries
# Path to thumbnails directory on host machine (for Docker volume mount) # Path to thumbnails directory on host machine (for Docker volume mount)
# Default: ../data/thumbnails (relative to infra/docker-compose.yml) # Default: ../data/thumbnails (relative to infra/docker-compose.yml)
THUMBNAILS_HOST_PATH=../data/thumbnails THUMBNAILS_HOST_PATH=./data/thumbnails
# ============================================================================= # =============================================================================
# Port Configuration # Port Configuration
# ============================================================================= # =============================================================================
# To change ports, edit docker-compose.yml directly: # To change ports, edit docker-compose.yml directly:
# - API: change "7080:8080" to "YOUR_PORT:8080" # - API: change "7080:7080" to "YOUR_PORT:7080"
# - Indexer: change "7081:8081" to "YOUR_PORT:8081" # - Indexer: change "7081:7081" to "YOUR_PORT:7081"
# - Backoffice: change "7082:8082" to "YOUR_PORT:8082" # - Backoffice: change "7082:7082" to "YOUR_PORT:7082"
# - Meilisearch: change "7700:7700" to "YOUR_PORT:7700"
# - PostgreSQL: change "6432:5432" to "YOUR_PORT:5432" # - PostgreSQL: change "6432:5432" to "YOUR_PORT:5432"

View File

@@ -0,0 +1,17 @@
name: Deploy with Docker Compose
on:
push:
branches:
- main # adapte la branche que tu veux déployer
jobs:
deploy:
runs-on: mac-orbstack-runner # le nom que tu as donné au runner
steps:
- name: Deploy stack
env:
DOCKER_BUILDKIT: 1
COMPOSE_DOCKER_CLI_BUILD: 1
run: |
BUILDKIT_PROGRESS=plain cd /Users/julienfroidefond/Sites/docker-stack && docker pull julienfroidefond32/stripstream-backoffice && docker pull julienfroidefond32/stripstream-api && docker pull julienfroidefond32/stripstream-indexer && ./scripts/stack.sh up stripstream

2
.gitignore vendored
View File

@@ -2,7 +2,7 @@ target/
.env .env
.DS_Store .DS_Store
tmp/ tmp/
libraries/ /libraries/
node_modules/ node_modules/
.next/ .next/
data/thumbnails data/thumbnails

View File

@@ -0,0 +1,149 @@
---
description: Implement tasks from an OpenSpec change (Experimental)
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name (e.g., `/opsx-apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx-continue`
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! You can archive this change with `/opsx-archive`.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,154 @@
---
description: Archive a completed change in the experimental workflow
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name after `/opsx-archive` (e.g., `/opsx-archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Prompt user for confirmation to continue
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Prompt user for confirmation to continue
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Spec sync status (synced / sync skipped / no delta specs)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs
All artifacts complete. All tasks complete.
```
**Output On Success (No Delta Specs)**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** No delta specs
All artifacts complete. All tasks complete.
```
**Output On Success With Warnings**
```
## Archive Complete (with warnings)
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** Sync skipped (user chose to skip)
**Warnings:**
- Archived with 2 incomplete artifacts
- Archived with 3 incomplete tasks
- Delta spec sync was skipped (user chose to skip)
Review the archive if this was not intentional.
```
**Output On Error (Archive Exists)**
```
## Archive Failed
**Change:** <change-name>
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
Target archive directory already exists.
**Options:**
1. Rename the existing archive
2. Delete the existing archive if it's a duplicate
3. Wait until a different date to archive
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,170 @@
---
description: Enter explore mode - think through ideas, investigate problems, clarify requirements
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
**Input**: The argument after `/opsx-explore` is whatever the user wants to think about. Could be:
- A vague idea: "real-time collaboration"
- A specific problem: "the auth system is getting unwieldy"
- A change name: "add-dark-mode" (to explore in context of that change)
- A comparison: "postgres vs sqlite for this"
- Nothing (just enter explore mode)
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
If the user mentioned a specific change name, read its artifacts for context.
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,103 @@
---
description: Propose a new change - create it and generate all artifacts in one step
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx-apply
---
**Input**: The argument after `/opsx-propose` is the change name (kebab-case), OR a description of what the user wants to build.
**Steps**
1. **If no input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx-apply` to start implementing."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx-explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx-apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx-apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -73,12 +73,14 @@ sqlx migrate add -r migration_name
### Docker Development ### Docker Development
`docker-compose.yml` est à la **racine** du projet (pas dans `infra/`).
```bash ```bash
# Start infrastructure only # Start infrastructure only
cd infra && docker compose up -d postgres meilisearch docker compose up -d postgres
# Start full stack # Start full stack
cd infra && docker compose up -d docker compose up -d
# View logs # View logs
docker compose logs -f api docker compose logs -f api
@@ -226,24 +228,21 @@ pub struct BookItem {
``` ```
stripstream-librarian/ stripstream-librarian/
├── apps/ ├── apps/
│ ├── api/ # REST API (axum) │ ├── api/ # REST API (axum) — port 7080
│ │ └── src/ │ │ └── src/ # books.rs, pages.rs, thumbnails.rs, state.rs, auth.rs...
│ ├── main.rs ├── indexer/ # Background indexing service — port 7081
│ │ ├── books.rs │ │ └── src/ # worker.rs, scanner.rs, batch.rs, scheduler.rs, watcher.rs...
│ ├── pages.rs └── backoffice/ # Next.js admin UI — port 7082
│ │ └── ...
│ ├── indexer/ # Background indexing service
│ │ └── src/
│ │ └── main.rs
│ └── backoffice/ # Next.js admin UI
├── crates/ ├── crates/
│ ├── core/ # Shared config │ ├── core/ # Shared config (env vars)
│ │ └── src/config.rs │ │ └── src/config.rs
│ └── parsers/ # Book parsing (CBZ, CBR, PDF) │ └── parsers/ # Book parsing (CBZ, CBR, PDF)
├── infra/ ├── infra/
── migrations/ # SQL migrations ── migrations/ # SQL migrations (sqlx)
│ └── docker-compose.yml ├── data/
└── libraries/ # Book storage (mounted volume) │ └── thumbnails/ # Thumbnails générés par l'API
├── libraries/ # Book storage (mounted volume)
└── docker-compose.yml # À la racine (pas dans infra/)
``` ```
### Key Files ### Key Files
@@ -251,8 +250,13 @@ stripstream-librarian/
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `apps/api/src/books.rs` | Book CRUD endpoints | | `apps/api/src/books.rs` | Book CRUD endpoints |
| `apps/api/src/pages.rs` | Page rendering & caching | | `apps/api/src/pages.rs` | Page rendering & caching (LRU + disk) |
| `apps/indexer/src/main.rs` | Indexing logic, batch processing | | `apps/api/src/thumbnails.rs` | Endpoints pour créer des jobs thumbnail (rebuild/regenerate) |
| `apps/api/src/state.rs` | AppState, Semaphore concurrent_renders |
| `apps/indexer/src/scanner.rs` | Phase 1 discovery : scan rapide sans I/O archive, skip dossiers inchangés |
| `apps/indexer/src/analyzer.rs` | Phase 2 analysis : `analyze_book` + génération thumbnails WebP |
| `apps/indexer/src/batch.rs` | Bulk DB ops via UNNEST |
| `apps/indexer/src/worker.rs` | Job loop, watcher, scheduler orchestration |
| `crates/parsers/src/lib.rs` | Format detection, metadata parsing | | `crates/parsers/src/lib.rs` | Format detection, metadata parsing |
| `crates/core/src/config.rs` | Configuration from environment | | `crates/core/src/config.rs` | Configuration from environment |
| `infra/migrations/*.sql` | Database schema | | `infra/migrations/*.sql` | Database schema |
@@ -269,7 +273,7 @@ impl IndexerConfig {
pub fn from_env() -> Result<Self> { pub fn from_env() -> Result<Self> {
Ok(Self { Ok(Self {
listen_addr: std::env::var("INDEXER_LISTEN_ADDR") listen_addr: std::env::var("INDEXER_LISTEN_ADDR")
.unwrap_or_else(|_| "0.0.0.0:8081".to_string()), .unwrap_or_else(|_| "0.0.0.0:7081".to_string()),
database_url: std::env::var("DATABASE_URL") database_url: std::env::var("DATABASE_URL")
.context("DATABASE_URL is required")?, .context("DATABASE_URL is required")?,
// ... // ...
@@ -298,4 +302,6 @@ fn remap_libraries_path(path: &str) -> String {
- **Workspace**: This is a Cargo workspace. Always specify the package when building specific apps. - **Workspace**: This is a Cargo workspace. Always specify the package when building specific apps.
- **Dependencies**: External crates are defined in workspace `Cargo.toml`, not individual `Cargo.toml`. - **Dependencies**: External crates are defined in workspace `Cargo.toml`, not individual `Cargo.toml`.
- **Database**: PostgreSQL is required. Run migrations before starting services. - **Database**: PostgreSQL is required. Run migrations before starting services.
- **External Tools**: The indexer relies on `unar` (for CBR) and `pdftoppm` (for PDF) being installed on the system. - **External Tools**: 4 system tools required — `unrar` (CBR page count), `unar` (CBR extraction), `pdfinfo` (PDF page count), `pdftoppm` (PDF page render). Note: `unrar` and `unar` are distinct tools.
- **Thumbnails**: generated by the **indexer** service (phase 2, `analyzer.rs`). The API only creates jobs in DB — it does not generate thumbnails directly.
- **Sub-AGENTS.md**: module-specific guidelines in `apps/api/`, `apps/indexer/`, `apps/backoffice/`, `crates/parsers/`.

73
CLAUDE.md Normal file
View File

@@ -0,0 +1,73 @@
# Stripstream Librarian
Gestionnaire de bibliothèque de bandes dessinées/ebooks. Workspace Cargo multi-crates avec backoffice Next.js.
## Architecture
| Service | Dossier | Port local |
|---------|---------|------------|
| API REST (axum) | `apps/api/` | 7080 |
| Indexer (background) | `apps/indexer/` | 7081 |
| Backoffice (Next.js) | `apps/backoffice/` | 7082 |
| PostgreSQL | infra | 6432 |
Crates partagés : `crates/core` (config env), `crates/parsers` (CBZ/CBR/PDF).
## Commandes
```bash
# Build
cargo build # workspace entier
cargo build -p api # crate spécifique
cargo build --release # version optimisée
# Linting / format
cargo clippy
cargo fmt
# Tests
cargo test
cargo test -p parsers
# Infra (dépendances uniquement) — docker-compose.yml est à la racine
docker compose up -d postgres
# Backoffice dev
cd apps/backoffice && npm install && npm run dev # http://localhost:7082
# Migrations
sqlx migrate run # DATABASE_URL doit être défini
```
## Environnement
```bash
cp .env.example .env # puis éditer les valeurs REQUIRED
```
Variables **requises** au démarrage : `DATABASE_URL`, `API_BOOTSTRAP_TOKEN`.
## Gotchas
- **Dépendances système** : 4 outils requis — `unrar` (CBR listing), `unar` (CBR extraction), `pdfinfo` (PDF page count), `pdftoppm` (PDF rendu). `unrar``unar`.
- **Port backoffice** : `npm run dev` écoute sur **7082**, pas 3000.
- **LIBRARIES_ROOT_PATH** : les chemins en DB commencent par `/libraries/` ; en dev local, définir cette variable pour remapper vers le dossier réel.
- **Thumbnails** : stockés dans `THUMBNAIL_DIRECTORY` (défaut `/data/thumbnails`), générés par **l'API** (pas l'indexer) — l'indexer déclenche un checkup via `POST /index/jobs/:id/thumbnails/checkup`.
- **Workspace Cargo** : les dépendances externes sont définies dans le `Cargo.toml` racine, pas dans les crates individuels.
- **Migrations** : dossier `infra/migrations/`, géré par sqlx. Toujours migrer avant de démarrer les services.
- **Recherche** : full-text via PostgreSQL (`ILIKE` + `pg_trgm`), pas de moteur de recherche externe.
## Fichiers clés
| Fichier | Rôle |
|---------|------|
| `crates/core/src/config.rs` | Config depuis env (API, Indexer, AdminUI) |
| `crates/parsers/src/lib.rs` | Détection format, extraction métadonnées |
| `apps/api/src/books.rs` | Endpoints CRUD livres |
| `apps/api/src/search.rs` | Recherche full-text PostgreSQL |
| `apps/api/src/pages.rs` | Rendu pages + cache LRU |
| `apps/indexer/src/scanner.rs` | Scan filesystem |
| `infra/migrations/*.sql` | Schéma DB |
> Voir `AGENTS.md` pour les conventions de code détaillées (error handling, patterns sqlx, async/tokio).
> Des `AGENTS.md` spécifiques existent dans `apps/api/`, `apps/indexer/`, `apps/backoffice/`, `crates/parsers/`.

817
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -3,13 +3,14 @@ members = [
"apps/api", "apps/api",
"apps/indexer", "apps/indexer",
"crates/core", "crates/core",
"crates/notifications",
"crates/parsers", "crates/parsers",
] ]
resolver = "2" resolver = "2"
[workspace.package] [workspace.package]
edition = "2021" edition = "2021"
version = "0.1.0" version = "1.27.0"
license = "MIT" license = "MIT"
[workspace.dependencies] [workspace.dependencies]
@@ -19,9 +20,10 @@ axum = "0.7"
base64 = "0.22" base64 = "0.22"
chrono = { version = "0.4", features = ["serde"] } chrono = { version = "0.4", features = ["serde"] }
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp"] } image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp"] }
jpeg-decoder = "0.3"
lru = "0.12" lru = "0.12"
rayon = "1.10" rayon = "1.10"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] } reqwest = { version = "0.12", default-features = false, features = ["json", "multipart", "rustls-tls"] }
rand = "0.8" rand = "0.8"
serde = { version = "1.0", features = ["derive"] } serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0" serde_json = "1.0"
@@ -32,6 +34,12 @@ tower = { version = "0.5", features = ["limit"] }
tracing = "0.1" tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] } tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
uuid = { version = "1.12", features = ["serde", "v4"] } uuid = { version = "1.12", features = ["serde", "v4"] }
natord = "1.0"
num_cpus = "1.16"
pdfium-render = { version = "0.8", default-features = false, features = ["pdfium_latest", "image_latest", "thread_safe"] }
unrar = "0.5"
walkdir = "2.5" walkdir = "2.5"
webp = "0.3"
utoipa = "4.0" utoipa = "4.0"
utoipa-swagger-ui = "6.0" utoipa-swagger-ui = "6.0"
scraper = "0.21"

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Julien Froidefond
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

10
PLAN.md
View File

@@ -12,7 +12,7 @@ Construire un serveur ultra performant pour indexer et servir des bibliotheques
- Backend/API: Rust (`axum`) - Backend/API: Rust (`axum`)
- Indexation: service Rust dedie (`indexer`) - Indexation: service Rust dedie (`indexer`)
- DB: PostgreSQL - DB: PostgreSQL
- Recherche: Meilisearch - Recherche: PostgreSQL full-text (ILIKE + pg_trgm)
- Deploiement: Docker Compose - Deploiement: Docker Compose
- Auth: token bootstrap env + tokens admin en DB (creables/revocables) - Auth: token bootstrap env + tokens admin en DB (creables/revocables)
- Expiration tokens admin: aucune par defaut (revocation manuelle) - Expiration tokens admin: aucune par defaut (revocation manuelle)
@@ -33,7 +33,7 @@ Construire un serveur ultra performant pour indexer et servir des bibliotheques
**DoD:** Build des crates OK. **DoD:** Build des crates OK.
### T2 - Infra Docker Compose ### T2 - Infra Docker Compose
- [x] Definir services `postgres`, `meilisearch`, `api`, `indexer` - [x] Definir services `postgres`, `api`, `indexer`
- [x] Volumes persistants - [x] Volumes persistants
- [x] Healthchecks - [x] Healthchecks
@@ -114,7 +114,7 @@ Construire un serveur ultra performant pour indexer et servir des bibliotheques
**DoD:** Pagination/filtres fonctionnels. **DoD:** Pagination/filtres fonctionnels.
### T13 - Recherche ### T13 - Recherche
- [x] Projection vers Meilisearch - [x] Recherche full-text PostgreSQL
- [x] `GET /search?q=...&library_id=...&type=...` - [x] `GET /search?q=...&library_id=...&type=...`
- [x] Fuzzy + filtres - [x] Fuzzy + filtres
@@ -264,10 +264,10 @@ Construire un serveur ultra performant pour indexer et servir des bibliotheques
- Bootstrap token = break-glass (peut etre desactive plus tard) - Bootstrap token = break-glass (peut etre desactive plus tard)
## Journal ## Journal
- 2026-03-05: `docker compose up -d --build` valide, stack complete en healthy (`postgres`, `meilisearch`, `api`, `indexer`, `admin-ui`). - 2026-03-05: `docker compose up -d --build` valide, stack complete en healthy (`postgres`, `api`, `indexer`, `admin-ui`).
- 2026-03-05: ajustements infra appliques pour demarrage stable (`unrar` -> `unrar-free`, image `rust:1-bookworm`, healthchecks `127.0.0.1`). - 2026-03-05: ajustements infra appliques pour demarrage stable (`unrar` -> `unrar-free`, image `rust:1-bookworm`, healthchecks `127.0.0.1`).
- 2026-03-05: ajout d'un service `migrate` dans Compose pour executer automatiquement `infra/migrations/0001_init.sql` au demarrage. - 2026-03-05: ajout d'un service `migrate` dans Compose pour executer automatiquement `infra/migrations/0001_init.sql` au demarrage.
- 2026-03-05: Lot 2 termine (jobs, scan incremental, parsers `cbz/cbr/pdf`, API livres, sync + recherche Meilisearch). - 2026-03-05: Lot 2 termine (jobs, scan incremental, parsers `cbz/cbr/pdf`, API livres, recherche PostgreSQL).
- 2026-03-05: verification de bout en bout OK sur une librairie de test (`/libraries/demo`) avec indexation, listing `/books` et recherche `/search` (1 CBZ detecte). - 2026-03-05: verification de bout en bout OK sur une librairie de test (`/libraries/demo`) avec indexation, listing `/books` et recherche `/search` (1 CBZ detecte).
- 2026-03-05: Lot 3 avancee: endpoint pages (`/books/:id/pages/:n`) actif avec cache LRU, ETag/Cache-Control, limite concurrence rendu et timeouts. - 2026-03-05: Lot 3 avancee: endpoint pages (`/books/:id/pages/:n`) actif avec cache LRU, ETag/Cache-Control, limite concurrence rendu et timeouts.
- 2026-03-05: hardening API: readiness expose sans auth via `route_layer`, metriques simples `/metrics`, rate limiting lecture (120 req/s). - 2026-03-05: hardening API: readiness expose sans auth via `route_layer`, metriques simples `/metrics`, rate limiting lecture (120 req/s).

233
README.md
View File

@@ -9,7 +9,7 @@ The project consists of the following components:
- **API** (`apps/api/`) - Rust-based REST API service - **API** (`apps/api/`) - Rust-based REST API service
- **Indexer** (`apps/indexer/`) - Rust-based background indexing service - **Indexer** (`apps/indexer/`) - Rust-based background indexing service
- **Backoffice** (`apps/backoffice/`) - Next.js web administration interface - **Backoffice** (`apps/backoffice/`) - Next.js web administration interface
- **Infrastructure** (`infra/`) - Docker Compose setup with PostgreSQL and Meilisearch - **Infrastructure** (`infra/`) - Docker Compose setup with PostgreSQL
## Quick Start ## Quick Start
@@ -27,28 +27,24 @@ The project consists of the following components:
``` ```
2. Edit `.env` and set secure values for: 2. Edit `.env` and set secure values for:
- `MEILI_MASTER_KEY` - Master key for Meilisearch
- `API_BOOTSTRAP_TOKEN` - Bootstrap token for initial API authentication - `API_BOOTSTRAP_TOKEN` - Bootstrap token for initial API authentication
### Running with Docker ### Running with Docker
```bash ```bash
cd infra
docker compose up -d docker compose up -d
``` ```
This will start: This will start:
- PostgreSQL (port 5432) - PostgreSQL (port 6432)
- Meilisearch (port 7700) - API service (port 7080)
- API service (port 8080) - Indexer service (port 7081)
- Indexer service (port 8081) - Backoffice web UI (port 7082)
- Backoffice web UI (port 8082)
### Accessing the Application ### Accessing the Application
- **Backoffice**: http://localhost:8082 - **Backoffice**: http://localhost:7082
- **API**: http://localhost:8080 - **API**: http://localhost:7080
- **Meilisearch**: http://localhost:7700
### Default Credentials ### Default Credentials
@@ -62,8 +58,7 @@ The default bootstrap token is configured in your `.env` file. Use this for init
```bash ```bash
# Start dependencies # Start dependencies
cd infra docker compose up -d postgres
docker compose up -d postgres meilisearch
# Run API # Run API
cd apps/api cd apps/api
@@ -82,53 +77,114 @@ npm install
npm run dev npm run dev
``` ```
The backoffice will be available at http://localhost:3000 The backoffice will be available at http://localhost:7082
## Features ## Features
### Libraries Management > For the full feature list, business rules, and API details, see [docs/FEATURES.md](docs/FEATURES.md).
- Create and manage multiple libraries
- Configure automatic scanning schedules (hourly, daily, weekly)
- Real-time file watcher for instant indexing
- Full and incremental rebuild options
### Books Management ### Libraries
- Support for CBZ, CBR, and PDF formats - Multi-library management with per-library configuration
- Automatic metadata extraction - Incremental and full scanning, real-time filesystem watcher
- Series and volume detection - Per-library metadata provider selection (Google Books, ComicVine, BedéThèque, AniList, Open Library)
- Full-text search with Meilisearch
### Jobs Monitoring ### Books & Series
- Real-time job progress tracking - **Formats**: CBZ, CBR, PDF, EPUB
- Detailed statistics (scanned, indexed, removed, errors) - Automatic metadata extraction (title, series, volume, authors, page count) from filenames and directory structure
- Job history and logs - Series aggregation with missing volume detection
- Cancel pending jobs - Thumbnail generation (WebP/JPEG/PNG) with lazy generation and bulk rebuild
- CBR → CBZ conversion
### Search ### Reading Progress
- Full-text search across titles, authors, and series - Per-book tracking: unread / reading / read with current page
- Library filtering - Series-level aggregated reading status
- Real-time suggestions - Bulk mark-as-read for series
### Search & Discovery
- Full-text search across titles, authors, and series (PostgreSQL `pg_trgm`)
- Author listing with book/series counts
- Filtering by reading status, series status, format, metadata provider
### External Metadata
- Search, match, approve/reject workflow with confidence scoring
- Batch auto-matching and scheduled metadata refresh
- Field locking to protect manual edits from sync
### Notifications
- **Telegram**: real-time notifications via Telegram Bot API
- 12 granular event toggles (scans, thumbnails, conversions, metadata)
- Book thumbnail images included in notifications where applicable
- Test connection from settings
### External Integrations
- **Komga**: import reading progress
- **Prowlarr**: search for missing volumes
- **qBittorrent**: add torrents directly from search results
### Background Jobs
- Rebuild, rescan, thumbnail generation, metadata batch, CBR conversion
- Real-time progress via Server-Sent Events (SSE)
- Job history, error tracking, cancellation
### Page Rendering
- On-demand page extraction from all formats
- Image processing (format, quality, max width, resampling filter)
- LRU in-memory + disk cache
### Security
- Token-based auth (`admin` / `read` scopes) with Argon2 hashing
- Rate limiting, token expiration and revocation
### Web UI (Backoffice)
- Dashboard with statistics, interactive charts (recharts), and reading progress
- Currently reading & recently read sections
- Library, book, series, author management
- Live job monitoring, metadata search modals, settings panel
- Notification settings with per-event toggle configuration
## Environment Variables ## Environment Variables
| Variable | Description | Default | Variables marquées **required** doivent être définies. Les autres ont une valeur par défaut.
|----------|-------------|---------|
| `API_LISTEN_ADDR` | API service bind address | `0.0.0.0:8080` | ### Partagées (API + Indexer)
| `INDEXER_LISTEN_ADDR` | Indexer service bind address | `0.0.0.0:8081` |
| `BACKOFFICE_PORT` | Backoffice web UI port | `8082` | | Variable | Description | Défaut |
| `DATABASE_URL` | PostgreSQL connection string | `postgres://stripstream:stripstream@postgres:5432/stripstream` | |----------|-------------|--------|
| `MEILI_URL` | Meilisearch connection URL | `http://meilisearch:7700` | | `DATABASE_URL` | **required** — Connexion PostgreSQL | — |
| `MEILI_MASTER_KEY` | Meilisearch master key (required) | - |
| `API_BOOTSTRAP_TOKEN` | Initial API admin token (required) | - | ### API
| `INDEXER_SCAN_INTERVAL_SECONDS` | Watcher scan interval | `5` |
| `LIBRARIES_ROOT_PATH` | Path to libraries directory | `/libraries` | | Variable | Description | Défaut |
|----------|-------------|--------|
| `API_BOOTSTRAP_TOKEN` | **required** — Token admin initial | — |
| `API_LISTEN_ADDR` | Adresse d'écoute | `0.0.0.0:7080` |
### Indexer
| Variable | Description | Défaut |
|----------|-------------|--------|
| `INDEXER_LISTEN_ADDR` | Adresse d'écoute | `0.0.0.0:7081` |
| `INDEXER_SCAN_INTERVAL_SECONDS` | Intervalle de scan du watcher | `5` |
| `THUMBNAIL_ENABLED` | Activer la génération de thumbnails | `true` |
| `THUMBNAIL_DIRECTORY` | Dossier de stockage des thumbnails | `/data/thumbnails` |
| `THUMBNAIL_WIDTH` | Largeur max des thumbnails (px) | `300` |
| `THUMBNAIL_HEIGHT` | Hauteur max des thumbnails (px) | `400` |
| `THUMBNAIL_QUALITY` | Qualité WebP (0100) | `80` |
| `THUMBNAIL_FORMAT` | Format de sortie | `webp` |
### Backoffice
| Variable | Description | Défaut |
|----------|-------------|--------|
| `API_BOOTSTRAP_TOKEN` | **required** — Token d'accès à l'API | — |
| `API_BASE_URL` | URL interne de l'API (dans le réseau Docker) | `http://api:7080` |
## API Documentation ## API Documentation
The API is documented with OpenAPI/Swagger. When running locally, access the docs at: The API is documented with OpenAPI/Swagger. When running locally, access the docs at:
``` ```
http://localhost:8080/api-docs http://localhost:7080/swagger-ui
``` ```
## Project Structure ## Project Structure
@@ -140,12 +196,95 @@ stripstream-librarian/
│ ├── indexer/ # Rust background indexer │ ├── indexer/ # Rust background indexer
│ └── backoffice/ # Next.js web UI │ └── backoffice/ # Next.js web UI
├── infra/ ├── infra/
│ ├── docker-compose.yml
│ └── migrations/ # SQL database migrations │ └── migrations/ # SQL database migrations
├── libraries/ # Book storage (mounted volume) ├── libraries/ # Book storage (mounted volume)
└── .env # Environment configuration └── .env # Environment configuration
``` ```
## Docker Registry
Images are built and pushed to Docker Hub with the naming convention `docker.io/{owner}/stripstream-{service}`.
### Publishing Images (Maintainers)
To build and push all service images to the registry:
```bash
# Login to Docker Hub first
docker login -u julienfroidefond32
# Build and push all images
./scripts/docker-push.sh
```
This script will:
- Build images for `api`, `indexer`, and `backoffice`
- Tag them with the current version (from `Cargo.toml`) and `latest`
- Push to the registry
### Using Published Images
To use the pre-built images in your own `docker-compose.yml`:
```yaml
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: stripstream
POSTGRES_USER: stripstream
POSTGRES_PASSWORD: stripstream
volumes:
- postgres_data:/var/lib/postgresql/data
api:
image: julienfroidefond32/stripstream-api:latest
ports:
- "7080:7080"
volumes:
- ./libraries:/libraries
- ./data/thumbnails:/data/thumbnails
environment:
# --- Required ---
DATABASE_URL: postgres://stripstream:stripstream@postgres:5432/stripstream
API_BOOTSTRAP_TOKEN: your_bootstrap_token # required — change this
# --- Optional (defaults shown) ---
# API_LISTEN_ADDR: 0.0.0.0:7080
indexer:
image: julienfroidefond32/stripstream-indexer:latest
ports:
- "7081:7081"
volumes:
- ./libraries:/libraries
- ./data/thumbnails:/data/thumbnails
environment:
# --- Required ---
DATABASE_URL: postgres://stripstream:stripstream@postgres:5432/stripstream
# --- Optional (defaults shown) ---
# INDEXER_LISTEN_ADDR: 0.0.0.0:7081
# INDEXER_SCAN_INTERVAL_SECONDS: 5
# THUMBNAIL_ENABLED: true
# THUMBNAIL_DIRECTORY: /data/thumbnails
# THUMBNAIL_WIDTH: 300
# THUMBNAIL_HEIGHT: 400
# THUMBNAIL_QUALITY: 80
# THUMBNAIL_FORMAT: webp
backoffice:
image: julienfroidefond32/stripstream-backoffice:latest
ports:
- "7082:7082"
environment:
# --- Required ---
API_BOOTSTRAP_TOKEN: your_bootstrap_token # must match api above
# --- Optional (defaults shown) ---
# API_BASE_URL: http://api:7080
volumes:
postgres_data:
```
## License ## License
[Your License Here] This project is licensed under the [MIT License](LICENSE).

73
apps/api/AGENTS.md Normal file
View File

@@ -0,0 +1,73 @@
# apps/api — REST API (axum)
Service HTTP sur le port **7080**. Voir `AGENTS.md` racine pour les conventions globales.
## Structure des fichiers
| Fichier | Rôle |
|---------|------|
| `main.rs` | Routes, initialisation AppState, Semaphore concurrent_renders |
| `state.rs` | `AppState` (pool, caches, métriques), `load_concurrent_renders` |
| `auth.rs` | Middlewares `require_admin` / `require_read`, authentification tokens |
| `error.rs` | `ApiError` avec constructeurs `bad_request`, `not_found`, `internal`, etc. |
| `books.rs` | CRUD livres, thumbnails |
| `pages.rs` | Rendu page + double cache (mémoire LRU + disque) |
| `libraries.rs` | CRUD bibliothèques, déclenchement scans |
| `index_jobs.rs` | Suivi jobs, SSE streaming progression |
| `thumbnails.rs` | Rebuild/regénération thumbnails |
| `tokens.rs` | Gestion tokens API (create/revoke) |
| `settings.rs` | Paramètres applicatifs (stockés en DB, clé `limits`) |
| `openapi.rs` | Doc OpenAPI via utoipa, accessible sur `/swagger-ui` |
## Patterns clés
### Handler type
```rust
async fn my_handler(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<Json<MyDto>, ApiError> {
// ...
}
```
### Erreurs API
```rust
// Constructeurs disponibles dans error.rs
ApiError::bad_request("message")
ApiError::not_found("resource not found")
ApiError::internal("unexpected error")
ApiError::unauthorized("missing token")
ApiError::forbidden("admin required")
// Conversion auto depuis sqlx::Error et std::io::Error
```
### Authentification
- **Bootstrap token** : comparaison directe (`API_BOOTSTRAP_TOKEN`), scope Admin
- **Tokens DB** : format `stl_<prefix>_<secret>`, hash argon2 en DB, scope `admin` ou `read`
- Middleware `require_admin` → routes admin ; `require_read` → routes lecture
### OpenAPI (utoipa)
```rust
#[utoipa::path(get, path = "/books/{id}", ...)]
async fn get_book(...) { }
// Ajouter le handler dans openapi.rs (ApiDoc)
```
### Cache pages (`pages.rs`)
- **Cache mémoire** : LRU 512 entrées (`AppState.page_cache`)
- **Cache disque** : `IMAGE_CACHE_DIR` (défaut `/tmp/stripstream-image-cache`), clé SHA256
- Concurrence limitée par `AppState.page_render_limit` (Semaphore, configurable en DB)
- `spawn_blocking` pour le rendu image (CPU-bound)
### Paramètre concurrent_renders
Stocké en DB : `SELECT value FROM app_settings WHERE key = 'limits'` → JSON `{"concurrent_renders": N}`.
Chargé au démarrage dans `load_concurrent_renders`.
## Gotchas
- **LIBRARIES_ROOT_PATH** : les `abs_path` en DB commencent par `/libraries/`. Appeler `remap_libraries_path()` avant tout accès fichier.
- **Rate limit lecture** : middleware `read_rate_limit` sur les routes read (100 req/5s par défaut).
- **Métriques** : `/metrics` expose `requests_total`, `page_cache_hits`, `page_cache_misses` (atomics dans `AppState.metrics`).
- **Swagger** : accessible sur `/swagger-ui`, spec JSON sur `/openapi.json`.

View File

@@ -13,10 +13,14 @@ async-stream = "0.3"
chrono.workspace = true chrono.workspace = true
futures = "0.3" futures = "0.3"
image.workspace = true image.workspace = true
jpeg-decoder.workspace = true
lru.workspace = true lru.workspace = true
notifications = { path = "../../crates/notifications" }
stripstream-core = { path = "../../crates/core" } stripstream-core = { path = "../../crates/core" }
parsers = { path = "../../crates/parsers" }
rand.workspace = true rand.workspace = true
tokio-stream = "0.1" tokio-stream = "0.1"
regex = "1"
reqwest.workspace = true reqwest.workspace = true
serde.workspace = true serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
@@ -28,8 +32,7 @@ tower-http = { version = "0.6", features = ["cors"] }
tracing.workspace = true tracing.workspace = true
tracing-subscriber.workspace = true tracing-subscriber.workspace = true
uuid.workspace = true uuid.workspace = true
zip = { version = "2.2", default-features = false, features = ["deflate"] }
utoipa.workspace = true utoipa.workspace = true
utoipa-swagger-ui = { workspace = true, features = ["axum"] } utoipa-swagger-ui = { workspace = true, features = ["axum"] }
webp = "0.3" webp.workspace = true
walkdir = "2" scraper.workspace = true

View File

@@ -1,30 +1,68 @@
FROM rust:1-bookworm AS builder FROM rust:1-bookworm AS builder
WORKDIR /app WORKDIR /app
# Install sccache for faster builds # Copy workspace manifests and create dummy source files to cache dependency builds
RUN cargo install sccache --locked
ENV RUSTC_WRAPPER=sccache
ENV SCCACHE_DIR=/sccache
COPY Cargo.toml ./ COPY Cargo.toml ./
COPY apps/api/Cargo.toml apps/api/Cargo.toml COPY apps/api/Cargo.toml apps/api/Cargo.toml
COPY apps/indexer/Cargo.toml apps/indexer/Cargo.toml COPY apps/indexer/Cargo.toml apps/indexer/Cargo.toml
COPY crates/core/Cargo.toml crates/core/Cargo.toml COPY crates/core/Cargo.toml crates/core/Cargo.toml
COPY crates/notifications/Cargo.toml crates/notifications/Cargo.toml
COPY crates/parsers/Cargo.toml crates/parsers/Cargo.toml COPY crates/parsers/Cargo.toml crates/parsers/Cargo.toml
RUN mkdir -p apps/api/src apps/indexer/src crates/core/src crates/notifications/src crates/parsers/src && \
echo "fn main() {}" > apps/api/src/main.rs && \
echo "fn main() {}" > apps/indexer/src/main.rs && \
echo "" > apps/indexer/src/lib.rs && \
echo "" > crates/core/src/lib.rs && \
echo "" > crates/notifications/src/lib.rs && \
echo "" > crates/parsers/src/lib.rs
# Build dependencies only (cached as long as Cargo.toml files don't change)
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/app/target \
cargo build --release -p api && \
cargo install sqlx-cli --no-default-features --features postgres --locked
# Copy real source code and build
COPY apps/api/src apps/api/src COPY apps/api/src apps/api/src
COPY apps/indexer/src apps/indexer/src COPY apps/indexer/src apps/indexer/src
COPY crates/core/src crates/core/src COPY crates/core/src crates/core/src
COPY crates/notifications/src crates/notifications/src
COPY crates/parsers/src crates/parsers/src COPY crates/parsers/src crates/parsers/src
# Build with sccache (cache persisted between builds via Docker cache mount) RUN --mount=type=cache,target=/usr/local/cargo/registry \
RUN --mount=type=cache,target=/sccache \ --mount=type=cache,target=/usr/local/cargo/git \
cargo build --release -p api --mount=type=cache,target=/app/target \
touch apps/api/src/main.rs crates/core/src/lib.rs crates/notifications/src/lib.rs crates/parsers/src/lib.rs && \
cargo build --release -p api && \
cp /app/target/release/api /usr/local/bin/api
FROM debian:bookworm-slim FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates wget unar poppler-utils locales && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates wget locales postgresql-client \
&& rm -rf /var/lib/apt/lists/*
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8 ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8 ENV LC_ALL=en_US.UTF-8
COPY --from=builder /app/target/release/api /usr/local/bin/api
EXPOSE 8080 # Download pdfium shared library (replaces pdftoppm subprocess)
CMD ["/usr/local/bin/api"] RUN ARCH=$(dpkg --print-architecture) && \
case "$ARCH" in \
amd64) PDFIUM_ARCH="linux-x64" ;; \
arm64) PDFIUM_ARCH="linux-arm64" ;; \
*) echo "Unsupported arch: $ARCH" && exit 1 ;; \
esac && \
wget -q "https://github.com/bblanchon/pdfium-binaries/releases/latest/download/pdfium-${PDFIUM_ARCH}.tgz" -O /tmp/pdfium.tgz && \
tar -xzf /tmp/pdfium.tgz -C /tmp && \
cp /tmp/lib/libpdfium.so /usr/local/lib/ && \
rm -rf /tmp/pdfium.tgz /tmp/lib /tmp/include && \
ldconfig
COPY --from=builder /usr/local/bin/api /usr/local/bin/api
COPY --from=builder /usr/local/cargo/bin/sqlx /usr/local/bin/sqlx
COPY infra/migrations /app/migrations
COPY apps/api/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
EXPOSE 7080
CMD ["/usr/local/bin/entrypoint.sh"]

63
apps/api/entrypoint.sh Normal file
View File

@@ -0,0 +1,63 @@
#!/bin/sh
set -e
# psql requires "postgresql://" but Rust/sqlx accepts both "postgres://" and "postgresql://"
PSQL_URL=$(echo "$DATABASE_URL" | sed 's|^postgres://|postgresql://|')
# Check 1: does the old schema exist (index_jobs table)?
HAS_OLD_TABLES=$(psql "$PSQL_URL" -tAc \
"SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_name='index_jobs')::text" \
2>/dev/null || echo "false")
# Check 2: is sqlx tracking present and non-empty?
HAS_SQLX_TABLE=$(psql "$PSQL_URL" -tAc \
"SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_name='_sqlx_migrations')::text" \
2>/dev/null || echo "false")
if [ "$HAS_SQLX_TABLE" = "true" ]; then
HAS_SQLX_ROWS=$(psql "$PSQL_URL" -tAc \
"SELECT EXISTS(SELECT 1 FROM _sqlx_migrations LIMIT 1)::text" \
2>/dev/null || echo "false")
else
HAS_SQLX_ROWS="false"
fi
echo "==> Migration check: old_tables=$HAS_OLD_TABLES sqlx_table=$HAS_SQLX_TABLE sqlx_rows=$HAS_SQLX_ROWS"
if [ "$HAS_OLD_TABLES" = "true" ] && [ "$HAS_SQLX_ROWS" = "false" ]; then
echo "==> Upgrade from pre-sqlx migration system detected: creating baseline..."
psql "$PSQL_URL" -c "
CREATE TABLE IF NOT EXISTS _sqlx_migrations (
version BIGINT PRIMARY KEY,
description TEXT NOT NULL,
installed_on TIMESTAMPTZ NOT NULL DEFAULT NOW(),
success BOOLEAN NOT NULL,
checksum BYTEA NOT NULL,
execution_time BIGINT NOT NULL
)
"
for f in /app/migrations/*.sql; do
filename=$(basename "$f")
# Strip leading zeros to get the integer version (e.g. "0005" -> "5")
version=$(echo "$filename" | sed 's/^0*//' | cut -d'_' -f1)
description=$(echo "$filename" | sed 's/^[0-9]*_//' | sed 's/\.sql$//')
checksum=$(sha384sum "$f" | awk '{print $1}')
psql "$PSQL_URL" -c "
INSERT INTO _sqlx_migrations (version, description, installed_on, success, checksum, execution_time)
VALUES ($version, '$description', NOW(), TRUE, decode('$checksum', 'hex'), 0)
ON CONFLICT (version) DO NOTHING
"
echo " baselined: $filename"
done
echo "==> Baseline complete."
fi
echo "==> Running migrations..."
sqlx migrate run --source /app/migrations
echo "==> Starting API..."
exec /usr/local/bin/api

View File

@@ -0,0 +1,51 @@
use axum::{
extract::State,
middleware::Next,
response::{IntoResponse, Response},
};
use std::time::Duration;
use std::sync::atomic::Ordering;
use tracing::info;
use crate::state::AppState;
pub async fn request_counter(
State(state): State<AppState>,
req: axum::extract::Request,
next: Next,
) -> Response {
state.metrics.requests_total.fetch_add(1, Ordering::Relaxed);
let method = req.method().clone();
let uri = req.uri().clone();
let start = std::time::Instant::now();
let response = next.run(req).await;
let status = response.status().as_u16();
let elapsed = start.elapsed();
info!("{} {} {} {}ms", method, uri.path(), status, elapsed.as_millis());
response
}
pub async fn read_rate_limit(
State(state): State<AppState>,
req: axum::extract::Request,
next: Next,
) -> Response {
let mut limiter = state.read_rate_limit.lock().await;
if limiter.window_started_at.elapsed() >= Duration::from_secs(1) {
limiter.window_started_at = std::time::Instant::now();
limiter.requests_in_window = 0;
}
let rate_limit = state.settings.read().await.rate_limit_per_second;
if limiter.requests_in_window >= rate_limit {
return (
axum::http::StatusCode::TOO_MANY_REQUESTS,
"rate limit exceeded",
)
.into_response();
}
limiter.requests_in_window += 1;
drop(limiter);
next.run(req).await
}

View File

@@ -8,7 +8,7 @@ use axum::{
use chrono::Utc; use chrono::Utc;
use sqlx::Row; use sqlx::Row;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::AppState};
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub enum Scope { pub enum Scope {
@@ -94,11 +94,15 @@ async fn authenticate(state: &AppState, token: &str) -> Result<Scope, ApiError>
} }
fn parse_prefix(token: &str) -> Option<&str> { fn parse_prefix(token: &str) -> Option<&str> {
let mut parts = token.split('_'); // Format: stl_{8-char prefix}_{secret}
let namespace = parts.next()?; // Base64 URL_SAFE peut contenir '_', donc on ne peut pas splitter aveuglément
let prefix = parts.next()?; let rest = token.strip_prefix("stl_")?;
let secret = parts.next()?; if rest.len() < 10 {
if namespace != "stl" || secret.is_empty() || prefix.len() < 6 { // 8 (prefix) + 1 ('_') + 1 (secret min)
return None;
}
let prefix = &rest[..8];
if rest.as_bytes().get(8) != Some(&b'_') {
return None; return None;
} }
Some(prefix) Some(prefix)

178
apps/api/src/authors.rs Normal file
View File

@@ -0,0 +1,178 @@
use axum::{extract::{Query, State}, Json};
use serde::{Deserialize, Serialize};
use sqlx::Row;
use utoipa::ToSchema;
use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)]
pub struct ListAuthorsQuery {
#[schema(value_type = Option<String>, example = "batman")]
pub q: Option<String>,
#[schema(value_type = Option<i64>, example = 1)]
pub page: Option<i64>,
#[schema(value_type = Option<i64>, example = 20)]
pub limit: Option<i64>,
/// Sort order: "name" (default), "books" (most books first)
#[schema(value_type = Option<String>, example = "books")]
pub sort: Option<String>,
}
#[derive(Serialize, ToSchema)]
pub struct AuthorItem {
pub name: String,
pub book_count: i64,
pub series_count: i64,
}
#[derive(Serialize, ToSchema)]
pub struct AuthorsPageResponse {
pub items: Vec<AuthorItem>,
pub total: i64,
pub page: i64,
pub limit: i64,
}
/// List all unique authors with book/series counts
#[utoipa::path(
get,
path = "/authors",
tag = "authors",
params(
("q" = Option<String>, Query, description = "Search by author name"),
("page" = Option<i64>, Query, description = "Page number (1-based)"),
("limit" = Option<i64>, Query, description = "Items per page (max 100)"),
("sort" = Option<String>, Query, description = "Sort: name (default) or books"),
),
responses(
(status = 200, body = AuthorsPageResponse),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn list_authors(
State(state): State<AppState>,
Query(query): Query<ListAuthorsQuery>,
) -> Result<Json<AuthorsPageResponse>, ApiError> {
let page = query.page.unwrap_or(1).max(1);
let limit = query.limit.unwrap_or(20).clamp(1, 100);
let offset = (page - 1) * limit;
let sort = query.sort.as_deref().unwrap_or("name");
let order_clause = match sort {
"books" => "book_count DESC, name ASC",
_ => "name ASC",
};
let q_pattern = query.q.as_deref()
.filter(|s| !s.trim().is_empty())
.map(|s| format!("%{s}%"));
// Aggregate unique authors from books.authors + books.author + series_metadata.authors
let sql = format!(
r#"
WITH all_authors AS (
SELECT DISTINCT UNNEST(
COALESCE(
NULLIF(authors, '{{}}'),
CASE WHEN author IS NOT NULL AND author != '' THEN ARRAY[author] ELSE ARRAY[]::text[] END
)
) AS name
FROM books
UNION
SELECT DISTINCT UNNEST(authors) AS name
FROM series_metadata
WHERE authors != '{{}}'
),
filtered AS (
SELECT name FROM all_authors
WHERE ($1::text IS NULL OR name ILIKE $1)
),
book_counts AS (
SELECT
f.name AS author_name,
COUNT(DISTINCT b.id) AS book_count
FROM filtered f
LEFT JOIN books b ON (
f.name = ANY(
COALESCE(
NULLIF(b.authors, '{{}}'),
CASE WHEN b.author IS NOT NULL AND b.author != '' THEN ARRAY[b.author] ELSE ARRAY[]::text[] END
)
)
)
GROUP BY f.name
),
series_counts AS (
SELECT
f.name AS author_name,
COUNT(DISTINCT (sm.library_id, sm.name)) AS series_count
FROM filtered f
LEFT JOIN series_metadata sm ON (
f.name = ANY(sm.authors) AND sm.authors != '{{}}'
)
GROUP BY f.name
)
SELECT
f.name,
COALESCE(bc.book_count, 0) AS book_count,
COALESCE(sc.series_count, 0) AS series_count
FROM filtered f
LEFT JOIN book_counts bc ON bc.author_name = f.name
LEFT JOIN series_counts sc ON sc.author_name = f.name
ORDER BY {order_clause}
LIMIT $2 OFFSET $3
"#
);
let count_sql = r#"
WITH all_authors AS (
SELECT DISTINCT UNNEST(
COALESCE(
NULLIF(authors, '{}'),
CASE WHEN author IS NOT NULL AND author != '' THEN ARRAY[author] ELSE ARRAY[]::text[] END
)
) AS name
FROM books
UNION
SELECT DISTINCT UNNEST(authors) AS name
FROM series_metadata
WHERE authors != '{}'
)
SELECT COUNT(*) AS total
FROM all_authors
WHERE ($1::text IS NULL OR name ILIKE $1)
"#;
let (rows, count_row) = tokio::join!(
sqlx::query(&sql)
.bind(q_pattern.as_deref())
.bind(limit)
.bind(offset)
.fetch_all(&state.pool),
sqlx::query(count_sql)
.bind(q_pattern.as_deref())
.fetch_one(&state.pool)
);
let rows = rows.map_err(|e| ApiError::internal(format!("authors query failed: {e}")))?;
let total: i64 = count_row
.map_err(|e| ApiError::internal(format!("authors count failed: {e}")))?
.get("total");
let items: Vec<AuthorItem> = rows
.iter()
.map(|r| AuthorItem {
name: r.get("name"),
book_count: r.get("book_count"),
series_count: r.get("series_count"),
})
.collect();
Ok(Json(AuthorsPageResponse {
items,
total,
page,
limit,
}))
}

View File

@@ -5,7 +5,7 @@ use sqlx::Row;
use uuid::Uuid; use uuid::Uuid;
use utoipa::ToSchema; use utoipa::ToSchema;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, index_jobs::IndexJobResponse, state::AppState};
#[derive(Deserialize, ToSchema)] #[derive(Deserialize, ToSchema)]
pub struct ListBooksQuery { pub struct ListBooksQuery {
@@ -13,12 +13,25 @@ pub struct ListBooksQuery {
pub library_id: Option<Uuid>, pub library_id: Option<Uuid>,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub kind: Option<String>, pub kind: Option<String>,
#[schema(value_type = Option<String>, example = "cbz")]
pub format: Option<String>,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub series: Option<String>, pub series: Option<String>,
#[schema(value_type = Option<String>, example = "unread,reading")]
pub reading_status: Option<String>,
/// Filter by exact author name (matches in authors array or scalar author field)
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub cursor: Option<Uuid>, pub author: Option<String>,
#[schema(value_type = Option<i64>, example = 1)]
pub page: Option<i64>,
#[schema(value_type = Option<i64>, example = 50)] #[schema(value_type = Option<i64>, example = 50)]
pub limit: Option<i64>, pub limit: Option<i64>,
/// Sort order: "title" (default) or "latest" (most recently added first)
#[schema(value_type = Option<String>, example = "latest")]
pub sort: Option<String>,
/// Filter by metadata provider: "linked" (any provider), "unlinked" (no provider), or a specific provider name
#[schema(value_type = Option<String>, example = "linked")]
pub metadata_provider: Option<String>,
} }
#[derive(Serialize, ToSchema)] #[derive(Serialize, ToSchema)]
@@ -28,8 +41,10 @@ pub struct BookItem {
#[schema(value_type = String)] #[schema(value_type = String)]
pub library_id: Uuid, pub library_id: Uuid,
pub kind: String, pub kind: String,
pub format: Option<String>,
pub title: String, pub title: String,
pub author: Option<String>, pub author: Option<String>,
pub authors: Vec<String>,
pub series: Option<String>, pub series: Option<String>,
pub volume: Option<i32>, pub volume: Option<i32>,
pub language: Option<String>, pub language: Option<String>,
@@ -37,13 +52,19 @@ pub struct BookItem {
pub thumbnail_url: Option<String>, pub thumbnail_url: Option<String>,
#[schema(value_type = String)] #[schema(value_type = String)]
pub updated_at: DateTime<Utc>, pub updated_at: DateTime<Utc>,
/// Reading status: "unread", "reading", or "read"
pub reading_status: String,
pub reading_current_page: Option<i32>,
#[schema(value_type = Option<String>)]
pub reading_last_read_at: Option<DateTime<Utc>>,
} }
#[derive(Serialize, ToSchema)] #[derive(Serialize, ToSchema)]
pub struct BooksPage { pub struct BooksPage {
pub items: Vec<BookItem>, pub items: Vec<BookItem>,
#[schema(value_type = Option<String>)] pub total: i64,
pub next_cursor: Option<Uuid>, pub page: i64,
pub limit: i64,
} }
#[derive(Serialize, ToSchema)] #[derive(Serialize, ToSchema)]
@@ -55,6 +76,7 @@ pub struct BookDetails {
pub kind: String, pub kind: String,
pub title: String, pub title: String,
pub author: Option<String>, pub author: Option<String>,
pub authors: Vec<String>,
pub series: Option<String>, pub series: Option<String>,
pub volume: Option<i32>, pub volume: Option<i32>,
pub language: Option<String>, pub language: Option<String>,
@@ -63,6 +85,17 @@ pub struct BookDetails {
pub file_path: Option<String>, pub file_path: Option<String>,
pub file_format: Option<String>, pub file_format: Option<String>,
pub file_parse_status: Option<String>, pub file_parse_status: Option<String>,
/// Reading status: "unread", "reading", or "read"
pub reading_status: String,
pub reading_current_page: Option<i32>,
#[schema(value_type = Option<String>)]
pub reading_last_read_at: Option<DateTime<Utc>>,
pub summary: Option<String>,
pub isbn: Option<String>,
pub publish_date: Option<String>,
/// Fields locked from external metadata sync
#[serde(skip_serializing_if = "Option::is_none")]
pub locked_fields: Option<serde_json::Value>,
} }
/// List books with optional filtering and pagination /// List books with optional filtering and pagination
@@ -72,10 +105,13 @@ pub struct BookDetails {
tag = "books", tag = "books",
params( params(
("library_id" = Option<String>, Query, description = "Filter by library ID"), ("library_id" = Option<String>, Query, description = "Filter by library ID"),
("kind" = Option<String>, Query, description = "Filter by book kind (cbz, cbr, pdf)"), ("kind" = Option<String>, Query, description = "Filter by book kind (cbz, cbr, pdf, epub)"),
("series" = Option<String>, Query, description = "Filter by series name (use 'unclassified' for books without series)"), ("series" = Option<String>, Query, description = "Filter by series name (use 'unclassified' for books without series)"),
("cursor" = Option<String>, Query, description = "Cursor for pagination"), ("reading_status" = Option<String>, Query, description = "Filter by reading status, comma-separated (e.g. 'unread,reading')"),
("limit" = Option<i64>, Query, description = "Max items to return (max 200)"), ("page" = Option<i64>, Query, description = "Page number (1-indexed, default 1)"),
("limit" = Option<i64>, Query, description = "Items per page (max 200, default 50)"),
("sort" = Option<String>, Query, description = "Sort order: 'title' (default) or 'latest' (most recently added first)"),
("metadata_provider" = Option<String>, Query, description = "Filter by metadata provider: 'linked' (any provider), 'unlinked' (no provider), or a specific provider name"),
), ),
responses( responses(
(status = 200, body = BooksPage), (status = 200, body = BooksPage),
@@ -88,82 +124,156 @@ pub async fn list_books(
Query(query): Query<ListBooksQuery>, Query(query): Query<ListBooksQuery>,
) -> Result<Json<BooksPage>, ApiError> { ) -> Result<Json<BooksPage>, ApiError> {
let limit = query.limit.unwrap_or(50).clamp(1, 200); let limit = query.limit.unwrap_or(50).clamp(1, 200);
let page = query.page.unwrap_or(1).max(1);
let offset = (page - 1) * limit;
// Build series filter condition // Parse reading_status CSV → Vec<String>
let series_condition = match query.series.as_deref() { let reading_statuses: Option<Vec<String>> = query.reading_status.as_deref().map(|s| {
Some("unclassified") => "AND (series IS NULL OR series = '')", s.split(',').map(|v| v.trim().to_string()).filter(|v| !v.is_empty()).collect()
Some(_series_name) => "AND series = $5", });
None => "",
// Conditions partagées COUNT et DATA — $1=library_id $2=kind $3=format, puis optionnels
let mut p: usize = 3;
let series_cond = match query.series.as_deref() {
Some("unclassified") => "AND (b.series IS NULL OR b.series = '')".to_string(),
Some(_) => { p += 1; format!("AND b.series = ${p}") }
None => String::new(),
};
let rs_cond = if reading_statuses.is_some() {
p += 1; format!("AND COALESCE(brp.status, 'unread') = ANY(${p})")
} else { String::new() };
let author_cond = if query.author.is_some() {
p += 1; format!("AND (${p} = ANY(COALESCE(NULLIF(b.authors, '{{}}'), CASE WHEN b.author IS NOT NULL AND b.author != '' THEN ARRAY[b.author] ELSE ARRAY[]::text[] END)) OR EXISTS (SELECT 1 FROM series_metadata sm WHERE sm.library_id = b.library_id AND sm.name = b.series AND ${p} = ANY(sm.authors)))")
} else { String::new() };
let metadata_cond = match query.metadata_provider.as_deref() {
Some("unlinked") => "AND eml.id IS NULL".to_string(),
Some("linked") => "AND eml.id IS NOT NULL".to_string(),
Some(_) => { p += 1; format!("AND eml.provider = ${p}") },
None => String::new(),
}; };
let sql = format!( let metadata_links_cte = r#"
r#" metadata_links AS (
SELECT id, library_id, kind, title, author, series, volume, language, page_count, thumbnail_path, updated_at SELECT DISTINCT ON (eml.series_name, eml.library_id)
FROM books eml.series_name, eml.library_id, eml.provider, eml.id
WHERE ($1::uuid IS NULL OR library_id = $1) FROM external_metadata_links eml
AND ($2::text IS NULL OR kind = $2) WHERE eml.status = 'approved'
AND ($3::uuid IS NULL OR id > $3) ORDER BY eml.series_name, eml.library_id, eml.created_at DESC
{} )"#;
ORDER BY
-- Extract text part before numbers (case insensitive) let count_sql = format!(
REGEXP_REPLACE(LOWER(title), '[0-9]+', '', 'g'), r#"WITH {metadata_links_cte}
-- Extract first number group and convert to integer for numeric sort SELECT COUNT(*) FROM books b
COALESCE( LEFT JOIN book_reading_progress brp ON brp.book_id = b.id
(REGEXP_MATCH(LOWER(title), '\d+'))[1]::int, LEFT JOIN metadata_links eml ON eml.series_name = b.series AND eml.library_id = b.library_id
0 WHERE ($1::uuid IS NULL OR b.library_id = $1)
), AND ($2::text IS NULL OR b.kind = $2)
-- Then by full title as fallback AND ($3::text IS NULL OR b.format = $3)
title ASC {series_cond}
LIMIT $4 {rs_cond}
"#, {author_cond}
series_condition {metadata_cond}"#
); );
let mut query_builder = sqlx::query(&sql) let order_clause = if query.sort.as_deref() == Some("latest") {
"b.updated_at DESC".to_string()
} else {
"b.volume NULLS LAST, REGEXP_REPLACE(LOWER(b.title), '[0-9].*$', ''), COALESCE((REGEXP_MATCH(LOWER(b.title), '\\d+'))[1]::int, 0), b.title ASC".to_string()
};
// DATA: mêmes params filtre, puis $N+1=limit $N+2=offset
let limit_p = p + 1;
let offset_p = p + 2;
let data_sql = format!(
r#"
WITH {metadata_links_cte}
SELECT b.id, b.library_id, b.kind, b.format, b.title, b.author, b.authors, b.series, b.volume, b.language, b.page_count, b.thumbnail_path, b.updated_at,
COALESCE(brp.status, 'unread') AS reading_status,
brp.current_page AS reading_current_page,
brp.last_read_at AS reading_last_read_at
FROM books b
LEFT JOIN book_reading_progress brp ON brp.book_id = b.id
LEFT JOIN metadata_links eml ON eml.series_name = b.series AND eml.library_id = b.library_id
WHERE ($1::uuid IS NULL OR b.library_id = $1)
AND ($2::text IS NULL OR b.kind = $2)
AND ($3::text IS NULL OR b.format = $3)
{series_cond}
{rs_cond}
{author_cond}
{metadata_cond}
ORDER BY {order_clause}
LIMIT ${limit_p} OFFSET ${offset_p}
"#
);
let mut count_builder = sqlx::query(&count_sql)
.bind(query.library_id) .bind(query.library_id)
.bind(query.kind.as_deref()) .bind(query.kind.as_deref())
.bind(query.cursor) .bind(query.format.as_deref());
.bind(limit + 1); let mut data_builder = sqlx::query(&data_sql)
.bind(query.library_id)
.bind(query.kind.as_deref())
.bind(query.format.as_deref());
// Bind series parameter if it's not unclassified if let Some(s) = query.series.as_deref() {
if let Some(series) = query.series.as_deref() { if s != "unclassified" {
if series != "unclassified" { count_builder = count_builder.bind(s);
query_builder = query_builder.bind(series); data_builder = data_builder.bind(s);
}
}
if let Some(ref statuses) = reading_statuses {
count_builder = count_builder.bind(statuses.clone());
data_builder = data_builder.bind(statuses.clone());
}
if let Some(ref author) = query.author {
count_builder = count_builder.bind(author.clone());
data_builder = data_builder.bind(author.clone());
}
if let Some(ref mp) = query.metadata_provider {
if mp != "linked" && mp != "unlinked" {
count_builder = count_builder.bind(mp.clone());
data_builder = data_builder.bind(mp.clone());
} }
} }
let rows = query_builder.fetch_all(&state.pool).await?; data_builder = data_builder.bind(limit).bind(offset);
let (count_row, rows) = tokio::try_join!(
count_builder.fetch_one(&state.pool),
data_builder.fetch_all(&state.pool),
)?;
let total: i64 = count_row.get(0);
let mut items: Vec<BookItem> = rows let mut items: Vec<BookItem> = rows
.iter() .iter()
.take(limit as usize)
.map(|row| { .map(|row| {
let thumbnail_path: Option<String> = row.get("thumbnail_path"); let thumbnail_path: Option<String> = row.get("thumbnail_path");
BookItem { BookItem {
id: row.get("id"), id: row.get("id"),
library_id: row.get("library_id"), library_id: row.get("library_id"),
kind: row.get("kind"), kind: row.get("kind"),
format: row.get("format"),
title: row.get("title"), title: row.get("title"),
author: row.get("author"), author: row.get("author"),
authors: row.get::<Vec<String>, _>("authors"),
series: row.get("series"), series: row.get("series"),
volume: row.get("volume"), volume: row.get("volume"),
language: row.get("language"), language: row.get("language"),
page_count: row.get("page_count"), page_count: row.get("page_count"),
thumbnail_url: thumbnail_path.map(|_p| format!("/books/{}/thumbnail", row.get::<Uuid, _>("id"))), thumbnail_url: thumbnail_path.map(|_p| format!("/books/{}/thumbnail", row.get::<Uuid, _>("id"))),
updated_at: row.get("updated_at"), updated_at: row.get("updated_at"),
reading_status: row.get("reading_status"),
reading_current_page: row.get("reading_current_page"),
reading_last_read_at: row.get("reading_last_read_at"),
} }
}) })
.collect(); .collect();
let next_cursor = if rows.len() > limit as usize {
items.last().map(|b| b.id)
} else {
None
};
Ok(Json(BooksPage { Ok(Json(BooksPage {
items: std::mem::take(&mut items), items: std::mem::take(&mut items),
next_cursor, total,
page,
limit,
})) }))
} }
@@ -188,8 +298,11 @@ pub async fn get_book(
) -> Result<Json<BookDetails>, ApiError> { ) -> Result<Json<BookDetails>, ApiError> {
let row = sqlx::query( let row = sqlx::query(
r#" r#"
SELECT b.id, b.library_id, b.kind, b.title, b.author, b.series, b.volume, b.language, b.page_count, b.thumbnail_path, SELECT b.id, b.library_id, b.kind, b.title, b.author, b.authors, b.series, b.volume, b.language, b.page_count, b.thumbnail_path, b.locked_fields, b.summary, b.isbn, b.publish_date,
bf.abs_path, bf.format, bf.parse_status bf.abs_path, bf.format, bf.parse_status,
COALESCE(brp.status, 'unread') AS reading_status,
brp.current_page AS reading_current_page,
brp.last_read_at AS reading_last_read_at
FROM books b FROM books b
LEFT JOIN LATERAL ( LEFT JOIN LATERAL (
SELECT abs_path, format, parse_status SELECT abs_path, format, parse_status
@@ -198,6 +311,7 @@ pub async fn get_book(
ORDER BY updated_at DESC ORDER BY updated_at DESC
LIMIT 1 LIMIT 1
) bf ON TRUE ) bf ON TRUE
LEFT JOIN book_reading_progress brp ON brp.book_id = b.id
WHERE b.id = $1 WHERE b.id = $1
"#, "#,
) )
@@ -213,6 +327,7 @@ pub async fn get_book(
kind: row.get("kind"), kind: row.get("kind"),
title: row.get("title"), title: row.get("title"),
author: row.get("author"), author: row.get("author"),
authors: row.get::<Vec<String>, _>("authors"),
series: row.get("series"), series: row.get("series"),
volume: row.get("volume"), volume: row.get("volume"),
language: row.get("language"), language: row.get("language"),
@@ -221,132 +336,272 @@ pub async fn get_book(
file_path: row.get("abs_path"), file_path: row.get("abs_path"),
file_format: row.get("format"), file_format: row.get("format"),
file_parse_status: row.get("parse_status"), file_parse_status: row.get("parse_status"),
reading_status: row.get("reading_status"),
reading_current_page: row.get("reading_current_page"),
reading_last_read_at: row.get("reading_last_read_at"),
summary: row.get("summary"),
isbn: row.get("isbn"),
publish_date: row.get("publish_date"),
locked_fields: Some(row.get::<serde_json::Value, _>("locked_fields")),
})) }))
} }
#[derive(Serialize, ToSchema)] // ─── Helpers ──────────────────────────────────────────────────────────────────
pub struct SeriesItem {
pub name: String, pub(crate) fn remap_libraries_path(path: &str) -> String {
pub book_count: i64, if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
#[schema(value_type = String)] if path.starts_with("/libraries/") {
pub first_book_id: Uuid, return path.replacen("/libraries", &root, 1);
}
}
path.to_string()
} }
#[derive(Serialize, ToSchema)] fn unmap_libraries_path(path: &str) -> String {
pub struct SeriesPage { if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
pub items: Vec<SeriesItem>, if path.starts_with(&root) {
#[schema(value_type = Option<String>)] return path.replacen(&root, "/libraries", 1);
pub next_cursor: Option<String>, }
}
path.to_string()
} }
#[derive(Deserialize, ToSchema)] // ─── Convert CBR → CBZ ───────────────────────────────────────────────────────
pub struct ListSeriesQuery {
#[schema(value_type = Option<String>)]
pub cursor: Option<String>,
#[schema(value_type = Option<i64>, example = 50)]
pub limit: Option<i64>,
}
/// List all series in a library with pagination /// Enqueue a CBR → CBZ conversion job for a single book
#[utoipa::path( #[utoipa::path(
get, post,
path = "/libraries/{library_id}/series", path = "/books/{id}/convert",
tag = "books", tag = "books",
params( params(
("library_id" = String, Path, description = "Library UUID"), ("id" = String, Path, description = "Book UUID"),
("cursor" = Option<String>, Query, description = "Cursor for pagination (series name)"),
("limit" = Option<i64>, Query, description = "Max items to return (max 200)"),
), ),
responses( responses(
(status = 200, body = SeriesPage), (status = 200, body = IndexJobResponse),
(status = 404, description = "Book not found"),
(status = 409, description = "Book is not CBR, or target CBZ already exists"),
(status = 401, description = "Unauthorized"), (status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"),
), ),
security(("Bearer" = [])) security(("Bearer" = []))
)] )]
pub async fn list_series( pub async fn convert_book(
State(state): State<AppState>, State(state): State<AppState>,
Path(library_id): Path<Uuid>, Path(book_id): Path<Uuid>,
Query(query): Query<ListSeriesQuery>, ) -> Result<Json<IndexJobResponse>, ApiError> {
) -> Result<Json<SeriesPage>, ApiError> { // Fetch book file info
let limit = query.limit.unwrap_or(50).clamp(1, 200); let row = sqlx::query(
let rows = sqlx::query(
r#" r#"
WITH sorted_books AS ( SELECT b.id, bf.abs_path, bf.format
SELECT FROM books b
COALESCE(NULLIF(series, ''), 'unclassified') as name, LEFT JOIN LATERAL (
id, SELECT abs_path, format
-- Natural sort order for books within series FROM book_files
ROW_NUMBER() OVER ( WHERE book_id = b.id
PARTITION BY COALESCE(NULLIF(series, ''), 'unclassified') ORDER BY updated_at DESC
ORDER BY LIMIT 1
REGEXP_REPLACE(LOWER(title), '[0-9]+', '', 'g'), ) bf ON TRUE
COALESCE((REGEXP_MATCH(LOWER(title), '\d+'))[1]::int, 0), WHERE b.id = $1
title ASC
) as rn
FROM books
WHERE library_id = $1
),
series_counts AS (
SELECT
name,
COUNT(*) as book_count
FROM sorted_books
GROUP BY name
)
SELECT
sc.name,
sc.book_count,
sb.id as first_book_id
FROM series_counts sc
JOIN sorted_books sb ON sb.name = sc.name AND sb.rn = 1
WHERE ($2::text IS NULL OR sc.name > $2)
ORDER BY
-- Natural sort: extract text part before numbers
REGEXP_REPLACE(LOWER(sc.name), '[0-9]+', '', 'g'),
-- Extract first number group and convert to integer
COALESCE(
(REGEXP_MATCH(LOWER(sc.name), '\d+'))[1]::int,
0
),
sc.name ASC
LIMIT $3
"#, "#,
) )
.bind(library_id) .bind(book_id)
.bind(query.cursor.as_deref()) .fetch_optional(&state.pool)
.bind(limit + 1)
.fetch_all(&state.pool)
.await?; .await?;
let mut items: Vec<SeriesItem> = rows let row = row.ok_or_else(|| ApiError::not_found("book not found"))?;
.iter() let abs_path: Option<String> = row.get("abs_path");
.take(limit as usize) let format: Option<String> = row.get("format");
.map(|row| SeriesItem {
name: row.get("name"), if format.as_deref() != Some("cbr") {
book_count: row.get("book_count"), return Err(ApiError {
first_book_id: row.get("first_book_id"), status: axum::http::StatusCode::CONFLICT,
}) message: "book is not in CBR format".to_string(),
});
}
let abs_path = abs_path.ok_or_else(|| ApiError::not_found("book file path not found"))?;
// Check for existing CBZ with same stem
let physical_path = remap_libraries_path(&abs_path);
let cbr_path = std::path::Path::new(&physical_path);
if let (Some(parent), Some(stem)) = (cbr_path.parent(), cbr_path.file_stem()) {
let cbz_path = parent.join(format!("{}.cbz", stem.to_string_lossy()));
if cbz_path.exists() {
return Err(ApiError {
status: axum::http::StatusCode::CONFLICT,
message: format!(
"CBZ file already exists: {}",
unmap_libraries_path(&cbz_path.to_string_lossy())
),
});
}
}
// Create the conversion job
let job_id = Uuid::new_v4();
sqlx::query(
"INSERT INTO index_jobs (id, book_id, type, status) VALUES ($1, $2, 'cbr_to_cbz', 'pending')",
)
.bind(job_id)
.bind(book_id)
.execute(&state.pool)
.await?;
let job_row = sqlx::query(
"SELECT id, library_id, book_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files FROM index_jobs WHERE id = $1",
)
.bind(job_id)
.fetch_one(&state.pool)
.await?;
Ok(Json(crate::index_jobs::map_row(job_row)))
}
// ─── Metadata editing ─────────────────────────────────────────────────────────
#[derive(Deserialize, ToSchema)]
pub struct UpdateBookRequest {
pub title: String,
pub author: Option<String>,
#[serde(default)]
pub authors: Vec<String>,
pub series: Option<String>,
pub volume: Option<i32>,
pub language: Option<String>,
pub summary: Option<String>,
pub isbn: Option<String>,
pub publish_date: Option<String>,
/// Fields locked from external metadata sync
#[serde(default)]
pub locked_fields: Option<serde_json::Value>,
}
/// Update metadata for a specific book
#[utoipa::path(
patch,
path = "/books/{id}",
tag = "books",
params(("id" = String, Path, description = "Book UUID")),
request_body = UpdateBookRequest,
responses(
(status = 200, body = BookDetails),
(status = 400, description = "Invalid request"),
(status = 404, description = "Book not found"),
(status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"),
),
security(("Bearer" = []))
)]
pub async fn update_book(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Json(body): Json<UpdateBookRequest>,
) -> Result<Json<BookDetails>, ApiError> {
let title = body.title.trim().to_string();
if title.is_empty() {
return Err(ApiError::bad_request("title cannot be empty"));
}
let author = body.author.as_deref().map(str::trim).filter(|s| !s.is_empty()).map(str::to_string);
let authors: Vec<String> = body.authors.iter()
.map(|a| a.trim().to_string())
.filter(|a| !a.is_empty())
.collect(); .collect();
let series = body.series.as_deref().map(str::trim).filter(|s| !s.is_empty()).map(str::to_string);
let language = body.language.as_deref().map(str::trim).filter(|s| !s.is_empty()).map(str::to_string);
let next_cursor = if rows.len() > limit as usize { let summary = body.summary.as_deref().map(str::trim).filter(|s| !s.is_empty()).map(str::to_string);
items.last().map(|s| s.name.clone()) let isbn = body.isbn.as_deref().map(str::trim).filter(|s| !s.is_empty()).map(str::to_string);
} else { let publish_date = body.publish_date.as_deref().map(str::trim).filter(|s| !s.is_empty()).map(str::to_string);
None let locked_fields = body.locked_fields.clone().unwrap_or(serde_json::json!({}));
}; let row = sqlx::query(
r#"
UPDATE books
SET title = $2, author = $3, authors = $4, series = $5, volume = $6, language = $7,
summary = $8, isbn = $9, publish_date = $10, locked_fields = $11, updated_at = NOW()
WHERE id = $1
RETURNING id, library_id, kind, title, author, authors, series, volume, language, page_count, thumbnail_path,
summary, isbn, publish_date,
COALESCE((SELECT status FROM book_reading_progress WHERE book_id = $1), 'unread') AS reading_status,
(SELECT current_page FROM book_reading_progress WHERE book_id = $1) AS reading_current_page,
(SELECT last_read_at FROM book_reading_progress WHERE book_id = $1) AS reading_last_read_at
"#,
)
.bind(id)
.bind(&title)
.bind(&author)
.bind(&authors)
.bind(&series)
.bind(body.volume)
.bind(&language)
.bind(&summary)
.bind(&isbn)
.bind(&publish_date)
.bind(&locked_fields)
.fetch_optional(&state.pool)
.await?;
Ok(Json(SeriesPage { let row = row.ok_or_else(|| ApiError::not_found("book not found"))?;
items: std::mem::take(&mut items), let thumbnail_path: Option<String> = row.get("thumbnail_path");
next_cursor,
Ok(Json(BookDetails {
id: row.get("id"),
library_id: row.get("library_id"),
kind: row.get("kind"),
title: row.get("title"),
author: row.get("author"),
authors: row.get::<Vec<String>, _>("authors"),
series: row.get("series"),
volume: row.get("volume"),
language: row.get("language"),
page_count: row.get("page_count"),
thumbnail_url: thumbnail_path.map(|_| format!("/books/{}/thumbnail", id)),
file_path: None,
file_format: None,
file_parse_status: None,
reading_status: row.get("reading_status"),
reading_current_page: row.get("reading_current_page"),
reading_last_read_at: row.get("reading_last_read_at"),
summary: row.get("summary"),
isbn: row.get("isbn"),
publish_date: row.get("publish_date"),
locked_fields: Some(locked_fields),
})) }))
} }
// ─── Thumbnail ────────────────────────────────────────────────────────────────
use axum::{ use axum::{
body::Body, body::Body,
http::{header, HeaderMap, HeaderValue, StatusCode}, http::{header, HeaderMap, HeaderValue, StatusCode},
response::IntoResponse, response::IntoResponse,
}; };
/// Detect content type from thumbnail file extension.
fn detect_thumbnail_content_type(path: &str) -> &'static str {
if path.ends_with(".jpg") || path.ends_with(".jpeg") {
"image/jpeg"
} else if path.ends_with(".png") {
"image/png"
} else {
"image/webp"
}
}
/// Get book thumbnail image
#[utoipa::path(
get,
path = "/books/{id}/thumbnail",
tag = "books",
params(
("id" = String, Path, description = "Book UUID"),
),
responses(
(status = 200, description = "WebP thumbnail image", content_type = "image/webp"),
(status = 404, description = "Book not found or thumbnail not available"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_thumbnail( pub async fn get_thumbnail(
State(state): State<AppState>, State(state): State<AppState>,
Path(book_id): Path<Uuid>, Path(book_id): Path<Uuid>,
@@ -360,20 +615,33 @@ pub async fn get_thumbnail(
let row = row.ok_or_else(|| ApiError::not_found("book not found"))?; let row = row.ok_or_else(|| ApiError::not_found("book not found"))?;
let thumbnail_path: Option<String> = row.get("thumbnail_path"); let thumbnail_path: Option<String> = row.get("thumbnail_path");
let data = if let Some(ref path) = thumbnail_path { let (data, content_type) = if let Some(ref path) = thumbnail_path {
std::fs::read(path) match std::fs::read(path) {
.map_err(|e| ApiError::internal(format!("cannot read thumbnail: {}", e)))? Ok(bytes) => {
let ct = detect_thumbnail_content_type(path);
(bytes, ct)
}
Err(_) => {
// File missing on disk (e.g. different mount in dev) — fall back to live render
crate::pages::render_book_page_1(&state, book_id, 300, 80).await?
}
}
} else { } else {
// Fallback: render page 1 on the fly (same as pages logic) // No stored thumbnail yet — render page 1 on the fly
crate::pages::render_book_page_1(&state, book_id, 300, 80).await? crate::pages::render_book_page_1(&state, book_id, 300, 80).await?
}; };
let etag_value = format!("\"{}_{:x}\"", book_id, data.len());
let mut headers = HeaderMap::new(); let mut headers = HeaderMap::new();
headers.insert(header::CONTENT_TYPE, HeaderValue::from_static("image/webp")); headers.insert(header::CONTENT_TYPE, HeaderValue::from_static(content_type));
headers.insert( headers.insert(
header::CACHE_CONTROL, header::CACHE_CONTROL,
HeaderValue::from_static("public, max-age=31536000, immutable"), HeaderValue::from_static("public, max-age=31536000, immutable"),
); );
if let Ok(v) = HeaderValue::from_str(&etag_value) {
headers.insert(header::ETAG, v);
}
Ok((StatusCode::OK, headers, Body::from(data))) Ok((StatusCode::OK, headers, Body::from(data)))
} }

View File

@@ -38,6 +38,13 @@ impl ApiError {
} }
} }
pub fn unprocessable_entity(message: impl Into<String>) -> Self {
Self {
status: StatusCode::UNPROCESSABLE_ENTITY,
message: message.into(),
}
}
pub fn not_found(message: impl Into<String>) -> Self { pub fn not_found(message: impl Into<String>) -> Self {
Self { Self {
status: StatusCode::NOT_FOUND, status: StatusCode::NOT_FOUND,
@@ -76,3 +83,9 @@ impl From<std::io::Error> for ApiError {
Self::internal(format!("IO error: {err}")) Self::internal(format!("IO error: {err}"))
} }
} }
impl From<reqwest::Error> for ApiError {
fn from(err: reqwest::Error) -> Self {
Self::internal(format!("HTTP client error: {err}"))
}
}

26
apps/api/src/handlers.rs Normal file
View File

@@ -0,0 +1,26 @@
use axum::{extract::State, Json};
use std::sync::atomic::Ordering;
use crate::{error::ApiError, state::AppState};
pub async fn health() -> &'static str {
"ok"
}
pub async fn docs_redirect() -> impl axum::response::IntoResponse {
axum::response::Redirect::to("/swagger-ui/")
}
pub async fn ready(State(state): State<AppState>) -> Result<Json<serde_json::Value>, ApiError> {
sqlx::query("SELECT 1").execute(&state.pool).await?;
Ok(Json(serde_json::json!({"status": "ready"})))
}
pub async fn metrics(State(state): State<AppState>) -> String {
format!(
"requests_total {}\npage_cache_hits {}\npage_cache_misses {}\n",
state.metrics.requests_total.load(Ordering::Relaxed),
state.metrics.page_cache_hits.load(Ordering::Relaxed),
state.metrics.page_cache_misses.load(Ordering::Relaxed),
)
}

View File

@@ -8,7 +8,7 @@ use tokio_stream::Stream;
use uuid::Uuid; use uuid::Uuid;
use utoipa::ToSchema; use utoipa::ToSchema;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)] #[derive(Deserialize, ToSchema)]
pub struct RebuildRequest { pub struct RebuildRequest {
@@ -16,6 +16,10 @@ pub struct RebuildRequest {
pub library_id: Option<Uuid>, pub library_id: Option<Uuid>,
#[schema(value_type = Option<bool>, example = false)] #[schema(value_type = Option<bool>, example = false)]
pub full: Option<bool>, pub full: Option<bool>,
/// Deep rescan: clears directory mtimes to force re-walking all directories,
/// discovering newly supported formats without deleting existing data.
#[schema(value_type = Option<bool>, example = false)]
pub rescan: Option<bool>,
} }
#[derive(Serialize, ToSchema)] #[derive(Serialize, ToSchema)]
@@ -24,6 +28,8 @@ pub struct IndexJobResponse {
pub id: Uuid, pub id: Uuid,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub library_id: Option<Uuid>, pub library_id: Option<Uuid>,
#[schema(value_type = Option<String>)]
pub book_id: Option<Uuid>,
pub r#type: String, pub r#type: String,
pub status: String, pub status: String,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
@@ -53,12 +59,18 @@ pub struct IndexJobDetailResponse {
pub id: Uuid, pub id: Uuid,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub library_id: Option<Uuid>, pub library_id: Option<Uuid>,
#[schema(value_type = Option<String>)]
pub book_id: Option<Uuid>,
pub r#type: String, pub r#type: String,
pub status: String, pub status: String,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub started_at: Option<DateTime<Utc>>, pub started_at: Option<DateTime<Utc>>,
#[schema(value_type = Option<String>)] #[schema(value_type = Option<String>)]
pub finished_at: Option<DateTime<Utc>>, pub finished_at: Option<DateTime<Utc>>,
#[schema(value_type = Option<String>)]
pub phase2_started_at: Option<DateTime<Utc>>,
#[schema(value_type = Option<String>)]
pub generating_thumbnails_started_at: Option<DateTime<Utc>>,
pub stats_json: Option<serde_json::Value>, pub stats_json: Option<serde_json::Value>,
pub error_opt: Option<String>, pub error_opt: Option<String>,
#[schema(value_type = String)] #[schema(value_type = String)]
@@ -109,7 +121,8 @@ pub async fn enqueue_rebuild(
) -> Result<Json<IndexJobResponse>, ApiError> { ) -> Result<Json<IndexJobResponse>, ApiError> {
let library_id = payload.as_ref().and_then(|p| p.0.library_id); let library_id = payload.as_ref().and_then(|p| p.0.library_id);
let is_full = payload.as_ref().and_then(|p| p.0.full).unwrap_or(false); let is_full = payload.as_ref().and_then(|p| p.0.full).unwrap_or(false);
let job_type = if is_full { "full_rebuild" } else { "rebuild" }; let is_rescan = payload.as_ref().and_then(|p| p.0.rescan).unwrap_or(false);
let job_type = if is_full { "full_rebuild" } else if is_rescan { "rescan" } else { "rebuild" };
let id = Uuid::new_v4(); let id = Uuid::new_v4();
sqlx::query( sqlx::query(
@@ -122,7 +135,7 @@ pub async fn enqueue_rebuild(
.await?; .await?;
let row = sqlx::query( let row = sqlx::query(
"SELECT id, library_id, type, status, started_at, finished_at, stats_json, error_opt, created_at FROM index_jobs WHERE id = $1", "SELECT id, library_id, book_id, type, status, started_at, finished_at, stats_json, error_opt, created_at FROM index_jobs WHERE id = $1",
) )
.bind(id) .bind(id)
.fetch_one(&state.pool) .fetch_one(&state.pool)
@@ -145,7 +158,7 @@ pub async fn enqueue_rebuild(
)] )]
pub async fn list_index_jobs(State(state): State<AppState>) -> Result<Json<Vec<IndexJobResponse>>, ApiError> { pub async fn list_index_jobs(State(state): State<AppState>) -> Result<Json<Vec<IndexJobResponse>>, ApiError> {
let rows = sqlx::query( let rows = sqlx::query(
"SELECT id, library_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files FROM index_jobs ORDER BY created_at DESC LIMIT 100", "SELECT id, library_id, book_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files FROM index_jobs ORDER BY created_at DESC LIMIT 100",
) )
.fetch_all(&state.pool) .fetch_all(&state.pool)
.await?; .await?;
@@ -174,7 +187,7 @@ pub async fn cancel_job(
id: axum::extract::Path<Uuid>, id: axum::extract::Path<Uuid>,
) -> Result<Json<IndexJobResponse>, ApiError> { ) -> Result<Json<IndexJobResponse>, ApiError> {
let rows_affected = sqlx::query( let rows_affected = sqlx::query(
"UPDATE index_jobs SET status = 'cancelled' WHERE id = $1 AND status IN ('pending', 'running', 'generating_thumbnails')", "UPDATE index_jobs SET status = 'cancelled' WHERE id = $1 AND status IN ('pending', 'running', 'extracting_pages', 'generating_thumbnails')",
) )
.bind(id.0) .bind(id.0)
.execute(&state.pool) .execute(&state.pool)
@@ -185,7 +198,7 @@ pub async fn cancel_job(
} }
let row = sqlx::query( let row = sqlx::query(
"SELECT id, library_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files FROM index_jobs WHERE id = $1", "SELECT id, library_id, book_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files FROM index_jobs WHERE id = $1",
) )
.bind(id.0) .bind(id.0)
.fetch_one(&state.pool) .fetch_one(&state.pool)
@@ -238,16 +251,16 @@ pub async fn list_folders(
base_path.to_path_buf() base_path.to_path_buf()
}; };
// Ensure the path is within the libraries root // Ensure the path is within the libraries root (avoid canonicalize — burns fd on Docker mounts)
let canonical_target = target_path.canonicalize().unwrap_or(target_path.clone()); let canonical_target = target_path.clone();
let canonical_base = base_path.canonicalize().unwrap_or(base_path.to_path_buf()); let canonical_base = base_path.to_path_buf();
if !canonical_target.starts_with(&canonical_base) { if !canonical_target.starts_with(&canonical_base) {
return Err(ApiError::bad_request("Path is outside libraries root")); return Err(ApiError::bad_request("Path is outside libraries root"));
} }
let mut folders = Vec::new(); let mut folders = Vec::new();
let depth = if params.get("path").is_some() { let depth = if params.contains_key("path") {
canonical_target.strip_prefix(&canonical_base) canonical_target.strip_prefix(&canonical_base)
.map(|p| p.components().count()) .map(|p| p.components().count())
.unwrap_or(0) .unwrap_or(0)
@@ -255,19 +268,31 @@ pub async fn list_folders(
0 0
}; };
if let Ok(entries) = std::fs::read_dir(&canonical_target) { let entries = std::fs::read_dir(&canonical_target)
for entry in entries.flatten() { .map_err(|e| ApiError::internal(format!("cannot read directory {}: {}", canonical_target.display(), e)))?;
if entry.file_type().map(|ft| ft.is_dir()).unwrap_or(false) {
for entry in entries {
let entry = match entry {
Ok(e) => e,
Err(e) => {
tracing::warn!("[FOLDERS] entry error in {}: {}", canonical_target.display(), e);
continue;
}
};
let is_dir = match entry.file_type() {
Ok(ft) => ft.is_dir(),
Err(e) => {
tracing::warn!("[FOLDERS] cannot stat {}: {}", entry.path().display(), e);
continue;
}
};
if is_dir {
let name = entry.file_name().to_string_lossy().to_string(); let name = entry.file_name().to_string_lossy().to_string();
// Check if this folder has children // Check if this folder has children (best-effort, default to true on error)
let has_children = if let Ok(sub_entries) = std::fs::read_dir(entry.path()) { let has_children = std::fs::read_dir(entry.path())
sub_entries.flatten().any(|e| { .map(|sub| sub.flatten().any(|e| e.file_type().map(|ft| ft.is_dir()).unwrap_or(false)))
e.file_type().map(|ft| ft.is_dir()).unwrap_or(false) .unwrap_or(true);
})
} else {
false
};
// Calculate the full path relative to libraries root // Calculate the full path relative to libraries root
let full_path = if let Ok(relative) = entry.path().strip_prefix(&canonical_base) { let full_path = if let Ok(relative) = entry.path().strip_prefix(&canonical_base) {
@@ -282,7 +307,6 @@ pub async fn list_folders(
depth, depth,
has_children, has_children,
}); });
}
} }
} }
@@ -294,6 +318,7 @@ pub fn map_row(row: sqlx::postgres::PgRow) -> IndexJobResponse {
IndexJobResponse { IndexJobResponse {
id: row.get("id"), id: row.get("id"),
library_id: row.get("library_id"), library_id: row.get("library_id"),
book_id: row.try_get("book_id").ok().flatten(),
r#type: row.get("type"), r#type: row.get("type"),
status: row.get("status"), status: row.get("status"),
started_at: row.get("started_at"), started_at: row.get("started_at"),
@@ -311,10 +336,13 @@ fn map_row_detail(row: sqlx::postgres::PgRow) -> IndexJobDetailResponse {
IndexJobDetailResponse { IndexJobDetailResponse {
id: row.get("id"), id: row.get("id"),
library_id: row.get("library_id"), library_id: row.get("library_id"),
book_id: row.try_get("book_id").ok().flatten(),
r#type: row.get("type"), r#type: row.get("type"),
status: row.get("status"), status: row.get("status"),
started_at: row.get("started_at"), started_at: row.get("started_at"),
finished_at: row.get("finished_at"), finished_at: row.get("finished_at"),
phase2_started_at: row.try_get("phase2_started_at").ok().flatten(),
generating_thumbnails_started_at: row.try_get("generating_thumbnails_started_at").ok().flatten(),
stats_json: row.get("stats_json"), stats_json: row.get("stats_json"),
error_opt: row.get("error_opt"), error_opt: row.get("error_opt"),
created_at: row.get("created_at"), created_at: row.get("created_at"),
@@ -339,9 +367,9 @@ fn map_row_detail(row: sqlx::postgres::PgRow) -> IndexJobDetailResponse {
)] )]
pub async fn get_active_jobs(State(state): State<AppState>) -> Result<Json<Vec<IndexJobResponse>>, ApiError> { pub async fn get_active_jobs(State(state): State<AppState>) -> Result<Json<Vec<IndexJobResponse>>, ApiError> {
let rows = sqlx::query( let rows = sqlx::query(
"SELECT id, library_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files "SELECT id, library_id, book_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, progress_percent, processed_files, total_files
FROM index_jobs FROM index_jobs
WHERE status IN ('pending', 'running', 'generating_thumbnails') WHERE status IN ('pending', 'running', 'extracting_pages', 'generating_thumbnails')
ORDER BY created_at ASC" ORDER BY created_at ASC"
) )
.fetch_all(&state.pool) .fetch_all(&state.pool)
@@ -371,8 +399,8 @@ pub async fn get_job_details(
id: axum::extract::Path<Uuid>, id: axum::extract::Path<Uuid>,
) -> Result<Json<IndexJobDetailResponse>, ApiError> { ) -> Result<Json<IndexJobDetailResponse>, ApiError> {
let row = sqlx::query( let row = sqlx::query(
"SELECT id, library_id, type, status, started_at, finished_at, stats_json, error_opt, created_at, "SELECT id, library_id, book_id, type, status, started_at, finished_at, phase2_started_at, generating_thumbnails_started_at,
current_file, progress_percent, total_files, processed_files stats_json, error_opt, created_at, current_file, progress_percent, total_files, processed_files
FROM index_jobs WHERE id = $1" FROM index_jobs WHERE id = $1"
) )
.bind(id.0) .bind(id.0)

398
apps/api/src/komga.rs Normal file
View File

@@ -0,0 +1,398 @@
use axum::{extract::State, Json};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::Row;
use std::collections::HashMap;
use utoipa::ToSchema;
use uuid::Uuid;
use crate::{error::ApiError, state::AppState};
// ─── Komga API types ─────────────────────────────────────────────────────────
#[derive(Deserialize)]
struct KomgaBooksResponse {
content: Vec<KomgaBook>,
#[serde(rename = "totalPages")]
total_pages: i32,
number: i32,
}
#[derive(Deserialize)]
struct KomgaBook {
name: String,
#[serde(rename = "seriesTitle")]
series_title: String,
metadata: KomgaBookMetadata,
}
#[derive(Deserialize)]
struct KomgaBookMetadata {
title: String,
}
// ─── Request / Response ──────────────────────────────────────────────────────
#[derive(Deserialize, ToSchema)]
pub struct KomgaSyncRequest {
pub url: String,
pub username: String,
pub password: String,
}
#[derive(Serialize, ToSchema)]
pub struct KomgaSyncResponse {
#[schema(value_type = String)]
pub id: Uuid,
pub komga_url: String,
pub total_komga_read: i64,
pub matched: i64,
pub already_read: i64,
pub newly_marked: i64,
pub matched_books: Vec<String>,
pub newly_marked_books: Vec<String>,
pub unmatched: Vec<String>,
#[schema(value_type = String)]
pub created_at: DateTime<Utc>,
}
#[derive(Serialize, ToSchema)]
pub struct KomgaSyncReportSummary {
#[schema(value_type = String)]
pub id: Uuid,
pub komga_url: String,
pub total_komga_read: i64,
pub matched: i64,
pub already_read: i64,
pub newly_marked: i64,
pub unmatched_count: i32,
#[schema(value_type = String)]
pub created_at: DateTime<Utc>,
}
// ─── Handlers ────────────────────────────────────────────────────────────────
/// Sync read books from a Komga server
#[utoipa::path(
post,
path = "/komga/sync",
tag = "komga",
request_body = KomgaSyncRequest,
responses(
(status = 200, body = KomgaSyncResponse),
(status = 400, description = "Bad request"),
(status = 401, description = "Unauthorized"),
(status = 500, description = "Komga connection or sync error"),
),
security(("Bearer" = []))
)]
pub async fn sync_komga_read_books(
State(state): State<AppState>,
Json(body): Json<KomgaSyncRequest>,
) -> Result<Json<KomgaSyncResponse>, ApiError> {
let url = body.url.trim_end_matches('/').to_string();
if url.is_empty() {
return Err(ApiError::bad_request("url is required"));
}
// Build HTTP client with basic auth
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.map_err(|e| ApiError::internal(format!("failed to build HTTP client: {e}")))?;
// Paginate through all READ books from Komga
let mut komga_books: Vec<(String, String)> = Vec::new(); // (series_title, title)
let mut page = 0;
let page_size = 100;
let max_pages = 500;
loop {
let resp = client
.post(format!("{url}/api/v1/books/list?page={page}&size={page_size}"))
.basic_auth(&body.username, Some(&body.password))
.header("Content-Type", "application/json")
.json(&serde_json::json!({ "condition": { "readStatus": { "operator": "is", "value": "READ" } } }))
.send()
.await
.map_err(|e| ApiError::internal(format!("Komga request failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(ApiError::internal(format!(
"Komga returned {status}: {text}"
)));
}
let data: KomgaBooksResponse = resp
.json()
.await
.map_err(|e| ApiError::internal(format!("Failed to parse Komga response: {e}")))?;
for book in &data.content {
let title = if !book.metadata.title.is_empty() {
&book.metadata.title
} else {
&book.name
};
komga_books.push((book.series_title.clone(), title.clone()));
}
if data.number >= data.total_pages - 1 || page >= max_pages {
break;
}
page += 1;
}
let total_komga_read = komga_books.len() as i64;
// Build local lookup maps
let rows = sqlx::query(
"SELECT id, title, COALESCE(series, '') as series, LOWER(title) as title_lower, LOWER(COALESCE(series, '')) as series_lower FROM books",
)
.fetch_all(&state.pool)
.await?;
type BookEntry = (Uuid, String, String);
// Primary: (series_lower, title_lower) -> Vec<(Uuid, title, series)>
let mut primary_map: HashMap<(String, String), Vec<BookEntry>> = HashMap::new();
// Secondary: title_lower -> Vec<(Uuid, title, series)>
let mut secondary_map: HashMap<String, Vec<BookEntry>> = HashMap::new();
for row in &rows {
let id: Uuid = row.get("id");
let title: String = row.get("title");
let series: String = row.get("series");
let title_lower: String = row.get("title_lower");
let series_lower: String = row.get("series_lower");
let entry = (id, title, series);
primary_map
.entry((series_lower, title_lower.clone()))
.or_default()
.push(entry.clone());
secondary_map.entry(title_lower).or_default().push(entry);
}
// Match Komga books to local books
let mut matched_entries: Vec<(Uuid, String)> = Vec::new(); // (id, display_title)
let mut unmatched: Vec<String> = Vec::new();
for (series_title, title) in &komga_books {
let title_lower = title.to_lowercase();
let series_lower = series_title.to_lowercase();
let found = if let Some(entries) = primary_map.get(&(series_lower.clone(), title_lower.clone())) {
Some(entries)
} else {
secondary_map.get(&title_lower)
};
if let Some(entries) = found {
for (id, local_title, local_series) in entries {
let display = if local_series.is_empty() {
local_title.clone()
} else {
format!("{local_series} - {local_title}")
};
matched_entries.push((*id, display));
}
} else if series_title.is_empty() {
unmatched.push(title.clone());
} else {
unmatched.push(format!("{series_title} - {title}"));
}
}
// Deduplicate by ID
matched_entries.sort_by(|a, b| a.0.cmp(&b.0));
matched_entries.dedup_by(|a, b| a.0 == b.0);
let matched_ids: Vec<Uuid> = matched_entries.iter().map(|(id, _)| *id).collect();
let matched = matched_ids.len() as i64;
let mut already_read: i64 = 0;
let mut already_read_ids: std::collections::HashSet<Uuid> = std::collections::HashSet::new();
if !matched_ids.is_empty() {
// Get already-read book IDs
let ar_rows = sqlx::query(
"SELECT book_id FROM book_reading_progress WHERE book_id = ANY($1) AND status = 'read'",
)
.bind(&matched_ids)
.fetch_all(&state.pool)
.await?;
for row in &ar_rows {
already_read_ids.insert(row.get("book_id"));
}
already_read = already_read_ids.len() as i64;
// Bulk upsert all matched books as read
sqlx::query(
r#"
INSERT INTO book_reading_progress (book_id, status, current_page, last_read_at, updated_at)
SELECT unnest($1::uuid[]), 'read', NULL, NOW(), NOW()
ON CONFLICT (book_id) DO UPDATE
SET status = 'read',
current_page = NULL,
last_read_at = NOW(),
updated_at = NOW()
WHERE book_reading_progress.status != 'read'
"#,
)
.bind(&matched_ids)
.execute(&state.pool)
.await?;
}
let newly_marked = matched - already_read;
// Build matched_books and newly_marked_books lists
let mut newly_marked_books: Vec<String> = Vec::new();
let mut matched_books: Vec<String> = Vec::new();
for (id, title) in &matched_entries {
if !already_read_ids.contains(id) {
newly_marked_books.push(title.clone());
}
matched_books.push(title.clone());
}
// Sort: newly marked first, then alphabetical
let newly_marked_set: std::collections::HashSet<&str> =
newly_marked_books.iter().map(|s| s.as_str()).collect();
matched_books.sort_by(|a, b| {
let a_new = newly_marked_set.contains(a.as_str());
let b_new = newly_marked_set.contains(b.as_str());
b_new.cmp(&a_new).then(a.cmp(b))
});
newly_marked_books.sort();
// Save sync report
let unmatched_json = serde_json::to_value(&unmatched).unwrap_or_default();
let matched_books_json = serde_json::to_value(&matched_books).unwrap_or_default();
let newly_marked_books_json = serde_json::to_value(&newly_marked_books).unwrap_or_default();
let report_row = sqlx::query(
r#"
INSERT INTO komga_sync_reports (komga_url, total_komga_read, matched, already_read, newly_marked, matched_books, newly_marked_books, unmatched)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING id, created_at
"#,
)
.bind(&url)
.bind(total_komga_read)
.bind(matched)
.bind(already_read)
.bind(newly_marked)
.bind(&matched_books_json)
.bind(&newly_marked_books_json)
.bind(&unmatched_json)
.fetch_one(&state.pool)
.await?;
Ok(Json(KomgaSyncResponse {
id: report_row.get("id"),
komga_url: url,
total_komga_read,
matched,
already_read,
newly_marked,
matched_books,
newly_marked_books,
unmatched,
created_at: report_row.get("created_at"),
}))
}
/// List Komga sync reports (most recent first)
#[utoipa::path(
get,
path = "/komga/reports",
tag = "komga",
responses(
(status = 200, body = Vec<KomgaSyncReportSummary>),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn list_sync_reports(
State(state): State<AppState>,
) -> Result<Json<Vec<KomgaSyncReportSummary>>, ApiError> {
let rows = sqlx::query(
r#"
SELECT id, komga_url, total_komga_read, matched, already_read, newly_marked,
jsonb_array_length(unmatched) as unmatched_count, created_at
FROM komga_sync_reports
ORDER BY created_at DESC
LIMIT 20
"#,
)
.fetch_all(&state.pool)
.await?;
let reports: Vec<KomgaSyncReportSummary> = rows
.iter()
.map(|row| KomgaSyncReportSummary {
id: row.get("id"),
komga_url: row.get("komga_url"),
total_komga_read: row.get("total_komga_read"),
matched: row.get("matched"),
already_read: row.get("already_read"),
newly_marked: row.get("newly_marked"),
unmatched_count: row.get("unmatched_count"),
created_at: row.get("created_at"),
})
.collect();
Ok(Json(reports))
}
/// Get a specific sync report with full unmatched list
#[utoipa::path(
get,
path = "/komga/reports/{id}",
tag = "komga",
params(("id" = String, Path, description = "Report UUID")),
responses(
(status = 200, body = KomgaSyncResponse),
(status = 404, description = "Report not found"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_sync_report(
State(state): State<AppState>,
axum::extract::Path(id): axum::extract::Path<Uuid>,
) -> Result<Json<KomgaSyncResponse>, ApiError> {
let row = sqlx::query(
r#"
SELECT id, komga_url, total_komga_read, matched, already_read, newly_marked, matched_books, newly_marked_books, unmatched, created_at
FROM komga_sync_reports
WHERE id = $1
"#,
)
.bind(id)
.fetch_optional(&state.pool)
.await?;
let row = row.ok_or_else(|| ApiError::not_found("report not found"))?;
let matched_books_json: serde_json::Value = row.try_get("matched_books").unwrap_or(serde_json::Value::Array(vec![]));
let matched_books: Vec<String> = serde_json::from_value(matched_books_json).unwrap_or_default();
let newly_marked_books_json: serde_json::Value = row.try_get("newly_marked_books").unwrap_or(serde_json::Value::Array(vec![]));
let newly_marked_books: Vec<String> = serde_json::from_value(newly_marked_books_json).unwrap_or_default();
let unmatched_json: serde_json::Value = row.get("unmatched");
let unmatched: Vec<String> = serde_json::from_value(unmatched_json).unwrap_or_default();
Ok(Json(KomgaSyncResponse {
id: row.get("id"),
komga_url: row.get("komga_url"),
total_komga_read: row.get("total_komga_read"),
matched: row.get("matched"),
already_read: row.get("already_read"),
newly_marked: row.get("newly_marked"),
matched_books,
newly_marked_books,
unmatched,
created_at: row.get("created_at"),
}))
}

View File

@@ -6,7 +6,7 @@ use sqlx::Row;
use uuid::Uuid; use uuid::Uuid;
use utoipa::ToSchema; use utoipa::ToSchema;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::AppState};
#[derive(Serialize, ToSchema)] #[derive(Serialize, ToSchema)]
pub struct LibraryResponse { pub struct LibraryResponse {
@@ -18,8 +18,18 @@ pub struct LibraryResponse {
pub book_count: i64, pub book_count: i64,
pub monitor_enabled: bool, pub monitor_enabled: bool,
pub scan_mode: String, pub scan_mode: String,
#[schema(value_type = Option<String>)]
pub next_scan_at: Option<chrono::DateTime<chrono::Utc>>, pub next_scan_at: Option<chrono::DateTime<chrono::Utc>>,
pub watcher_enabled: bool, pub watcher_enabled: bool,
pub metadata_provider: Option<String>,
pub fallback_metadata_provider: Option<String>,
pub metadata_refresh_mode: String,
#[schema(value_type = Option<String>)]
pub next_metadata_refresh_at: Option<chrono::DateTime<chrono::Utc>>,
pub series_count: i64,
/// First book IDs from up to 5 distinct series (for thumbnail fan display)
#[schema(value_type = Vec<String>)]
pub thumbnail_book_ids: Vec<Uuid>,
} }
#[derive(Deserialize, ToSchema)] #[derive(Deserialize, ToSchema)]
@@ -38,14 +48,27 @@ pub struct CreateLibraryRequest {
responses( responses(
(status = 200, body = Vec<LibraryResponse>), (status = 200, body = Vec<LibraryResponse>),
(status = 401, description = "Unauthorized"), (status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"),
), ),
security(("Bearer" = [])) security(("Bearer" = []))
)] )]
pub async fn list_libraries(State(state): State<AppState>) -> Result<Json<Vec<LibraryResponse>>, ApiError> { pub async fn list_libraries(State(state): State<AppState>) -> Result<Json<Vec<LibraryResponse>>, ApiError> {
let rows = sqlx::query( let rows = sqlx::query(
"SELECT l.id, l.name, l.root_path, l.enabled, l.monitor_enabled, l.scan_mode, l.next_scan_at, l.watcher_enabled, "SELECT l.id, l.name, l.root_path, l.enabled, l.monitor_enabled, l.scan_mode, l.next_scan_at, l.watcher_enabled, l.metadata_provider, l.fallback_metadata_provider, l.metadata_refresh_mode, l.next_metadata_refresh_at,
(SELECT COUNT(*) FROM books b WHERE b.library_id = l.id) as book_count (SELECT COUNT(*) FROM books b WHERE b.library_id = l.id) as book_count,
(SELECT COUNT(DISTINCT COALESCE(NULLIF(b.series, ''), 'unclassified')) FROM books b WHERE b.library_id = l.id) as series_count,
COALESCE((
SELECT ARRAY_AGG(first_id ORDER BY series_name)
FROM (
SELECT DISTINCT ON (COALESCE(NULLIF(b.series, ''), 'unclassified'))
COALESCE(NULLIF(b.series, ''), 'unclassified') as series_name,
b.id as first_id
FROM books b
WHERE b.library_id = l.id
ORDER BY COALESCE(NULLIF(b.series, ''), 'unclassified'),
b.volume NULLS LAST, b.title ASC
LIMIT 5
) sub
), ARRAY[]::uuid[]) as thumbnail_book_ids
FROM libraries l ORDER BY l.created_at DESC" FROM libraries l ORDER BY l.created_at DESC"
) )
.fetch_all(&state.pool) .fetch_all(&state.pool)
@@ -59,10 +82,16 @@ pub async fn list_libraries(State(state): State<AppState>) -> Result<Json<Vec<Li
root_path: row.get("root_path"), root_path: row.get("root_path"),
enabled: row.get("enabled"), enabled: row.get("enabled"),
book_count: row.get("book_count"), book_count: row.get("book_count"),
series_count: row.get("series_count"),
monitor_enabled: row.get("monitor_enabled"), monitor_enabled: row.get("monitor_enabled"),
scan_mode: row.get("scan_mode"), scan_mode: row.get("scan_mode"),
next_scan_at: row.get("next_scan_at"), next_scan_at: row.get("next_scan_at"),
watcher_enabled: row.get("watcher_enabled"), watcher_enabled: row.get("watcher_enabled"),
metadata_provider: row.get("metadata_provider"),
fallback_metadata_provider: row.get("fallback_metadata_provider"),
metadata_refresh_mode: row.get("metadata_refresh_mode"),
next_metadata_refresh_at: row.get("next_metadata_refresh_at"),
thumbnail_book_ids: row.get("thumbnail_book_ids"),
}) })
.collect(); .collect();
@@ -110,10 +139,16 @@ pub async fn create_library(
root_path, root_path,
enabled: true, enabled: true,
book_count: 0, book_count: 0,
series_count: 0,
monitor_enabled: false, monitor_enabled: false,
scan_mode: "manual".to_string(), scan_mode: "manual".to_string(),
next_scan_at: None, next_scan_at: None,
watcher_enabled: false, watcher_enabled: false,
metadata_provider: None,
fallback_metadata_provider: None,
metadata_refresh_mode: "manual".to_string(),
next_metadata_refresh_at: None,
thumbnail_book_ids: vec![],
})) }))
} }
@@ -155,14 +190,19 @@ fn canonicalize_library_root(root_path: &str) -> Result<PathBuf, ApiError> {
return Err(ApiError::bad_request("root_path must be absolute")); return Err(ApiError::bad_request("root_path must be absolute"));
} }
let canonical = std::fs::canonicalize(path) // Avoid fs::canonicalize — it opens extra file descriptors to resolve symlinks
.map_err(|_| ApiError::bad_request("root_path does not exist or is inaccessible"))?; // and can fail on Docker volume mounts (ro, cached) when fd limits are low.
if !path.exists() {
if !canonical.is_dir() { return Err(ApiError::bad_request(format!(
"root_path does not exist: {}",
root_path
)));
}
if !path.is_dir() {
return Err(ApiError::bad_request("root_path must point to a directory")); return Err(ApiError::bad_request("root_path must point to a directory"));
} }
Ok(canonical) Ok(path.to_path_buf())
} }
use crate::index_jobs::{IndexJobResponse, RebuildRequest}; use crate::index_jobs::{IndexJobResponse, RebuildRequest};
@@ -180,7 +220,6 @@ use crate::index_jobs::{IndexJobResponse, RebuildRequest};
(status = 200, body = IndexJobResponse), (status = 200, body = IndexJobResponse),
(status = 404, description = "Library not found"), (status = 404, description = "Library not found"),
(status = 401, description = "Unauthorized"), (status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"),
), ),
security(("Bearer" = [])) security(("Bearer" = []))
)] )]
@@ -200,7 +239,8 @@ pub async fn scan_library(
} }
let is_full = payload.as_ref().and_then(|p| p.full).unwrap_or(false); let is_full = payload.as_ref().and_then(|p| p.full).unwrap_or(false);
let job_type = if is_full { "full_rebuild" } else { "rebuild" }; let is_rescan = payload.as_ref().and_then(|p| p.rescan).unwrap_or(false);
let job_type = if is_full { "full_rebuild" } else if is_rescan { "rescan" } else { "rebuild" };
// Create indexing job for this library // Create indexing job for this library
let job_id = Uuid::new_v4(); let job_id = Uuid::new_v4();
@@ -229,6 +269,8 @@ pub struct UpdateMonitoringRequest {
#[schema(value_type = String, example = "hourly")] #[schema(value_type = String, example = "hourly")]
pub scan_mode: String, // 'manual', 'hourly', 'daily', 'weekly' pub scan_mode: String, // 'manual', 'hourly', 'daily', 'weekly'
pub watcher_enabled: Option<bool>, pub watcher_enabled: Option<bool>,
#[schema(value_type = Option<String>, example = "daily")]
pub metadata_refresh_mode: Option<String>, // 'manual', 'hourly', 'daily', 'weekly'
} }
/// Update monitoring settings for a library /// Update monitoring settings for a library
@@ -259,6 +301,12 @@ pub async fn update_monitoring(
return Err(ApiError::bad_request("scan_mode must be one of: manual, hourly, daily, weekly")); return Err(ApiError::bad_request("scan_mode must be one of: manual, hourly, daily, weekly"));
} }
// Validate metadata_refresh_mode
let metadata_refresh_mode = input.metadata_refresh_mode.as_deref().unwrap_or("manual");
if !valid_modes.contains(&metadata_refresh_mode) {
return Err(ApiError::bad_request("metadata_refresh_mode must be one of: manual, hourly, daily, weekly"));
}
// Calculate next_scan_at if monitoring is enabled // Calculate next_scan_at if monitoring is enabled
let next_scan_at = if input.monitor_enabled { let next_scan_at = if input.monitor_enabled {
let interval_minutes = match input.scan_mode.as_str() { let interval_minutes = match input.scan_mode.as_str() {
@@ -272,16 +320,31 @@ pub async fn update_monitoring(
None None
}; };
// Calculate next_metadata_refresh_at
let next_metadata_refresh_at = if metadata_refresh_mode != "manual" {
let interval_minutes = match metadata_refresh_mode {
"hourly" => 60,
"daily" => 1440,
"weekly" => 10080,
_ => 1440,
};
Some(chrono::Utc::now() + chrono::Duration::minutes(interval_minutes))
} else {
None
};
let watcher_enabled = input.watcher_enabled.unwrap_or(false); let watcher_enabled = input.watcher_enabled.unwrap_or(false);
let result = sqlx::query( let result = sqlx::query(
"UPDATE libraries SET monitor_enabled = $2, scan_mode = $3, next_scan_at = $4, watcher_enabled = $5 WHERE id = $1 RETURNING id, name, root_path, enabled, monitor_enabled, scan_mode, next_scan_at, watcher_enabled" "UPDATE libraries SET monitor_enabled = $2, scan_mode = $3, next_scan_at = $4, watcher_enabled = $5, metadata_refresh_mode = $6, next_metadata_refresh_at = $7 WHERE id = $1 RETURNING id, name, root_path, enabled, monitor_enabled, scan_mode, next_scan_at, watcher_enabled, metadata_provider, fallback_metadata_provider, metadata_refresh_mode, next_metadata_refresh_at"
) )
.bind(library_id) .bind(library_id)
.bind(input.monitor_enabled) .bind(input.monitor_enabled)
.bind(input.scan_mode) .bind(input.scan_mode)
.bind(next_scan_at) .bind(next_scan_at)
.bind(watcher_enabled) .bind(watcher_enabled)
.bind(metadata_refresh_mode)
.bind(next_metadata_refresh_at)
.fetch_optional(&state.pool) .fetch_optional(&state.pool)
.await?; .await?;
@@ -294,15 +357,121 @@ pub async fn update_monitoring(
.fetch_one(&state.pool) .fetch_one(&state.pool)
.await?; .await?;
let series_count: i64 = sqlx::query_scalar("SELECT COUNT(DISTINCT COALESCE(NULLIF(series, ''), 'unclassified')) FROM books WHERE library_id = $1")
.bind(library_id)
.fetch_one(&state.pool)
.await?;
let thumbnail_book_ids: Vec<Uuid> = sqlx::query_scalar(
"SELECT b.id FROM books b
WHERE b.library_id = $1
ORDER BY COALESCE(NULLIF(b.series, ''), 'unclassified'), b.volume NULLS LAST, b.title ASC
LIMIT 5"
)
.bind(library_id)
.fetch_all(&state.pool)
.await
.unwrap_or_default();
Ok(Json(LibraryResponse { Ok(Json(LibraryResponse {
id: row.get("id"), id: row.get("id"),
name: row.get("name"), name: row.get("name"),
root_path: row.get("root_path"), root_path: row.get("root_path"),
enabled: row.get("enabled"), enabled: row.get("enabled"),
book_count, book_count,
series_count,
monitor_enabled: row.get("monitor_enabled"), monitor_enabled: row.get("monitor_enabled"),
scan_mode: row.get("scan_mode"), scan_mode: row.get("scan_mode"),
next_scan_at: row.get("next_scan_at"), next_scan_at: row.get("next_scan_at"),
watcher_enabled: row.get("watcher_enabled"), watcher_enabled: row.get("watcher_enabled"),
metadata_provider: row.get("metadata_provider"),
fallback_metadata_provider: row.get("fallback_metadata_provider"),
metadata_refresh_mode: row.get("metadata_refresh_mode"),
next_metadata_refresh_at: row.get("next_metadata_refresh_at"),
thumbnail_book_ids,
}))
}
#[derive(Deserialize, ToSchema)]
pub struct UpdateMetadataProviderRequest {
pub metadata_provider: Option<String>,
pub fallback_metadata_provider: Option<String>,
}
/// Update the metadata provider for a library
#[utoipa::path(
patch,
path = "/libraries/{id}/metadata-provider",
tag = "libraries",
params(
("id" = String, Path, description = "Library UUID"),
),
request_body = UpdateMetadataProviderRequest,
responses(
(status = 200, body = LibraryResponse),
(status = 404, description = "Library not found"),
(status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"),
),
security(("Bearer" = []))
)]
pub async fn update_metadata_provider(
State(state): State<AppState>,
AxumPath(library_id): AxumPath<Uuid>,
Json(input): Json<UpdateMetadataProviderRequest>,
) -> Result<Json<LibraryResponse>, ApiError> {
let provider = input.metadata_provider.as_deref().filter(|s| !s.is_empty());
let fallback = input.fallback_metadata_provider.as_deref().filter(|s| !s.is_empty());
let result = sqlx::query(
"UPDATE libraries SET metadata_provider = $2, fallback_metadata_provider = $3 WHERE id = $1 RETURNING id, name, root_path, enabled, monitor_enabled, scan_mode, next_scan_at, watcher_enabled, metadata_provider, fallback_metadata_provider, metadata_refresh_mode, next_metadata_refresh_at"
)
.bind(library_id)
.bind(provider)
.bind(fallback)
.fetch_optional(&state.pool)
.await?;
let Some(row) = result else {
return Err(ApiError::not_found("library not found"));
};
let book_count: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM books WHERE library_id = $1")
.bind(library_id)
.fetch_one(&state.pool)
.await?;
let series_count: i64 = sqlx::query_scalar("SELECT COUNT(DISTINCT COALESCE(NULLIF(series, ''), 'unclassified')) FROM books WHERE library_id = $1")
.bind(library_id)
.fetch_one(&state.pool)
.await?;
let thumbnail_book_ids: Vec<Uuid> = sqlx::query_scalar(
"SELECT b.id FROM books b
WHERE b.library_id = $1
ORDER BY COALESCE(NULLIF(b.series, ''), 'unclassified'), b.volume NULLS LAST, b.title ASC
LIMIT 5"
)
.bind(library_id)
.fetch_all(&state.pool)
.await
.unwrap_or_default();
Ok(Json(LibraryResponse {
id: row.get("id"),
name: row.get("name"),
root_path: row.get("root_path"),
enabled: row.get("enabled"),
book_count,
series_count,
monitor_enabled: row.get("monitor_enabled"),
scan_mode: row.get("scan_mode"),
next_scan_at: row.get("next_scan_at"),
watcher_enabled: row.get("watcher_enabled"),
metadata_provider: row.get("metadata_provider"),
fallback_metadata_provider: row.get("fallback_metadata_provider"),
metadata_refresh_mode: row.get("metadata_refresh_mode"),
next_metadata_refresh_at: row.get("next_metadata_refresh_at"),
thumbnail_book_ids,
})) }))
} }

View File

@@ -1,91 +1,48 @@
mod auth; mod auth;
mod authors;
mod books; mod books;
mod error; mod error;
mod handlers;
mod index_jobs; mod index_jobs;
mod komga;
mod libraries; mod libraries;
mod metadata;
mod metadata_batch;
mod metadata_refresh;
mod metadata_providers;
mod api_middleware;
mod openapi; mod openapi;
mod pages; mod pages;
mod prowlarr;
mod qbittorrent;
mod reading_progress;
mod search; mod search;
mod series;
mod settings; mod settings;
mod state;
mod stats;
mod telegram;
mod thumbnails; mod thumbnails;
mod tokens; mod tokens;
use std::{ use std::sync::Arc;
num::NonZeroUsize, use std::time::Instant;
sync::{
atomic::{AtomicU64, Ordering},
Arc,
},
time::{Duration, Instant},
};
use axum::{ use axum::{
middleware, middleware,
response::IntoResponse,
routing::{delete, get}, routing::{delete, get},
Json, Router, Router,
}; };
use utoipa::OpenApi; use utoipa::OpenApi;
use utoipa_swagger_ui::SwaggerUi; use utoipa_swagger_ui::SwaggerUi;
use lru::LruCache; use lru::LruCache;
use std::num::NonZeroUsize;
use stripstream_core::config::ApiConfig; use stripstream_core::config::ApiConfig;
use sqlx::postgres::PgPoolOptions; use sqlx::postgres::PgPoolOptions;
use tokio::sync::{Mutex, Semaphore}; use tokio::sync::{Mutex, RwLock, Semaphore};
use tracing::info; use tracing::info;
use sqlx::{Pool, Postgres, Row};
use serde_json::Value;
#[derive(Clone)] use crate::state::{load_concurrent_renders, load_dynamic_settings, AppState, Metrics, ReadRateLimit};
struct AppState {
pool: sqlx::PgPool,
bootstrap_token: Arc<str>,
meili_url: Arc<str>,
meili_master_key: Arc<str>,
page_cache: Arc<Mutex<LruCache<String, Arc<Vec<u8>>>>>,
page_render_limit: Arc<Semaphore>,
metrics: Arc<Metrics>,
read_rate_limit: Arc<Mutex<ReadRateLimit>>,
}
struct Metrics {
requests_total: AtomicU64,
page_cache_hits: AtomicU64,
page_cache_misses: AtomicU64,
}
struct ReadRateLimit {
window_started_at: Instant,
requests_in_window: u32,
}
impl Metrics {
fn new() -> Self {
Self {
requests_total: AtomicU64::new(0),
page_cache_hits: AtomicU64::new(0),
page_cache_misses: AtomicU64::new(0),
}
}
}
async fn load_concurrent_renders(pool: &Pool<Postgres>) -> usize {
let default_concurrency = 8;
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: Value = row.get("value");
value
.get("concurrent_renders")
.and_then(|v: &Value| v.as_u64())
.map(|v| v as usize)
.unwrap_or(default_concurrency)
}
_ => default_concurrency,
}
}
#[tokio::main] #[tokio::main]
async fn main() -> anyhow::Result<()> { async fn main() -> anyhow::Result<()> {
@@ -105,11 +62,21 @@ async fn main() -> anyhow::Result<()> {
let concurrent_renders = load_concurrent_renders(&pool).await; let concurrent_renders = load_concurrent_renders(&pool).await;
info!("Using concurrent_renders limit: {}", concurrent_renders); info!("Using concurrent_renders limit: {}", concurrent_renders);
let dynamic_settings = load_dynamic_settings(&pool).await;
info!(
"Dynamic settings: rate_limit={}, timeout={}s, format={}, quality={}, filter={}, max_width={}, cache_dir={}",
dynamic_settings.rate_limit_per_second,
dynamic_settings.timeout_seconds,
dynamic_settings.image_format,
dynamic_settings.image_quality,
dynamic_settings.image_filter,
dynamic_settings.image_max_width,
dynamic_settings.cache_directory,
);
let state = AppState { let state = AppState {
pool, pool,
bootstrap_token: Arc::from(config.api_bootstrap_token), bootstrap_token: Arc::from(config.api_bootstrap_token),
meili_url: Arc::from(config.meili_url),
meili_master_key: Arc::from(config.meili_master_key),
page_cache: Arc::new(Mutex::new(LruCache::new(NonZeroUsize::new(512).expect("non-zero")))), page_cache: Arc::new(Mutex::new(LruCache::new(NonZeroUsize::new(512).expect("non-zero")))),
page_render_limit: Arc::new(Semaphore::new(concurrent_renders)), page_render_limit: Arc::new(Semaphore::new(concurrent_renders)),
metrics: Arc::new(Metrics::new()), metrics: Arc::new(Metrics::new()),
@@ -117,13 +84,17 @@ async fn main() -> anyhow::Result<()> {
window_started_at: Instant::now(), window_started_at: Instant::now(),
requests_in_window: 0, requests_in_window: 0,
})), })),
settings: Arc::new(RwLock::new(dynamic_settings)),
}; };
let admin_routes = Router::new() let admin_routes = Router::new()
.route("/libraries", get(libraries::list_libraries).post(libraries::create_library)) .route("/libraries", axum::routing::post(libraries::create_library))
.route("/libraries/:id", delete(libraries::delete_library)) .route("/libraries/:id", delete(libraries::delete_library))
.route("/libraries/:id/scan", axum::routing::post(libraries::scan_library))
.route("/libraries/:id/monitoring", axum::routing::patch(libraries::update_monitoring)) .route("/libraries/:id/monitoring", axum::routing::patch(libraries::update_monitoring))
.route("/libraries/:id/metadata-provider", axum::routing::patch(libraries::update_metadata_provider))
.route("/books/:id", axum::routing::patch(books::update_book))
.route("/books/:id/convert", axum::routing::post(books::convert_book))
.route("/libraries/:library_id/series/:name", axum::routing::patch(series::update_series))
.route("/index/rebuild", axum::routing::post(index_jobs::enqueue_rebuild)) .route("/index/rebuild", axum::routing::post(index_jobs::enqueue_rebuild))
.route("/index/thumbnails/rebuild", axum::routing::post(thumbnails::start_thumbnails_rebuild)) .route("/index/thumbnails/rebuild", axum::routing::post(thumbnails::start_thumbnails_rebuild))
.route("/index/thumbnails/regenerate", axum::routing::post(thumbnails::start_thumbnails_regenerate)) .route("/index/thumbnails/regenerate", axum::routing::post(thumbnails::start_thumbnails_regenerate))
@@ -131,12 +102,32 @@ async fn main() -> anyhow::Result<()> {
.route("/index/jobs/active", get(index_jobs::get_active_jobs)) .route("/index/jobs/active", get(index_jobs::get_active_jobs))
.route("/index/jobs/:id", get(index_jobs::get_job_details)) .route("/index/jobs/:id", get(index_jobs::get_job_details))
.route("/index/jobs/:id/stream", get(index_jobs::stream_job_progress)) .route("/index/jobs/:id/stream", get(index_jobs::stream_job_progress))
.route("/index/jobs/:id/thumbnails/checkup", axum::routing::post(thumbnails::start_checkup))
.route("/index/jobs/:id/errors", get(index_jobs::get_job_errors)) .route("/index/jobs/:id/errors", get(index_jobs::get_job_errors))
.route("/index/cancel/:id", axum::routing::post(index_jobs::cancel_job)) .route("/index/cancel/:id", axum::routing::post(index_jobs::cancel_job))
.route("/folders", get(index_jobs::list_folders)) .route("/folders", get(index_jobs::list_folders))
.route("/admin/tokens", get(tokens::list_tokens).post(tokens::create_token)) .route("/admin/tokens", get(tokens::list_tokens).post(tokens::create_token))
.route("/admin/tokens/:id", delete(tokens::revoke_token)) .route("/admin/tokens/:id", delete(tokens::revoke_token))
.route("/admin/tokens/:id/delete", axum::routing::post(tokens::delete_token))
.route("/prowlarr/search", axum::routing::post(prowlarr::search_prowlarr))
.route("/prowlarr/test", get(prowlarr::test_prowlarr))
.route("/qbittorrent/add", axum::routing::post(qbittorrent::add_torrent))
.route("/qbittorrent/test", get(qbittorrent::test_qbittorrent))
.route("/telegram/test", get(telegram::test_telegram))
.route("/komga/sync", axum::routing::post(komga::sync_komga_read_books))
.route("/komga/reports", get(komga::list_sync_reports))
.route("/komga/reports/:id", get(komga::get_sync_report))
.route("/metadata/search", axum::routing::post(metadata::search_metadata))
.route("/metadata/match", axum::routing::post(metadata::create_metadata_match))
.route("/metadata/approve/:id", axum::routing::post(metadata::approve_metadata))
.route("/metadata/reject/:id", axum::routing::post(metadata::reject_metadata))
.route("/metadata/links", get(metadata::get_metadata_links))
.route("/metadata/missing/:id", get(metadata::get_missing_books))
.route("/metadata/links/:id", delete(metadata::delete_metadata_link))
.route("/metadata/batch", axum::routing::post(metadata_batch::start_batch))
.route("/metadata/batch/:id/report", get(metadata_batch::get_batch_report))
.route("/metadata/batch/:id/results", get(metadata_batch::get_batch_results))
.route("/metadata/refresh", axum::routing::post(metadata_refresh::start_refresh))
.route("/metadata/refresh/:id/report", get(metadata_refresh::get_refresh_report))
.merge(settings::settings_routes()) .merge(settings::settings_routes())
.route_layer(middleware::from_fn_with_state( .route_layer(middleware::from_fn_with_state(
state.clone(), state.clone(),
@@ -144,27 +135,39 @@ async fn main() -> anyhow::Result<()> {
)); ));
let read_routes = Router::new() let read_routes = Router::new()
.route("/libraries", get(libraries::list_libraries))
.route("/libraries/:id/scan", axum::routing::post(libraries::scan_library))
.route("/books", get(books::list_books)) .route("/books", get(books::list_books))
.route("/books/ongoing", get(series::ongoing_books))
.route("/books/:id", get(books::get_book)) .route("/books/:id", get(books::get_book))
.route("/books/:id/thumbnail", get(books::get_thumbnail)) .route("/books/:id/thumbnail", get(books::get_thumbnail))
.route("/books/:id/pages/:n", get(pages::get_page)) .route("/books/:id/pages/:n", get(pages::get_page))
.route("/libraries/:library_id/series", get(books::list_series)) .route("/books/:id/progress", get(reading_progress::get_reading_progress).patch(reading_progress::update_reading_progress))
.route("/libraries/:library_id/series", get(series::list_series))
.route("/libraries/:library_id/series/:name/metadata", get(series::get_series_metadata))
.route("/series", get(series::list_all_series))
.route("/series/ongoing", get(series::ongoing_series))
.route("/series/statuses", get(series::series_statuses))
.route("/series/provider-statuses", get(series::provider_statuses))
.route("/series/mark-read", axum::routing::post(reading_progress::mark_series_read))
.route("/authors", get(authors::list_authors))
.route("/stats", get(stats::get_stats))
.route("/search", get(search::search_books)) .route("/search", get(search::search_books))
.route_layer(middleware::from_fn_with_state(state.clone(), read_rate_limit)) .route_layer(middleware::from_fn_with_state(state.clone(), api_middleware::read_rate_limit))
.route_layer(middleware::from_fn_with_state( .route_layer(middleware::from_fn_with_state(
state.clone(), state.clone(),
auth::require_read, auth::require_read,
)); ));
let app = Router::new() let app = Router::new()
.route("/health", get(health)) .route("/health", get(handlers::health))
.route("/ready", get(ready)) .route("/ready", get(handlers::ready))
.route("/metrics", get(metrics)) .route("/metrics", get(handlers::metrics))
.route("/docs", get(docs_redirect)) .route("/docs", get(handlers::docs_redirect))
.merge(SwaggerUi::new("/swagger-ui").url("/openapi.json", openapi::ApiDoc::openapi())) .merge(SwaggerUi::new("/swagger-ui").url("/openapi.json", openapi::ApiDoc::openapi()))
.merge(admin_routes) .merge(admin_routes)
.merge(read_routes) .merge(read_routes)
.layer(middleware::from_fn_with_state(state.clone(), request_counter)) .layer(middleware::from_fn_with_state(state.clone(), api_middleware::request_counter))
.with_state(state); .with_state(state);
let listener = tokio::net::TcpListener::bind(&config.listen_addr).await?; let listener = tokio::net::TcpListener::bind(&config.listen_addr).await?;
@@ -173,57 +176,3 @@ async fn main() -> anyhow::Result<()> {
Ok(()) Ok(())
} }
async fn health() -> &'static str {
"ok"
}
async fn docs_redirect() -> impl axum::response::IntoResponse {
axum::response::Redirect::to("/swagger-ui/")
}
async fn ready(axum::extract::State(state): axum::extract::State<AppState>) -> Result<Json<serde_json::Value>, error::ApiError> {
sqlx::query("SELECT 1").execute(&state.pool).await?;
Ok(Json(serde_json::json!({"status": "ready"})))
}
async fn metrics(axum::extract::State(state): axum::extract::State<AppState>) -> String {
format!(
"requests_total {}\npage_cache_hits {}\npage_cache_misses {}\n",
state.metrics.requests_total.load(Ordering::Relaxed),
state.metrics.page_cache_hits.load(Ordering::Relaxed),
state.metrics.page_cache_misses.load(Ordering::Relaxed),
)
}
async fn request_counter(
axum::extract::State(state): axum::extract::State<AppState>,
req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
state.metrics.requests_total.fetch_add(1, Ordering::Relaxed);
next.run(req).await
}
async fn read_rate_limit(
axum::extract::State(state): axum::extract::State<AppState>,
req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
let mut limiter = state.read_rate_limit.lock().await;
if limiter.window_started_at.elapsed() >= Duration::from_secs(1) {
limiter.window_started_at = Instant::now();
limiter.requests_in_window = 0;
}
if limiter.requests_in_window >= 120 {
return (
axum::http::StatusCode::TOO_MANY_REQUESTS,
"rate limit exceeded",
)
.into_response();
}
limiter.requests_in_window += 1;
drop(limiter);
next.run(req).await
}

1097
apps/api/src/metadata.rs Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,342 @@
use super::{BookCandidate, MetadataProvider, ProviderConfig, SeriesCandidate};
pub struct AniListProvider;
impl MetadataProvider for AniListProvider {
fn name(&self) -> &str {
"anilist"
}
fn search_series(
&self,
query: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<SeriesCandidate>, String>> + Send + '_>,
> {
let query = query.to_string();
let config = config.clone();
Box::pin(async move { search_series_impl(&query, &config).await })
}
fn get_series_books(
&self,
external_id: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<BookCandidate>, String>> + Send + '_>,
> {
let external_id = external_id.to_string();
let config = config.clone();
Box::pin(async move { get_series_books_impl(&external_id, &config).await })
}
}
const SEARCH_QUERY: &str = r#"
query ($search: String) {
Page(perPage: 20) {
media(search: $search, type: MANGA, sort: SEARCH_MATCH) {
id
title { romaji english native }
description(asHtml: false)
coverImage { large medium }
startDate { year }
status
volumes
chapters
staff { edges { node { name { full } } role } }
siteUrl
genres
}
}
}
"#;
const DETAIL_QUERY: &str = r#"
query ($id: Int) {
Media(id: $id, type: MANGA) {
id
title { romaji english native }
description(asHtml: false)
coverImage { large medium }
startDate { year }
status
volumes
chapters
staff { edges { node { name { full } } role } }
siteUrl
genres
}
}
"#;
async fn graphql_request(
client: &reqwest::Client,
query: &str,
variables: serde_json::Value,
) -> Result<serde_json::Value, String> {
let resp = client
.post("https://graphql.anilist.co")
.header("Content-Type", "application/json")
.json(&serde_json::json!({
"query": query,
"variables": variables,
}))
.send()
.await
.map_err(|e| format!("AniList request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(format!("AniList returned {status}: {text}"));
}
resp.json()
.await
.map_err(|e| format!("Failed to parse AniList response: {e}"))
}
async fn search_series_impl(
query: &str,
_config: &ProviderConfig,
) -> Result<Vec<SeriesCandidate>, String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))?;
let data = graphql_request(
&client,
SEARCH_QUERY,
serde_json::json!({ "search": query }),
)
.await?;
let media = match data
.get("data")
.and_then(|d| d.get("Page"))
.and_then(|p| p.get("media"))
.and_then(|m| m.as_array())
{
Some(media) => media,
None => return Ok(vec![]),
};
let query_lower = query.to_lowercase();
let mut candidates: Vec<SeriesCandidate> = media
.iter()
.filter_map(|m| {
let id = m.get("id").and_then(|id| id.as_i64())?;
let title_obj = m.get("title")?;
let title = title_obj
.get("english")
.and_then(|t| t.as_str())
.or_else(|| title_obj.get("romaji").and_then(|t| t.as_str()))?
.to_string();
let description = m
.get("description")
.and_then(|d| d.as_str())
.map(|d| d.replace("\\n", "\n").trim().to_string())
.filter(|d| !d.is_empty());
let cover_url = m
.get("coverImage")
.and_then(|ci| ci.get("large").or_else(|| ci.get("medium")))
.and_then(|u| u.as_str())
.map(String::from);
let start_year = m
.get("startDate")
.and_then(|sd| sd.get("year"))
.and_then(|y| y.as_i64())
.map(|y| y as i32);
let volumes = m
.get("volumes")
.and_then(|v| v.as_i64())
.map(|v| v as i32);
let chapters = m
.get("chapters")
.and_then(|v| v.as_i64())
.map(|v| v as i32);
let status = m
.get("status")
.and_then(|s| s.as_str())
.unwrap_or("UNKNOWN")
.to_string();
let site_url = m
.get("siteUrl")
.and_then(|u| u.as_str())
.map(String::from);
let authors = extract_authors(m);
let confidence = compute_confidence(&title, &query_lower);
// Use volumes if known, otherwise fall back to chapters count
let (total_volumes, volume_source) = match volumes {
Some(v) => (Some(v), "volumes"),
None => match chapters {
Some(c) => (Some(c), "chapters"),
None => (None, "unknown"),
},
};
Some(SeriesCandidate {
external_id: id.to_string(),
title,
authors,
description,
publishers: vec![],
start_year,
total_volumes,
cover_url,
external_url: site_url,
confidence,
metadata_json: serde_json::json!({
"status": status,
"chapters": chapters,
"volumes": volumes,
"volume_source": volume_source,
}),
})
})
.collect();
candidates.sort_by(|a, b| b.confidence.partial_cmp(&a.confidence).unwrap_or(std::cmp::Ordering::Equal));
candidates.truncate(10);
Ok(candidates)
}
async fn get_series_books_impl(
external_id: &str,
_config: &ProviderConfig,
) -> Result<Vec<BookCandidate>, String> {
let id: i64 = external_id
.parse()
.map_err(|_| "invalid AniList ID".to_string())?;
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))?;
let data = graphql_request(
&client,
DETAIL_QUERY,
serde_json::json!({ "id": id }),
)
.await?;
let media = match data.get("data").and_then(|d| d.get("Media")) {
Some(m) => m,
None => return Ok(vec![]),
};
let title_obj = media.get("title").cloned().unwrap_or(serde_json::json!({}));
let title = title_obj
.get("english")
.and_then(|t| t.as_str())
.or_else(|| title_obj.get("romaji").and_then(|t| t.as_str()))
.unwrap_or("")
.to_string();
let volumes = media
.get("volumes")
.and_then(|v| v.as_i64())
.map(|v| v as i32);
let chapters = media
.get("chapters")
.and_then(|v| v.as_i64())
.map(|v| v as i32);
// Use volumes if known, otherwise fall back to chapters count
let total = volumes.or(chapters);
let cover_url = media
.get("coverImage")
.and_then(|ci| ci.get("large").or_else(|| ci.get("medium")))
.and_then(|u| u.as_str())
.map(String::from);
let description = media
.get("description")
.and_then(|d| d.as_str())
.map(|d| d.replace("\\n", "\n").trim().to_string());
let authors = extract_authors(media);
// AniList doesn't have per-volume data — generate entries from volumes count (or chapters as fallback)
let mut books = Vec::new();
if let Some(total) = total {
for vol in 1..=total {
books.push(BookCandidate {
external_book_id: format!("{}-vol-{}", external_id, vol),
title: format!("{} Vol. {}", title, vol),
volume_number: Some(vol),
authors: authors.clone(),
isbn: None,
summary: if vol == 1 { description.clone() } else { None },
cover_url: if vol == 1 { cover_url.clone() } else { None },
page_count: None,
language: Some("ja".to_string()),
publish_date: None,
metadata_json: serde_json::json!({}),
});
}
}
Ok(books)
}
fn extract_authors(media: &serde_json::Value) -> Vec<String> {
let mut authors = Vec::new();
if let Some(edges) = media
.get("staff")
.and_then(|s| s.get("edges"))
.and_then(|e| e.as_array())
{
for edge in edges {
let role = edge
.get("role")
.and_then(|r| r.as_str())
.unwrap_or("");
let role_lower = role.to_lowercase();
if role_lower.contains("story") || role_lower.contains("art") || role_lower.contains("original") {
if let Some(name) = edge
.get("node")
.and_then(|n| n.get("name"))
.and_then(|n| n.get("full"))
.and_then(|f| f.as_str())
{
if !authors.contains(&name.to_string()) {
authors.push(name.to_string());
}
}
}
}
}
authors
}
fn compute_confidence(title: &str, query: &str) -> f32 {
let title_lower = title.to_lowercase();
if title_lower == query {
1.0
} else if title_lower.starts_with(query) || query.starts_with(&title_lower) {
0.8
} else if title_lower.contains(query) || query.contains(&title_lower) {
0.7
} else {
let common: usize = query.chars().filter(|c| title_lower.contains(*c)).count();
let max_len = query.len().max(title_lower.len()).max(1);
(common as f32 / max_len as f32).clamp(0.1, 0.6)
}
}

View File

@@ -0,0 +1,671 @@
use scraper::{Html, Selector};
use super::{BookCandidate, MetadataProvider, ProviderConfig, SeriesCandidate};
pub struct BedethequeProvider;
impl MetadataProvider for BedethequeProvider {
fn name(&self) -> &str {
"bedetheque"
}
fn search_series(
&self,
query: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<SeriesCandidate>, String>> + Send + '_>,
> {
let query = query.to_string();
let config = config.clone();
Box::pin(async move { search_series_impl(&query, &config).await })
}
fn get_series_books(
&self,
external_id: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<BookCandidate>, String>> + Send + '_>,
> {
let external_id = external_id.to_string();
let config = config.clone();
Box::pin(async move { get_series_books_impl(&external_id, &config).await })
}
}
fn build_client() -> Result<reqwest::Client, String> {
reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(20))
.user_agent("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) Gecko/20100101 Firefox/108.0")
.default_headers({
let mut h = reqwest::header::HeaderMap::new();
h.insert(
reqwest::header::ACCEPT,
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
.parse()
.unwrap(),
);
h.insert(
reqwest::header::ACCEPT_LANGUAGE,
"fr-FR,fr;q=0.9,en;q=0.5".parse().unwrap(),
);
h.insert(reqwest::header::REFERER, "https://www.bedetheque.com/".parse().unwrap());
h
})
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))
}
/// Remove diacritics for URL construction (bedetheque uses ASCII slugs)
fn normalize_for_url(s: &str) -> String {
s.chars()
.map(|c| match c {
'é' | 'è' | 'ê' | 'ë' | 'É' | 'È' | 'Ê' | 'Ë' => 'e',
'à' | 'â' | 'ä' | 'À' | 'Â' | 'Ä' => 'a',
'ù' | 'û' | 'ü' | 'Ù' | 'Û' | 'Ü' => 'u',
'ô' | 'ö' | 'Ô' | 'Ö' => 'o',
'î' | 'ï' | 'Î' | 'Ï' => 'i',
'ç' | 'Ç' => 'c',
'ñ' | 'Ñ' => 'n',
_ => c,
})
.collect()
}
fn urlencoded(s: &str) -> String {
let mut result = String::new();
for byte in s.bytes() {
match byte {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
result.push(byte as char);
}
b' ' => result.push('+'),
_ => result.push_str(&format!("%{:02X}", byte)),
}
}
result
}
// ---------------------------------------------------------------------------
// Search
// ---------------------------------------------------------------------------
async fn search_series_impl(
query: &str,
_config: &ProviderConfig,
) -> Result<Vec<SeriesCandidate>, String> {
let client = build_client()?;
// Use the full-text search page
let url = format!(
"https://www.bedetheque.com/search/tout?RechTexte={}&RechWhere=0",
urlencoded(&normalize_for_url(query))
);
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("Bedetheque request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
return Err(format!("Bedetheque returned {status}"));
}
let html = resp
.text()
.await
.map_err(|e| format!("Failed to read Bedetheque response: {e}"))?;
// Detect IP blacklist
if html.contains("<title></title>") || html.contains("<title> </title>") {
return Err("Bedetheque: IP may be rate-limited, please retry later".to_string());
}
// Parse HTML in a block so the non-Send Html type is dropped before any .await
let candidates = {
let document = Html::parse_document(&html);
let link_sel =
Selector::parse("a[href*='/serie-']").map_err(|e| format!("selector error: {e}"))?;
let query_lower = query.to_lowercase();
let mut seen = std::collections::HashSet::new();
let mut candidates = Vec::new();
for el in document.select(&link_sel) {
let href = match el.value().attr("href") {
Some(h) => h.to_string(),
None => continue,
};
let (series_id, _slug) = match parse_serie_href(&href) {
Some(v) => v,
None => continue,
};
if !seen.insert(series_id.clone()) {
continue;
}
let title = el.text().collect::<String>().trim().to_string();
if title.is_empty() {
continue;
}
let confidence = compute_confidence(&title, &query_lower);
let cover_url = format!(
"https://www.bedetheque.com/cache/thb_series/PlancheS_{}.jpg",
series_id
);
candidates.push(SeriesCandidate {
external_id: series_id.clone(),
title: title.clone(),
authors: vec![],
description: None,
publishers: vec![],
start_year: None,
total_volumes: None,
cover_url: Some(cover_url),
external_url: Some(href),
confidence,
metadata_json: serde_json::json!({}),
});
}
candidates.sort_by(|a, b| {
b.confidence
.partial_cmp(&a.confidence)
.unwrap_or(std::cmp::Ordering::Equal)
});
candidates.truncate(10);
candidates
}; // document is dropped here — safe to .await below
// For the top candidates, fetch series details to enrich metadata
// (limit to top 3 to avoid hammering the site)
let mut enriched = Vec::new();
for mut c in candidates {
if enriched.len() < 3 {
if let Ok(details) = fetch_series_details(&client, &c.external_id, c.external_url.as_deref()).await {
if let Some(desc) = details.description {
c.description = Some(desc);
}
if !details.authors.is_empty() {
c.authors = details.authors;
}
if !details.publishers.is_empty() {
c.publishers = details.publishers;
}
if let Some(year) = details.start_year {
c.start_year = Some(year);
}
if let Some(count) = details.album_count {
c.total_volumes = Some(count);
}
c.metadata_json = serde_json::json!({
"description": c.description,
"authors": c.authors,
"publishers": c.publishers,
"start_year": c.start_year,
"genres": details.genres,
"status": details.status,
"origin": details.origin,
"language": details.language,
});
}
}
enriched.push(c);
}
Ok(enriched)
}
/// Parse serie URL to extract (id, slug)
fn parse_serie_href(href: &str) -> Option<(String, String)> {
// Patterns:
// https://www.bedetheque.com/serie-3-BD-Blacksad.html
// /serie-3-BD-Blacksad.html
let re = regex::Regex::new(r"/serie-(\d+)-[A-Za-z]+-(.+?)(?:__\d+)?\.html").ok()?;
let caps = re.captures(href)?;
Some((caps[1].to_string(), caps[2].to_string()))
}
struct SeriesDetails {
description: Option<String>,
authors: Vec<String>,
publishers: Vec<String>,
start_year: Option<i32>,
album_count: Option<i32>,
genres: Vec<String>,
status: Option<String>,
origin: Option<String>,
language: Option<String>,
}
async fn fetch_series_details(
client: &reqwest::Client,
series_id: &str,
series_url: Option<&str>,
) -> Result<SeriesDetails, String> {
// Build URL — append __10000 to get all albums on one page
let url = match series_url {
Some(u) => {
// Replace .html with __10000.html
u.replace(".html", "__10000.html")
}
None => format!(
"https://www.bedetheque.com/serie-{}-BD-Serie__10000.html",
series_id
),
};
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("Failed to fetch series page: {e}"))?;
if !resp.status().is_success() {
return Err(format!("Series page returned {}", resp.status()));
}
let html = resp
.text()
.await
.map_err(|e| format!("Failed to read series page: {e}"))?;
let doc = Html::parse_document(&html);
let mut details = SeriesDetails {
description: None,
authors: vec![],
publishers: vec![],
start_year: None,
album_count: None,
genres: vec![],
status: None,
origin: None,
language: None,
};
// Description from <meta name="description"> — format: "Tout sur la série {name} : {description}"
if let Ok(sel) = Selector::parse(r#"meta[name="description"]"#) {
if let Some(el) = doc.select(&sel).next() {
if let Some(content) = el.value().attr("content") {
let desc = content.trim().to_string();
// Strip the "Tout sur la série ... : " prefix
let cleaned = if let Some(pos) = desc.find(" : ") {
desc[pos + 3..].trim().to_string()
} else {
desc
};
if !cleaned.is_empty() {
details.description = Some(cleaned);
}
}
}
}
// Extract authors from itemprop="author" and itemprop="illustrator" (deduplicated)
{
let mut authors_set = std::collections::HashSet::new();
for attr in ["author", "illustrator"] {
if let Ok(sel) = Selector::parse(&format!(r#"[itemprop="{attr}"]"#)) {
for el in doc.select(&sel) {
let name = el.text().collect::<String>().trim().to_string();
// Names are "Last, First" — normalize to "First Last"
let normalized = if let Some((last, first)) = name.split_once(',') {
format!("{} {}", first.trim(), last.trim())
} else {
name
};
if !normalized.is_empty() && is_real_author(&normalized) {
authors_set.insert(normalized);
}
}
}
}
details.authors = authors_set.into_iter().collect();
details.authors.sort();
}
// Extract publishers from itemprop="publisher" (deduplicated)
{
let mut publishers_set = std::collections::HashSet::new();
if let Ok(sel) = Selector::parse(r#"[itemprop="publisher"]"#) {
for el in doc.select(&sel) {
let name = el.text().collect::<String>().trim().to_string();
if !name.is_empty() {
publishers_set.insert(name);
}
}
}
details.publishers = publishers_set.into_iter().collect();
details.publishers.sort();
}
// Extract series-level info from <li><label>X :</label>value</li> blocks
// Genre: <li><label>Genre :</label><span class="style-serie">Animalier, Aventure, Humour</span></li>
if let Ok(sel) = Selector::parse("span.style-serie") {
if let Some(el) = doc.select(&sel).next() {
let text = el.text().collect::<String>();
details.genres = text
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
}
}
// Parution: <li><label>Parution :</label><span class="parution-serie">Série finie</span></li>
if let Ok(sel) = Selector::parse("span.parution-serie") {
if let Some(el) = doc.select(&sel).next() {
let text = el.text().collect::<String>().trim().to_string();
if !text.is_empty() {
details.status = Some(text);
}
}
}
// Origine and Langue from page text (no dedicated CSS class)
let page_text = doc.root_element().text().collect::<String>();
if let Some(val) = extract_info_value(&page_text, "Origine") {
let val = val.lines().next().unwrap_or(val).trim();
if !val.is_empty() {
details.origin = Some(val.to_string());
}
}
if let Some(val) = extract_info_value(&page_text, "Langue") {
let val = val.lines().next().unwrap_or(val).trim();
if !val.is_empty() {
details.language = Some(val.to_string());
}
}
// Album count from serie-info text (e.g. "Tomes : 8")
if let Ok(re) = regex::Regex::new(r"Tomes?\s*:\s*(\d+)") {
if let Some(caps) = re.captures(&page_text) {
if let Ok(n) = caps[1].parse::<i32>() {
details.album_count = Some(n);
}
}
}
// Start year from first <meta itemprop="datePublished" content="YYYY-MM-DD">
if let Ok(sel) = Selector::parse(r#"[itemprop="datePublished"]"#) {
if let Some(el) = doc.select(&sel).next() {
if let Some(content) = el.value().attr("content") {
// content is "YYYY-MM-DD"
if let Some(year_str) = content.split('-').next() {
if let Ok(year) = year_str.parse::<i32>() {
details.start_year = Some(year);
}
}
}
}
}
Ok(details)
}
/// Extract value after a label like "Scénario : Jean-Claude" → "Jean-Claude"
fn extract_info_value<'a>(text: &'a str, label: &str) -> Option<&'a str> {
// Handle both "Label :" and "Label:"
let patterns = [
format!("{} :", label),
format!("{}:", label),
format!("{} :", &label.to_lowercase()),
];
for pat in &patterns {
if let Some(pos) = text.find(pat.as_str()) {
let val = text[pos + pat.len()..].trim();
if !val.is_empty() {
return Some(val);
}
}
}
None
}
// ---------------------------------------------------------------------------
// Get series books
// ---------------------------------------------------------------------------
async fn get_series_books_impl(
external_id: &str,
_config: &ProviderConfig,
) -> Result<Vec<BookCandidate>, String> {
let client = build_client()?;
// We need to find the series URL — try a direct fetch
// external_id is the numeric series ID
// We try to fetch the series page to get the album list
let url = format!(
"https://www.bedetheque.com/serie-{}-BD-Serie__10000.html",
external_id
);
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("Failed to fetch series: {e}"))?;
// If the generic slug fails, try without the slug part (bedetheque redirects)
let html = if resp.status().is_success() {
resp.text().await.map_err(|e| format!("Failed to read: {e}"))?
} else {
// Try alternative URL pattern
let alt_url = format!(
"https://www.bedetheque.com/serie-{}__10000.html",
external_id
);
let resp2 = client
.get(&alt_url)
.send()
.await
.map_err(|e| format!("Failed to fetch series (alt): {e}"))?;
if !resp2.status().is_success() {
return Err(format!("Series page not found for id {external_id}"));
}
resp2.text().await.map_err(|e| format!("Failed to read: {e}"))?
};
if html.contains("<title></title>") {
return Err("Bedetheque: IP may be rate-limited".to_string());
}
let doc = Html::parse_document(&html);
let mut books = Vec::new();
// Each album block starts before a .album-main div.
// The cover image (<img itemprop="image">) is OUTSIDE .album-main (sibling),
// so we iterate over a broader parent. But the simplest approach: parse all
// itemprop elements relative to each .album-main, plus pick covers separately.
let album_sel = Selector::parse(".album-main").map_err(|e| format!("selector: {e}"))?;
// Pre-collect cover images — they appear in <img itemprop="image"> before each .album-main
// and link to an album URL containing the book ID
let cover_sel = Selector::parse(r#"img[itemprop="image"]"#).map_err(|e| format!("selector: {e}"))?;
let covers: Vec<String> = doc.select(&cover_sel)
.filter_map(|el| el.value().attr("src").map(|s| {
if s.starts_with("http") { s.to_string() } else { format!("https://www.bedetheque.com{}", s) }
}))
.collect();
static RE_TOME: std::sync::LazyLock<regex::Regex> =
std::sync::LazyLock::new(|| regex::Regex::new(r"(?i)-Tome-\d+-").unwrap());
static RE_BOOK_ID: std::sync::LazyLock<regex::Regex> =
std::sync::LazyLock::new(|| regex::Regex::new(r"-(\d+)\.html").unwrap());
static RE_VOLUME: std::sync::LazyLock<regex::Regex> =
std::sync::LazyLock::new(|| regex::Regex::new(r"(?i)Tome-(\d+)-").unwrap());
for (idx, album_el) in doc.select(&album_sel).enumerate() {
// Title from <a class="titre" title="..."> — the title attribute is clean
let title_sel = Selector::parse("a.titre").ok();
let title_el = title_sel.as_ref().and_then(|s| album_el.select(s).next());
let title = title_el
.and_then(|el| el.value().attr("title"))
.unwrap_or("")
.trim()
.to_string();
if title.is_empty() {
continue;
}
// External book ID from album URL (e.g. "...-1063.html")
let album_url = title_el.and_then(|el| el.value().attr("href")).unwrap_or("");
// Only keep main tomes — their URLs contain "Tome-{N}-"
// Skip hors-série (HS), intégrales (INT/INTFL), romans, coffrets, etc.
if !RE_TOME.is_match(album_url) {
continue;
}
let external_book_id = RE_BOOK_ID
.captures(album_url)
.map(|c| c[1].to_string())
.unwrap_or_default();
// Volume number from URL pattern "Tome-{N}-" or from itemprop name
let volume_number = RE_VOLUME
.captures(album_url)
.and_then(|c| c[1].parse::<i32>().ok())
.or_else(|| extract_volume_from_title(&title));
// Authors from itemprop="author" and itemprop="illustrator"
let mut authors = Vec::new();
let author_sel = Selector::parse(r#"[itemprop="author"]"#).ok();
let illustrator_sel = Selector::parse(r#"[itemprop="illustrator"]"#).ok();
for sel in [&author_sel, &illustrator_sel].into_iter().flatten() {
for el in album_el.select(sel) {
let name = el.text().collect::<String>().trim().to_string();
// Names are "Last, First" format — normalize to "First Last"
let normalized = if let Some((last, first)) = name.split_once(',') {
format!("{} {}", first.trim(), last.trim())
} else {
name
};
if !normalized.is_empty() && is_real_author(&normalized) && !authors.contains(&normalized) {
authors.push(normalized);
}
}
}
// ISBN from <span itemprop="isbn">
let isbn = Selector::parse(r#"[itemprop="isbn"]"#)
.ok()
.and_then(|s| album_el.select(&s).next())
.map(|el| el.text().collect::<String>().trim().to_string())
.filter(|s| !s.is_empty());
// Page count from <span itemprop="numberOfPages">
let page_count = Selector::parse(r#"[itemprop="numberOfPages"]"#)
.ok()
.and_then(|s| album_el.select(&s).next())
.and_then(|el| el.text().collect::<String>().trim().parse::<i32>().ok());
// Publish date from <meta itemprop="datePublished" content="YYYY-MM-DD">
let publish_date = Selector::parse(r#"[itemprop="datePublished"]"#)
.ok()
.and_then(|s| album_el.select(&s).next())
.and_then(|el| el.value().attr("content").map(|c| c.trim().to_string()))
.filter(|s| !s.is_empty());
// Cover from pre-collected covers (same index)
let cover_url = covers.get(idx).cloned();
books.push(BookCandidate {
external_book_id,
title,
volume_number,
authors,
isbn,
summary: None,
cover_url,
page_count,
language: Some("fr".to_string()),
publish_date,
metadata_json: serde_json::json!({}),
});
}
books.sort_by_key(|b| b.volume_number.unwrap_or(999));
Ok(books)
}
/// Filter out placeholder author names from Bédéthèque
fn is_real_author(name: &str) -> bool {
!name.starts_with('<') && !name.ends_with('>') && name != "Collectif"
}
fn extract_volume_from_title(title: &str) -> Option<i32> {
let patterns = [
r"(?i)(?:tome|t\.)\s*(\d+)",
r"(?i)(?:vol(?:ume)?\.?)\s*(\d+)",
r"#\s*(\d+)",
];
for pattern in &patterns {
if let Ok(re) = regex::Regex::new(pattern) {
if let Some(caps) = re.captures(title) {
if let Ok(n) = caps[1].parse::<i32>() {
return Some(n);
}
}
}
}
None
}
/// Normalize a title by removing French articles (leading or in parentheses)
/// and extra whitespace/punctuation, so that "Les Légendaires - Résistance"
/// and "Légendaires (Les) - Résistance" produce the same canonical form.
fn normalize_title(s: &str) -> String {
let lower = s.to_lowercase();
// Remove articles in parentheses: "(les)", "(la)", "(le)", "(l')", "(un)", "(une)", "(des)"
let re_parens = regex::Regex::new(r"\s*\((?:les?|la|l'|une?|des|du|d')\)").unwrap();
let cleaned = re_parens.replace_all(&lower, "");
// Remove leading articles: "les ", "la ", "le ", "l'", "un ", "une ", "des ", "du ", "d'"
let re_leading = regex::Regex::new(r"^(?:les?|la|l'|une?|des|du|d')\s+").unwrap();
let cleaned = re_leading.replace(&cleaned, "");
// Collapse multiple spaces/dashes into single
let re_spaces = regex::Regex::new(r"\s+").unwrap();
re_spaces.replace_all(cleaned.trim(), " ").to_string()
}
fn compute_confidence(title: &str, query: &str) -> f32 {
let title_lower = title.to_lowercase();
let query_lower = query.to_lowercase();
if title_lower == query_lower {
return 1.0;
}
// Try with normalized forms (handles Bedetheque's "Name (Article)" convention)
let title_norm = normalize_title(title);
let query_norm = normalize_title(query);
if title_norm == query_norm {
return 1.0;
}
if title_lower.starts_with(&query_lower) || query_lower.starts_with(&title_lower)
|| title_norm.starts_with(&query_norm) || query_norm.starts_with(&title_norm)
{
0.85
} else if title_lower.contains(&query_lower) || query_lower.contains(&title_lower)
|| title_norm.contains(&query_norm) || query_norm.contains(&title_norm)
{
0.7
} else {
let common: usize = query_lower
.chars()
.filter(|c| title_lower.contains(*c))
.count();
let max_len = query_lower.len().max(title_lower.len()).max(1);
(common as f32 / max_len as f32).clamp(0.1, 0.6)
}
}

View File

@@ -0,0 +1,267 @@
use super::{BookCandidate, MetadataProvider, ProviderConfig, SeriesCandidate};
pub struct ComicVineProvider;
impl MetadataProvider for ComicVineProvider {
fn name(&self) -> &str {
"comicvine"
}
fn search_series(
&self,
query: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<SeriesCandidate>, String>> + Send + '_>,
> {
let query = query.to_string();
let config = config.clone();
Box::pin(async move { search_series_impl(&query, &config).await })
}
fn get_series_books(
&self,
external_id: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<BookCandidate>, String>> + Send + '_>,
> {
let external_id = external_id.to_string();
let config = config.clone();
Box::pin(async move { get_series_books_impl(&external_id, &config).await })
}
}
fn build_client() -> Result<reqwest::Client, String> {
reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.user_agent("StripstreamLibrarian/1.0")
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))
}
async fn search_series_impl(
query: &str,
config: &ProviderConfig,
) -> Result<Vec<SeriesCandidate>, String> {
let api_key = config
.api_key
.as_deref()
.filter(|k| !k.is_empty())
.ok_or_else(|| "ComicVine requires an API key. Configure it in Settings > Integrations.".to_string())?;
let client = build_client()?;
let url = format!(
"https://comicvine.gamespot.com/api/search/?api_key={}&format=json&resources=volume&query={}&limit=20",
api_key,
urlencoded(query)
);
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("ComicVine request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(format!("ComicVine returned {status}: {text}"));
}
let data: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("Failed to parse ComicVine response: {e}"))?;
let results = match data.get("results").and_then(|r| r.as_array()) {
Some(results) => results,
None => return Ok(vec![]),
};
let query_lower = query.to_lowercase();
let mut candidates: Vec<SeriesCandidate> = results
.iter()
.filter_map(|vol| {
let name = vol.get("name").and_then(|n| n.as_str())?.to_string();
let id = vol.get("id").and_then(|id| id.as_i64())?;
let description = vol
.get("description")
.and_then(|d| d.as_str())
.map(strip_html);
let publisher = vol
.get("publisher")
.and_then(|p| p.get("name"))
.and_then(|n| n.as_str())
.map(String::from);
let start_year = vol
.get("start_year")
.and_then(|y| y.as_str())
.and_then(|y| y.parse::<i32>().ok());
let count_of_issues = vol
.get("count_of_issues")
.and_then(|c| c.as_i64())
.map(|c| c as i32);
let cover_url = vol
.get("image")
.and_then(|img| img.get("medium_url").or_else(|| img.get("small_url")))
.and_then(|u| u.as_str())
.map(String::from);
let site_url = vol
.get("site_detail_url")
.and_then(|u| u.as_str())
.map(String::from);
let confidence = compute_confidence(&name, &query_lower);
Some(SeriesCandidate {
external_id: id.to_string(),
title: name,
authors: vec![],
description,
publishers: publisher.into_iter().collect(),
start_year,
total_volumes: count_of_issues,
cover_url,
external_url: site_url,
confidence,
metadata_json: serde_json::json!({}),
})
})
.collect();
candidates.sort_by(|a, b| b.confidence.partial_cmp(&a.confidence).unwrap_or(std::cmp::Ordering::Equal));
candidates.truncate(10);
Ok(candidates)
}
async fn get_series_books_impl(
external_id: &str,
config: &ProviderConfig,
) -> Result<Vec<BookCandidate>, String> {
let api_key = config
.api_key
.as_deref()
.filter(|k| !k.is_empty())
.ok_or_else(|| "ComicVine requires an API key".to_string())?;
let client = build_client()?;
let url = format!(
"https://comicvine.gamespot.com/api/issues/?api_key={}&format=json&filter=volume:{}&sort=issue_number:asc&limit=100&field_list=id,name,issue_number,description,image,cover_date,site_detail_url",
api_key,
external_id
);
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("ComicVine request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(format!("ComicVine returned {status}: {text}"));
}
let data: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("Failed to parse ComicVine response: {e}"))?;
let results = match data.get("results").and_then(|r| r.as_array()) {
Some(results) => results,
None => return Ok(vec![]),
};
let books: Vec<BookCandidate> = results
.iter()
.filter_map(|issue| {
let id = issue.get("id").and_then(|id| id.as_i64())?;
let name = issue
.get("name")
.and_then(|n| n.as_str())
.unwrap_or("")
.to_string();
let issue_number = issue
.get("issue_number")
.and_then(|n| n.as_str())
.and_then(|n| n.parse::<f64>().ok())
.map(|n| n as i32);
let description = issue
.get("description")
.and_then(|d| d.as_str())
.map(strip_html);
let cover_url = issue
.get("image")
.and_then(|img| img.get("medium_url").or_else(|| img.get("small_url")))
.and_then(|u| u.as_str())
.map(String::from);
let cover_date = issue
.get("cover_date")
.and_then(|d| d.as_str())
.map(String::from);
Some(BookCandidate {
external_book_id: id.to_string(),
title: name,
volume_number: issue_number,
authors: vec![],
isbn: None,
summary: description,
cover_url,
page_count: None,
language: None,
publish_date: cover_date,
metadata_json: serde_json::json!({}),
})
})
.collect();
Ok(books)
}
fn strip_html(s: &str) -> String {
let mut result = String::new();
let mut in_tag = false;
for ch in s.chars() {
match ch {
'<' => in_tag = true,
'>' => in_tag = false,
_ if !in_tag => result.push(ch),
_ => {}
}
}
result.trim().to_string()
}
fn compute_confidence(title: &str, query: &str) -> f32 {
let title_lower = title.to_lowercase();
if title_lower == query {
1.0
} else if title_lower.starts_with(query) || query.starts_with(&title_lower) {
0.8
} else if title_lower.contains(query) || query.contains(&title_lower) {
0.7
} else {
let common: usize = query.chars().filter(|c| title_lower.contains(*c)).count();
let max_len = query.len().max(title_lower.len()).max(1);
(common as f32 / max_len as f32).clamp(0.1, 0.6)
}
}
fn urlencoded(s: &str) -> String {
let mut result = String::new();
for byte in s.bytes() {
match byte {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
result.push(byte as char);
}
_ => result.push_str(&format!("%{:02X}", byte)),
}
}
result
}

View File

@@ -0,0 +1,472 @@
use super::{BookCandidate, MetadataProvider, ProviderConfig, SeriesCandidate};
pub struct GoogleBooksProvider;
impl MetadataProvider for GoogleBooksProvider {
fn name(&self) -> &str {
"google_books"
}
fn search_series(
&self,
query: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<SeriesCandidate>, String>> + Send + '_>,
> {
let query = query.to_string();
let config = config.clone();
Box::pin(async move { search_series_impl(&query, &config).await })
}
fn get_series_books(
&self,
external_id: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<BookCandidate>, String>> + Send + '_>,
> {
let external_id = external_id.to_string();
let config = config.clone();
Box::pin(async move { get_series_books_impl(&external_id, &config).await })
}
}
async fn search_series_impl(
query: &str,
config: &ProviderConfig,
) -> Result<Vec<SeriesCandidate>, String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))?;
let search_query = format!("intitle:{}", query);
let mut url = format!(
"https://www.googleapis.com/books/v1/volumes?q={}&maxResults=20&printType=books&langRestrict={}",
urlencoded(&search_query),
urlencoded(&config.language),
);
if let Some(ref key) = config.api_key {
url.push_str(&format!("&key={}", key));
}
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("Google Books request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(format!("Google Books returned {status}: {text}"));
}
let data: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("Failed to parse Google Books response: {e}"))?;
let items = match data.get("items").and_then(|i| i.as_array()) {
Some(items) => items,
None => return Ok(vec![]),
};
// Group volumes by series name to produce series candidates
let query_lower = query.to_lowercase();
let mut series_map: std::collections::HashMap<String, SeriesCandidateBuilder> =
std::collections::HashMap::new();
for item in items {
let volume_info = match item.get("volumeInfo") {
Some(vi) => vi,
None => continue,
};
let title = volume_info
.get("title")
.and_then(|t| t.as_str())
.unwrap_or("")
.to_string();
let authors: Vec<String> = volume_info
.get("authors")
.and_then(|a| a.as_array())
.map(|arr| {
arr.iter()
.filter_map(|v| v.as_str().map(String::from))
.collect()
})
.unwrap_or_default();
let publisher = volume_info
.get("publisher")
.and_then(|p| p.as_str())
.map(String::from);
let published_date = volume_info
.get("publishedDate")
.and_then(|d| d.as_str())
.map(String::from);
let description = volume_info
.get("description")
.and_then(|d| d.as_str())
.map(String::from);
// Extract series info from title or seriesInfo
let series_name = volume_info
.get("seriesInfo")
.and_then(|si| si.get("title"))
.and_then(|t| t.as_str())
.map(String::from)
.unwrap_or_else(|| extract_series_name(&title));
let cover_url = volume_info
.get("imageLinks")
.and_then(|il| {
il.get("thumbnail")
.or_else(|| il.get("smallThumbnail"))
})
.and_then(|u| u.as_str())
.map(|s| s.replace("http://", "https://"));
let google_id = item
.get("id")
.and_then(|id| id.as_str())
.unwrap_or("")
.to_string();
let entry = series_map
.entry(series_name.clone())
.or_insert_with(|| SeriesCandidateBuilder {
title: series_name.clone(),
authors: vec![],
description: None,
publishers: vec![],
start_year: None,
volume_count: 0,
cover_url: None,
external_id: google_id.clone(),
external_url: None,
metadata_json: serde_json::json!({}),
});
entry.volume_count += 1;
// Merge authors
for a in &authors {
if !entry.authors.contains(a) {
entry.authors.push(a.clone());
}
}
// Set description if not yet set
if entry.description.is_none() {
entry.description = description;
}
// Merge publisher
if let Some(ref pub_name) = publisher {
if !entry.publishers.contains(pub_name) {
entry.publishers.push(pub_name.clone());
}
}
// Extract year
if let Some(ref date) = published_date {
if let Some(year) = extract_year(date) {
if entry.start_year.is_none() || entry.start_year.unwrap() > year {
entry.start_year = Some(year);
}
}
}
if entry.cover_url.is_none() {
entry.cover_url = cover_url;
}
entry.external_url = Some(format!(
"https://books.google.com/books?id={}",
google_id
));
}
let mut candidates: Vec<SeriesCandidate> = series_map
.into_values()
.map(|b| {
let confidence = compute_confidence(&b.title, &query_lower);
SeriesCandidate {
external_id: b.external_id,
title: b.title,
authors: b.authors,
description: b.description,
publishers: b.publishers,
start_year: b.start_year,
total_volumes: if b.volume_count > 1 {
Some(b.volume_count)
} else {
None
},
cover_url: b.cover_url,
external_url: b.external_url,
confidence,
metadata_json: b.metadata_json,
}
})
.collect();
candidates.sort_by(|a, b| b.confidence.partial_cmp(&a.confidence).unwrap_or(std::cmp::Ordering::Equal));
candidates.truncate(10);
Ok(candidates)
}
async fn get_series_books_impl(
external_id: &str,
config: &ProviderConfig,
) -> Result<Vec<BookCandidate>, String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))?;
// First fetch the volume to get its series info
let mut url = format!(
"https://www.googleapis.com/books/v1/volumes/{}",
external_id
);
if let Some(ref key) = config.api_key {
url.push_str(&format!("?key={}", key));
}
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("Google Books request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(format!("Google Books returned {status}: {text}"));
}
let volume: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("Failed to parse Google Books response: {e}"))?;
let volume_info = volume.get("volumeInfo").cloned().unwrap_or(serde_json::json!({}));
let title = volume_info
.get("title")
.and_then(|t| t.as_str())
.unwrap_or("");
// Search for more volumes in this series
let series_name = extract_series_name(title);
let search_query = format!("intitle:{}", series_name);
let mut search_url = format!(
"https://www.googleapis.com/books/v1/volumes?q={}&maxResults=40&printType=books&langRestrict={}",
urlencoded(&search_query),
urlencoded(&config.language),
);
if let Some(ref key) = config.api_key {
search_url.push_str(&format!("&key={}", key));
}
let resp = client
.get(&search_url)
.send()
.await
.map_err(|e| format!("Google Books search failed: {e}"))?;
if !resp.status().is_success() {
// Return just the single volume as a book
return Ok(vec![volume_to_book_candidate(&volume)]);
}
let data: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("Failed to parse search response: {e}"))?;
let items = match data.get("items").and_then(|i| i.as_array()) {
Some(items) => items,
None => return Ok(vec![volume_to_book_candidate(&volume)]),
};
let mut books: Vec<BookCandidate> = items
.iter()
.map(volume_to_book_candidate)
.collect();
// Sort by volume number
books.sort_by_key(|b| b.volume_number.unwrap_or(999));
Ok(books)
}
fn volume_to_book_candidate(item: &serde_json::Value) -> BookCandidate {
let volume_info = item.get("volumeInfo").cloned().unwrap_or(serde_json::json!({}));
let title = volume_info
.get("title")
.and_then(|t| t.as_str())
.unwrap_or("")
.to_string();
let authors: Vec<String> = volume_info
.get("authors")
.and_then(|a| a.as_array())
.map(|arr| {
arr.iter()
.filter_map(|v| v.as_str().map(String::from))
.collect()
})
.unwrap_or_default();
let isbn = volume_info
.get("industryIdentifiers")
.and_then(|ids| ids.as_array())
.and_then(|arr| {
arr.iter()
.find(|id| {
id.get("type")
.and_then(|t| t.as_str())
.map(|t| t == "ISBN_13" || t == "ISBN_10")
.unwrap_or(false)
})
.and_then(|id| id.get("identifier").and_then(|i| i.as_str()))
})
.map(String::from);
let summary = volume_info
.get("description")
.and_then(|d| d.as_str())
.map(String::from);
let cover_url = volume_info
.get("imageLinks")
.and_then(|il| il.get("thumbnail").or_else(|| il.get("smallThumbnail")))
.and_then(|u| u.as_str())
.map(|s| s.replace("http://", "https://"));
let page_count = volume_info
.get("pageCount")
.and_then(|p| p.as_i64())
.map(|p| p as i32);
let language = volume_info
.get("language")
.and_then(|l| l.as_str())
.map(String::from);
let publish_date = volume_info
.get("publishedDate")
.and_then(|d| d.as_str())
.map(String::from);
let google_id = item
.get("id")
.and_then(|id| id.as_str())
.unwrap_or("")
.to_string();
let volume_number = extract_volume_number(&title);
BookCandidate {
external_book_id: google_id,
title,
volume_number,
authors,
isbn,
summary,
cover_url,
page_count,
language,
publish_date,
metadata_json: serde_json::json!({}),
}
}
fn extract_series_name(title: &str) -> String {
// Remove trailing volume indicators like "Vol. 1", "Tome 2", "#3", "- Volume 1"
let re_patterns = [
r"(?i)\s*[-–—]\s*(?:vol(?:ume)?\.?\s*|tome\s*|t\.\s*|#)\s*\d+.*$",
r"(?i)\s*,?\s*(?:vol(?:ume)?\.?\s*|tome\s*|t\.\s*|#)\s*\d+.*$",
r"\s*\(\d+\)\s*$",
r"\s+\d+\s*$",
];
let mut result = title.to_string();
for pattern in &re_patterns {
if let Ok(re) = regex::Regex::new(pattern) {
let cleaned = re.replace(&result, "").to_string();
if !cleaned.is_empty() {
result = cleaned;
break;
}
}
}
result.trim().to_string()
}
fn extract_volume_number(title: &str) -> Option<i32> {
let patterns = [
r"(?i)(?:vol(?:ume)?\.?\s*|tome\s*|t\.\s*|#)\s*(\d+)",
r"\((\d+)\)\s*$",
r"\b(\d+)\s*$",
];
for pattern in &patterns {
if let Ok(re) = regex::Regex::new(pattern) {
if let Some(caps) = re.captures(title) {
if let Some(num) = caps.get(1).and_then(|m| m.as_str().parse::<i32>().ok()) {
return Some(num);
}
}
}
}
None
}
fn extract_year(date: &str) -> Option<i32> {
date.get(..4).and_then(|s| s.parse::<i32>().ok())
}
fn compute_confidence(title: &str, query: &str) -> f32 {
let title_lower = title.to_lowercase();
if title_lower == query {
1.0
} else if title_lower.starts_with(query) || query.starts_with(&title_lower) {
0.8
} else if title_lower.contains(query) || query.contains(&title_lower) {
0.7
} else {
// Simple character overlap ratio
let common: usize = query
.chars()
.filter(|c| title_lower.contains(*c))
.count();
let max_len = query.len().max(title_lower.len()).max(1);
(common as f32 / max_len as f32).clamp(0.1, 0.6)
}
}
fn urlencoded(s: &str) -> String {
let mut result = String::new();
for byte in s.bytes() {
match byte {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
result.push(byte as char);
}
_ => {
result.push_str(&format!("%{:02X}", byte));
}
}
}
result
}
struct SeriesCandidateBuilder {
title: String,
authors: Vec<String>,
description: Option<String>,
publishers: Vec<String>,
start_year: Option<i32>,
volume_count: i32,
cover_url: Option<String>,
external_id: String,
external_url: Option<String>,
metadata_json: serde_json::Value,
}

View File

@@ -0,0 +1,295 @@
pub mod anilist;
pub mod bedetheque;
pub mod comicvine;
pub mod google_books;
pub mod open_library;
use serde::{Deserialize, Serialize};
/// Configuration passed to providers (API keys, etc.)
#[derive(Debug, Clone, Default)]
pub struct ProviderConfig {
pub api_key: Option<String>,
/// Preferred language for metadata results (ISO 639-1: "en", "fr", "es"). Defaults to "en".
pub language: String,
}
/// A candidate series returned by a provider search
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SeriesCandidate {
pub external_id: String,
pub title: String,
pub authors: Vec<String>,
pub description: Option<String>,
pub publishers: Vec<String>,
pub start_year: Option<i32>,
pub total_volumes: Option<i32>,
pub cover_url: Option<String>,
pub external_url: Option<String>,
pub confidence: f32,
pub metadata_json: serde_json::Value,
}
/// A candidate book within a series
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BookCandidate {
pub external_book_id: String,
pub title: String,
pub volume_number: Option<i32>,
pub authors: Vec<String>,
pub isbn: Option<String>,
pub summary: Option<String>,
pub cover_url: Option<String>,
pub page_count: Option<i32>,
pub language: Option<String>,
pub publish_date: Option<String>,
pub metadata_json: serde_json::Value,
}
/// Trait that all metadata providers must implement
pub trait MetadataProvider: Send + Sync {
#[allow(dead_code)]
fn name(&self) -> &str;
fn search_series(
&self,
query: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<SeriesCandidate>, String>> + Send + '_>,
>;
fn get_series_books(
&self,
external_id: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<BookCandidate>, String>> + Send + '_>,
>;
}
/// Factory function to get a provider by name
pub fn get_provider(name: &str) -> Option<Box<dyn MetadataProvider>> {
match name {
"google_books" => Some(Box::new(google_books::GoogleBooksProvider)),
"open_library" => Some(Box::new(open_library::OpenLibraryProvider)),
"comicvine" => Some(Box::new(comicvine::ComicVineProvider)),
"anilist" => Some(Box::new(anilist::AniListProvider)),
"bedetheque" => Some(Box::new(bedetheque::BedethequeProvider)),
_ => None,
}
}
// ---------------------------------------------------------------------------
// End-to-end provider tests
//
// These tests hit real external APIs — run them explicitly with:
// cargo test -p api providers_e2e -- --ignored --nocapture
// ---------------------------------------------------------------------------
#[cfg(test)]
mod providers_e2e {
use super::*;
fn config_fr() -> ProviderConfig {
ProviderConfig { api_key: None, language: "fr".to_string() }
}
fn config_en() -> ProviderConfig {
ProviderConfig { api_key: None, language: "en".to_string() }
}
fn print_candidate(name: &str, c: &SeriesCandidate) {
println!("\n=== {name} — best candidate ===");
println!(" title: {:?}", c.title);
println!(" external_id: {:?}", c.external_id);
println!(" authors: {:?}", c.authors);
println!(" description: {:?}", c.description.as_deref().map(|d| &d[..d.len().min(120)]));
println!(" publishers: {:?}", c.publishers);
println!(" start_year: {:?}", c.start_year);
println!(" total_volumes: {:?}", c.total_volumes);
println!(" cover_url: {}", c.cover_url.is_some());
println!(" external_url: {}", c.external_url.is_some());
println!(" confidence: {:.2}", c.confidence);
println!(" metadata_json: {}", serde_json::to_string_pretty(&c.metadata_json).unwrap_or_default());
}
fn print_books(name: &str, books: &[BookCandidate]) {
println!("\n=== {name}{} books ===", books.len());
for (i, b) in books.iter().take(5).enumerate() {
println!(
" [{}] vol={:?} title={:?} authors={} isbn={:?} pages={:?} lang={:?} date={:?} cover={}",
i, b.volume_number, b.title, b.authors.len(), b.isbn, b.page_count, b.language, b.publish_date, b.cover_url.is_some()
);
}
if books.len() > 5 { println!(" ... and {} more", books.len() - 5); }
let with_vol = books.iter().filter(|b| b.volume_number.is_some()).count();
let with_isbn = books.iter().filter(|b| b.isbn.is_some()).count();
let with_authors = books.iter().filter(|b| !b.authors.is_empty()).count();
let with_date = books.iter().filter(|b| b.publish_date.is_some()).count();
let with_cover = books.iter().filter(|b| b.cover_url.is_some()).count();
let with_pages = books.iter().filter(|b| b.page_count.is_some()).count();
println!(" --- field coverage ---");
println!(" volume_number: {with_vol}/{}", books.len());
println!(" isbn: {with_isbn}/{}", books.len());
println!(" authors: {with_authors}/{}", books.len());
println!(" publish_date: {with_date}/{}", books.len());
println!(" cover_url: {with_cover}/{}", books.len());
println!(" page_count: {with_pages}/{}", books.len());
}
// --- Google Books ---
#[tokio::test]
#[ignore]
async fn google_books_search_and_books() {
let p = get_provider("google_books").unwrap();
let cfg = config_en();
let candidates = p.search_series("Blacksad", &cfg).await.unwrap();
assert!(!candidates.is_empty(), "google_books: no results for Blacksad");
print_candidate("google_books", &candidates[0]);
let books = p.get_series_books(&candidates[0].external_id, &cfg).await.unwrap();
print_books("google_books", &books);
assert!(!books.is_empty(), "google_books: no books returned");
}
// --- Open Library ---
#[tokio::test]
#[ignore]
async fn open_library_search_and_books() {
let p = get_provider("open_library").unwrap();
let cfg = config_en();
let candidates = p.search_series("Sandman Neil Gaiman", &cfg).await.unwrap();
assert!(!candidates.is_empty(), "open_library: no results for Sandman");
print_candidate("open_library", &candidates[0]);
let books = p.get_series_books(&candidates[0].external_id, &cfg).await.unwrap();
print_books("open_library", &books);
assert!(!books.is_empty(), "open_library: no books returned");
}
// --- AniList ---
#[tokio::test]
#[ignore]
async fn anilist_search_finished() {
let p = get_provider("anilist").unwrap();
let cfg = config_fr();
let candidates = p.search_series("Death Note", &cfg).await.unwrap();
assert!(!candidates.is_empty(), "anilist: no results for Death Note");
print_candidate("anilist (finished)", &candidates[0]);
let best = &candidates[0];
assert!(best.total_volumes.is_some(), "anilist: finished series should have total_volumes");
assert!(best.description.is_some(), "anilist: should have description");
assert!(!best.authors.is_empty(), "anilist: should have authors");
let status = best.metadata_json.get("status").and_then(|s| s.as_str());
assert_eq!(status, Some("FINISHED"), "anilist: Death Note should be FINISHED");
let books = p.get_series_books(&best.external_id, &cfg).await.unwrap();
print_books("anilist (Death Note)", &books);
assert!(books.len() >= 12, "anilist: Death Note should have ≥12 volumes, got {}", books.len());
}
#[tokio::test]
#[ignore]
async fn anilist_search_ongoing() {
let p = get_provider("anilist").unwrap();
let cfg = config_fr();
let candidates = p.search_series("One Piece", &cfg).await.unwrap();
assert!(!candidates.is_empty(), "anilist: no results for One Piece");
print_candidate("anilist (ongoing)", &candidates[0]);
let best = &candidates[0];
let status = best.metadata_json.get("status").and_then(|s| s.as_str());
assert_eq!(status, Some("RELEASING"), "anilist: One Piece should be RELEASING");
let volume_source = best.metadata_json.get("volume_source").and_then(|s| s.as_str());
println!(" volume_source: {:?}", volume_source);
println!(" total_volumes: {:?}", best.total_volumes);
}
// --- Bédéthèque ---
#[tokio::test]
#[ignore]
async fn bedetheque_search_and_books() {
let p = get_provider("bedetheque").unwrap();
let cfg = config_fr();
let candidates = p.search_series("De Cape et de Crocs", &cfg).await.unwrap();
assert!(!candidates.is_empty(), "bedetheque: no results");
print_candidate("bedetheque", &candidates[0]);
let best = &candidates[0];
assert!(best.description.is_some(), "bedetheque: should have description");
assert!(!best.authors.is_empty(), "bedetheque: should have authors");
assert!(!best.publishers.is_empty(), "bedetheque: should have publishers");
assert!(best.start_year.is_some(), "bedetheque: should have start_year");
assert!(best.total_volumes.is_some(), "bedetheque: should have total_volumes");
// Enriched metadata_json
let mj = &best.metadata_json;
assert!(mj.get("genres").and_then(|g| g.as_array()).map(|a| !a.is_empty()).unwrap_or(false), "bedetheque: should have genres");
assert!(mj.get("status").and_then(|s| s.as_str()).is_some(), "bedetheque: should have status");
let books = p.get_series_books(&best.external_id, &cfg).await.unwrap();
print_books("bedetheque", &books);
assert!(books.len() >= 12, "bedetheque: De Cape et de Crocs should have ≥12 volumes, got {}", books.len());
}
// --- ComicVine (needs API key) ---
#[tokio::test]
#[ignore]
async fn comicvine_no_key() {
let p = get_provider("comicvine").unwrap();
let cfg = config_en();
let result = p.search_series("Batman", &cfg).await;
println!("\n=== comicvine (no key) ===");
match result {
Ok(c) => println!(" returned {} candidates (unexpected without key)", c.len()),
Err(e) => println!(" expected error: {e}"),
}
}
// --- Cross-provider comparison ---
#[tokio::test]
#[ignore]
async fn cross_provider_blacksad() {
println!("\n{}", "=".repeat(60));
println!(" Cross-provider comparison: Blacksad");
println!("{}\n", "=".repeat(60));
let providers: Vec<(&str, ProviderConfig)> = vec![
("google_books", config_en()),
("open_library", config_en()),
("anilist", config_fr()),
("bedetheque", config_fr()),
];
for (name, cfg) in &providers {
let p = get_provider(name).unwrap();
match p.search_series("Blacksad", cfg).await {
Ok(candidates) if !candidates.is_empty() => {
let b = &candidates[0];
println!("[{name}] title={:?} authors={} desc={} pubs={} year={:?} vols={:?} cover={} url={} conf={:.2}",
b.title, b.authors.len(), b.description.is_some(), b.publishers.len(),
b.start_year, b.total_volumes, b.cover_url.is_some(), b.external_url.is_some(), b.confidence);
}
Ok(_) => println!("[{name}] no results"),
Err(e) => println!("[{name}] error: {e}"),
}
}
}
}

View File

@@ -0,0 +1,351 @@
use super::{BookCandidate, MetadataProvider, ProviderConfig, SeriesCandidate};
pub struct OpenLibraryProvider;
impl MetadataProvider for OpenLibraryProvider {
fn name(&self) -> &str {
"open_library"
}
fn search_series(
&self,
query: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<SeriesCandidate>, String>> + Send + '_>,
> {
let query = query.to_string();
let config = config.clone();
Box::pin(async move { search_series_impl(&query, &config).await })
}
fn get_series_books(
&self,
external_id: &str,
config: &ProviderConfig,
) -> std::pin::Pin<
Box<dyn std::future::Future<Output = Result<Vec<BookCandidate>, String>> + Send + '_>,
> {
let external_id = external_id.to_string();
let config = config.clone();
Box::pin(async move { get_series_books_impl(&external_id, &config).await })
}
}
async fn search_series_impl(
query: &str,
config: &ProviderConfig,
) -> Result<Vec<SeriesCandidate>, String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))?;
// Open Library uses 3-letter language codes
let ol_lang = match config.language.as_str() {
"fr" => "fre",
"es" => "spa",
_ => "eng",
};
let url = format!(
"https://openlibrary.org/search.json?title={}&limit=20&language={}",
urlencoded(query),
ol_lang,
);
let resp = client
.get(&url)
.send()
.await
.map_err(|e| format!("Open Library request failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(format!("Open Library returned {status}: {text}"));
}
let data: serde_json::Value = resp
.json()
.await
.map_err(|e| format!("Failed to parse Open Library response: {e}"))?;
let docs = match data.get("docs").and_then(|d| d.as_array()) {
Some(docs) => docs,
None => return Ok(vec![]),
};
let query_lower = query.to_lowercase();
let mut series_map: std::collections::HashMap<String, SeriesCandidateBuilder> =
std::collections::HashMap::new();
for doc in docs {
let title = doc
.get("title")
.and_then(|t| t.as_str())
.unwrap_or("")
.to_string();
let authors: Vec<String> = doc
.get("author_name")
.and_then(|a| a.as_array())
.map(|arr| arr.iter().filter_map(|v| v.as_str().map(String::from)).collect())
.unwrap_or_default();
let publishers: Vec<String> = doc
.get("publisher")
.and_then(|a| a.as_array())
.map(|arr| {
let mut pubs: Vec<String> = arr.iter().filter_map(|v| v.as_str().map(String::from)).collect();
pubs.truncate(3);
pubs
})
.unwrap_or_default();
let first_publish_year = doc
.get("first_publish_year")
.and_then(|y| y.as_i64())
.map(|y| y as i32);
let cover_i = doc.get("cover_i").and_then(|c| c.as_i64());
let cover_url = cover_i.map(|id| format!("https://covers.openlibrary.org/b/id/{}-M.jpg", id));
let key = doc
.get("key")
.and_then(|k| k.as_str())
.unwrap_or("")
.to_string();
let series_name = extract_series_name(&title);
let entry = series_map
.entry(series_name.clone())
.or_insert_with(|| SeriesCandidateBuilder {
title: series_name.clone(),
authors: vec![],
description: None,
publishers: vec![],
start_year: None,
volume_count: 0,
cover_url: None,
external_id: key.clone(),
external_url: if key.is_empty() {
None
} else {
Some(format!("https://openlibrary.org{}", key))
},
});
entry.volume_count += 1;
for a in &authors {
if !entry.authors.contains(a) {
entry.authors.push(a.clone());
}
}
for p in &publishers {
if !entry.publishers.contains(p) {
entry.publishers.push(p.clone());
}
}
if (entry.start_year.is_none() || first_publish_year.is_some_and(|y| entry.start_year.unwrap() > y))
&& first_publish_year.is_some()
{
entry.start_year = first_publish_year;
}
if entry.cover_url.is_none() {
entry.cover_url = cover_url;
}
}
let mut candidates: Vec<SeriesCandidate> = series_map
.into_values()
.map(|b| {
let confidence = compute_confidence(&b.title, &query_lower);
SeriesCandidate {
external_id: b.external_id,
title: b.title,
authors: b.authors,
description: b.description,
publishers: b.publishers,
start_year: b.start_year,
total_volumes: if b.volume_count > 1 { Some(b.volume_count) } else { None },
cover_url: b.cover_url,
external_url: b.external_url,
confidence,
metadata_json: serde_json::json!({}),
}
})
.collect();
candidates.sort_by(|a, b| b.confidence.partial_cmp(&a.confidence).unwrap_or(std::cmp::Ordering::Equal));
candidates.truncate(10);
Ok(candidates)
}
async fn get_series_books_impl(
external_id: &str,
_config: &ProviderConfig,
) -> Result<Vec<BookCandidate>, String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(15))
.build()
.map_err(|e| format!("failed to build HTTP client: {e}"))?;
// Fetch the work to get its title for series search
let url = format!("https://openlibrary.org{}.json", external_id);
let resp = client.get(&url).send().await.map_err(|e| format!("Open Library request failed: {e}"))?;
let work: serde_json::Value = if resp.status().is_success() {
resp.json().await.map_err(|e| format!("Failed to parse response: {e}"))?
} else {
serde_json::json!({})
};
let title = work.get("title").and_then(|t| t.as_str()).unwrap_or("");
let series_name = extract_series_name(title);
// Search for editions of this series
let search_url = format!(
"https://openlibrary.org/search.json?title={}&limit=40",
urlencoded(&series_name)
);
let resp = client.get(&search_url).send().await.map_err(|e| format!("Open Library search failed: {e}"))?;
if !resp.status().is_success() {
return Ok(vec![]);
}
let data: serde_json::Value = resp.json().await.map_err(|e| format!("Failed to parse response: {e}"))?;
let docs = match data.get("docs").and_then(|d| d.as_array()) {
Some(docs) => docs,
None => return Ok(vec![]),
};
let mut books: Vec<BookCandidate> = docs
.iter()
.map(|doc| {
let title = doc.get("title").and_then(|t| t.as_str()).unwrap_or("").to_string();
let authors: Vec<String> = doc
.get("author_name")
.and_then(|a| a.as_array())
.map(|arr| arr.iter().filter_map(|v| v.as_str().map(String::from)).collect())
.unwrap_or_default();
let isbn = doc
.get("isbn")
.and_then(|a| a.as_array())
.and_then(|arr| arr.first())
.and_then(|v| v.as_str())
.map(String::from);
let page_count = doc
.get("number_of_pages_median")
.and_then(|n| n.as_i64())
.map(|n| n as i32);
let cover_i = doc.get("cover_i").and_then(|c| c.as_i64());
let cover_url = cover_i.map(|id| format!("https://covers.openlibrary.org/b/id/{}-M.jpg", id));
let language = doc
.get("language")
.and_then(|a| a.as_array())
.and_then(|arr| arr.first())
.and_then(|v| v.as_str())
.map(String::from);
let publish_date = doc
.get("first_publish_year")
.and_then(|y| y.as_i64())
.map(|y| y.to_string());
let key = doc.get("key").and_then(|k| k.as_str()).unwrap_or("").to_string();
let volume_number = extract_volume_number(&title);
BookCandidate {
external_book_id: key,
title,
volume_number,
authors,
isbn,
summary: None,
cover_url,
page_count,
language,
publish_date,
metadata_json: serde_json::json!({}),
}
})
.collect();
books.sort_by_key(|b| b.volume_number.unwrap_or(999));
Ok(books)
}
fn extract_series_name(title: &str) -> String {
let re_patterns = [
r"(?i)\s*[-–—]\s*(?:vol(?:ume)?\.?\s*|tome\s*|t\.\s*|#)\s*\d+.*$",
r"(?i)\s*,?\s*(?:vol(?:ume)?\.?\s*|tome\s*|t\.\s*|#)\s*\d+.*$",
r"\s*\(\d+\)\s*$",
r"\s+\d+\s*$",
];
let mut result = title.to_string();
for pattern in &re_patterns {
if let Ok(re) = regex::Regex::new(pattern) {
let cleaned = re.replace(&result, "").to_string();
if !cleaned.is_empty() {
result = cleaned;
break;
}
}
}
result.trim().to_string()
}
fn extract_volume_number(title: &str) -> Option<i32> {
let patterns = [
r"(?i)(?:vol(?:ume)?\.?\s*|tome\s*|t\.\s*|#)\s*(\d+)",
r"\((\d+)\)\s*$",
r"\b(\d+)\s*$",
];
for pattern in &patterns {
if let Ok(re) = regex::Regex::new(pattern) {
if let Some(caps) = re.captures(title) {
if let Some(num) = caps.get(1).and_then(|m| m.as_str().parse::<i32>().ok()) {
return Some(num);
}
}
}
}
None
}
fn compute_confidence(title: &str, query: &str) -> f32 {
let title_lower = title.to_lowercase();
if title_lower == query {
1.0
} else if title_lower.starts_with(query) || query.starts_with(&title_lower) {
0.8
} else if title_lower.contains(query) || query.contains(&title_lower) {
0.7
} else {
let common: usize = query.chars().filter(|c| title_lower.contains(*c)).count();
let max_len = query.len().max(title_lower.len()).max(1);
(common as f32 / max_len as f32).clamp(0.1, 0.6)
}
}
fn urlencoded(s: &str) -> String {
let mut result = String::new();
for byte in s.bytes() {
match byte {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
result.push(byte as char);
}
_ => result.push_str(&format!("%{:02X}", byte)),
}
}
result
}
struct SeriesCandidateBuilder {
title: String,
authors: Vec<String>,
description: Option<String>,
publishers: Vec<String>,
start_year: Option<i32>,
volume_count: i32,
cover_url: Option<String>,
external_id: String,
external_url: Option<String>,
}

View File

@@ -0,0 +1,825 @@
use axum::{
extract::{Path as AxumPath, State},
Json,
};
use serde::{Deserialize, Serialize};
use sqlx::{PgPool, Row};
use uuid::Uuid;
use utoipa::ToSchema;
use tracing::{info, warn};
use crate::{error::ApiError, metadata_providers, state::AppState};
use crate::metadata_batch::{load_provider_config_from_pool, is_job_cancelled, update_progress};
// ---------------------------------------------------------------------------
// DTOs
// ---------------------------------------------------------------------------
#[derive(Deserialize, ToSchema)]
pub struct MetadataRefreshRequest {
pub library_id: String,
}
/// A single field change: old → new
#[derive(Serialize, Clone)]
struct FieldDiff {
field: String,
#[serde(skip_serializing_if = "Option::is_none")]
old: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
new: Option<serde_json::Value>,
}
/// Per-book changes
#[derive(Serialize, Clone)]
struct BookDiff {
book_id: String,
title: String,
volume: Option<i32>,
changes: Vec<FieldDiff>,
}
/// Per-series change report
#[derive(Serialize, Clone)]
struct SeriesRefreshResult {
series_name: String,
provider: String,
status: String, // "updated", "unchanged", "error"
series_changes: Vec<FieldDiff>,
book_changes: Vec<BookDiff>,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
}
/// Response DTO for the report endpoint
#[derive(Serialize, ToSchema)]
pub struct MetadataRefreshReportDto {
#[schema(value_type = String)]
pub job_id: Uuid,
pub status: String,
pub total_links: i64,
pub refreshed: i64,
pub unchanged: i64,
pub errors: i64,
pub changes: serde_json::Value,
}
// ---------------------------------------------------------------------------
// POST /metadata/refresh — Trigger a metadata refresh job
// ---------------------------------------------------------------------------
#[utoipa::path(
post,
path = "/metadata/refresh",
tag = "metadata",
request_body = MetadataRefreshRequest,
responses(
(status = 200, description = "Job created"),
(status = 400, description = "Bad request"),
),
security(("Bearer" = []))
)]
pub async fn start_refresh(
State(state): State<AppState>,
Json(body): Json<MetadataRefreshRequest>,
) -> Result<Json<serde_json::Value>, ApiError> {
let library_id: Uuid = body
.library_id
.parse()
.map_err(|_| ApiError::bad_request("invalid library_id"))?;
// Verify library exists
sqlx::query("SELECT 1 FROM libraries WHERE id = $1")
.bind(library_id)
.fetch_optional(&state.pool)
.await?
.ok_or_else(|| ApiError::not_found("library not found"))?;
// Check no existing running metadata_refresh job for this library
let existing: Option<Uuid> = sqlx::query_scalar(
"SELECT id FROM index_jobs WHERE library_id = $1 AND type = 'metadata_refresh' AND status IN ('pending', 'running') LIMIT 1",
)
.bind(library_id)
.fetch_optional(&state.pool)
.await?;
if let Some(existing_id) = existing {
return Ok(Json(serde_json::json!({
"id": existing_id.to_string(),
"status": "already_running",
})));
}
// Check there are approved links to refresh
let link_count: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM external_metadata_links WHERE library_id = $1 AND status = 'approved'",
)
.bind(library_id)
.fetch_one(&state.pool)
.await?;
if link_count == 0 {
return Err(ApiError::bad_request("No approved metadata links to refresh for this library"));
}
let job_id = Uuid::new_v4();
sqlx::query(
"INSERT INTO index_jobs (id, library_id, type, status) VALUES ($1, $2, 'metadata_refresh', 'pending')",
)
.bind(job_id)
.bind(library_id)
.execute(&state.pool)
.await?;
// Spawn the background processing task
let pool = state.pool.clone();
let library_name: Option<String> = sqlx::query_scalar("SELECT name FROM libraries WHERE id = $1")
.bind(library_id)
.fetch_optional(&state.pool)
.await
.ok()
.flatten();
tokio::spawn(async move {
if let Err(e) = process_metadata_refresh(&pool, job_id, library_id).await {
warn!("[METADATA_REFRESH] job {job_id} failed: {e}");
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'failed', error_opt = $2, finished_at = NOW() WHERE id = $1",
)
.bind(job_id)
.bind(e.to_string())
.execute(&pool)
.await;
notifications::notify(
pool.clone(),
notifications::NotificationEvent::MetadataRefreshFailed {
library_name,
error: e.to_string(),
},
);
}
});
Ok(Json(serde_json::json!({
"id": job_id.to_string(),
"status": "pending",
})))
}
// ---------------------------------------------------------------------------
// GET /metadata/refresh/:id/report — Refresh report from stats_json
// ---------------------------------------------------------------------------
#[utoipa::path(
get,
path = "/metadata/refresh/{id}/report",
tag = "metadata",
params(("id" = String, Path, description = "Job UUID")),
responses(
(status = 200, body = MetadataRefreshReportDto),
(status = 404, description = "Job not found"),
),
security(("Bearer" = []))
)]
pub async fn get_refresh_report(
State(state): State<AppState>,
AxumPath(job_id): AxumPath<Uuid>,
) -> Result<Json<MetadataRefreshReportDto>, ApiError> {
let row = sqlx::query(
"SELECT status, stats_json, total_files FROM index_jobs WHERE id = $1 AND type = 'metadata_refresh'",
)
.bind(job_id)
.fetch_optional(&state.pool)
.await?
.ok_or_else(|| ApiError::not_found("job not found"))?;
let job_status: String = row.get("status");
let stats: Option<serde_json::Value> = row.get("stats_json");
let total_files: Option<i32> = row.get("total_files");
let (refreshed, unchanged, errors, changes) = if let Some(ref s) = stats {
(
s.get("refreshed").and_then(|v| v.as_i64()).unwrap_or(0),
s.get("unchanged").and_then(|v| v.as_i64()).unwrap_or(0),
s.get("errors").and_then(|v| v.as_i64()).unwrap_or(0),
s.get("changes").cloned().unwrap_or(serde_json::json!([])),
)
} else {
(0, 0, 0, serde_json::json!([]))
};
Ok(Json(MetadataRefreshReportDto {
job_id,
status: job_status,
total_links: total_files.unwrap_or(0) as i64,
refreshed,
unchanged,
errors,
changes,
}))
}
// ---------------------------------------------------------------------------
// Background processing
// ---------------------------------------------------------------------------
async fn process_metadata_refresh(
pool: &PgPool,
job_id: Uuid,
library_id: Uuid,
) -> Result<(), String> {
// Set job to running
sqlx::query("UPDATE index_jobs SET status = 'running', started_at = NOW() WHERE id = $1")
.bind(job_id)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
// Get all approved links for this library
let links: Vec<(Uuid, String, String, String)> = sqlx::query_as(
r#"
SELECT id, series_name, provider, external_id
FROM external_metadata_links
WHERE library_id = $1 AND status = 'approved'
ORDER BY series_name
"#,
)
.bind(library_id)
.fetch_all(pool)
.await
.map_err(|e| e.to_string())?;
let total = links.len() as i32;
sqlx::query("UPDATE index_jobs SET total_files = $2 WHERE id = $1")
.bind(job_id)
.bind(total)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
let mut processed = 0i32;
let mut refreshed = 0i32;
let mut unchanged = 0i32;
let mut errors = 0i32;
let mut all_results: Vec<SeriesRefreshResult> = Vec::new();
for (link_id, series_name, provider_name, external_id) in &links {
// Check cancellation
if is_job_cancelled(pool, job_id).await {
sqlx::query(
"UPDATE index_jobs SET status = 'cancelled', finished_at = NOW() WHERE id = $1",
)
.bind(job_id)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
return Ok(());
}
match refresh_link(pool, *link_id, library_id, series_name, provider_name, external_id).await {
Ok(result) => {
if result.status == "updated" {
refreshed += 1;
info!("[METADATA_REFRESH] job={job_id} updated series='{series_name}' via {provider_name}");
} else {
unchanged += 1;
}
all_results.push(result);
}
Err(e) => {
errors += 1;
warn!("[METADATA_REFRESH] job={job_id} error on series='{series_name}': {e}");
all_results.push(SeriesRefreshResult {
series_name: series_name.clone(),
provider: provider_name.clone(),
status: "error".to_string(),
series_changes: vec![],
book_changes: vec![],
error: Some(e),
});
}
}
processed += 1;
update_progress(pool, job_id, processed, total, series_name).await;
// Rate limit: 1s delay between provider calls
tokio::time::sleep(std::time::Duration::from_millis(1000)).await;
}
// Only keep series that have changes or errors (filter out "unchanged")
let changes_only: Vec<&SeriesRefreshResult> = all_results
.iter()
.filter(|r| r.status != "unchanged")
.collect();
// Build stats summary
let stats = serde_json::json!({
"total_links": total,
"refreshed": refreshed,
"unchanged": unchanged,
"errors": errors,
"changes": changes_only,
});
sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, stats_json = $2 WHERE id = $1",
)
.bind(job_id)
.bind(stats)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
info!("[METADATA_REFRESH] job={job_id} completed: {refreshed} updated, {unchanged} unchanged, {errors} errors");
let library_name: Option<String> = sqlx::query_scalar("SELECT name FROM libraries WHERE id = $1")
.bind(library_id)
.fetch_optional(pool)
.await
.ok()
.flatten();
notifications::notify(
pool.clone(),
notifications::NotificationEvent::MetadataRefreshCompleted {
library_name,
refreshed,
unchanged,
errors,
},
);
Ok(())
}
/// Refresh a single approved metadata link: re-fetch from provider, compare, sync, return diff
async fn refresh_link(
pool: &PgPool,
link_id: Uuid,
library_id: Uuid,
series_name: &str,
provider_name: &str,
external_id: &str,
) -> Result<SeriesRefreshResult, String> {
let provider = metadata_providers::get_provider(provider_name)
.ok_or_else(|| format!("Unknown provider: {provider_name}"))?;
let config = load_provider_config_from_pool(pool, provider_name).await;
let mut series_changes: Vec<FieldDiff> = Vec::new();
let mut book_changes: Vec<BookDiff> = Vec::new();
// ── Series-level refresh ──────────────────────────────────────────────
let candidates = provider
.search_series(series_name, &config)
.await
.map_err(|e| format!("provider search error: {e}"))?;
let candidate = candidates
.iter()
.find(|c| c.external_id == external_id)
.or_else(|| candidates.first());
if let Some(candidate) = candidate {
// Update link metadata_json
sqlx::query(
r#"
UPDATE external_metadata_links
SET metadata_json = $2,
total_volumes_external = $3,
updated_at = NOW()
WHERE id = $1
"#,
)
.bind(link_id)
.bind(&candidate.metadata_json)
.bind(candidate.total_volumes)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
// Diff + sync series metadata
series_changes = sync_series_with_diff(pool, library_id, series_name, candidate).await?;
}
// ── Book-level refresh ────────────────────────────────────────────────
let books = provider
.get_series_books(external_id, &config)
.await
.map_err(|e| format!("provider books error: {e}"))?;
// Delete existing external_book_metadata for this link
sqlx::query("DELETE FROM external_book_metadata WHERE link_id = $1")
.bind(link_id)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
// Pre-fetch local books
let local_books: Vec<(Uuid, Option<i32>, String)> = sqlx::query_as(
r#"
SELECT id, volume, title FROM books
WHERE library_id = $1
AND COALESCE(NULLIF(series, ''), 'unclassified') = $2
ORDER BY volume NULLS LAST,
REGEXP_REPLACE(LOWER(title), '[0-9].*$', ''),
COALESCE((REGEXP_MATCH(LOWER(title), '\d+'))[1]::int, 0),
title ASC
"#,
)
.bind(library_id)
.bind(series_name)
.fetch_all(pool)
.await
.map_err(|e| e.to_string())?;
let local_books_with_pos: Vec<(Uuid, i32, String)> = local_books
.iter()
.enumerate()
.map(|(idx, (id, vol, title))| (*id, vol.unwrap_or((idx + 1) as i32), title.clone()))
.collect();
let mut matched_local_ids = std::collections::HashSet::new();
for (ext_idx, book) in books.iter().enumerate() {
let ext_vol = book.volume_number.unwrap_or((ext_idx + 1) as i32);
// Match by volume number
let mut local_book_id: Option<Uuid> = local_books_with_pos
.iter()
.find(|(id, v, _)| *v == ext_vol && !matched_local_ids.contains(id))
.map(|(id, _, _)| *id);
// Match by title containment
if local_book_id.is_none() {
let ext_title_lower = book.title.to_lowercase();
local_book_id = local_books_with_pos
.iter()
.find(|(id, _, local_title)| {
if matched_local_ids.contains(id) {
return false;
}
let local_lower = local_title.to_lowercase();
local_lower.contains(&ext_title_lower) || ext_title_lower.contains(&local_lower)
})
.map(|(id, _, _)| *id);
}
if let Some(id) = local_book_id {
matched_local_ids.insert(id);
}
// Insert external_book_metadata
sqlx::query(
r#"
INSERT INTO external_book_metadata
(link_id, book_id, external_book_id, volume_number, title, authors, isbn, summary, cover_url, page_count, language, publish_date, metadata_json)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
"#,
)
.bind(link_id)
.bind(local_book_id)
.bind(&book.external_book_id)
.bind(book.volume_number)
.bind(&book.title)
.bind(&book.authors)
.bind(&book.isbn)
.bind(&book.summary)
.bind(&book.cover_url)
.bind(book.page_count)
.bind(&book.language)
.bind(&book.publish_date)
.bind(&book.metadata_json)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
// Diff + push metadata to matched local book
if let Some(book_id) = local_book_id {
let diffs = sync_book_with_diff(pool, book_id, book).await?;
if !diffs.is_empty() {
let local_title = local_books_with_pos
.iter()
.find(|(id, _, _)| *id == book_id)
.map(|(_, _, t)| t.clone())
.unwrap_or_default();
book_changes.push(BookDiff {
book_id: book_id.to_string(),
title: local_title,
volume: book.volume_number,
changes: diffs,
});
}
}
}
// Update synced_at on the link
sqlx::query("UPDATE external_metadata_links SET synced_at = NOW(), updated_at = NOW() WHERE id = $1")
.bind(link_id)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
let has_changes = !series_changes.is_empty() || !book_changes.is_empty();
Ok(SeriesRefreshResult {
series_name: series_name.to_string(),
provider: provider_name.to_string(),
status: if has_changes { "updated".to_string() } else { "unchanged".to_string() },
series_changes,
book_changes,
error: None,
})
}
// ---------------------------------------------------------------------------
// Diff helpers
// ---------------------------------------------------------------------------
/// Compare old/new for a nullable string field. Returns Some(FieldDiff) only if value actually changed.
fn diff_opt_str(field: &str, old: Option<&str>, new: Option<&str>) -> Option<FieldDiff> {
let new_val = new.filter(|s| !s.is_empty());
// Only report a change if there is a new non-empty value AND it differs from old
match (old, new_val) {
(Some(o), Some(n)) if o != n => Some(FieldDiff {
field: field.to_string(),
old: Some(serde_json::Value::String(o.to_string())),
new: Some(serde_json::Value::String(n.to_string())),
}),
(None, Some(n)) => Some(FieldDiff {
field: field.to_string(),
old: None,
new: Some(serde_json::Value::String(n.to_string())),
}),
_ => None,
}
}
fn diff_opt_i32(field: &str, old: Option<i32>, new: Option<i32>) -> Option<FieldDiff> {
match (old, new) {
(Some(o), Some(n)) if o != n => Some(FieldDiff {
field: field.to_string(),
old: Some(serde_json::json!(o)),
new: Some(serde_json::json!(n)),
}),
(None, Some(n)) => Some(FieldDiff {
field: field.to_string(),
old: None,
new: Some(serde_json::json!(n)),
}),
_ => None,
}
}
fn diff_str_vec(field: &str, old: &[String], new: &[String]) -> Option<FieldDiff> {
if new.is_empty() {
return None;
}
if old != new {
Some(FieldDiff {
field: field.to_string(),
old: Some(serde_json::json!(old)),
new: Some(serde_json::json!(new)),
})
} else {
None
}
}
// ---------------------------------------------------------------------------
// Series sync with diff tracking
// ---------------------------------------------------------------------------
async fn sync_series_with_diff(
pool: &PgPool,
library_id: Uuid,
series_name: &str,
candidate: &metadata_providers::SeriesCandidate,
) -> Result<Vec<FieldDiff>, String> {
let new_description = candidate.metadata_json
.get("description")
.and_then(|d| d.as_str())
.or(candidate.description.as_deref());
let new_authors = &candidate.authors;
let new_publishers = &candidate.publishers;
let new_start_year = candidate.start_year;
let new_total_volumes = candidate.total_volumes;
let new_status = if let Some(raw) = candidate.metadata_json.get("status").and_then(|s| s.as_str()) {
Some(crate::metadata::normalize_series_status(pool, raw).await)
} else {
None
};
let new_status = new_status.as_deref();
// Fetch existing series metadata for diffing
let existing = sqlx::query(
r#"SELECT description, publishers, start_year, total_volumes, status, authors, locked_fields
FROM series_metadata WHERE library_id = $1 AND name = $2"#,
)
.bind(library_id)
.bind(series_name)
.fetch_optional(pool)
.await
.map_err(|e| e.to_string())?;
let locked = existing
.as_ref()
.map(|r| r.get::<serde_json::Value, _>("locked_fields"))
.unwrap_or(serde_json::json!({}));
let is_locked = |field: &str| -> bool {
locked.get(field).and_then(|v| v.as_bool()).unwrap_or(false)
};
// Build diffs (only for unlocked fields that actually change)
let mut diffs: Vec<FieldDiff> = Vec::new();
if !is_locked("description") {
let old_desc: Option<String> = existing.as_ref().and_then(|r| r.get("description"));
if let Some(d) = diff_opt_str("description", old_desc.as_deref(), new_description) {
diffs.push(d);
}
}
if !is_locked("authors") {
let old_authors: Vec<String> = existing.as_ref().map(|r| r.get("authors")).unwrap_or_default();
if let Some(d) = diff_str_vec("authors", &old_authors, new_authors) {
diffs.push(d);
}
}
if !is_locked("publishers") {
let old_publishers: Vec<String> = existing.as_ref().map(|r| r.get("publishers")).unwrap_or_default();
if let Some(d) = diff_str_vec("publishers", &old_publishers, new_publishers) {
diffs.push(d);
}
}
if !is_locked("start_year") {
let old_year: Option<i32> = existing.as_ref().and_then(|r| r.get("start_year"));
if let Some(d) = diff_opt_i32("start_year", old_year, new_start_year) {
diffs.push(d);
}
}
if !is_locked("total_volumes") {
let old_vols: Option<i32> = existing.as_ref().and_then(|r| r.get("total_volumes"));
if let Some(d) = diff_opt_i32("total_volumes", old_vols, new_total_volumes) {
diffs.push(d);
}
}
if !is_locked("status") {
let old_status: Option<String> = existing.as_ref().and_then(|r| r.get("status"));
if let Some(d) = diff_opt_str("status", old_status.as_deref(), new_status) {
diffs.push(d);
}
}
// Now do the actual upsert
sqlx::query(
r#"
INSERT INTO series_metadata (library_id, name, description, publishers, start_year, total_volumes, status, authors, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, NOW(), NOW())
ON CONFLICT (library_id, name)
DO UPDATE SET
description = CASE
WHEN (series_metadata.locked_fields->>'description')::boolean IS TRUE THEN series_metadata.description
ELSE COALESCE(NULLIF(EXCLUDED.description, ''), series_metadata.description)
END,
publishers = CASE
WHEN (series_metadata.locked_fields->>'publishers')::boolean IS TRUE THEN series_metadata.publishers
WHEN array_length(EXCLUDED.publishers, 1) > 0 THEN EXCLUDED.publishers
ELSE series_metadata.publishers
END,
start_year = CASE
WHEN (series_metadata.locked_fields->>'start_year')::boolean IS TRUE THEN series_metadata.start_year
ELSE COALESCE(EXCLUDED.start_year, series_metadata.start_year)
END,
total_volumes = CASE
WHEN (series_metadata.locked_fields->>'total_volumes')::boolean IS TRUE THEN series_metadata.total_volumes
ELSE COALESCE(EXCLUDED.total_volumes, series_metadata.total_volumes)
END,
status = CASE
WHEN (series_metadata.locked_fields->>'status')::boolean IS TRUE THEN series_metadata.status
ELSE COALESCE(EXCLUDED.status, series_metadata.status)
END,
authors = CASE
WHEN (series_metadata.locked_fields->>'authors')::boolean IS TRUE THEN series_metadata.authors
WHEN array_length(EXCLUDED.authors, 1) > 0 THEN EXCLUDED.authors
ELSE series_metadata.authors
END,
updated_at = NOW()
"#,
)
.bind(library_id)
.bind(series_name)
.bind(new_description)
.bind(new_publishers)
.bind(new_start_year)
.bind(new_total_volumes)
.bind(new_status)
.bind(new_authors)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
Ok(diffs)
}
// ---------------------------------------------------------------------------
// Book sync with diff tracking
// ---------------------------------------------------------------------------
async fn sync_book_with_diff(
pool: &PgPool,
book_id: Uuid,
ext_book: &metadata_providers::BookCandidate,
) -> Result<Vec<FieldDiff>, String> {
// Fetch current book state
let current = sqlx::query(
"SELECT summary, isbn, publish_date, language, authors, locked_fields FROM books WHERE id = $1",
)
.bind(book_id)
.fetch_one(pool)
.await
.map_err(|e| e.to_string())?;
let locked = current.get::<serde_json::Value, _>("locked_fields");
let is_locked = |field: &str| -> bool {
locked.get(field).and_then(|v| v.as_bool()).unwrap_or(false)
};
// Build diffs
let mut diffs: Vec<FieldDiff> = Vec::new();
if !is_locked("summary") {
let old: Option<String> = current.get("summary");
if let Some(d) = diff_opt_str("summary", old.as_deref(), ext_book.summary.as_deref()) {
diffs.push(d);
}
}
if !is_locked("isbn") {
let old: Option<String> = current.get("isbn");
if let Some(d) = diff_opt_str("isbn", old.as_deref(), ext_book.isbn.as_deref()) {
diffs.push(d);
}
}
if !is_locked("publish_date") {
let old: Option<String> = current.get("publish_date");
if let Some(d) = diff_opt_str("publish_date", old.as_deref(), ext_book.publish_date.as_deref()) {
diffs.push(d);
}
}
if !is_locked("language") {
let old: Option<String> = current.get("language");
if let Some(d) = diff_opt_str("language", old.as_deref(), ext_book.language.as_deref()) {
diffs.push(d);
}
}
if !is_locked("authors") {
let old: Vec<String> = current.get("authors");
if let Some(d) = diff_str_vec("authors", &old, &ext_book.authors) {
diffs.push(d);
}
}
// Do the actual update
sqlx::query(
r#"
UPDATE books SET
summary = CASE
WHEN (locked_fields->>'summary')::boolean IS TRUE THEN summary
ELSE COALESCE(NULLIF($2, ''), summary)
END,
isbn = CASE
WHEN (locked_fields->>'isbn')::boolean IS TRUE THEN isbn
ELSE COALESCE(NULLIF($3, ''), isbn)
END,
publish_date = CASE
WHEN (locked_fields->>'publish_date')::boolean IS TRUE THEN publish_date
ELSE COALESCE(NULLIF($4, ''), publish_date)
END,
language = CASE
WHEN (locked_fields->>'language')::boolean IS TRUE THEN language
ELSE COALESCE(NULLIF($5, ''), language)
END,
authors = CASE
WHEN (locked_fields->>'authors')::boolean IS TRUE THEN authors
WHEN CARDINALITY($6::text[]) > 0 THEN $6
ELSE authors
END,
author = CASE
WHEN (locked_fields->>'authors')::boolean IS TRUE THEN author
WHEN CARDINALITY($6::text[]) > 0 THEN $6[1]
ELSE author
END,
updated_at = NOW()
WHERE id = $1
"#,
)
.bind(book_id)
.bind(&ext_book.summary)
.bind(&ext_book.isbn)
.bind(&ext_book.publish_date)
.bind(&ext_book.language)
.bind(&ext_book.authors)
.execute(pool)
.await
.map_err(|e| e.to_string())?;
Ok(diffs)
}

View File

@@ -6,7 +6,18 @@ use utoipa::OpenApi;
paths( paths(
crate::books::list_books, crate::books::list_books,
crate::books::get_book, crate::books::get_book,
crate::books::list_series, crate::reading_progress::get_reading_progress,
crate::reading_progress::update_reading_progress,
crate::reading_progress::mark_series_read,
crate::books::get_thumbnail,
crate::series::list_series,
crate::series::list_all_series,
crate::series::ongoing_series,
crate::series::ongoing_books,
crate::books::convert_book,
crate::books::update_book,
crate::series::get_series_metadata,
crate::series::update_series,
crate::pages::get_page, crate::pages::get_page,
crate::search::search_books, crate::search::search_books,
crate::index_jobs::enqueue_rebuild, crate::index_jobs::enqueue_rebuild,
@@ -24,9 +35,43 @@ use utoipa::OpenApi;
crate::libraries::delete_library, crate::libraries::delete_library,
crate::libraries::scan_library, crate::libraries::scan_library,
crate::libraries::update_monitoring, crate::libraries::update_monitoring,
crate::libraries::update_metadata_provider,
crate::tokens::list_tokens, crate::tokens::list_tokens,
crate::tokens::create_token, crate::tokens::create_token,
crate::tokens::revoke_token, crate::tokens::revoke_token,
crate::tokens::delete_token,
crate::authors::list_authors,
crate::stats::get_stats,
crate::settings::get_settings,
crate::settings::get_setting,
crate::settings::update_setting,
crate::settings::clear_cache,
crate::settings::get_cache_stats,
crate::settings::get_thumbnail_stats,
crate::metadata::search_metadata,
crate::metadata::create_metadata_match,
crate::metadata::approve_metadata,
crate::metadata::reject_metadata,
crate::metadata::get_metadata_links,
crate::metadata::get_missing_books,
crate::metadata::delete_metadata_link,
crate::series::series_statuses,
crate::series::provider_statuses,
crate::settings::list_status_mappings,
crate::settings::upsert_status_mapping,
crate::settings::delete_status_mapping,
crate::prowlarr::search_prowlarr,
crate::prowlarr::test_prowlarr,
crate::qbittorrent::add_torrent,
crate::qbittorrent::test_qbittorrent,
crate::metadata_batch::start_batch,
crate::metadata_batch::get_batch_report,
crate::metadata_batch::get_batch_results,
crate::metadata_refresh::start_refresh,
crate::metadata_refresh::get_refresh_report,
crate::komga::sync_komga_read_books,
crate::komga::list_sync_reports,
crate::komga::get_sync_report,
), ),
components( components(
schemas( schemas(
@@ -34,10 +79,22 @@ use utoipa::OpenApi;
crate::books::BookItem, crate::books::BookItem,
crate::books::BooksPage, crate::books::BooksPage,
crate::books::BookDetails, crate::books::BookDetails,
crate::books::SeriesItem, crate::reading_progress::ReadingProgressResponse,
crate::reading_progress::UpdateReadingProgressRequest,
crate::reading_progress::MarkSeriesReadRequest,
crate::reading_progress::MarkSeriesReadResponse,
crate::series::SeriesItem,
crate::series::SeriesPage,
crate::series::ListAllSeriesQuery,
crate::series::OngoingQuery,
crate::books::UpdateBookRequest,
crate::series::SeriesMetadata,
crate::series::UpdateSeriesRequest,
crate::series::UpdateSeriesResponse,
crate::pages::PageQuery, crate::pages::PageQuery,
crate::search::SearchQuery, crate::search::SearchQuery,
crate::search::SearchResponse, crate::search::SearchResponse,
crate::search::SeriesHit,
crate::index_jobs::RebuildRequest, crate::index_jobs::RebuildRequest,
crate::thumbnails::ThumbnailsRebuildRequest, crate::thumbnails::ThumbnailsRebuildRequest,
crate::index_jobs::IndexJobResponse, crate::index_jobs::IndexJobResponse,
@@ -48,9 +105,58 @@ use utoipa::OpenApi;
crate::libraries::LibraryResponse, crate::libraries::LibraryResponse,
crate::libraries::CreateLibraryRequest, crate::libraries::CreateLibraryRequest,
crate::libraries::UpdateMonitoringRequest, crate::libraries::UpdateMonitoringRequest,
crate::libraries::UpdateMetadataProviderRequest,
crate::tokens::CreateTokenRequest, crate::tokens::CreateTokenRequest,
crate::tokens::TokenResponse, crate::tokens::TokenResponse,
crate::tokens::CreatedTokenResponse, crate::tokens::CreatedTokenResponse,
crate::settings::UpdateSettingRequest,
crate::settings::ClearCacheResponse,
crate::settings::CacheStats,
crate::settings::ThumbnailStats,
crate::settings::StatusMappingDto,
crate::settings::UpsertStatusMappingRequest,
crate::authors::ListAuthorsQuery,
crate::authors::AuthorItem,
crate::authors::AuthorsPageResponse,
crate::stats::StatsResponse,
crate::stats::StatsOverview,
crate::stats::ReadingStatusStats,
crate::stats::FormatCount,
crate::stats::LanguageCount,
crate::stats::LibraryStats,
crate::stats::TopSeries,
crate::stats::MonthlyAdditions,
crate::stats::MetadataStats,
crate::stats::ProviderCount,
crate::metadata::ApproveRequest,
crate::metadata::ApproveResponse,
crate::metadata::SyncReport,
crate::metadata::SeriesSyncReport,
crate::metadata::BookSyncReport,
crate::metadata::FieldChange,
crate::metadata::MetadataSearchRequest,
crate::metadata::SeriesCandidateDto,
crate::metadata::MetadataMatchRequest,
crate::metadata::ExternalMetadataLinkDto,
crate::metadata::MissingBooksDto,
crate::metadata::MissingBookItem,
crate::qbittorrent::QBittorrentAddRequest,
crate::qbittorrent::QBittorrentAddResponse,
crate::qbittorrent::QBittorrentTestResponse,
crate::prowlarr::ProwlarrSearchRequest,
crate::prowlarr::ProwlarrRelease,
crate::prowlarr::ProwlarrCategory,
crate::prowlarr::ProwlarrSearchResponse,
crate::prowlarr::MissingVolumeInput,
crate::prowlarr::ProwlarrTestResponse,
crate::metadata_batch::MetadataBatchRequest,
crate::metadata_batch::MetadataBatchReportDto,
crate::metadata_batch::MetadataBatchResultDto,
crate::metadata_refresh::MetadataRefreshRequest,
crate::metadata_refresh::MetadataRefreshReportDto,
crate::komga::KomgaSyncRequest,
crate::komga::KomgaSyncResponse,
crate::komga::KomgaSyncReportSummary,
ErrorResponse, ErrorResponse,
) )
), ),
@@ -58,10 +164,20 @@ use utoipa::OpenApi;
("Bearer" = []) ("Bearer" = [])
), ),
tags( tags(
(name = "books", description = "Read-only endpoints for browsing and searching books"), (name = "books", description = "Book browsing, details and management"),
(name = "libraries", description = "Library management endpoints (Admin only)"), (name = "series", description = "Series browsing, filtering and management"),
(name = "search", description = "Full-text search across books and series"),
(name = "reading-progress", description = "Reading progress tracking per book"),
(name = "authors", description = "Author browsing and listing"),
(name = "stats", description = "Collection statistics and dashboard data"),
(name = "libraries", description = "Library listing, scanning, and management (create/delete/settings: Admin only)"),
(name = "indexing", description = "Search index management and job control (Admin only)"), (name = "indexing", description = "Search index management and job control (Admin only)"),
(name = "metadata", description = "External metadata providers and matching (Admin only)"),
(name = "komga", description = "Komga read-status sync (Admin only)"),
(name = "tokens", description = "API token management (Admin only)"), (name = "tokens", description = "API token management (Admin only)"),
(name = "settings", description = "Application settings and cache management (Admin only)"),
(name = "prowlarr", description = "Prowlarr indexer integration (Admin only)"),
(name = "qbittorrent", description = "qBittorrent download client integration (Admin only)"),
), ),
modifiers(&SecurityAddon) modifiers(&SecurityAddon)
)] )]
@@ -106,15 +222,24 @@ mod tests {
.to_pretty_json() .to_pretty_json()
.expect("Failed to serialize OpenAPI"); .expect("Failed to serialize OpenAPI");
// Check that there are no references to non-existent schemas // Check that all $ref targets exist in components/schemas
assert!( let doc: serde_json::Value =
!json.contains("\"/components/schemas/Uuid\""), serde_json::from_str(&json).expect("OpenAPI JSON should be valid");
"Uuid schema should not be referenced" let empty = serde_json::Map::new();
); let schemas = doc["components"]["schemas"]
assert!( .as_object()
!json.contains("\"/components/schemas/DateTime\""), .unwrap_or(&empty);
"DateTime schema should not be referenced" let prefix = "#/components/schemas/";
); let mut broken: Vec<String> = Vec::new();
for part in json.split(prefix).skip(1) {
if let Some(name) = part.split('"').next() {
if !schemas.contains_key(name) {
broken.push(name.to_string());
}
}
}
broken.dedup();
assert!(broken.is_empty(), "Unresolved schema refs: {:?}", broken);
// Save to file for inspection // Save to file for inspection
std::fs::write("/tmp/openapi.json", &json).expect("Failed to write file"); std::fs::write("/tmp/openapi.json", &json).expect("Failed to write file");

View File

@@ -1,5 +1,5 @@
use std::{ use std::{
io::{Read, Write}, io::Write,
path::{Path, PathBuf}, path::{Path, PathBuf},
sync::{atomic::Ordering, Arc}, sync::{atomic::Ordering, Arc},
time::Duration, time::Duration,
@@ -16,11 +16,10 @@ use serde::Deserialize;
use utoipa::ToSchema; use utoipa::ToSchema;
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use sqlx::Row; use sqlx::Row;
use tracing::{debug, error, info, instrument, warn}; use tracing::{error, info, instrument, warn};
use uuid::Uuid; use uuid::Uuid;
use walkdir::WalkDir;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::AppState};
fn remap_libraries_path(path: &str) -> String { fn remap_libraries_path(path: &str) -> String {
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") { if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
@@ -31,10 +30,12 @@ fn remap_libraries_path(path: &str) -> String {
path.to_string() path.to_string()
} }
fn get_image_cache_dir() -> PathBuf { fn parse_filter(s: &str) -> image::imageops::FilterType {
std::env::var("IMAGE_CACHE_DIR") match s {
.map(PathBuf::from) "lanczos3" => image::imageops::FilterType::Lanczos3,
.unwrap_or_else(|_| PathBuf::from("/tmp/stripstream-image-cache")) "nearest" => image::imageops::FilterType::Nearest,
_ => image::imageops::FilterType::Triangle, // Triangle (bilinear) is fast and good enough for comics
}
} }
fn get_cache_key(abs_path: &str, page: u32, format: &str, quality: u8, width: u32) -> String { fn get_cache_key(abs_path: &str, page: u32, format: &str, quality: u8, width: u32) -> String {
@@ -47,8 +48,7 @@ fn get_cache_key(abs_path: &str, page: u32, format: &str, quality: u8, width: u3
format!("{:x}", hasher.finalize()) format!("{:x}", hasher.finalize())
} }
fn get_cache_path(cache_key: &str, format: &OutputFormat) -> PathBuf { fn get_cache_path(cache_key: &str, format: &OutputFormat, cache_dir: &Path) -> PathBuf {
let cache_dir = get_image_cache_dir();
let prefix = &cache_key[..2]; let prefix = &cache_key[..2];
let ext = format.extension(); let ext = format.extension();
cache_dir.join(prefix).join(format!("{}.{}", cache_key, ext)) cache_dir.join(prefix).join(format!("{}.{}", cache_key, ext))
@@ -64,7 +64,7 @@ fn write_to_disk_cache(cache_path: &Path, data: &[u8]) -> Result<(), std::io::Er
} }
let mut file = std::fs::File::create(cache_path)?; let mut file = std::fs::File::create(cache_path)?;
file.write_all(data)?; file.write_all(data)?;
file.sync_data()?; // No sync_data() — this is a cache, durability is not critical
Ok(()) Ok(())
} }
@@ -80,6 +80,8 @@ pub struct PageQuery {
#[derive(Clone, Copy, Debug)] #[derive(Clone, Copy, Debug)]
enum OutputFormat { enum OutputFormat {
/// Serve raw bytes from the archive — no decode, no re-encode.
Original,
Jpeg, Jpeg,
Png, Png,
Webp, Webp,
@@ -87,16 +89,19 @@ enum OutputFormat {
impl OutputFormat { impl OutputFormat {
fn parse(value: Option<&str>) -> Result<Self, ApiError> { fn parse(value: Option<&str>) -> Result<Self, ApiError> {
match value.unwrap_or("webp") { match value {
"jpeg" | "jpg" => Ok(Self::Jpeg), None => Ok(Self::Original),
"png" => Ok(Self::Png), Some("original") => Ok(Self::Original),
"webp" => Ok(Self::Webp), Some("jpeg") | Some("jpg") => Ok(Self::Jpeg),
_ => Err(ApiError::bad_request("format must be webp|jpeg|png")), Some("png") => Ok(Self::Png),
Some("webp") => Ok(Self::Webp),
_ => Err(ApiError::bad_request("format must be original|webp|jpeg|png")),
} }
} }
fn content_type(&self) -> &'static str { fn content_type(&self) -> &'static str {
match self { match self {
Self::Original => "application/octet-stream", // will be overridden by detected type
Self::Jpeg => "image/jpeg", Self::Jpeg => "image/jpeg",
Self::Png => "image/png", Self::Png => "image/png",
Self::Webp => "image/webp", Self::Webp => "image/webp",
@@ -105,6 +110,7 @@ impl OutputFormat {
fn extension(&self) -> &'static str { fn extension(&self) -> &'static str {
match self { match self {
Self::Original => "orig",
Self::Jpeg => "jpg", Self::Jpeg => "jpg",
Self::Png => "png", Self::Png => "png",
Self::Webp => "webp", Self::Webp => "webp",
@@ -112,6 +118,17 @@ impl OutputFormat {
} }
} }
/// Detect content type from raw image bytes.
fn detect_content_type(data: &[u8]) -> &'static str {
match image::guess_format(data) {
Ok(ImageFormat::Jpeg) => "image/jpeg",
Ok(ImageFormat::Png) => "image/png",
Ok(ImageFormat::WebP) => "image/webp",
Ok(ImageFormat::Avif) => "image/avif",
_ => "application/octet-stream",
}
}
/// Get a specific page image from a book with optional format conversion /// Get a specific page image from a book with optional format conversion
#[utoipa::path( #[utoipa::path(
get, get,
@@ -132,36 +149,38 @@ impl OutputFormat {
), ),
security(("Bearer" = [])) security(("Bearer" = []))
)] )]
#[instrument(skip(state), fields(book_id = %book_id, page = n))] #[instrument(skip(state, headers), fields(book_id = %book_id, page = n))]
pub async fn get_page( pub async fn get_page(
State(state): State<AppState>, State(state): State<AppState>,
AxumPath((book_id, n)): AxumPath<(Uuid, u32)>, AxumPath((book_id, n)): AxumPath<(Uuid, u32)>,
Query(query): Query<PageQuery>, Query(query): Query<PageQuery>,
headers: HeaderMap,
) -> Result<Response, ApiError> { ) -> Result<Response, ApiError> {
info!("Processing image request");
if n == 0 { if n == 0 {
warn!("Invalid page number: 0");
return Err(ApiError::bad_request("page index starts at 1")); return Err(ApiError::bad_request("page index starts at 1"));
} }
let (default_quality, max_width, filter_str, timeout_secs, cache_dir) = {
let s = state.settings.read().await;
(s.image_quality, s.image_max_width, s.image_filter.clone(), s.timeout_seconds, s.cache_directory.clone())
};
let format = OutputFormat::parse(query.format.as_deref())?; let format = OutputFormat::parse(query.format.as_deref())?;
let quality = query.quality.unwrap_or(80).clamp(1, 100); let quality = query.quality.unwrap_or(default_quality).clamp(1, 100);
let width = query.width.unwrap_or(0); let width = query.width.unwrap_or(0);
if width > 2160 { if width > max_width {
warn!("Invalid width: {}", width); return Err(ApiError::bad_request(format!("width must be <= {}", max_width)));
return Err(ApiError::bad_request("width must be <= 2160"));
} }
let filter = parse_filter(&filter_str);
let cache_dir_path = std::path::PathBuf::from(&cache_dir);
let memory_cache_key = format!("{book_id}:{n}:{}:{quality}:{width}", format.extension()); let memory_cache_key = format!("{book_id}:{n}:{}:{quality}:{width}", format.extension());
if let Some(cached) = state.page_cache.lock().await.get(&memory_cache_key).cloned() { if let Some(cached) = state.page_cache.lock().await.get(&memory_cache_key).cloned() {
state.metrics.page_cache_hits.fetch_add(1, Ordering::Relaxed); state.metrics.page_cache_hits.fetch_add(1, Ordering::Relaxed);
debug!("Memory cache hit for key: {}", memory_cache_key); return Ok(image_response(cached, format, None, &headers));
return Ok(image_response(cached, format.content_type(), None));
} }
state.metrics.page_cache_misses.fetch_add(1, Ordering::Relaxed); state.metrics.page_cache_misses.fetch_add(1, Ordering::Relaxed);
debug!("Memory cache miss for key: {}", memory_cache_key);
let row = sqlx::query( let row = sqlx::query(
r#" r#"
@@ -183,7 +202,6 @@ pub async fn get_page(
let row = match row { let row = match row {
Some(r) => r, Some(r) => r,
None => { None => {
error!("Book file not found for book_id: {}", book_id);
return Err(ApiError::not_found("book file not found")); return Err(ApiError::not_found("book file not found"));
} }
}; };
@@ -192,18 +210,22 @@ pub async fn get_page(
let abs_path = remap_libraries_path(&abs_path); let abs_path = remap_libraries_path(&abs_path);
let input_format: String = row.get("format"); let input_format: String = row.get("format");
info!("Processing book file: {} (format: {})", abs_path, input_format);
let disk_cache_key = get_cache_key(&abs_path, n, format.extension(), quality, width); let disk_cache_key = get_cache_key(&abs_path, n, format.extension(), quality, width);
let cache_path = get_cache_path(&disk_cache_key, &format); let cache_path = get_cache_path(&disk_cache_key, &format, &cache_dir_path);
// If-None-Match: return 304 if the client already has this version
if let Some(if_none_match) = headers.get(header::IF_NONE_MATCH) {
let expected_etag = format!("\"{}\"", disk_cache_key);
if if_none_match.as_bytes() == expected_etag.as_bytes() {
return Ok(StatusCode::NOT_MODIFIED.into_response());
}
}
if let Some(cached_bytes) = read_from_disk_cache(&cache_path) { if let Some(cached_bytes) = read_from_disk_cache(&cache_path) {
info!("Disk cache hit for: {}", cache_path.display());
let bytes = Arc::new(cached_bytes); let bytes = Arc::new(cached_bytes);
state.page_cache.lock().await.put(memory_cache_key, bytes.clone()); state.page_cache.lock().await.put(memory_cache_key, bytes.clone());
return Ok(image_response(bytes, format.content_type(), Some(&disk_cache_key))); return Ok(image_response(bytes, format, Some(&disk_cache_key), &headers));
} }
debug!("Disk cache miss for: {}", cache_path.display());
let _permit = state let _permit = state
.page_render_limit .page_render_limit
@@ -215,15 +237,14 @@ pub async fn get_page(
ApiError::internal("render limiter unavailable") ApiError::internal("render limiter unavailable")
})?; })?;
info!("Rendering page {} from {}", n, abs_path);
let abs_path_clone = abs_path.clone(); let abs_path_clone = abs_path.clone();
let format_clone = format; let format_clone = format;
let start_time = std::time::Instant::now(); let start_time = std::time::Instant::now();
let bytes = tokio::time::timeout( let bytes = tokio::time::timeout(
Duration::from_secs(60), Duration::from_secs(timeout_secs),
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
render_page(&abs_path_clone, &input_format, n, &format_clone, quality, width) render_page(&abs_path_clone, &input_format, n, &format_clone, quality, width, filter)
}), }),
) )
.await .await
@@ -240,18 +261,37 @@ pub async fn get_page(
match bytes { match bytes {
Ok(data) => { Ok(data) => {
info!("Successfully rendered page {} in {:?}", n, duration); info!("Rendered page {} in {:?}", n, duration);
if let Err(e) = write_to_disk_cache(&cache_path, &data) { if let Err(e) = write_to_disk_cache(&cache_path, &data) {
warn!("Failed to write to disk cache: {}", e); warn!("Failed to write to disk cache: {}", e);
} else {
info!("Cached rendered image to: {}", cache_path.display());
} }
let bytes = Arc::new(data); let bytes = Arc::new(data);
state.page_cache.lock().await.put(memory_cache_key, bytes.clone()); state.page_cache.lock().await.put(memory_cache_key.clone(), bytes.clone());
Ok(image_response(bytes, format.content_type(), Some(&disk_cache_key))) // Prefetch next 2 pages in background (fire-and-forget)
for next_page in [n + 1, n + 2] {
let state2 = state.clone();
let abs_path2 = abs_path.clone();
let cache_dir2 = cache_dir_path.clone();
let format2 = format;
tokio::spawn(async move {
prefetch_page(state2, &PrefetchParams {
book_id,
abs_path: &abs_path2,
page: next_page,
format: format2,
quality,
width,
filter,
timeout_secs,
cache_dir: &cache_dir2,
}).await;
});
}
Ok(image_response(bytes, format, Some(&disk_cache_key), &headers))
} }
Err(e) => { Err(e) => {
error!("Failed to render page {} from {}: {:?}", n, abs_path, e); error!("Failed to render page {} from {}: {:?}", n, abs_path, e);
@@ -260,11 +300,84 @@ pub async fn get_page(
} }
} }
fn image_response(bytes: Arc<Vec<u8>>, content_type: &str, etag_suffix: Option<&str>) -> Response { struct PrefetchParams<'a> {
let mut headers = HeaderMap::new(); book_id: Uuid,
headers.insert(header::CONTENT_TYPE, HeaderValue::from_str(content_type).unwrap_or(HeaderValue::from_static("application/octet-stream"))); abs_path: &'a str,
headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("public, max-age=31536000, immutable")); page: u32,
format: OutputFormat,
quality: u8,
width: u32,
filter: image::imageops::FilterType,
timeout_secs: u64,
cache_dir: &'a Path,
}
/// Prefetch a single page into disk+memory cache (best-effort, ignores errors).
async fn prefetch_page(state: AppState, params: &PrefetchParams<'_>) {
let book_id = params.book_id;
let page = params.page;
let format = params.format;
let quality = params.quality;
let width = params.width;
let filter = params.filter;
let timeout_secs = params.timeout_secs;
let abs_path = params.abs_path;
let cache_dir = params.cache_dir;
let mem_key = format!("{book_id}:{page}:{}:{quality}:{width}", format.extension());
// Already in memory cache?
if state.page_cache.lock().await.contains(&mem_key) {
return;
}
// Already on disk?
let disk_key = get_cache_key(abs_path, page, format.extension(), quality, width);
let cache_path = get_cache_path(&disk_key, &format, cache_dir);
if cache_path.exists() {
return;
}
// Acquire render permit (don't block too long — if busy, skip)
let permit = tokio::time::timeout(
Duration::from_millis(100),
state.page_render_limit.clone().acquire_owned(),
)
.await;
let _permit = match permit {
Ok(Ok(p)) => p,
_ => return,
};
// Fetch the book format from the path extension as a shortcut
let input_format = match abs_path.rsplit('.').next().map(|e| e.to_ascii_lowercase()) {
Some(ref e) if e == "cbz" => "cbz",
Some(ref e) if e == "cbr" => "cbr",
Some(ref e) if e == "pdf" => "pdf",
Some(ref e) if e == "epub" => "epub",
_ => return,
}
.to_string();
let abs_clone = abs_path.to_string();
let fmt = format;
let result = tokio::time::timeout(
Duration::from_secs(timeout_secs),
tokio::task::spawn_blocking(move || {
render_page(&abs_clone, &input_format, page, &fmt, quality, width, filter)
}),
)
.await;
if let Ok(Ok(Ok(data))) = result {
let _ = write_to_disk_cache(&cache_path, &data);
let bytes = Arc::new(data);
state.page_cache.lock().await.put(mem_key, bytes);
}
}
fn image_response(bytes: Arc<Vec<u8>>, format: OutputFormat, etag_suffix: Option<&str>, req_headers: &HeaderMap) -> Response {
let content_type = match format {
OutputFormat::Original => detect_content_type(&bytes),
_ => format.content_type(),
};
let etag = if let Some(suffix) = etag_suffix { let etag = if let Some(suffix) = etag_suffix {
format!("\"{}\"", suffix) format!("\"{}\"", suffix)
} else { } else {
@@ -273,19 +386,37 @@ fn image_response(bytes: Arc<Vec<u8>>, content_type: &str, etag_suffix: Option<&
format!("\"{:x}\"", hasher.finalize()) format!("\"{:x}\"", hasher.finalize())
}; };
// Check If-None-Match for 304
if let Some(if_none_match) = req_headers.get(header::IF_NONE_MATCH) {
if if_none_match.as_bytes() == etag.as_bytes() {
let mut headers = HeaderMap::new();
headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("public, max-age=31536000, immutable"));
if let Ok(v) = HeaderValue::from_str(&etag) {
headers.insert(header::ETAG, v);
}
return (StatusCode::NOT_MODIFIED, headers).into_response();
}
}
let mut headers = HeaderMap::new();
headers.insert(header::CONTENT_TYPE, HeaderValue::from_str(content_type).unwrap_or(HeaderValue::from_static("application/octet-stream")));
headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("public, max-age=31536000, immutable"));
if let Ok(v) = HeaderValue::from_str(&etag) { if let Ok(v) = HeaderValue::from_str(&etag) {
headers.insert(header::ETAG, v); headers.insert(header::ETAG, v);
} }
(StatusCode::OK, headers, Body::from((*bytes).clone())).into_response() // Use Bytes to avoid cloning the Vec — shares the Arc's allocation via zero-copy
let body_bytes = axum::body::Bytes::from(Arc::unwrap_or_clone(bytes));
(StatusCode::OK, headers, Body::from(body_bytes)).into_response()
} }
/// Render page 1 of a book (for thumbnail fallback or thumbnail checkup). Uses thumbnail dimensions by default. /// Render page 1 of a book (for thumbnail fallback or thumbnail checkup). Uses thumbnail dimensions by default.
/// Render page 1 as a thumbnail fallback. Returns (bytes, content_type).
pub async fn render_book_page_1( pub async fn render_book_page_1(
state: &AppState, state: &AppState,
book_id: Uuid, book_id: Uuid,
width: u32, width: u32,
quality: u8, quality: u8,
) -> Result<Vec<u8>, ApiError> { ) -> Result<(Vec<u8>, &'static str), ApiError> {
let row = sqlx::query( let row = sqlx::query(
r#"SELECT abs_path, format FROM book_files WHERE book_id = $1 ORDER BY updated_at DESC LIMIT 1"#, r#"SELECT abs_path, format FROM book_files WHERE book_id = $1 ORDER BY updated_at DESC LIMIT 1"#,
) )
@@ -306,17 +437,24 @@ pub async fn render_book_page_1(
.await .await
.map_err(|_| ApiError::internal("render limiter unavailable"))?; .map_err(|_| ApiError::internal("render limiter unavailable"))?;
let (timeout_secs, filter_str) = {
let s = state.settings.read().await;
(s.timeout_seconds, s.image_filter.clone())
};
let filter = parse_filter(&filter_str);
let abs_path_clone = abs_path.clone(); let abs_path_clone = abs_path.clone();
let bytes = tokio::time::timeout( let bytes = tokio::time::timeout(
Duration::from_secs(60), Duration::from_secs(timeout_secs),
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
render_page( render_page(
&abs_path_clone, &abs_path_clone,
&input_format, &input_format,
1, 1,
&OutputFormat::Webp, &OutputFormat::Original,
quality, quality,
width, width,
filter,
) )
}), }),
) )
@@ -324,7 +462,9 @@ pub async fn render_book_page_1(
.map_err(|_| ApiError::internal("page rendering timeout"))? .map_err(|_| ApiError::internal("page rendering timeout"))?
.map_err(|e| ApiError::internal(format!("render task failed: {e}")))?; .map_err(|e| ApiError::internal(format!("render task failed: {e}")))?;
bytes let bytes = bytes?;
let content_type = detect_content_type(&bytes);
Ok((bytes, content_type))
} }
fn render_page( fn render_page(
@@ -334,200 +474,115 @@ fn render_page(
out_format: &OutputFormat, out_format: &OutputFormat,
quality: u8, quality: u8,
width: u32, width: u32,
filter: image::imageops::FilterType,
) -> Result<Vec<u8>, ApiError> { ) -> Result<Vec<u8>, ApiError> {
let page_bytes = match input_format { let format = match input_format {
"cbz" => extract_cbz_page(abs_path, page_number)?, "cbz" => parsers::BookFormat::Cbz,
"cbr" => extract_cbr_page(abs_path, page_number)?, "cbr" => parsers::BookFormat::Cbr,
"pdf" => render_pdf_page(abs_path, page_number, width)?, "pdf" => parsers::BookFormat::Pdf,
"epub" => parsers::BookFormat::Epub,
_ => return Err(ApiError::bad_request("unsupported source format")), _ => return Err(ApiError::bad_request("unsupported source format")),
}; };
transcode_image(&page_bytes, out_format, quality, width) let pdf_render_width = if width > 0 { width } else { 1200 };
} let page_bytes = parsers::extract_page(
std::path::Path::new(abs_path),
fn extract_cbz_page(abs_path: &str, page_number: u32) -> Result<Vec<u8>, ApiError> { format,
debug!("Opening CBZ archive: {}", abs_path); page_number,
let file = std::fs::File::open(abs_path).map_err(|e| { pdf_render_width,
error!("Cannot open CBZ file {}: {}", abs_path, e); )
ApiError::internal(format!("cannot open cbz: {e}")) .map_err(|e| {
error!("Failed to extract page {} from {}: {}", page_number, abs_path, e);
ApiError::internal(format!("page extraction failed: {e}"))
})?; })?;
let mut archive = zip::ZipArchive::new(file).map_err(|e| { // Original mode or source matches output with no resize → return raw bytes (zero transcoding)
error!("Invalid CBZ archive {}: {}", abs_path, e); if matches!(out_format, OutputFormat::Original) && width == 0 {
ApiError::internal(format!("invalid cbz: {e}")) return Ok(page_bytes);
})?; }
if width == 0 {
let mut image_names: Vec<String> = Vec::new(); if let Ok(source_fmt) = image::guess_format(&page_bytes) {
for i in 0..archive.len() { if format_matches(&source_fmt, out_format) {
let entry = archive.by_index(i).map_err(|e| { return Ok(page_bytes);
error!("Failed to read CBZ entry {} in {}: {}", i, abs_path, e); }
ApiError::internal(format!("cbz entry read failed: {e}"))
})?;
let name = entry.name().to_ascii_lowercase();
if is_image_name(&name) {
image_names.push(entry.name().to_string());
} }
} }
image_names.sort();
debug!("Found {} images in CBZ {}", image_names.len(), abs_path);
let index = page_number as usize - 1; transcode_image(&page_bytes, out_format, quality, width, filter)
let selected = image_names.get(index).ok_or_else(|| {
error!("Page {} out of range in {} (total: {})", page_number, abs_path, image_names.len());
ApiError::not_found("page out of range")
})?;
debug!("Extracting page {} ({}) from {}", page_number, selected, abs_path);
let mut entry = archive.by_name(selected).map_err(|e| {
error!("Failed to read CBZ page {} from {}: {}", selected, abs_path, e);
ApiError::internal(format!("cbz page read failed: {e}"))
})?;
let mut buf = Vec::new();
entry.read_to_end(&mut buf).map_err(|e| {
error!("Failed to load CBZ page {} from {}: {}", selected, abs_path, e);
ApiError::internal(format!("cbz page load failed: {e}"))
})?;
Ok(buf)
} }
fn extract_cbr_page(abs_path: &str, page_number: u32) -> Result<Vec<u8>, ApiError> {
info!("Opening CBR archive: {}", abs_path);
let index = page_number as usize - 1; /// Fast JPEG decode with DCT scaling: decodes directly at reduced resolution.
let tmp_dir = std::env::temp_dir().join(format!("stripstream-cbr-{}", Uuid::new_v4())); fn fast_jpeg_decode(input: &[u8], target_w: u32, target_h: u32) -> Option<image::DynamicImage> {
debug!("Creating temp dir for CBR extraction: {}", tmp_dir.display()); if image::guess_format(input).ok()? != ImageFormat::Jpeg {
return None;
std::fs::create_dir_all(&tmp_dir).map_err(|e| { }
error!("Cannot create temp dir: {}", e); let mut decoder = jpeg_decoder::Decoder::new(std::io::Cursor::new(input));
ApiError::internal(format!("temp dir error: {}", e)) decoder.read_info().ok()?;
})?; decoder.scale(target_w as u16, target_h as u16).ok()?;
let pixels = decoder.decode().ok()?;
// Extract directly - skip listing which fails on UTF-16 encoded filenames let info = decoder.info()?;
let extract_output = std::process::Command::new("env") let w = info.width as u32;
.args(["LC_ALL=en_US.UTF-8", "LANG=en_US.UTF-8", "unar", "-o"]) let h = info.height as u32;
.arg(&tmp_dir) match info.pixel_format {
.arg(abs_path) jpeg_decoder::PixelFormat::RGB24 => {
.output() let buf = image::RgbImage::from_raw(w, h, pixels)?;
.map_err(|e| { Some(image::DynamicImage::ImageRgb8(buf))
let _ = std::fs::remove_dir_all(&tmp_dir); }
error!("unar extract failed: {}", e); jpeg_decoder::PixelFormat::L8 => {
ApiError::internal(format!("unar extract failed: {e}")) let buf = image::GrayImage::from_raw(w, h, pixels)?;
})?; Some(image::DynamicImage::ImageLuma8(buf))
}
if !extract_output.status.success() { _ => None,
let _ = std::fs::remove_dir_all(&tmp_dir);
let stderr = String::from_utf8_lossy(&extract_output.stderr);
error!("unar extract failed {}: {}", abs_path, stderr);
return Err(ApiError::internal("unar extract failed"));
} }
// Find and read the requested image (recursive search for CBR files with subdirectories)
let mut image_files: Vec<_> = WalkDir::new(&tmp_dir)
.into_iter()
.filter_map(|e| e.ok())
.filter(|e| {
let name = e.file_name().to_string_lossy().to_lowercase();
is_image_name(&name)
})
.collect();
image_files.sort_by_key(|e| e.path().to_string_lossy().to_lowercase());
let selected = image_files.get(index).ok_or_else(|| {
let _ = std::fs::remove_dir_all(&tmp_dir);
error!("Page {} not found (total: {})", page_number, image_files.len());
ApiError::not_found("page out of range")
})?;
let data = std::fs::read(selected.path()).map_err(|e| {
let _ = std::fs::remove_dir_all(&tmp_dir);
error!("read failed: {}", e);
ApiError::internal(format!("read error: {}", e))
})?;
let _ = std::fs::remove_dir_all(&tmp_dir);
info!("Successfully extracted CBR page {} ({} bytes)", page_number, data.len());
Ok(data)
} }
fn render_pdf_page(abs_path: &str, page_number: u32, width: u32) -> Result<Vec<u8>, ApiError> { fn transcode_image(input: &[u8], out_format: &OutputFormat, quality: u8, width: u32, filter: image::imageops::FilterType) -> Result<Vec<u8>, ApiError> {
let tmp_dir = std::env::temp_dir().join(format!("stripstream-pdf-{}", Uuid::new_v4()));
debug!("Creating temp dir for PDF rendering: {}", tmp_dir.display());
std::fs::create_dir_all(&tmp_dir).map_err(|e| {
error!("Cannot create temp dir {}: {}", tmp_dir.display(), e);
ApiError::internal(format!("cannot create temp dir: {e}"))
})?;
let output_prefix = tmp_dir.join("page");
let mut cmd = std::process::Command::new("pdftoppm");
cmd.arg("-f")
.arg(page_number.to_string())
.arg("-singlefile")
.arg("-png");
if width > 0 {
cmd.arg("-scale-to-x").arg(width.to_string()).arg("-scale-to-y").arg("-1");
}
cmd.arg(abs_path).arg(&output_prefix);
debug!("Running pdftoppm for page {} of {} (width: {})", page_number, abs_path, width);
let output = cmd
.output()
.map_err(|e| {
error!("pdftoppm command failed for {} page {}: {}", abs_path, page_number, e);
ApiError::internal(format!("pdf render failed: {e}"))
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
let _ = std::fs::remove_dir_all(&tmp_dir);
error!("pdftoppm failed for {} page {}: {}", abs_path, page_number, stderr);
return Err(ApiError::internal("pdf render command failed"));
}
let image_path = output_prefix.with_extension("png");
debug!("Reading rendered PDF page from: {}", image_path.display());
let bytes = std::fs::read(&image_path).map_err(|e| {
error!("Failed to read rendered PDF output {}: {}", image_path.display(), e);
ApiError::internal(format!("render output missing: {e}"))
})?;
let _ = std::fs::remove_dir_all(&tmp_dir);
debug!("Successfully rendered PDF page {} to {} bytes", page_number, bytes.len());
Ok(bytes)
}
fn transcode_image(input: &[u8], out_format: &OutputFormat, quality: u8, width: u32) -> Result<Vec<u8>, ApiError> {
debug!("Transcoding image: {} bytes, format: {:?}, quality: {}, width: {}", input.len(), out_format, quality, width);
let source_format = image::guess_format(input).ok(); let source_format = image::guess_format(input).ok();
debug!("Source format detected: {:?}", source_format);
let needs_transcode = source_format.map(|f| !format_matches(&f, out_format)).unwrap_or(true); // Resolve "Original" to the actual source format for encoding
let effective_format = match out_format {
OutputFormat::Original => match source_format {
Some(ImageFormat::Png) => OutputFormat::Png,
Some(ImageFormat::WebP) => OutputFormat::Webp,
_ => OutputFormat::Jpeg, // default to JPEG for original resize
},
other => *other,
};
let needs_transcode = source_format.map(|f| !format_matches(&f, &effective_format)).unwrap_or(true);
if width == 0 && !needs_transcode { if width == 0 && !needs_transcode {
debug!("No transcoding needed, returning original");
return Ok(input.to_vec()); return Ok(input.to_vec());
} }
debug!("Loading image from memory..."); // For JPEG with resize: use DCT scaling to decode at ~target size (much faster)
let mut image = image::load_from_memory(input).map_err(|e| { let mut image = if width > 0 {
error!("Failed to load image from memory: {} (input size: {} bytes)", e, input.len()); fast_jpeg_decode(input, width, u32::MAX)
ApiError::internal(format!("invalid source image: {e}")) .unwrap_or_else(|| {
})?; image::load_from_memory(input).unwrap_or_default()
})
} else {
image::load_from_memory(input).map_err(|e| {
ApiError::internal(format!("invalid source image: {e}"))
})?
};
if width > 0 { if width > 0 {
debug!("Resizing image to width: {}", width); image = image.resize(width, u32::MAX, filter);
image = image.resize(width, u32::MAX, image::imageops::FilterType::Lanczos3);
} }
debug!("Converting to RGBA...");
let rgba = image.to_rgba8(); let rgba = image.to_rgba8();
let (w, h) = rgba.dimensions(); let (w, h) = rgba.dimensions();
debug!("Image dimensions: {}x{}", w, h);
let mut out = Vec::new(); let mut out = Vec::new();
match out_format { match effective_format {
OutputFormat::Jpeg => { OutputFormat::Jpeg | OutputFormat::Original => {
// JPEG doesn't support alpha — convert RGBA to RGB
let rgb = image::DynamicImage::ImageRgba8(rgba.clone()).to_rgb8();
let mut encoder = JpegEncoder::new_with_quality(&mut out, quality); let mut encoder = JpegEncoder::new_with_quality(&mut out, quality);
encoder encoder
.encode(&rgba, w, h, ColorType::Rgba8.into()) .encode(&rgb, w, h, ColorType::Rgb8.into())
.map_err(|e| ApiError::internal(format!("jpeg encode failed: {e}")))?; .map_err(|e| ApiError::internal(format!("jpeg encode failed: {e}")))?;
} }
OutputFormat::Png => { OutputFormat::Png => {
@@ -542,7 +597,7 @@ fn transcode_image(input: &[u8], out_format: &OutputFormat, quality: u8, width:
.flat_map(|p| [p[0], p[1], p[2]]) .flat_map(|p| [p[0], p[1], p[2]])
.collect(); .collect();
let webp_data = webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h) let webp_data = webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h)
.encode(f32::max(quality as f32, 85.0)); .encode(quality as f32);
out.extend_from_slice(&webp_data); out.extend_from_slice(&webp_data);
} }
} }
@@ -550,28 +605,11 @@ fn transcode_image(input: &[u8], out_format: &OutputFormat, quality: u8, width:
} }
fn format_matches(source: &ImageFormat, target: &OutputFormat) -> bool { fn format_matches(source: &ImageFormat, target: &OutputFormat) -> bool {
match (source, target) { matches!(
(ImageFormat::Jpeg, OutputFormat::Jpeg) => true, (source, target),
(ImageFormat::Png, OutputFormat::Png) => true, (ImageFormat::Jpeg, OutputFormat::Jpeg)
(ImageFormat::WebP, OutputFormat::Webp) => true, | (ImageFormat::Png, OutputFormat::Png)
_ => false, | (ImageFormat::WebP, OutputFormat::Webp)
} )
} }
fn is_image_name(name: &str) -> bool {
let lower = name.to_lowercase();
lower.ends_with(".jpg")
|| lower.ends_with(".jpeg")
|| lower.ends_with(".png")
|| lower.ends_with(".webp")
|| lower.ends_with(".avif")
|| lower.ends_with(".gif")
|| lower.ends_with(".tif")
|| lower.ends_with(".tiff")
|| lower.ends_with(".bmp")
}
#[allow(dead_code)]
fn _is_absolute_path(value: &str) -> bool {
Path::new(value).is_absolute()
}

363
apps/api/src/prowlarr.rs Normal file
View File

@@ -0,0 +1,363 @@
use axum::{extract::State, Json};
use serde::{Deserialize, Serialize};
use sqlx::Row;
use utoipa::ToSchema;
use crate::{error::ApiError, state::AppState};
// ─── Types ──────────────────────────────────────────────────────────────────
#[derive(Deserialize, ToSchema)]
pub struct MissingVolumeInput {
pub volume_number: Option<i32>,
#[allow(dead_code)]
pub title: Option<String>,
}
#[derive(Deserialize, ToSchema)]
pub struct ProwlarrSearchRequest {
pub series_name: String,
pub volume_number: Option<i32>,
pub custom_query: Option<String>,
pub missing_volumes: Option<Vec<MissingVolumeInput>>,
}
#[derive(Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "camelCase")]
pub struct ProwlarrRawRelease {
pub guid: String,
pub title: String,
pub size: i64,
pub download_url: Option<String>,
pub indexer: Option<String>,
pub seeders: Option<i32>,
pub leechers: Option<i32>,
pub publish_date: Option<String>,
pub protocol: Option<String>,
pub info_url: Option<String>,
pub categories: Option<Vec<ProwlarrCategory>>,
}
#[derive(Serialize, ToSchema)]
#[serde(rename_all = "camelCase")]
pub struct ProwlarrRelease {
pub guid: String,
pub title: String,
pub size: i64,
pub download_url: Option<String>,
pub indexer: Option<String>,
pub seeders: Option<i32>,
pub leechers: Option<i32>,
pub publish_date: Option<String>,
pub protocol: Option<String>,
pub info_url: Option<String>,
pub categories: Option<Vec<ProwlarrCategory>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub matched_missing_volumes: Option<Vec<i32>>,
}
#[derive(Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "camelCase")]
pub struct ProwlarrCategory {
pub id: i32,
pub name: Option<String>,
}
#[derive(Serialize, ToSchema)]
pub struct ProwlarrSearchResponse {
pub results: Vec<ProwlarrRelease>,
pub query: String,
}
#[derive(Serialize, ToSchema)]
pub struct ProwlarrTestResponse {
pub success: bool,
pub message: String,
pub indexer_count: Option<i32>,
}
// ─── Config helper ──────────────────────────────────────────────────────────
#[derive(Deserialize)]
struct ProwlarrConfig {
url: String,
api_key: String,
categories: Option<Vec<i32>>,
}
async fn load_prowlarr_config(
pool: &sqlx::PgPool,
) -> Result<(String, String, Vec<i32>), ApiError> {
let row = sqlx::query("SELECT value FROM app_settings WHERE key = 'prowlarr'")
.fetch_optional(pool)
.await?;
let row = row.ok_or_else(|| ApiError::bad_request("Prowlarr is not configured"))?;
let value: serde_json::Value = row.get("value");
let config: ProwlarrConfig = serde_json::from_value(value)
.map_err(|e| ApiError::internal(format!("invalid prowlarr config: {e}")))?;
if config.url.is_empty() || config.api_key.is_empty() {
return Err(ApiError::bad_request(
"Prowlarr URL and API key must be configured in settings",
));
}
let url = config.url.trim_end_matches('/').to_string();
let categories = config.categories.unwrap_or_else(|| vec![7030, 7020]);
Ok((url, config.api_key, categories))
}
// ─── Volume matching ─────────────────────────────────────────────────────────
/// Extract volume numbers from a release title.
/// Looks for patterns like: T01, Tome 01, Vol. 01, v01, #01,
/// or standalone numbers that appear after common separators.
fn extract_volumes_from_title(title: &str) -> Vec<i32> {
let lower = title.to_lowercase();
let mut volumes = Vec::new();
// Patterns: T01, Tome 01, Tome01, Vol 01, Vol.01, v01, #01
let prefixes = ["tome", "vol.", "vol ", "t", "v", "#"];
let chars: Vec<char> = lower.chars().collect();
let len = chars.len();
for prefix in &prefixes {
let mut start = 0;
while let Some(pos) = lower[start..].find(prefix) {
let abs_pos = start + pos;
let after = abs_pos + prefix.len();
// For single-char prefixes (t, v, #), ensure it's at a word boundary
if prefix.len() == 1 && *prefix != "#" {
if abs_pos > 0 && chars[abs_pos - 1].is_alphanumeric() {
start = after;
continue;
}
}
// Skip optional spaces after prefix
let mut i = after;
while i < len && chars[i] == ' ' {
i += 1;
}
// Read digits
let digit_start = i;
while i < len && chars[i].is_ascii_digit() {
i += 1;
}
if i > digit_start {
if let Ok(num) = lower[digit_start..i].parse::<i32>() {
if !volumes.contains(&num) {
volumes.push(num);
}
}
}
start = after;
}
}
volumes
}
/// Match releases against missing volume numbers.
fn match_missing_volumes(
releases: Vec<ProwlarrRawRelease>,
missing: &[MissingVolumeInput],
) -> Vec<ProwlarrRelease> {
let missing_numbers: Vec<i32> = missing
.iter()
.filter_map(|m| m.volume_number)
.collect();
releases
.into_iter()
.map(|r| {
let matched = if missing_numbers.is_empty() {
None
} else {
let title_volumes = extract_volumes_from_title(&r.title);
let matched: Vec<i32> = title_volumes
.into_iter()
.filter(|v| missing_numbers.contains(v))
.collect();
if matched.is_empty() {
None
} else {
Some(matched)
}
};
ProwlarrRelease {
guid: r.guid,
title: r.title,
size: r.size,
download_url: r.download_url,
indexer: r.indexer,
seeders: r.seeders,
leechers: r.leechers,
publish_date: r.publish_date,
protocol: r.protocol,
info_url: r.info_url,
categories: r.categories,
matched_missing_volumes: matched,
}
})
.collect()
}
// ─── Handlers ───────────────────────────────────────────────────────────────
/// Search for releases on Prowlarr
#[utoipa::path(
post,
path = "/prowlarr/search",
tag = "prowlarr",
request_body = ProwlarrSearchRequest,
responses(
(status = 200, body = ProwlarrSearchResponse),
(status = 400, description = "Bad request or Prowlarr not configured"),
(status = 401, description = "Unauthorized"),
(status = 500, description = "Prowlarr connection error"),
),
security(("Bearer" = []))
)]
pub async fn search_prowlarr(
State(state): State<AppState>,
Json(body): Json<ProwlarrSearchRequest>,
) -> Result<Json<ProwlarrSearchResponse>, ApiError> {
let (url, api_key, categories) = load_prowlarr_config(&state.pool).await?;
let query = if let Some(custom) = &body.custom_query {
custom.clone()
} else if let Some(vol) = body.volume_number {
format!("\"{}\" {}", body.series_name, vol)
} else {
format!("\"{}\"", body.series_name)
};
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.map_err(|e| ApiError::internal(format!("failed to build HTTP client: {e}")))?;
let mut params: Vec<(&str, String)> = vec![
("query", query.clone()),
("type", "search".to_string()),
];
for cat in &categories {
params.push(("categories", cat.to_string()));
}
let resp = client
.get(format!("{url}/api/v1/search"))
.query(&params)
.header("X-Api-Key", &api_key)
.send()
.await
.map_err(|e| ApiError::internal(format!("Prowlarr request failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(ApiError::internal(format!(
"Prowlarr returned {status}: {text}"
)));
}
let raw_text = resp
.text()
.await
.map_err(|e| ApiError::internal(format!("Failed to read Prowlarr response: {e}")))?;
tracing::debug!("Prowlarr raw response length: {} chars", raw_text.len());
let raw_releases: Vec<ProwlarrRawRelease> = serde_json::from_str(&raw_text)
.map_err(|e| {
tracing::error!("Failed to parse Prowlarr response: {e}");
tracing::error!("Raw response (first 500 chars): {}", &raw_text[..raw_text.len().min(500)]);
ApiError::internal(format!("Failed to parse Prowlarr response: {e}"))
})?;
let results = if let Some(missing) = &body.missing_volumes {
match_missing_volumes(raw_releases, missing)
} else {
raw_releases
.into_iter()
.map(|r| ProwlarrRelease {
guid: r.guid,
title: r.title,
size: r.size,
download_url: r.download_url,
indexer: r.indexer,
seeders: r.seeders,
leechers: r.leechers,
publish_date: r.publish_date,
protocol: r.protocol,
info_url: r.info_url,
categories: r.categories,
matched_missing_volumes: None,
})
.collect()
};
Ok(Json(ProwlarrSearchResponse { results, query }))
}
/// Test connection to Prowlarr
#[utoipa::path(
get,
path = "/prowlarr/test",
tag = "prowlarr",
responses(
(status = 200, body = ProwlarrTestResponse),
(status = 400, description = "Prowlarr not configured"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn test_prowlarr(
State(state): State<AppState>,
) -> Result<Json<ProwlarrTestResponse>, ApiError> {
let (url, api_key, _categories) = load_prowlarr_config(&state.pool).await?;
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| ApiError::internal(format!("failed to build HTTP client: {e}")))?;
let resp = client
.get(format!("{url}/api/v1/indexer"))
.header("X-Api-Key", &api_key)
.send()
.await;
match resp {
Ok(r) if r.status().is_success() => {
let indexers: Vec<serde_json::Value> = r.json().await.unwrap_or_default();
Ok(Json(ProwlarrTestResponse {
success: true,
message: format!("Connected successfully ({} indexers)", indexers.len()),
indexer_count: Some(indexers.len() as i32),
}))
}
Ok(r) => {
let status = r.status();
let text = r.text().await.unwrap_or_default();
Ok(Json(ProwlarrTestResponse {
success: false,
message: format!("Prowlarr returned {status}: {text}"),
indexer_count: None,
}))
}
Err(e) => Ok(Json(ProwlarrTestResponse {
success: false,
message: format!("Connection failed: {e}"),
indexer_count: None,
})),
}
}

218
apps/api/src/qbittorrent.rs Normal file
View File

@@ -0,0 +1,218 @@
use axum::{extract::State, Json};
use serde::{Deserialize, Serialize};
use sqlx::Row;
use utoipa::ToSchema;
use crate::{error::ApiError, state::AppState};
// ─── Types ──────────────────────────────────────────────────────────────────
#[derive(Deserialize, ToSchema)]
pub struct QBittorrentAddRequest {
pub url: String,
}
#[derive(Serialize, ToSchema)]
pub struct QBittorrentAddResponse {
pub success: bool,
pub message: String,
}
#[derive(Serialize, ToSchema)]
pub struct QBittorrentTestResponse {
pub success: bool,
pub message: String,
pub version: Option<String>,
}
// ─── Config helper ──────────────────────────────────────────────────────────
#[derive(Deserialize)]
struct QBittorrentConfig {
url: String,
username: String,
password: String,
}
async fn load_qbittorrent_config(
pool: &sqlx::PgPool,
) -> Result<(String, String, String), ApiError> {
let row = sqlx::query("SELECT value FROM app_settings WHERE key = 'qbittorrent'")
.fetch_optional(pool)
.await?;
let row = row.ok_or_else(|| ApiError::bad_request("qBittorrent is not configured"))?;
let value: serde_json::Value = row.get("value");
let config: QBittorrentConfig = serde_json::from_value(value)
.map_err(|e| ApiError::internal(format!("invalid qbittorrent config: {e}")))?;
if config.url.is_empty() || config.username.is_empty() {
return Err(ApiError::bad_request(
"qBittorrent URL and username must be configured in settings",
));
}
let url = config.url.trim_end_matches('/').to_string();
Ok((url, config.username, config.password))
}
// ─── Login helper ───────────────────────────────────────────────────────────
async fn qbittorrent_login(
client: &reqwest::Client,
base_url: &str,
username: &str,
password: &str,
) -> Result<String, ApiError> {
let resp = client
.post(format!("{base_url}/api/v2/auth/login"))
.form(&[("username", username), ("password", password)])
.send()
.await
.map_err(|e| ApiError::internal(format!("qBittorrent login request failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(ApiError::internal(format!(
"qBittorrent login failed ({status}): {text}"
)));
}
// Extract SID from Set-Cookie header
let cookie_header = resp
.headers()
.get("set-cookie")
.and_then(|v| v.to_str().ok())
.unwrap_or("");
let sid = cookie_header
.split(';')
.next()
.and_then(|s| s.strip_prefix("SID="))
.ok_or_else(|| ApiError::internal("Failed to get SID cookie from qBittorrent"))?
.to_string();
Ok(sid)
}
// ─── Handlers ───────────────────────────────────────────────────────────────
/// Add a torrent to qBittorrent
#[utoipa::path(
post,
path = "/qbittorrent/add",
tag = "qbittorrent",
request_body = QBittorrentAddRequest,
responses(
(status = 200, body = QBittorrentAddResponse),
(status = 400, description = "Bad request or qBittorrent not configured"),
(status = 401, description = "Unauthorized"),
(status = 500, description = "qBittorrent connection error"),
),
security(("Bearer" = []))
)]
pub async fn add_torrent(
State(state): State<AppState>,
Json(body): Json<QBittorrentAddRequest>,
) -> Result<Json<QBittorrentAddResponse>, ApiError> {
if body.url.is_empty() {
return Err(ApiError::bad_request("url is required"));
}
let (base_url, username, password) = load_qbittorrent_config(&state.pool).await?;
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| ApiError::internal(format!("failed to build HTTP client: {e}")))?;
let sid = qbittorrent_login(&client, &base_url, &username, &password).await?;
let resp = client
.post(format!("{base_url}/api/v2/torrents/add"))
.header("Cookie", format!("SID={sid}"))
.form(&[("urls", &body.url)])
.send()
.await
.map_err(|e| ApiError::internal(format!("qBittorrent add request failed: {e}")))?;
if resp.status().is_success() {
Ok(Json(QBittorrentAddResponse {
success: true,
message: "Torrent added to qBittorrent".to_string(),
}))
} else {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
Ok(Json(QBittorrentAddResponse {
success: false,
message: format!("qBittorrent returned {status}: {text}"),
}))
}
}
/// Test connection to qBittorrent
#[utoipa::path(
get,
path = "/qbittorrent/test",
tag = "qbittorrent",
responses(
(status = 200, body = QBittorrentTestResponse),
(status = 400, description = "qBittorrent not configured"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn test_qbittorrent(
State(state): State<AppState>,
) -> Result<Json<QBittorrentTestResponse>, ApiError> {
let (base_url, username, password) = load_qbittorrent_config(&state.pool).await?;
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| ApiError::internal(format!("failed to build HTTP client: {e}")))?;
let sid = match qbittorrent_login(&client, &base_url, &username, &password).await {
Ok(sid) => sid,
Err(e) => {
return Ok(Json(QBittorrentTestResponse {
success: false,
message: format!("Login failed: {}", e.message),
version: None,
}));
}
};
let resp = client
.get(format!("{base_url}/api/v2/app/version"))
.header("Cookie", format!("SID={sid}"))
.send()
.await;
match resp {
Ok(r) if r.status().is_success() => {
let version = r.text().await.unwrap_or_default();
Ok(Json(QBittorrentTestResponse {
success: true,
message: format!("Connected successfully ({})", version.trim()),
version: Some(version.trim().to_string()),
}))
}
Ok(r) => {
let status = r.status();
let text = r.text().await.unwrap_or_default();
Ok(Json(QBittorrentTestResponse {
success: false,
message: format!("qBittorrent returned {status}: {text}"),
version: None,
}))
}
Err(e) => Ok(Json(QBittorrentTestResponse {
success: false,
message: format!("Connection failed: {e}"),
version: None,
})),
}
}

View File

@@ -0,0 +1,247 @@
use axum::{extract::{Path, State}, Json};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::Row;
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, state::AppState};
#[derive(Serialize, ToSchema)]
pub struct ReadingProgressResponse {
/// Reading status: "unread", "reading", or "read"
pub status: String,
/// Current page (only set when status is "reading")
pub current_page: Option<i32>,
#[schema(value_type = Option<String>)]
pub last_read_at: Option<DateTime<Utc>>,
}
#[derive(Deserialize, ToSchema)]
pub struct UpdateReadingProgressRequest {
/// Reading status: "unread", "reading", or "read"
pub status: String,
/// Required when status is "reading", must be > 0
pub current_page: Option<i32>,
}
/// Get reading progress for a book
#[utoipa::path(
get,
path = "/books/{id}/progress",
tag = "reading-progress",
params(
("id" = String, Path, description = "Book UUID"),
),
responses(
(status = 200, body = ReadingProgressResponse),
(status = 404, description = "Book not found"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_reading_progress(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<Json<ReadingProgressResponse>, ApiError> {
// Verify book exists
let exists: bool = sqlx::query_scalar("SELECT EXISTS(SELECT 1 FROM books WHERE id = $1)")
.bind(id)
.fetch_one(&state.pool)
.await?;
if !exists {
return Err(ApiError::not_found("book not found"));
}
let row = sqlx::query(
"SELECT status, current_page, last_read_at FROM book_reading_progress WHERE book_id = $1",
)
.bind(id)
.fetch_optional(&state.pool)
.await?;
let response = match row {
Some(r) => ReadingProgressResponse {
status: r.get("status"),
current_page: r.get("current_page"),
last_read_at: r.get("last_read_at"),
},
None => ReadingProgressResponse {
status: "unread".to_string(),
current_page: None,
last_read_at: None,
},
};
Ok(Json(response))
}
/// Update reading progress for a book
#[utoipa::path(
patch,
path = "/books/{id}/progress",
tag = "reading-progress",
params(
("id" = String, Path, description = "Book UUID"),
),
request_body = UpdateReadingProgressRequest,
responses(
(status = 200, body = ReadingProgressResponse),
(status = 404, description = "Book not found"),
(status = 422, description = "Validation error (missing or invalid current_page for status 'reading')"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn update_reading_progress(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Json(body): Json<UpdateReadingProgressRequest>,
) -> Result<Json<ReadingProgressResponse>, ApiError> {
// Validate status value
if !["unread", "reading", "read"].contains(&body.status.as_str()) {
return Err(ApiError::bad_request(format!(
"invalid status '{}': must be one of unread, reading, read",
body.status
)));
}
// Validate current_page for "reading" status
if body.status == "reading" {
match body.current_page {
None => {
return Err(ApiError::unprocessable_entity(
"current_page is required when status is 'reading'",
))
}
Some(p) if p <= 0 => {
return Err(ApiError::unprocessable_entity(
"current_page must be greater than 0",
))
}
_ => {}
}
}
// Verify book exists
let exists: bool = sqlx::query_scalar("SELECT EXISTS(SELECT 1 FROM books WHERE id = $1)")
.bind(id)
.fetch_one(&state.pool)
.await?;
if !exists {
return Err(ApiError::not_found("book not found"));
}
// current_page is only stored for "reading" status
let current_page = if body.status == "reading" {
body.current_page
} else {
None
};
let row = sqlx::query(
r#"
INSERT INTO book_reading_progress (book_id, status, current_page, last_read_at, updated_at)
VALUES ($1, $2, $3, NOW(), NOW())
ON CONFLICT (book_id) DO UPDATE
SET status = EXCLUDED.status,
current_page = EXCLUDED.current_page,
last_read_at = NOW(),
updated_at = NOW()
RETURNING status, current_page, last_read_at
"#,
)
.bind(id)
.bind(&body.status)
.bind(current_page)
.fetch_one(&state.pool)
.await?;
Ok(Json(ReadingProgressResponse {
status: row.get("status"),
current_page: row.get("current_page"),
last_read_at: row.get("last_read_at"),
}))
}
#[derive(Deserialize, ToSchema)]
pub struct MarkSeriesReadRequest {
/// Series name (use "unclassified" for books without series)
pub series: String,
/// Status to set: "read" or "unread"
pub status: String,
}
#[derive(Serialize, ToSchema)]
pub struct MarkSeriesReadResponse {
pub updated: i64,
}
/// Mark all books in a series as read or unread
#[utoipa::path(
post,
path = "/series/mark-read",
tag = "reading-progress",
request_body = MarkSeriesReadRequest,
responses(
(status = 200, body = MarkSeriesReadResponse),
(status = 422, description = "Invalid status"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn mark_series_read(
State(state): State<AppState>,
Json(body): Json<MarkSeriesReadRequest>,
) -> Result<Json<MarkSeriesReadResponse>, ApiError> {
if !["read", "unread"].contains(&body.status.as_str()) {
return Err(ApiError::bad_request(
"status must be 'read' or 'unread'",
));
}
let series_filter = if body.series == "unclassified" {
"(series IS NULL OR series = '')"
} else {
"series = $1"
};
let sql = if body.status == "unread" {
// Delete progress records to reset to unread
format!(
r#"
WITH target_books AS (
SELECT id FROM books WHERE {series_filter}
)
DELETE FROM book_reading_progress
WHERE book_id IN (SELECT id FROM target_books)
"#
)
} else {
format!(
r#"
INSERT INTO book_reading_progress (book_id, status, current_page, last_read_at, updated_at)
SELECT id, 'read', NULL, NOW(), NOW()
FROM books
WHERE {series_filter}
ON CONFLICT (book_id) DO UPDATE
SET status = 'read',
current_page = NULL,
last_read_at = NOW(),
updated_at = NOW()
"#
)
};
let result = if body.series == "unclassified" {
sqlx::query(&sql).execute(&state.pool).await?
} else {
sqlx::query(&sql).bind(&body.series).execute(&state.pool).await?
};
Ok(Json(MarkSeriesReadResponse {
updated: result.rows_affected() as i64,
}))
}

View File

@@ -1,8 +1,10 @@
use axum::{extract::{Query, State}, Json}; use axum::{extract::{Query, State}, Json};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use sqlx::Row;
use utoipa::ToSchema; use utoipa::ToSchema;
use uuid::Uuid;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)] #[derive(Deserialize, ToSchema)]
pub struct SearchQuery { pub struct SearchQuery {
@@ -18,24 +20,36 @@ pub struct SearchQuery {
pub limit: Option<usize>, pub limit: Option<usize>,
} }
#[derive(Serialize, ToSchema)]
pub struct SeriesHit {
#[schema(value_type = String)]
pub library_id: Uuid,
pub name: String,
pub book_count: i64,
pub books_read_count: i64,
#[schema(value_type = String)]
pub first_book_id: Uuid,
}
#[derive(Serialize, ToSchema)] #[derive(Serialize, ToSchema)]
pub struct SearchResponse { pub struct SearchResponse {
pub hits: serde_json::Value, pub hits: serde_json::Value,
pub series_hits: Vec<SeriesHit>,
pub estimated_total_hits: Option<u64>, pub estimated_total_hits: Option<u64>,
pub processing_time_ms: Option<u64>, pub processing_time_ms: Option<u64>,
} }
/// Search books across all libraries using Meilisearch /// Search books across all libraries
#[utoipa::path( #[utoipa::path(
get, get,
path = "/search", path = "/search",
tag = "books", tag = "search",
params( params(
("q" = String, Query, description = "Search query"), ("q" = String, Query, description = "Search query (books + series via PostgreSQL full-text)"),
("library_id" = Option<String>, Query, description = "Filter by library ID"), ("library_id" = Option<String>, Query, description = "Filter by library ID"),
("type" = Option<String>, Query, description = "Filter by type (cbz, cbr, pdf)"), ("type" = Option<String>, Query, description = "Filter by type (cbz, cbr, pdf, epub)"),
("kind" = Option<String>, Query, description = "Filter by kind (alias for type)"), ("kind" = Option<String>, Query, description = "Filter by kind (alias for type)"),
("limit" = Option<usize>, Query, description = "Max results (max 100)"), ("limit" = Option<usize>, Query, description = "Max results per type (max 100)"),
), ),
responses( responses(
(status = 200, body = SearchResponse), (status = 200, body = SearchResponse),
@@ -51,51 +65,127 @@ pub async fn search_books(
return Err(ApiError::bad_request("q is required")); return Err(ApiError::bad_request("q is required"));
} }
let mut filters: Vec<String> = Vec::new(); let limit_val = query.limit.unwrap_or(20).clamp(1, 100) as i64;
if let Some(library_id) = query.library_id.as_deref() { let q_pattern = format!("%{}%", query.q);
filters.push(format!("library_id = '{}'", library_id.replace('"', ""))); let library_id_uuid: Option<Uuid> = query.library_id.as_deref()
} .and_then(|s| s.parse().ok());
let kind_filter = query.r#type.as_deref().or(query.kind.as_deref()); let kind_filter: Option<&str> = query.r#type.as_deref().or(query.kind.as_deref());
if let Some(kind) = kind_filter {
filters.push(format!("kind = '{}'", kind.replace('"', "")));
}
let body = serde_json::json!({ let start = std::time::Instant::now();
"q": query.q,
"limit": query.limit.unwrap_or(20).clamp(1, 100),
"filter": if filters.is_empty() { serde_json::Value::Null } else { serde_json::Value::String(filters.join(" AND ")) }
});
let client = reqwest::Client::new(); // Book search via PostgreSQL ILIKE on title, authors, series
let url = format!("{}/indexes/books/search", state.meili_url.trim_end_matches('/')); let books_sql = r#"
let response = client SELECT b.id, b.library_id, b.kind, b.title,
.post(url) COALESCE(b.authors, CASE WHEN b.author IS NOT NULL AND b.author != '' THEN ARRAY[b.author] ELSE ARRAY[]::text[] END) as authors,
.header("Authorization", format!("Bearer {}", state.meili_master_key)) b.series, b.volume, b.language
.json(&body) FROM books b
.send() LEFT JOIN series_metadata sm
.await ON sm.library_id = b.library_id
.map_err(|e| ApiError::internal(format!("meili request failed: {e}")))?; AND sm.name = COALESCE(NULLIF(b.series, ''), 'unclassified')
WHERE (
b.title ILIKE $1
OR b.series ILIKE $1
OR EXISTS (SELECT 1 FROM unnest(
COALESCE(b.authors, CASE WHEN b.author IS NOT NULL AND b.author != '' THEN ARRAY[b.author] ELSE ARRAY[]::text[] END)
|| COALESCE(sm.authors, ARRAY[]::text[])
) AS a WHERE a ILIKE $1)
)
AND ($2::uuid IS NULL OR b.library_id = $2)
AND ($3::text IS NULL OR b.kind = $3)
ORDER BY
CASE WHEN b.title ILIKE $1 THEN 0 ELSE 1 END,
b.title ASC
LIMIT $4
"#;
if !response.status().is_success() { let series_sql = r#"
let body = response.text().await.unwrap_or_else(|_| "unknown meili error".to_string()); WITH sorted_books AS (
if body.contains("index_not_found") { SELECT
return Ok(Json(SearchResponse { library_id,
hits: serde_json::json!([]), COALESCE(NULLIF(series, ''), 'unclassified') as name,
estimated_total_hits: Some(0), id,
processing_time_ms: Some(0), ROW_NUMBER() OVER (
})); PARTITION BY library_id, COALESCE(NULLIF(series, ''), 'unclassified')
} ORDER BY
return Err(ApiError::internal(format!("meili error: {body}"))); REGEXP_REPLACE(LOWER(title), '[0-9]+', '', 'g'),
} COALESCE((REGEXP_MATCH(LOWER(title), '\d+'))[1]::int, 0),
title ASC
) as rn
FROM books
WHERE ($2::uuid IS NULL OR library_id = $2)
),
series_counts AS (
SELECT
sb.library_id,
sb.name,
COUNT(*) as book_count,
COUNT(brp.book_id) FILTER (WHERE brp.status = 'read') as books_read_count
FROM sorted_books sb
LEFT JOIN book_reading_progress brp ON brp.book_id = sb.id
GROUP BY sb.library_id, sb.name
)
SELECT sc.library_id, sc.name, sc.book_count, sc.books_read_count, sb.id as first_book_id
FROM series_counts sc
JOIN sorted_books sb ON sb.library_id = sc.library_id AND sb.name = sc.name AND sb.rn = 1
WHERE sc.name ILIKE $1
ORDER BY sc.name ASC
LIMIT $4
"#;
let payload: serde_json::Value = response let (books_rows, series_rows) = tokio::join!(
.json() sqlx::query(books_sql)
.await .bind(&q_pattern)
.map_err(|e| ApiError::internal(format!("invalid meili response: {e}")))?; .bind(library_id_uuid)
.bind(kind_filter)
.bind(limit_val)
.fetch_all(&state.pool),
sqlx::query(series_sql)
.bind(&q_pattern)
.bind(library_id_uuid)
.bind(kind_filter) // unused in series query but keeps bind positions consistent
.bind(limit_val)
.fetch_all(&state.pool)
);
let elapsed_ms = start.elapsed().as_millis() as u64;
// Build book hits as JSON array (same shape as before)
let books_rows = books_rows.map_err(|e| ApiError::internal(format!("book search failed: {e}")))?;
let hits: Vec<serde_json::Value> = books_rows
.iter()
.map(|row| {
serde_json::json!({
"id": row.get::<Uuid, _>("id").to_string(),
"library_id": row.get::<Uuid, _>("library_id").to_string(),
"kind": row.get::<String, _>("kind"),
"title": row.get::<String, _>("title"),
"authors": row.get::<Vec<String>, _>("authors"),
"series": row.get::<Option<String>, _>("series"),
"volume": row.get::<Option<i32>, _>("volume"),
"language": row.get::<Option<String>, _>("language"),
})
})
.collect();
let estimated_total_hits = hits.len() as u64;
// Series hits
let series_hits: Vec<SeriesHit> = series_rows
.unwrap_or_default()
.iter()
.map(|row| SeriesHit {
library_id: row.get("library_id"),
name: row.get("name"),
book_count: row.get("book_count"),
books_read_count: row.get("books_read_count"),
first_book_id: row.get("first_book_id"),
})
.collect();
Ok(Json(SearchResponse { Ok(Json(SearchResponse {
hits: payload.get("hits").cloned().unwrap_or_else(|| serde_json::json!([])), hits: serde_json::Value::Array(hits),
estimated_total_hits: payload.get("estimatedTotalHits").and_then(|v| v.as_u64()), series_hits,
processing_time_ms: payload.get("processingTimeMs").and_then(|v| v.as_u64()), estimated_total_hits: Some(estimated_total_hits),
processing_time_ms: Some(elapsed_ms),
})) }))
} }

1028
apps/api/src/series.rs Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,33 +1,35 @@
use axum::{ use axum::{
extract::State, extract::{Path as AxumPath, State},
routing::{get, post}, routing::{delete, get, post},
Json, Router, Json, Router,
}; };
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::Value; use serde_json::Value;
use sqlx::Row; use sqlx::Row;
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::{AppState, load_dynamic_settings}};
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
pub struct UpdateSettingRequest { pub struct UpdateSettingRequest {
pub value: Value, pub value: Value,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
pub struct ClearCacheResponse { pub struct ClearCacheResponse {
pub success: bool, pub success: bool,
pub message: String, pub message: String,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
pub struct CacheStats { pub struct CacheStats {
pub total_size_mb: f64, pub total_size_mb: f64,
pub file_count: u64, pub file_count: u64,
pub directory: String, pub directory: String,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
pub struct ThumbnailStats { pub struct ThumbnailStats {
pub total_size_mb: f64, pub total_size_mb: f64,
pub file_count: u64, pub file_count: u64,
@@ -41,9 +43,28 @@ pub fn settings_routes() -> Router<AppState> {
.route("/settings/cache/clear", post(clear_cache)) .route("/settings/cache/clear", post(clear_cache))
.route("/settings/cache/stats", get(get_cache_stats)) .route("/settings/cache/stats", get(get_cache_stats))
.route("/settings/thumbnail/stats", get(get_thumbnail_stats)) .route("/settings/thumbnail/stats", get(get_thumbnail_stats))
.route(
"/settings/status-mappings",
get(list_status_mappings).post(upsert_status_mapping),
)
.route(
"/settings/status-mappings/:id",
delete(delete_status_mapping),
)
} }
async fn get_settings(State(state): State<AppState>) -> Result<Json<Value>, ApiError> { /// List all settings
#[utoipa::path(
get,
path = "/settings",
tag = "settings",
responses(
(status = 200, description = "All settings as key/value object"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_settings(State(state): State<AppState>) -> Result<Json<Value>, ApiError> {
let rows = sqlx::query(r#"SELECT key, value FROM app_settings"#) let rows = sqlx::query(r#"SELECT key, value FROM app_settings"#)
.fetch_all(&state.pool) .fetch_all(&state.pool)
.await?; .await?;
@@ -58,7 +79,20 @@ async fn get_settings(State(state): State<AppState>) -> Result<Json<Value>, ApiE
Ok(Json(Value::Object(settings))) Ok(Json(Value::Object(settings)))
} }
async fn get_setting( /// Get a single setting by key
#[utoipa::path(
get,
path = "/settings/{key}",
tag = "settings",
params(("key" = String, Path, description = "Setting key")),
responses(
(status = 200, description = "Setting value"),
(status = 404, description = "Setting not found"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_setting(
State(state): State<AppState>, State(state): State<AppState>,
axum::extract::Path(key): axum::extract::Path<String>, axum::extract::Path(key): axum::extract::Path<String>,
) -> Result<Json<Value>, ApiError> { ) -> Result<Json<Value>, ApiError> {
@@ -76,7 +110,20 @@ async fn get_setting(
} }
} }
async fn update_setting( /// Create or update a setting
#[utoipa::path(
post,
path = "/settings/{key}",
tag = "settings",
params(("key" = String, Path, description = "Setting key")),
request_body = UpdateSettingRequest,
responses(
(status = 200, description = "Updated setting value"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn update_setting(
State(state): State<AppState>, State(state): State<AppState>,
axum::extract::Path(key): axum::extract::Path<String>, axum::extract::Path(key): axum::extract::Path<String>,
Json(body): Json<UpdateSettingRequest>, Json(body): Json<UpdateSettingRequest>,
@@ -96,12 +143,29 @@ async fn update_setting(
.await?; .await?;
let value: Value = row.get("value"); let value: Value = row.get("value");
// Rechargement des settings dynamiques si la clé affecte le comportement runtime
if key == "limits" || key == "image_processing" || key == "cache" {
let new_settings = load_dynamic_settings(&state.pool).await;
*state.settings.write().await = new_settings;
}
Ok(Json(value)) Ok(Json(value))
} }
async fn clear_cache(State(_state): State<AppState>) -> Result<Json<ClearCacheResponse>, ApiError> { /// Clear the image page cache
let cache_dir = std::env::var("IMAGE_CACHE_DIR") #[utoipa::path(
.unwrap_or_else(|_| "/tmp/stripstream-image-cache".to_string()); post,
path = "/settings/cache/clear",
tag = "settings",
responses(
(status = 200, body = ClearCacheResponse),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn clear_cache(State(state): State<AppState>) -> Result<Json<ClearCacheResponse>, ApiError> {
let cache_dir = state.settings.read().await.cache_directory.clone();
let result = tokio::task::spawn_blocking(move || { let result = tokio::task::spawn_blocking(move || {
if std::path::Path::new(&cache_dir).exists() { if std::path::Path::new(&cache_dir).exists() {
@@ -128,9 +192,19 @@ async fn clear_cache(State(_state): State<AppState>) -> Result<Json<ClearCacheRe
Ok(Json(result)) Ok(Json(result))
} }
async fn get_cache_stats(State(_state): State<AppState>) -> Result<Json<CacheStats>, ApiError> { /// Get image page cache statistics
let cache_dir = std::env::var("IMAGE_CACHE_DIR") #[utoipa::path(
.unwrap_or_else(|_| "/tmp/stripstream-image-cache".to_string()); get,
path = "/settings/cache/stats",
tag = "settings",
responses(
(status = 200, body = CacheStats),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_cache_stats(State(state): State<AppState>) -> Result<Json<CacheStats>, ApiError> {
let cache_dir = state.settings.read().await.cache_directory.clone();
let cache_dir_clone = cache_dir.clone(); let cache_dir_clone = cache_dir.clone();
let stats = tokio::task::spawn_blocking(move || { let stats = tokio::task::spawn_blocking(move || {
@@ -208,7 +282,18 @@ fn compute_dir_stats(path: &std::path::Path) -> (u64, u64) {
(total_size, file_count) (total_size, file_count)
} }
async fn get_thumbnail_stats(State(_state): State<AppState>) -> Result<Json<ThumbnailStats>, ApiError> { /// Get thumbnail storage statistics
#[utoipa::path(
get,
path = "/settings/thumbnail/stats",
tag = "settings",
responses(
(status = 200, body = ThumbnailStats),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_thumbnail_stats(State(_state): State<AppState>) -> Result<Json<ThumbnailStats>, ApiError> {
let settings = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#) let settings = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#)
.fetch_optional(&_state.pool) .fetch_optional(&_state.pool)
.await?; .await?;
@@ -248,3 +333,125 @@ async fn get_thumbnail_stats(State(_state): State<AppState>) -> Result<Json<Thum
Ok(Json(stats)) Ok(Json(stats))
} }
// ---------------------------------------------------------------------------
// Status Mappings
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
pub struct StatusMappingDto {
pub id: String,
pub provider_status: String,
pub mapped_status: Option<String>,
}
#[derive(Debug, Clone, Deserialize, ToSchema)]
pub struct UpsertStatusMappingRequest {
pub provider_status: String,
pub mapped_status: String,
}
/// List all status mappings
#[utoipa::path(
get,
path = "/settings/status-mappings",
tag = "settings",
responses(
(status = 200, body = Vec<StatusMappingDto>),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn list_status_mappings(
State(state): State<AppState>,
) -> Result<Json<Vec<StatusMappingDto>>, ApiError> {
let rows = sqlx::query(
"SELECT id, provider_status, mapped_status FROM status_mappings ORDER BY mapped_status NULLS LAST, provider_status",
)
.fetch_all(&state.pool)
.await?;
let mappings = rows
.iter()
.map(|row| StatusMappingDto {
id: row.get::<Uuid, _>("id").to_string(),
provider_status: row.get("provider_status"),
mapped_status: row.get::<Option<String>, _>("mapped_status"),
})
.collect();
Ok(Json(mappings))
}
/// Create or update a status mapping
#[utoipa::path(
post,
path = "/settings/status-mappings",
tag = "settings",
request_body = UpsertStatusMappingRequest,
responses(
(status = 200, body = StatusMappingDto),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn upsert_status_mapping(
State(state): State<AppState>,
Json(body): Json<UpsertStatusMappingRequest>,
) -> Result<Json<StatusMappingDto>, ApiError> {
let provider_status = body.provider_status.to_lowercase();
let row = sqlx::query(
r#"
INSERT INTO status_mappings (provider_status, mapped_status)
VALUES ($1, $2)
ON CONFLICT (provider_status)
DO UPDATE SET mapped_status = $2, updated_at = NOW()
RETURNING id, provider_status, mapped_status
"#,
)
.bind(&provider_status)
.bind(&body.mapped_status)
.fetch_one(&state.pool)
.await?;
Ok(Json(StatusMappingDto {
id: row.get::<Uuid, _>("id").to_string(),
provider_status: row.get("provider_status"),
mapped_status: row.get::<Option<String>, _>("mapped_status"),
}))
}
/// Unmap a status mapping (sets mapped_status to NULL, keeps the provider status known)
#[utoipa::path(
delete,
path = "/settings/status-mappings/{id}",
tag = "settings",
params(("id" = String, Path, description = "Mapping UUID")),
responses(
(status = 200, body = StatusMappingDto),
(status = 401, description = "Unauthorized"),
(status = 404, description = "Not found"),
),
security(("Bearer" = []))
)]
pub async fn delete_status_mapping(
State(state): State<AppState>,
AxumPath(id): AxumPath<Uuid>,
) -> Result<Json<StatusMappingDto>, ApiError> {
let row = sqlx::query(
"UPDATE status_mappings SET mapped_status = NULL, updated_at = NOW() WHERE id = $1 RETURNING id, provider_status, mapped_status",
)
.bind(id)
.fetch_optional(&state.pool)
.await?;
match row {
Some(row) => Ok(Json(StatusMappingDto {
id: row.get::<Uuid, _>("id").to_string(),
provider_status: row.get("provider_status"),
mapped_status: row.get::<Option<String>, _>("mapped_status"),
})),
None => Err(ApiError::not_found("status mapping not found")),
}
}

134
apps/api/src/state.rs Normal file
View File

@@ -0,0 +1,134 @@
use std::sync::{
atomic::AtomicU64,
Arc,
};
use std::time::Instant;
use lru::LruCache;
use sqlx::{Pool, Postgres, Row};
use tokio::sync::{Mutex, RwLock, Semaphore};
#[derive(Clone)]
pub struct AppState {
pub pool: sqlx::PgPool,
pub bootstrap_token: Arc<str>,
pub page_cache: Arc<Mutex<LruCache<String, Arc<Vec<u8>>>>>,
pub page_render_limit: Arc<Semaphore>,
pub metrics: Arc<Metrics>,
pub read_rate_limit: Arc<Mutex<ReadRateLimit>>,
pub settings: Arc<RwLock<DynamicSettings>>,
}
#[derive(Clone)]
pub struct DynamicSettings {
pub rate_limit_per_second: u32,
pub timeout_seconds: u64,
pub image_format: String,
pub image_quality: u8,
pub image_filter: String,
pub image_max_width: u32,
pub cache_directory: String,
}
impl Default for DynamicSettings {
fn default() -> Self {
Self {
rate_limit_per_second: 120,
timeout_seconds: 12,
image_format: "webp".to_string(),
image_quality: 85,
image_filter: "triangle".to_string(),
image_max_width: 2160,
cache_directory: std::env::var("IMAGE_CACHE_DIR")
.unwrap_or_else(|_| "/tmp/stripstream-image-cache".to_string()),
}
}
}
pub struct Metrics {
pub requests_total: AtomicU64,
pub page_cache_hits: AtomicU64,
pub page_cache_misses: AtomicU64,
}
pub struct ReadRateLimit {
pub window_started_at: Instant,
pub requests_in_window: u32,
}
impl Metrics {
pub fn new() -> Self {
Self {
requests_total: AtomicU64::new(0),
page_cache_hits: AtomicU64::new(0),
page_cache_misses: AtomicU64::new(0),
}
}
}
pub async fn load_concurrent_renders(pool: &Pool<Postgres>) -> usize {
let default_concurrency = 8;
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
value
.get("concurrent_renders")
.and_then(|v: &serde_json::Value| v.as_u64())
.map(|v| v as usize)
.unwrap_or(default_concurrency)
}
_ => default_concurrency,
}
}
pub async fn load_dynamic_settings(pool: &Pool<Postgres>) -> DynamicSettings {
let mut s = DynamicSettings::default();
if let Ok(Some(row)) = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
.fetch_optional(pool)
.await
{
let v: serde_json::Value = row.get("value");
if let Some(n) = v.get("rate_limit_per_second").and_then(|x| x.as_u64()) {
s.rate_limit_per_second = n as u32;
}
if let Some(n) = v.get("timeout_seconds").and_then(|x| x.as_u64()) {
s.timeout_seconds = n;
}
}
if let Ok(Some(row)) = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'image_processing'"#)
.fetch_optional(pool)
.await
{
let v: serde_json::Value = row.get("value");
if let Some(s2) = v.get("format").and_then(|x| x.as_str()) {
s.image_format = s2.to_string();
}
if let Some(n) = v.get("quality").and_then(|x| x.as_u64()) {
s.image_quality = n.clamp(1, 100) as u8;
}
if let Some(s2) = v.get("filter").and_then(|x| x.as_str()) {
s.image_filter = s2.to_string();
}
if let Some(n) = v.get("max_width").and_then(|x| x.as_u64()) {
s.image_max_width = n as u32;
}
}
if let Ok(Some(row)) = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'cache'"#)
.fetch_optional(pool)
.await
{
let v: serde_json::Value = row.get("value");
if let Some(dir) = v.get("directory").and_then(|x| x.as_str()) {
s.cache_directory = dir.to_string();
}
}
s
}

701
apps/api/src/stats.rs Normal file
View File

@@ -0,0 +1,701 @@
use axum::{
extract::{Query, State},
Json,
};
use serde::{Deserialize, Serialize};
use sqlx::Row;
use utoipa::{IntoParams, ToSchema};
use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, IntoParams)]
pub struct StatsQuery {
/// Granularity: "day", "week" or "month" (default: "month")
pub period: Option<String>,
}
#[derive(Serialize, ToSchema)]
pub struct StatsOverview {
pub total_books: i64,
pub total_series: i64,
pub total_libraries: i64,
pub total_pages: i64,
pub total_size_bytes: i64,
pub total_authors: i64,
}
#[derive(Serialize, ToSchema)]
pub struct ReadingStatusStats {
pub unread: i64,
pub reading: i64,
pub read: i64,
}
#[derive(Serialize, ToSchema)]
pub struct FormatCount {
pub format: String,
pub count: i64,
}
#[derive(Serialize, ToSchema)]
pub struct LanguageCount {
pub language: Option<String>,
pub count: i64,
}
#[derive(Serialize, ToSchema)]
pub struct LibraryStats {
pub library_name: String,
pub book_count: i64,
pub size_bytes: i64,
pub read_count: i64,
pub reading_count: i64,
pub unread_count: i64,
}
#[derive(Serialize, ToSchema)]
pub struct TopSeries {
pub series: String,
pub book_count: i64,
pub read_count: i64,
pub total_pages: i64,
}
#[derive(Serialize, ToSchema)]
pub struct MonthlyAdditions {
pub month: String,
pub books_added: i64,
}
#[derive(Serialize, ToSchema)]
pub struct MetadataStats {
pub total_series: i64,
pub series_linked: i64,
pub series_unlinked: i64,
pub books_with_summary: i64,
pub books_with_isbn: i64,
pub by_provider: Vec<ProviderCount>,
}
#[derive(Serialize, ToSchema)]
pub struct ProviderCount {
pub provider: String,
pub count: i64,
}
#[derive(Serialize, ToSchema)]
pub struct CurrentlyReadingItem {
pub book_id: String,
pub title: String,
pub series: Option<String>,
pub current_page: i32,
pub page_count: i32,
}
#[derive(Serialize, ToSchema)]
pub struct RecentlyReadItem {
pub book_id: String,
pub title: String,
pub series: Option<String>,
pub last_read_at: String,
}
#[derive(Serialize, ToSchema)]
pub struct MonthlyReading {
pub month: String,
pub books_read: i64,
}
#[derive(Serialize, ToSchema)]
pub struct JobTimePoint {
pub label: String,
pub scan: i64,
pub rebuild: i64,
pub thumbnail: i64,
pub other: i64,
}
#[derive(Serialize, ToSchema)]
pub struct StatsResponse {
pub overview: StatsOverview,
pub reading_status: ReadingStatusStats,
pub currently_reading: Vec<CurrentlyReadingItem>,
pub recently_read: Vec<RecentlyReadItem>,
pub reading_over_time: Vec<MonthlyReading>,
pub by_format: Vec<FormatCount>,
pub by_language: Vec<LanguageCount>,
pub by_library: Vec<LibraryStats>,
pub top_series: Vec<TopSeries>,
pub additions_over_time: Vec<MonthlyAdditions>,
pub jobs_over_time: Vec<JobTimePoint>,
pub metadata: MetadataStats,
}
/// Get collection statistics for the dashboard
#[utoipa::path(
get,
path = "/stats",
tag = "stats",
params(StatsQuery),
responses(
(status = 200, body = StatsResponse),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn get_stats(
State(state): State<AppState>,
Query(query): Query<StatsQuery>,
) -> Result<Json<StatsResponse>, ApiError> {
let period = query.period.as_deref().unwrap_or("month");
// Overview + reading status in one query
let overview_row = sqlx::query(
r#"
SELECT
COUNT(*) AS total_books,
COUNT(DISTINCT NULLIF(series, '')) AS total_series,
COUNT(DISTINCT library_id) AS total_libraries,
COALESCE(SUM(page_count), 0)::BIGINT AS total_pages,
(SELECT COUNT(DISTINCT a) FROM (
SELECT DISTINCT UNNEST(authors) AS a FROM books WHERE authors != '{}'
UNION
SELECT DISTINCT author FROM books WHERE author IS NOT NULL AND author != ''
) sub) AS total_authors,
COUNT(*) FILTER (WHERE COALESCE(brp.status, 'unread') = 'unread') AS unread,
COUNT(*) FILTER (WHERE brp.status = 'reading') AS reading,
COUNT(*) FILTER (WHERE brp.status = 'read') AS read
FROM books b
LEFT JOIN book_reading_progress brp ON brp.book_id = b.id
"#,
)
.fetch_one(&state.pool)
.await?;
// Total size from book_files
let size_row = sqlx::query(
r#"
SELECT COALESCE(SUM(bf.size_bytes), 0)::BIGINT AS total_size_bytes
FROM (
SELECT DISTINCT ON (book_id) size_bytes
FROM book_files
ORDER BY book_id, updated_at DESC
) bf
"#,
)
.fetch_one(&state.pool)
.await?;
let overview = StatsOverview {
total_books: overview_row.get("total_books"),
total_series: overview_row.get("total_series"),
total_libraries: overview_row.get("total_libraries"),
total_pages: overview_row.get("total_pages"),
total_size_bytes: size_row.get("total_size_bytes"),
total_authors: overview_row.get("total_authors"),
};
let reading_status = ReadingStatusStats {
unread: overview_row.get("unread"),
reading: overview_row.get("reading"),
read: overview_row.get("read"),
};
// By format
let format_rows = sqlx::query(
r#"
SELECT COALESCE(bf.format, b.kind) AS fmt, COUNT(*) AS count
FROM books b
LEFT JOIN LATERAL (
SELECT format FROM book_files WHERE book_id = b.id ORDER BY updated_at DESC LIMIT 1
) bf ON TRUE
GROUP BY fmt
ORDER BY count DESC
"#,
)
.fetch_all(&state.pool)
.await?;
let by_format: Vec<FormatCount> = format_rows
.iter()
.map(|r| FormatCount {
format: r.get::<Option<String>, _>("fmt").unwrap_or_else(|| "unknown".to_string()),
count: r.get("count"),
})
.collect();
// By language
let lang_rows = sqlx::query(
r#"
SELECT language, COUNT(*) AS count
FROM books
GROUP BY language
ORDER BY count DESC
"#,
)
.fetch_all(&state.pool)
.await?;
let by_language: Vec<LanguageCount> = lang_rows
.iter()
.map(|r| LanguageCount {
language: r.get("language"),
count: r.get("count"),
})
.collect();
// By library
let lib_rows = sqlx::query(
r#"
SELECT
l.name AS library_name,
COUNT(b.id) AS book_count,
COALESCE(SUM(bf.size_bytes), 0)::BIGINT AS size_bytes,
COUNT(*) FILTER (WHERE brp.status = 'read') AS read_count,
COUNT(*) FILTER (WHERE brp.status = 'reading') AS reading_count,
COUNT(*) FILTER (WHERE COALESCE(brp.status, 'unread') = 'unread') AS unread_count
FROM libraries l
LEFT JOIN books b ON b.library_id = l.id
LEFT JOIN book_reading_progress brp ON brp.book_id = b.id
LEFT JOIN LATERAL (
SELECT size_bytes FROM book_files WHERE book_id = b.id ORDER BY updated_at DESC LIMIT 1
) bf ON TRUE
GROUP BY l.id, l.name
ORDER BY book_count DESC
"#,
)
.fetch_all(&state.pool)
.await?;
let by_library: Vec<LibraryStats> = lib_rows
.iter()
.map(|r| LibraryStats {
library_name: r.get("library_name"),
book_count: r.get("book_count"),
size_bytes: r.get("size_bytes"),
read_count: r.get("read_count"),
reading_count: r.get("reading_count"),
unread_count: r.get("unread_count"),
})
.collect();
// Top series (by book count)
let series_rows = sqlx::query(
r#"
SELECT
b.series,
COUNT(*) AS book_count,
COUNT(*) FILTER (WHERE brp.status = 'read') AS read_count,
COALESCE(SUM(b.page_count), 0)::BIGINT AS total_pages
FROM books b
LEFT JOIN book_reading_progress brp ON brp.book_id = b.id
WHERE b.series IS NOT NULL AND b.series != ''
GROUP BY b.series
ORDER BY book_count DESC
LIMIT 10
"#,
)
.fetch_all(&state.pool)
.await?;
let top_series: Vec<TopSeries> = series_rows
.iter()
.map(|r| TopSeries {
series: r.get("series"),
book_count: r.get("book_count"),
read_count: r.get("read_count"),
total_pages: r.get("total_pages"),
})
.collect();
// Additions over time (with gap filling)
let additions_rows = match period {
"day" => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM-DD') AS month,
COALESCE(cnt.books_added, 0) AS books_added
FROM generate_series(CURRENT_DATE - INTERVAL '6 days', CURRENT_DATE, '1 day') AS d(dt)
LEFT JOIN (
SELECT created_at::date AS dt, COUNT(*) AS books_added
FROM books
WHERE created_at >= CURRENT_DATE - INTERVAL '6 days'
GROUP BY created_at::date
) cnt ON cnt.dt = d.dt
ORDER BY month ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
"week" => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM-DD') AS month,
COALESCE(cnt.books_added, 0) AS books_added
FROM generate_series(
DATE_TRUNC('week', NOW() - INTERVAL '2 months'),
DATE_TRUNC('week', NOW()),
'1 week'
) AS d(dt)
LEFT JOIN (
SELECT DATE_TRUNC('week', created_at) AS dt, COUNT(*) AS books_added
FROM books
WHERE created_at >= DATE_TRUNC('week', NOW() - INTERVAL '2 months')
GROUP BY DATE_TRUNC('week', created_at)
) cnt ON cnt.dt = d.dt
ORDER BY month ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
_ => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM') AS month,
COALESCE(cnt.books_added, 0) AS books_added
FROM generate_series(
DATE_TRUNC('month', NOW()) - INTERVAL '11 months',
DATE_TRUNC('month', NOW()),
'1 month'
) AS d(dt)
LEFT JOIN (
SELECT DATE_TRUNC('month', created_at) AS dt, COUNT(*) AS books_added
FROM books
WHERE created_at >= DATE_TRUNC('month', NOW()) - INTERVAL '11 months'
GROUP BY DATE_TRUNC('month', created_at)
) cnt ON cnt.dt = d.dt
ORDER BY month ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
};
let additions_over_time: Vec<MonthlyAdditions> = additions_rows
.iter()
.map(|r| MonthlyAdditions {
month: r.get("month"),
books_added: r.get("books_added"),
})
.collect();
// Metadata stats
let meta_row = sqlx::query(
r#"
SELECT
(SELECT COUNT(DISTINCT NULLIF(series, '')) FROM books) AS total_series,
(SELECT COUNT(DISTINCT series_name) FROM external_metadata_links WHERE status = 'approved') AS series_linked,
(SELECT COUNT(*) FROM books WHERE summary IS NOT NULL AND summary != '') AS books_with_summary,
(SELECT COUNT(*) FROM books WHERE isbn IS NOT NULL AND isbn != '') AS books_with_isbn
"#,
)
.fetch_one(&state.pool)
.await?;
let meta_total_series: i64 = meta_row.get("total_series");
let meta_series_linked: i64 = meta_row.get("series_linked");
let provider_rows = sqlx::query(
r#"
SELECT provider, COUNT(DISTINCT series_name) AS count
FROM external_metadata_links
WHERE status = 'approved'
GROUP BY provider
ORDER BY count DESC
"#,
)
.fetch_all(&state.pool)
.await?;
let by_provider: Vec<ProviderCount> = provider_rows
.iter()
.map(|r| ProviderCount {
provider: r.get("provider"),
count: r.get("count"),
})
.collect();
let metadata = MetadataStats {
total_series: meta_total_series,
series_linked: meta_series_linked,
series_unlinked: meta_total_series - meta_series_linked,
books_with_summary: meta_row.get("books_with_summary"),
books_with_isbn: meta_row.get("books_with_isbn"),
by_provider,
};
// Currently reading books
let reading_rows = sqlx::query(
r#"
SELECT b.id AS book_id, b.title, b.series, brp.current_page, b.page_count
FROM book_reading_progress brp
JOIN books b ON b.id = brp.book_id
WHERE brp.status = 'reading' AND brp.current_page IS NOT NULL
ORDER BY brp.updated_at DESC
LIMIT 20
"#,
)
.fetch_all(&state.pool)
.await?;
let currently_reading: Vec<CurrentlyReadingItem> = reading_rows
.iter()
.map(|r| {
let id: uuid::Uuid = r.get("book_id");
CurrentlyReadingItem {
book_id: id.to_string(),
title: r.get("title"),
series: r.get("series"),
current_page: r.get::<Option<i32>, _>("current_page").unwrap_or(0),
page_count: r.get::<Option<i32>, _>("page_count").unwrap_or(0),
}
})
.collect();
// Recently read books
let recent_rows = sqlx::query(
r#"
SELECT b.id AS book_id, b.title, b.series,
TO_CHAR(brp.last_read_at, 'YYYY-MM-DD') AS last_read_at
FROM book_reading_progress brp
JOIN books b ON b.id = brp.book_id
WHERE brp.status = 'read' AND brp.last_read_at IS NOT NULL
ORDER BY brp.last_read_at DESC
LIMIT 10
"#,
)
.fetch_all(&state.pool)
.await?;
let recently_read: Vec<RecentlyReadItem> = recent_rows
.iter()
.map(|r| {
let id: uuid::Uuid = r.get("book_id");
RecentlyReadItem {
book_id: id.to_string(),
title: r.get("title"),
series: r.get("series"),
last_read_at: r.get::<Option<String>, _>("last_read_at").unwrap_or_default(),
}
})
.collect();
// Reading activity over time (with gap filling)
let reading_time_rows = match period {
"day" => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM-DD') AS month,
COALESCE(cnt.books_read, 0) AS books_read
FROM generate_series(CURRENT_DATE - INTERVAL '6 days', CURRENT_DATE, '1 day') AS d(dt)
LEFT JOIN (
SELECT brp.last_read_at::date AS dt, COUNT(*) AS books_read
FROM book_reading_progress brp
WHERE brp.status = 'read'
AND brp.last_read_at >= CURRENT_DATE - INTERVAL '6 days'
GROUP BY brp.last_read_at::date
) cnt ON cnt.dt = d.dt
ORDER BY month ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
"week" => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM-DD') AS month,
COALESCE(cnt.books_read, 0) AS books_read
FROM generate_series(
DATE_TRUNC('week', NOW() - INTERVAL '2 months'),
DATE_TRUNC('week', NOW()),
'1 week'
) AS d(dt)
LEFT JOIN (
SELECT DATE_TRUNC('week', brp.last_read_at) AS dt, COUNT(*) AS books_read
FROM book_reading_progress brp
WHERE brp.status = 'read'
AND brp.last_read_at >= DATE_TRUNC('week', NOW() - INTERVAL '2 months')
GROUP BY DATE_TRUNC('week', brp.last_read_at)
) cnt ON cnt.dt = d.dt
ORDER BY month ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
_ => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM') AS month,
COALESCE(cnt.books_read, 0) AS books_read
FROM generate_series(
DATE_TRUNC('month', NOW()) - INTERVAL '11 months',
DATE_TRUNC('month', NOW()),
'1 month'
) AS d(dt)
LEFT JOIN (
SELECT DATE_TRUNC('month', brp.last_read_at) AS dt, COUNT(*) AS books_read
FROM book_reading_progress brp
WHERE brp.status = 'read'
AND brp.last_read_at >= DATE_TRUNC('month', NOW()) - INTERVAL '11 months'
GROUP BY DATE_TRUNC('month', brp.last_read_at)
) cnt ON cnt.dt = d.dt
ORDER BY month ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
};
let reading_over_time: Vec<MonthlyReading> = reading_time_rows
.iter()
.map(|r| MonthlyReading {
month: r.get::<Option<String>, _>("month").unwrap_or_default(),
books_read: r.get("books_read"),
})
.collect();
// Jobs over time (with gap filling, grouped by type category)
let jobs_rows = match period {
"day" => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM-DD') AS label,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'scan'), 0)::BIGINT AS scan,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'rebuild'), 0)::BIGINT AS rebuild,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'thumbnail'), 0)::BIGINT AS thumbnail,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'other'), 0)::BIGINT AS other
FROM generate_series(CURRENT_DATE - INTERVAL '6 days', CURRENT_DATE, '1 day') AS d(dt)
LEFT JOIN (
SELECT
finished_at::date AS dt,
CASE
WHEN type = 'scan' THEN 'scan'
WHEN type IN ('rebuild', 'full_rebuild', 'rescan') THEN 'rebuild'
WHEN type IN ('thumbnail_rebuild', 'thumbnail_regenerate') THEN 'thumbnail'
ELSE 'other'
END AS cat,
COUNT(*) AS c
FROM index_jobs
WHERE status IN ('success', 'failed')
AND finished_at >= CURRENT_DATE - INTERVAL '6 days'
GROUP BY finished_at::date, cat
) cnt ON cnt.dt = d.dt
GROUP BY d.dt
ORDER BY label ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
"week" => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM-DD') AS label,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'scan'), 0)::BIGINT AS scan,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'rebuild'), 0)::BIGINT AS rebuild,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'thumbnail'), 0)::BIGINT AS thumbnail,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'other'), 0)::BIGINT AS other
FROM generate_series(
DATE_TRUNC('week', NOW() - INTERVAL '2 months'),
DATE_TRUNC('week', NOW()),
'1 week'
) AS d(dt)
LEFT JOIN (
SELECT
DATE_TRUNC('week', finished_at) AS dt,
CASE
WHEN type = 'scan' THEN 'scan'
WHEN type IN ('rebuild', 'full_rebuild', 'rescan') THEN 'rebuild'
WHEN type IN ('thumbnail_rebuild', 'thumbnail_regenerate') THEN 'thumbnail'
ELSE 'other'
END AS cat,
COUNT(*) AS c
FROM index_jobs
WHERE status IN ('success', 'failed')
AND finished_at >= DATE_TRUNC('week', NOW() - INTERVAL '2 months')
GROUP BY DATE_TRUNC('week', finished_at), cat
) cnt ON cnt.dt = d.dt
GROUP BY d.dt
ORDER BY label ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
_ => {
sqlx::query(
r#"
SELECT
TO_CHAR(d.dt, 'YYYY-MM') AS label,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'scan'), 0)::BIGINT AS scan,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'rebuild'), 0)::BIGINT AS rebuild,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'thumbnail'), 0)::BIGINT AS thumbnail,
COALESCE(SUM(cnt.c) FILTER (WHERE cnt.cat = 'other'), 0)::BIGINT AS other
FROM generate_series(
DATE_TRUNC('month', NOW()) - INTERVAL '11 months',
DATE_TRUNC('month', NOW()),
'1 month'
) AS d(dt)
LEFT JOIN (
SELECT
DATE_TRUNC('month', finished_at) AS dt,
CASE
WHEN type = 'scan' THEN 'scan'
WHEN type IN ('rebuild', 'full_rebuild', 'rescan') THEN 'rebuild'
WHEN type IN ('thumbnail_rebuild', 'thumbnail_regenerate') THEN 'thumbnail'
ELSE 'other'
END AS cat,
COUNT(*) AS c
FROM index_jobs
WHERE status IN ('success', 'failed')
AND finished_at >= DATE_TRUNC('month', NOW()) - INTERVAL '11 months'
GROUP BY DATE_TRUNC('month', finished_at), cat
) cnt ON cnt.dt = d.dt
GROUP BY d.dt
ORDER BY label ASC
"#,
)
.fetch_all(&state.pool)
.await?
}
};
let jobs_over_time: Vec<JobTimePoint> = jobs_rows
.iter()
.map(|r| JobTimePoint {
label: r.get("label"),
scan: r.get("scan"),
rebuild: r.get("rebuild"),
thumbnail: r.get("thumbnail"),
other: r.get("other"),
})
.collect();
Ok(Json(StatsResponse {
overview,
reading_status,
currently_reading,
recently_read,
reading_over_time,
by_format,
by_language,
by_library,
top_series,
additions_over_time,
jobs_over_time,
metadata,
}))
}

46
apps/api/src/telegram.rs Normal file
View File

@@ -0,0 +1,46 @@
use axum::{extract::State, Json};
use serde::Serialize;
use utoipa::ToSchema;
use crate::{error::ApiError, state::AppState};
#[derive(Serialize, ToSchema)]
pub struct TelegramTestResponse {
pub success: bool,
pub message: String,
}
/// Test Telegram connection by sending a test message
#[utoipa::path(
get,
path = "/telegram/test",
tag = "notifications",
responses(
(status = 200, body = TelegramTestResponse),
(status = 400, description = "Telegram not configured"),
(status = 401, description = "Unauthorized"),
),
security(("Bearer" = []))
)]
pub async fn test_telegram(
State(state): State<AppState>,
) -> Result<Json<TelegramTestResponse>, ApiError> {
let config = notifications::load_telegram_config(&state.pool)
.await
.ok_or_else(|| {
ApiError::bad_request(
"Telegram is not configured or disabled. Set bot_token, chat_id, and enable it.",
)
})?;
match notifications::send_test_message(&config).await {
Ok(()) => Ok(Json(TelegramTestResponse {
success: true,
message: "Test message sent successfully".to_string(),
})),
Err(e) => Ok(Json(TelegramTestResponse {
success: false,
message: format!("Failed to send: {e}"),
})),
}
}

View File

@@ -1,310 +1,12 @@
use std::path::Path;
use std::sync::atomic::{AtomicI32, Ordering};
use std::sync::Arc;
use anyhow::Context;
use axum::{ use axum::{
extract::{Path as AxumPath, State}, extract::State,
http::StatusCode,
Json, Json,
}; };
use futures::stream::{self, StreamExt};
use image::GenericImageView;
use serde::Deserialize; use serde::Deserialize;
use sqlx::Row;
use tracing::{info, warn};
use uuid::Uuid; use uuid::Uuid;
use utoipa::ToSchema; use utoipa::ToSchema;
use crate::{error::ApiError, index_jobs, pages, AppState}; use crate::{error::ApiError, index_jobs, state::AppState};
#[derive(Clone)]
struct ThumbnailConfig {
enabled: bool,
width: u32,
height: u32,
quality: u8,
directory: String,
}
async fn load_thumbnail_concurrency(pool: &sqlx::PgPool) -> usize {
let default_concurrency = 4;
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
value
.get("concurrent_renders")
.and_then(|v| v.as_u64())
.map(|v| v as usize)
.unwrap_or(default_concurrency)
}
_ => default_concurrency,
}
}
async fn load_thumbnail_config(pool: &sqlx::PgPool) -> ThumbnailConfig {
let fallback = ThumbnailConfig {
enabled: true,
width: 300,
height: 400,
quality: 80,
directory: "/data/thumbnails".to_string(),
};
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
ThumbnailConfig {
enabled: value
.get("enabled")
.and_then(|v| v.as_bool())
.unwrap_or(fallback.enabled),
width: value
.get("width")
.and_then(|v| v.as_u64())
.map(|v| v as u32)
.unwrap_or(fallback.width),
height: value
.get("height")
.and_then(|v| v.as_u64())
.map(|v| v as u32)
.unwrap_or(fallback.height),
quality: value
.get("quality")
.and_then(|v| v.as_u64())
.map(|v| v as u8)
.unwrap_or(fallback.quality),
directory: value
.get("directory")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| fallback.directory.clone()),
}
}
_ => fallback,
}
}
fn generate_thumbnail(image_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<Vec<u8>> {
let img = image::load_from_memory(image_bytes).context("failed to load image")?;
let (orig_w, orig_h) = img.dimensions();
let ratio_w = config.width as f32 / orig_w as f32;
let ratio_h = config.height as f32 / orig_h as f32;
let ratio = ratio_w.min(ratio_h);
let new_w = (orig_w as f32 * ratio) as u32;
let new_h = (orig_h as f32 * ratio) as u32;
let resized = img.resize(new_w, new_h, image::imageops::FilterType::Lanczos3);
let rgba = resized.to_rgba8();
let (w, h) = rgba.dimensions();
let rgb_data: Vec<u8> = rgba.pixels().flat_map(|p| [p[0], p[1], p[2]]).collect();
let quality = f32::max(config.quality as f32, 85.0);
let webp_data =
webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h).encode(quality);
Ok(webp_data.to_vec())
}
fn save_thumbnail(book_id: Uuid, thumbnail_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<String> {
let dir = Path::new(&config.directory);
std::fs::create_dir_all(dir)?;
let filename = format!("{}.webp", book_id);
let path = dir.join(&filename);
std::fs::write(&path, thumbnail_bytes)?;
Ok(path.to_string_lossy().to_string())
}
async fn run_checkup(state: AppState, job_id: Uuid) {
let pool = &state.pool;
let row = sqlx::query("SELECT library_id, type FROM index_jobs WHERE id = $1")
.bind(job_id)
.fetch_optional(pool)
.await;
let (library_id, job_type) = match row {
Ok(Some(r)) => (
r.get::<Option<Uuid>, _>("library_id"),
r.get::<String, _>("type"),
),
_ => {
warn!("thumbnails checkup: job {} not found", job_id);
return;
}
};
// Regenerate or full_rebuild: clear existing thumbnails in scope so they get regenerated
if job_type == "thumbnail_regenerate" || job_type == "full_rebuild" {
let config = load_thumbnail_config(pool).await;
if job_type == "full_rebuild" {
// For full_rebuild: delete orphaned thumbnail files (books were deleted, new ones have new UUIDs)
// Get all existing book IDs to keep their thumbnails
let existing_book_ids: std::collections::HashSet<Uuid> = sqlx::query_scalar(
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL)"#,
)
.bind(library_id)
.fetch_all(pool)
.await
.unwrap_or_default()
.into_iter()
.collect();
// Delete thumbnail files that don't correspond to existing books
let thumbnail_dir = Path::new(&config.directory);
if thumbnail_dir.exists() {
let mut deleted_count = 0;
if let Ok(entries) = std::fs::read_dir(thumbnail_dir) {
for entry in entries.flatten() {
if let Some(file_name) = entry.file_name().to_str() {
if file_name.ends_with(".webp") {
if let Some(book_id_str) = file_name.strip_suffix(".webp") {
if let Ok(book_id) = Uuid::parse_str(book_id_str) {
if !existing_book_ids.contains(&book_id) {
if let Err(e) = std::fs::remove_file(entry.path()) {
warn!("Failed to delete orphaned thumbnail {}: {}", entry.path().display(), e);
} else {
deleted_count += 1;
}
}
}
}
}
}
}
}
info!("thumbnails full_rebuild: deleted {} orphaned thumbnail files", deleted_count);
}
} else {
// For regenerate: delete thumbnail files for books with thumbnails
let book_ids_to_clear: Vec<Uuid> = sqlx::query_scalar(
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL) AND thumbnail_path IS NOT NULL"#,
)
.bind(library_id)
.fetch_all(pool)
.await
.unwrap_or_default();
let mut deleted_count = 0;
for book_id in &book_ids_to_clear {
let filename = format!("{}.webp", book_id);
let thumbnail_path = Path::new(&config.directory).join(&filename);
if thumbnail_path.exists() {
if let Err(e) = std::fs::remove_file(&thumbnail_path) {
warn!("Failed to delete thumbnail file {}: {}", thumbnail_path.display(), e);
} else {
deleted_count += 1;
}
}
}
info!("thumbnails regenerate: deleted {} thumbnail files", deleted_count);
}
// Clear thumbnail_path in database
let cleared = sqlx::query(
r#"UPDATE books SET thumbnail_path = NULL WHERE (library_id = $1 OR $1 IS NULL)"#,
)
.bind(library_id)
.execute(pool)
.await;
if let Ok(res) = cleared {
info!("thumbnails {}: cleared {} books in database", job_type, res.rows_affected());
}
}
let book_ids: Vec<Uuid> = sqlx::query_scalar(
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL) AND thumbnail_path IS NULL"#,
)
.bind(library_id)
.fetch_all(pool)
.await
.unwrap_or_default();
let config = load_thumbnail_config(pool).await;
if !config.enabled || book_ids.is_empty() {
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(pool)
.await;
return;
}
let total = book_ids.len() as i32;
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'generating_thumbnails', total_files = $2, processed_files = 0, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.bind(total)
.execute(pool)
.await;
let concurrency = load_thumbnail_concurrency(pool).await;
let processed_count = Arc::new(AtomicI32::new(0));
let pool_clone = pool.clone();
let job_id_clone = job_id;
let config_clone = config.clone();
let state_clone = state.clone();
let total_clone = total;
stream::iter(book_ids)
.for_each_concurrent(concurrency, |book_id| {
let processed_count = processed_count.clone();
let pool = pool_clone.clone();
let job_id = job_id_clone;
let config = config_clone.clone();
let state = state_clone.clone();
let total = total_clone;
async move {
match pages::render_book_page_1(&state, book_id, config.width, config.quality).await {
Ok(page_bytes) => {
match generate_thumbnail(&page_bytes, &config) {
Ok(thumb_bytes) => {
if let Ok(path) = save_thumbnail(book_id, &thumb_bytes, &config) {
if sqlx::query("UPDATE books SET thumbnail_path = $1 WHERE id = $2")
.bind(&path)
.bind(book_id)
.execute(&pool)
.await
.is_ok()
{
let processed = processed_count.fetch_add(1, Ordering::Relaxed) + 1;
let percent = (processed as f64 / total as f64 * 100.0) as i32;
let _ = sqlx::query(
"UPDATE index_jobs SET processed_files = $2, progress_percent = $3 WHERE id = $1",
)
.bind(job_id)
.bind(processed)
.bind(percent)
.execute(&pool)
.await;
}
}
}
Err(e) => warn!("thumbnail generate failed for book {}: {:?}", book_id, e),
}
}
Err(e) => warn!("render page 1 failed for book {}: {:?}", book_id, e),
}
}
})
.await;
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(pool)
.await;
info!("thumbnails checkup finished for job {} ({} books)", job_id, total);
}
#[derive(Deserialize, ToSchema)] #[derive(Deserialize, ToSchema)]
pub struct ThumbnailsRebuildRequest { pub struct ThumbnailsRebuildRequest {
@@ -312,14 +14,14 @@ pub struct ThumbnailsRebuildRequest {
pub library_id: Option<Uuid>, pub library_id: Option<Uuid>,
} }
/// POST /index/thumbnails/rebuild — create a job and generate thumbnails for books that don't have one (optional library scope). /// POST /index/thumbnails/rebuild — create a job to generate thumbnails for books that don't have one.
#[utoipa::path( #[utoipa::path(
post, post,
path = "/index/thumbnails/rebuild", path = "/index/thumbnails/rebuild",
tag = "indexing", tag = "indexing",
request_body = Option<ThumbnailsRebuildRequest>, request_body = Option<ThumbnailsRebuildRequest>,
responses( responses(
(status = 200, body = index_jobs::IndexJobResponse), (status = 200, body = IndexJobResponse),
(status = 401, description = "Unauthorized"), (status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"), (status = 403, description = "Forbidden - Admin scope required"),
), ),
@@ -346,14 +48,14 @@ pub async fn start_thumbnails_rebuild(
Ok(Json(index_jobs::map_row(row))) Ok(Json(index_jobs::map_row(row)))
} }
/// POST /index/thumbnails/regenerate — create a job and regenerate all thumbnails in scope (clears then regenerates). /// POST /index/thumbnails/regenerate — create a job to regenerate all thumbnails (clears then regenerates).
#[utoipa::path( #[utoipa::path(
post, post,
path = "/index/thumbnails/regenerate", path = "/index/thumbnails/regenerate",
tag = "indexing", tag = "indexing",
request_body = Option<ThumbnailsRebuildRequest>, request_body = Option<ThumbnailsRebuildRequest>,
responses( responses(
(status = 200, body = index_jobs::IndexJobResponse), (status = 200, body = IndexJobResponse),
(status = 401, description = "Unauthorized"), (status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"), (status = 403, description = "Forbidden - Admin scope required"),
), ),
@@ -379,13 +81,3 @@ pub async fn start_thumbnails_regenerate(
Ok(Json(index_jobs::map_row(row))) Ok(Json(index_jobs::map_row(row)))
} }
/// POST /index/jobs/:id/thumbnails/checkup — start thumbnail generation for books missing thumbnails (called by indexer at end of build).
pub async fn start_checkup(
State(state): State<AppState>,
AxumPath(job_id): AxumPath<Uuid>,
) -> Result<StatusCode, ApiError> {
let state = state.clone();
tokio::spawn(async move { run_checkup(state, job_id).await });
Ok(StatusCode::ACCEPTED)
}

View File

@@ -8,7 +8,7 @@ use sqlx::Row;
use uuid::Uuid; use uuid::Uuid;
use utoipa::ToSchema; use utoipa::ToSchema;
use crate::{error::ApiError, AppState}; use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)] #[derive(Deserialize, ToSchema)]
pub struct CreateTokenRequest { pub struct CreateTokenRequest {
@@ -170,3 +170,35 @@ pub async fn revoke_token(
Ok(Json(serde_json::json!({"revoked": true, "id": id}))) Ok(Json(serde_json::json!({"revoked": true, "id": id})))
} }
/// Permanently delete a revoked API token
#[utoipa::path(
post,
path = "/admin/tokens/{id}/delete",
tag = "tokens",
params(
("id" = String, Path, description = "Token UUID"),
),
responses(
(status = 200, description = "Token permanently deleted"),
(status = 404, description = "Token not found or not revoked"),
(status = 401, description = "Unauthorized"),
(status = 403, description = "Forbidden - Admin scope required"),
),
security(("Bearer" = []))
)]
pub async fn delete_token(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<Json<serde_json::Value>, ApiError> {
let result = sqlx::query("DELETE FROM api_tokens WHERE id = $1 AND revoked_at IS NOT NULL")
.bind(id)
.execute(&state.pool)
.await?;
if result.rows_affected() == 0 {
return Err(ApiError::not_found("token not found or not revoked"));
}
Ok(Json(serde_json::json!({"deleted": true, "id": id})))
}

View File

@@ -1,4 +1,4 @@
API_BASE_URL=http://localhost:8080 API_BASE_URL=http://localhost:7080
API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token
NEXT_PUBLIC_API_BASE_URL=http://localhost:8080 NEXT_PUBLIC_API_BASE_URL=http://localhost:7080
NEXT_PUBLIC_API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token NEXT_PUBLIC_API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token

66
apps/backoffice/AGENTS.md Normal file
View File

@@ -0,0 +1,66 @@
# apps/backoffice — Interface d'administration (Next.js)
App Next.js 16 avec React 19, Tailwind CSS v4, TypeScript. Port de dev : **7082** (`npm run dev`).
## Structure
```
app/
├── layout.tsx # Layout global (nav sticky glassmorphism, ThemeProvider)
├── page.tsx # Dashboard
├── books/ # Liste et détail des livres
├── libraries/ # Gestion bibliothèques
├── jobs/ # Monitoring jobs
├── tokens/ # Tokens API
├── settings/ # Paramètres
├── components/ # Composants métier
│ ├── ui/ # Composants génériques (Button, Card, Badge, Icon, Input, ProgressBar, StatBox...)
│ ├── BookCard.tsx
│ ├── JobProgress.tsx
│ ├── JobsList.tsx
│ ├── LibraryForm.tsx
│ ├── FolderBrowser.tsx / FolderPicker.tsx
│ └── ...
└── globals.css # Variables CSS, Tailwind base
lib/
└── api.ts # Client API : types DTO + fonctions fetch vers l'API Rust
```
## Client API (lib/api.ts)
Tous les appels vers l'API Rust passent par `lib/api.ts`. Les types DTO sont définis là :
- `LibraryDto`, `IndexJobDto`, `BookDto`, `TokenDto`, `FolderItem`
Ajouter les nouveaux endpoints et types dans ce fichier.
## Composants UI
Les composants génériques sont dans `app/components/ui/`. Utiliser ces composants plutôt que des éléments HTML bruts :
```tsx
import { Button, Card, Badge, Icon, Input, ProgressBar, StatBox } from "@/app/components/ui";
```
## Conventions
- **App Router** : toutes les pages sont des Server Components par défaut. Utiliser `"use client"` seulement pour l'interactivité.
- **Tailwind v4** : config dans `postcss.config.js` + `tailwind.config.js`. Variables CSS dans `globals.css`.
- **Thème** : `ThemeProvider` + `ThemeToggle` pour dark/light mode via `next-themes`.
- **Icônes** : composant `<Icon name="..." size="sm|md|lg" />` dans `ui/Icon.tsx` — pas de librairie externe.
- **Navigation** : routes typées dans `layout.tsx` (`"/" | "/books" | "/libraries" | "/jobs" | "/tokens" | "/settings"`).
## Commandes
```bash
npm install
npm run dev # http://localhost:7082
npm run build
npm run start # Production sur http://localhost:7082
```
## Gotchas
- **Port 7082** : pas le port Next.js par défaut (3000). Défini dans `package.json` scripts (`-p 7082`).
- **API_BASE_URL** : en prod, configuré via env. En dev local, l'API doit tourner sur `http://localhost:7080`.
- **React 19 + Next.js 16** : utiliser les nouvelles APIs (actions serveur, `use()` hook) si disponibles.
- **Pas de gestion d'état global** : fetch direct depuis les Server Components ou `useState`/`useEffect` dans les Client Components.

View File

@@ -12,11 +12,11 @@ RUN npm run build
FROM node:22-alpine AS runner FROM node:22-alpine AS runner
WORKDIR /app WORKDIR /app
ENV NODE_ENV=production ENV NODE_ENV=production
ENV PORT=8082 ENV PORT=7082
ENV HOST=0.0.0.0 ENV HOST=0.0.0.0
RUN apk add --no-cache wget RUN apk add --no-cache wget
COPY --from=builder /app/.next/standalone ./ COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public COPY --from=builder /app/public ./public
EXPOSE 8082 EXPOSE 7082
CMD ["node", "server.js"] CMD ["node", "server.js"]

View File

@@ -0,0 +1,17 @@
import { NextRequest, NextResponse } from "next/server";
import { convertBook } from "@/lib/api";
export async function POST(
_request: NextRequest,
{ params }: { params: Promise<{ bookId: string }> }
) {
const { bookId } = await params;
try {
const data = await convertBook(bookId);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to start conversion";
const status = message.includes("409") ? 409 : 500;
return NextResponse.json({ error: message }, { status });
}
}

View File

@@ -1,35 +1,25 @@
import { NextRequest, NextResponse } from "next/server"; import { NextRequest, NextResponse } from "next/server";
import { config } from "@/lib/api";
export async function GET( export async function GET(
request: NextRequest, request: NextRequest,
{ params }: { params: Promise<{ bookId: string; pageNum: string }> } { params }: { params: Promise<{ bookId: string; pageNum: string }> }
) { ) {
const { bookId, pageNum } = await params; const { bookId, pageNum } = await params;
// Récupérer les query params (format, width, quality)
const { searchParams } = new URL(request.url);
const format = searchParams.get("format") || "webp";
const width = searchParams.get("width") || "";
const quality = searchParams.get("quality") || "";
// Construire l'URL vers l'API backend
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiUrl = new URL(`${apiBaseUrl}/books/${bookId}/pages/${pageNum}`);
apiUrl.searchParams.set("format", format);
if (width) apiUrl.searchParams.set("width", width);
if (quality) apiUrl.searchParams.set("quality", quality);
// Faire la requête à l'API
const token = process.env.API_BOOTSTRAP_TOKEN;
if (!token) {
return new NextResponse("API token not configured", { status: 500 });
}
try { try {
const { baseUrl, token } = config();
const { searchParams } = new URL(request.url);
const format = searchParams.get("format") || "webp";
const width = searchParams.get("width") || "";
const quality = searchParams.get("quality") || "";
const apiUrl = new URL(`${baseUrl}/books/${bookId}/pages/${pageNum}`);
apiUrl.searchParams.set("format", format);
if (width) apiUrl.searchParams.set("width", width);
if (quality) apiUrl.searchParams.set("quality", quality);
const response = await fetch(apiUrl.toString(), { const response = await fetch(apiUrl.toString(), {
headers: { headers: { Authorization: `Bearer ${token}` },
Authorization: `Bearer ${token}`,
},
}); });
if (!response.ok) { if (!response.ok) {
@@ -38,12 +28,9 @@ export async function GET(
}); });
} }
// Récupérer le content-type et les données
const contentType = response.headers.get("content-type") || "image/webp"; const contentType = response.headers.get("content-type") || "image/webp";
const imageBuffer = await response.arrayBuffer();
// Retourner l'image avec le bon content-type return new NextResponse(response.body, {
return new NextResponse(imageBuffer, {
headers: { headers: {
"Content-Type": contentType, "Content-Type": contentType,
"Cache-Control": "public, max-age=300", "Cache-Control": "public, max-age=300",

View File

@@ -0,0 +1,17 @@
import { NextRequest, NextResponse } from "next/server";
import { updateReadingProgress } from "@/lib/api";
export async function PATCH(
request: NextRequest,
{ params }: { params: Promise<{ bookId: string }> }
) {
const { bookId } = await params;
try {
const body = await request.json();
const data = await updateReadingProgress(bookId, body.status, body.current_page ?? undefined);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to update reading progress";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,17 @@
import { NextRequest, NextResponse } from "next/server";
import { updateBook } from "@/lib/api";
export async function PATCH(
request: NextRequest,
{ params }: { params: Promise<{ bookId: string }> }
) {
const { bookId } = await params;
try {
const body = await request.json();
const data = await updateBook(bookId, body);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to update book";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -1,4 +1,5 @@
import { NextRequest, NextResponse } from "next/server"; import { NextRequest, NextResponse } from "next/server";
import { config } from "@/lib/api";
export async function GET( export async function GET(
request: NextRequest, request: NextRequest,
@@ -6,21 +7,27 @@ export async function GET(
) { ) {
const { bookId } = await params; const { bookId } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiUrl = `${apiBaseUrl}/books/${bookId}/thumbnail`;
const token = process.env.API_BOOTSTRAP_TOKEN;
if (!token) {
return new NextResponse("API token not configured", { status: 500 });
}
try { try {
const response = await fetch(apiUrl, { const { baseUrl, token } = config();
headers: { const ifNoneMatch = request.headers.get("if-none-match");
Authorization: `Bearer ${token}`,
}, const fetchHeaders: Record<string, string> = {
Authorization: `Bearer ${token}`,
};
if (ifNoneMatch) {
fetchHeaders["If-None-Match"] = ifNoneMatch;
}
const response = await fetch(`${baseUrl}/books/${bookId}/thumbnail`, {
headers: fetchHeaders,
next: { revalidate: 86400 },
}); });
// Forward 304 Not Modified as-is
if (response.status === 304) {
return new NextResponse(null, { status: 304 });
}
if (!response.ok) { if (!response.ok) {
return new NextResponse(`Failed to fetch thumbnail: ${response.status}`, { return new NextResponse(`Failed to fetch thumbnail: ${response.status}`, {
status: response.status status: response.status
@@ -28,14 +35,17 @@ export async function GET(
} }
const contentType = response.headers.get("content-type") || "image/webp"; const contentType = response.headers.get("content-type") || "image/webp";
const imageBuffer = await response.arrayBuffer(); const etag = response.headers.get("etag");
return new NextResponse(imageBuffer, { const headers: Record<string, string> = {
headers: { "Content-Type": contentType,
"Content-Type": contentType, "Cache-Control": "public, max-age=31536000, immutable",
"Cache-Control": "public, max-age=31536000, immutable", };
}, if (etag) {
}); headers["ETag"] = etag;
}
return new NextResponse(response.body, { headers });
} catch (error) { } catch (error) {
console.error("Error fetching thumbnail:", error); console.error("Error fetching thumbnail:", error);
return new NextResponse("Failed to fetch thumbnail", { status: 500 }); return new NextResponse("Failed to fetch thumbnail", { status: 500 });

View File

@@ -1,39 +1,13 @@
import { NextRequest, NextResponse } from "next/server"; import { NextRequest, NextResponse } from "next/server";
import { listFolders } from "@/lib/api";
export async function GET(request: NextRequest) { export async function GET(request: NextRequest) {
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try { try {
const { searchParams } = new URL(request.url); const { searchParams } = new URL(request.url);
const path = searchParams.get("path"); const path = searchParams.get("path") || undefined;
const data = await listFolders(path);
let apiUrl = `${apiBaseUrl}/folders`;
if (path) {
apiUrl += `?path=${encodeURIComponent(path)}`;
}
const response = await fetch(apiUrl, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
return NextResponse.json(data); return NextResponse.json(data);
} catch (error) { } catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to fetch folders" }, { status: 500 }); return NextResponse.json({ error: "Failed to fetch folders" }, { status: 500 });
} }
} }

View File

@@ -1,36 +1,15 @@
import { NextRequest, NextResponse } from "next/server"; import { NextRequest, NextResponse } from "next/server";
import { cancelJob } from "@/lib/api";
export async function POST( export async function POST(
request: NextRequest, _request: NextRequest,
{ params }: { params: Promise<{ id: string }> } { params }: { params: Promise<{ id: string }> }
) { ) {
const { id } = await params; const { id } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try { try {
const response = await fetch(`${apiBaseUrl}/index/cancel/${id}`, { const data = await cancelJob(id);
method: "POST",
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
return NextResponse.json(data); return NextResponse.json(data);
} catch (error) { } catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to cancel job" }, { status: 500 }); return NextResponse.json({ error: "Failed to cancel job" }, { status: 500 });
} }
} }

View File

@@ -1,35 +1,15 @@
import { NextRequest, NextResponse } from "next/server"; import { NextRequest, NextResponse } from "next/server";
import { apiFetch, IndexJobDto } from "@/lib/api";
export async function GET( export async function GET(
request: NextRequest, _request: NextRequest,
{ params }: { params: Promise<{ id: string }> } { params }: { params: Promise<{ id: string }> }
) { ) {
const { id } = await params; const { id } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try { try {
const response = await fetch(`${apiBaseUrl}/index/jobs/${id}`, { const data = await apiFetch<IndexJobDto>(`/index/jobs/${id}`);
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
return NextResponse.json(data); return NextResponse.json(data);
} catch (error) { } catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to fetch job" }, { status: 500 }); return NextResponse.json({ error: "Failed to fetch job" }, { status: 500 });
} }
} }

View File

@@ -1,19 +1,12 @@
import { NextRequest } from "next/server"; import { NextRequest } from "next/server";
import { config } from "@/lib/api";
export async function GET( export async function GET(
request: NextRequest, request: NextRequest,
{ params }: { params: Promise<{ id: string }> } { params }: { params: Promise<{ id: string }> }
) { ) {
const { id } = await params; const { id } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080"; const { baseUrl, token } = config();
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return new Response(
`data: ${JSON.stringify({ error: "API token not configured" })}\n\n`,
{ status: 500, headers: { "Content-Type": "text/event-stream" } }
);
}
const stream = new ReadableStream({ const stream = new ReadableStream({
async start(controller) { async start(controller) {
@@ -22,18 +15,18 @@ export async function GET(
let lastData: string | null = null; let lastData: string | null = null;
let isActive = true; let isActive = true;
let consecutiveErrors = 0;
const fetchJob = async () => { const fetchJob = async () => {
if (!isActive) return; if (!isActive) return;
try { try {
const response = await fetch(`${apiBaseUrl}/index/jobs/${id}`, { const response = await fetch(`${baseUrl}/index/jobs/${id}`, {
headers: { headers: { Authorization: `Bearer ${token}` },
Authorization: `Bearer ${apiToken}`,
},
}); });
if (response.ok && isActive) { if (response.ok && isActive) {
consecutiveErrors = 0;
const data = await response.json(); const data = await response.json();
const dataStr = JSON.stringify(data); const dataStr = JSON.stringify(data);
@@ -63,7 +56,11 @@ export async function GET(
} }
} catch (error) { } catch (error) {
if (isActive) { if (isActive) {
console.error("SSE fetch error:", error); consecutiveErrors++;
// Only log first failure and every 60th to avoid spam
if (consecutiveErrors === 1 || consecutiveErrors % 60 === 0) {
console.warn(`SSE fetch error (${consecutiveErrors} consecutive):`, error);
}
} }
} }
}; };

View File

@@ -0,0 +1,11 @@
import { NextResponse } from "next/server";
import { apiFetch, IndexJobDto } from "@/lib/api";
export async function GET() {
try {
const data = await apiFetch<IndexJobDto[]>("/index/jobs/active");
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Failed to fetch active jobs" }, { status: 500 });
}
}

View File

@@ -1,31 +0,0 @@
import { NextRequest, NextResponse } from "next/server";
export async function GET(request: NextRequest) {
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try {
const response = await fetch(`${apiBaseUrl}/index/status`, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
return NextResponse.json(data);
} catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to fetch jobs" }, { status: 500 });
}
}

View File

@@ -1,15 +1,8 @@
import { NextRequest } from "next/server"; import { NextRequest } from "next/server";
import { config } from "@/lib/api";
export async function GET(request: NextRequest) { export async function GET(request: NextRequest) {
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080"; const { baseUrl, token } = config();
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return new Response(
`data: ${JSON.stringify({ error: "API token not configured" })}\n\n`,
{ status: 500, headers: { "Content-Type": "text/event-stream" } }
);
}
const stream = new ReadableStream({ const stream = new ReadableStream({
async start(controller) { async start(controller) {
@@ -17,57 +10,63 @@ export async function GET(request: NextRequest) {
let lastData: string | null = null; let lastData: string | null = null;
let isActive = true; let isActive = true;
let consecutiveErrors = 0;
let intervalId: ReturnType<typeof setInterval> | null = null;
const fetchJobs = async () => { const fetchJobs = async () => {
if (!isActive) return; if (!isActive) return;
try { try {
const response = await fetch(`${apiBaseUrl}/index/status`, { const response = await fetch(`${baseUrl}/index/status`, {
headers: { headers: { Authorization: `Bearer ${token}` },
Authorization: `Bearer ${apiToken}`,
},
}); });
if (response.ok && isActive) { if (response.ok && isActive) {
consecutiveErrors = 0;
const data = await response.json(); const data = await response.json();
const dataStr = JSON.stringify(data); const dataStr = JSON.stringify(data);
// Send if data changed // Send only if data changed
if (dataStr !== lastData && isActive) { if (dataStr !== lastData && isActive) {
lastData = dataStr; lastData = dataStr;
try { try {
controller.enqueue( controller.enqueue(
new TextEncoder().encode(`data: ${dataStr}\n\n`) new TextEncoder().encode(`data: ${dataStr}\n\n`)
); );
} catch (err) { } catch {
// Controller closed, ignore
isActive = false; isActive = false;
} }
} }
// Adapt interval: 2s when active jobs exist, 15s when idle
const hasActiveJobs = data.some((j: { status: string }) =>
j.status === "running" || j.status === "pending" || j.status === "extracting_pages" || j.status === "generating_thumbnails"
);
const nextInterval = hasActiveJobs ? 2000 : 15000;
restartInterval(nextInterval);
} }
} catch (error) { } catch (error) {
if (isActive) { if (isActive) {
console.error("SSE fetch error:", error); consecutiveErrors++;
if (consecutiveErrors === 1 || consecutiveErrors % 30 === 0) {
console.warn(`SSE fetch error (${consecutiveErrors} consecutive):`, error);
}
} }
} }
}; };
// Initial fetch const restartInterval = (ms: number) => {
await fetchJobs(); if (intervalId !== null) clearInterval(intervalId);
intervalId = setInterval(fetchJobs, ms);
};
// Poll every 2 seconds // Initial fetch + start polling
const interval = setInterval(async () => { await fetchJobs();
if (!isActive) {
clearInterval(interval);
return;
}
await fetchJobs();
}, 2000);
// Cleanup // Cleanup
request.signal.addEventListener("abort", () => { request.signal.addEventListener("abort", () => {
isActive = false; isActive = false;
clearInterval(interval); if (intervalId !== null) clearInterval(intervalId);
controller.close(); controller.close();
}); });
}, },

View File

@@ -0,0 +1,16 @@
import { NextResponse, NextRequest } from "next/server";
import { getKomgaReport } from "@/lib/api";
export async function GET(
_request: NextRequest,
{ params }: { params: Promise<{ id: string }> },
) {
try {
const { id } = await params;
const data = await getKomgaReport(id);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to fetch report";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,12 @@
import { NextResponse } from "next/server";
import { listKomgaReports } from "@/lib/api";
export async function GET() {
try {
const data = await listKomgaReports();
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to fetch reports";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,16 @@
import { NextResponse, NextRequest } from "next/server";
import { apiFetch } from "@/lib/api";
export async function POST(request: NextRequest) {
try {
const body = await request.json();
const data = await apiFetch("/komga/sync", {
method: "POST",
body: JSON.stringify(body),
});
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to sync with Komga";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,20 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch, LibraryDto } from "@/lib/api";
export async function PATCH(
request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const { id } = await params;
try {
const body = await request.json();
const data = await apiFetch<LibraryDto>(`/libraries/${id}/metadata-provider`, {
method: "PATCH",
body: JSON.stringify(body),
});
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to update metadata provider";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,18 @@
import { NextRequest, NextResponse } from "next/server";
import { updateLibraryMonitoring } from "@/lib/api";
export async function PATCH(
request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const { id } = await params;
try {
const { monitor_enabled, scan_mode, watcher_enabled, metadata_refresh_mode } = await request.json();
const data = await updateLibraryMonitoring(id, monitor_enabled, scan_mode, watcher_enabled, metadata_refresh_mode);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to update monitoring settings";
console.error("[monitoring PATCH]", message);
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,16 @@
import { NextRequest, NextResponse } from "next/server";
import { fetchSeriesMetadata } from "@/lib/api";
export async function GET(
_request: NextRequest,
{ params }: { params: Promise<{ id: string; name: string }> }
) {
const { id, name } = await params;
try {
const data = await fetchSeriesMetadata(id, name);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to fetch series metadata";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,17 @@
import { NextRequest, NextResponse } from "next/server";
import { updateSeries } from "@/lib/api";
export async function PATCH(
request: NextRequest,
{ params }: { params: Promise<{ id: string; name: string }> }
) {
const { id, name } = await params;
try {
const body = await request.json();
const data = await updateSeries(id, name, body);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to update series";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,17 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch } from "@/lib/api";
export async function POST(request: NextRequest) {
try {
const body = await request.json();
const { id, ...rest } = body;
const data = await apiFetch<{ status: string; books_synced: number }>(`/metadata/approve/${id}`, {
method: "POST",
body: JSON.stringify(rest),
});
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to approve metadata";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,17 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch, MetadataBatchReportDto } from "@/lib/api";
export async function GET(request: NextRequest) {
try {
const { searchParams } = new URL(request.url);
const id = searchParams.get("id");
if (!id) {
return NextResponse.json({ error: "id is required" }, { status: 400 });
}
const data = await apiFetch<MetadataBatchReportDto>(`/metadata/batch/${id}/report`);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to fetch report";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,19 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch, MetadataBatchResultDto } from "@/lib/api";
export async function GET(request: NextRequest) {
try {
const { searchParams } = new URL(request.url);
const id = searchParams.get("id");
if (!id) {
return NextResponse.json({ error: "id is required" }, { status: 400 });
}
const status = searchParams.get("status") || "";
const params = status ? `?status=${status}` : "";
const data = await apiFetch<MetadataBatchResultDto[]>(`/metadata/batch/${id}/results${params}`);
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to fetch results";
return NextResponse.json({ error: message }, { status: 500 });
}
}

View File

@@ -0,0 +1,16 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch } from "@/lib/api";
export async function POST(request: NextRequest) {
try {
const body = await request.json();
const data = await apiFetch<{ id: string; status: string }>("/metadata/batch", {
method: "POST",
body: JSON.stringify(body),
});
return NextResponse.json(data);
} catch (error) {
const message = error instanceof Error ? error.message : "Failed to start batch";
return NextResponse.json({ error: message }, { status: 500 });
}
}

Some files were not shown because too many files have changed in this diff Show More