Compare commits

..

10 Commits

Author SHA1 Message Date
473e849dfa feat(backoffice): add page preview carousel on book detail page
Shows 5 pages at a time in a full-width grid with prev/next navigation.
Pages are fetched via the existing proxy route with webp format.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:18:47 +01:00
cfc896e92f feat: two-phase indexation with direct thumbnail generation in indexer
Phase 1 (discovery): walkdir + filename-only metadata, zero archive I/O.
Books are visible immediately in the UI while Phase 2 runs in background.

Phase 2 (analysis): open each archive once via analyze_book() to extract
page_count and first page bytes, then generate WebP thumbnail directly in
the indexer — removing the HTTP roundtrip to the API checkup endpoint.

- Add parse_metadata_fast() (infallible, no archive I/O)
- Add analyze_book() returning (page_count, first_page_bytes) in one pass
- Add looks_like_image() magic bytes check for unrar p stdout validation
- Add lsar fallback in list_cbr_images() for UTF-16BE encoded filenames
- Add directory_mtimes table to skip unchanged dirs on incremental scans
- Add analyzer.rs: generate_thumbnail, analyze_library_books, regenerate_thumbnails
- Remove run_checkup() from API; indexer handles thumbnail jobs directly
- Remove api_base_url/api_bootstrap_token from IndexerConfig and AppState
- Add unar + poppler-utils to indexer Dockerfile
- Fix smoke.sh: wait for job completion, check thumbnail_url field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 22:13:05 +01:00
36af34443e refactor: improve API error handling and response structure
- Refactor error handling across various API endpoints to ensure consistent response formats.
- Enhance response structure to include more informative error messages and status codes.
- Update relevant tests to reflect changes in error handling and response formats.
2026-03-09 21:24:22 +01:00
85cad1a7e7 refactor: streamline API calls and enhance configuration management
- Refactor multiple API routes to utilize a centralized configuration function for base URL and token management, improving code consistency and maintainability.
- Replace direct environment variable access with a unified config function in the `lib/api.ts` file.
- Remove redundant error handling and streamline response handling in various API endpoints.
- Delete unused job-related API routes and settings, simplifying the overall API structure.
2026-03-09 14:16:01 +01:00
0f5094575a docs: add AGENTS.md per module and unify ports to 70XX
- Add CLAUDE.md at root and AGENTS.md in apps/api, apps/indexer,
  apps/backoffice, crates/parsers with module-specific guidelines
- Unify all service ports to 70XX (no more internal/external split):
  API 7080, Indexer 7081, Backoffice 7082
- Update docker-compose.yml, Dockerfiles, config.rs defaults,
  .env.example, backoffice routes, bench.sh, smoke.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 13:57:39 +01:00
131c50b1a1 chore: remove docker-compose configuration
- Delete the docker-compose.yml file, which contained service definitions for postgres, meilisearch, migrate, api, indexer, backoffice, and associated volumes.
- This change may indicate a shift in deployment strategy or service management.
2026-03-08 21:34:28 +01:00
6d4c400017 refactor: update AppState references to use state module
- Change all instances of AppState to reference the new state module across multiple files for consistency.
- Clean up imports in auth, books, index_jobs, libraries, pages, search, settings, thumbnails, and tokens modules.
- Simplify main.rs by removing unused code and organizing middleware and route handlers under the new handlers module.
2026-03-08 21:19:22 +01:00
539dc77d57 feat: enhance thumbnail management with full rebuild functionality
- Extend thumbnail regeneration logic to support full rebuilds, allowing for the deletion of orphaned thumbnails.
- Implement database updates to clear thumbnail paths for books during regeneration and full rebuild processes.
- Improve logging to provide detailed insights on the number of deleted thumbnails and cleared database entries.
- Refactor code for better organization and clarity in handling thumbnail files.
2026-03-08 21:10:34 +01:00
9c7120c3dc feat: enhance library scanning and metadata parsing
- Introduce a structured approach to collect book file information before parsing.
- Implement parallel processing for metadata extraction to improve performance.
- Refactor file handling to utilize a new FileInfo struct for better organization.
- Update database interactions to use collected file information for batch inserts.
- Improve logging for scanning and parsing processes to provide better insights.
2026-03-08 21:07:03 +01:00
b1844a4f01 feat: enhance concurrency settings for rendering and thumbnail generation
- Introduce dynamic loading of concurrent render limits from the database for both page rendering and thumbnail generation.
- Update API to utilize the loaded concurrency settings, defaulting to 8 for page renders and 4 for thumbnails.
- Modify front-end settings page to reflect changes in concurrency limits and provide user guidance on their impact.
- Ensure that changes to limits require a server restart to take effect, with clear messaging in the UI.
2026-03-08 21:03:04 +01:00
94 changed files with 7570 additions and 2085 deletions

View File

@@ -0,0 +1,152 @@
---
name: "OPSX: Apply"
description: Implement tasks from an OpenSpec change (Experimental)
category: Workflow
tags: [workflow, artifacts, experimental]
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue`
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! You can archive this change with `/opsx:archive`.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,157 @@
---
name: "OPSX: Archive"
description: Archive a completed change in the experimental workflow
category: Workflow
tags: [workflow, archive, experimental]
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Prompt user for confirmation to continue
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Prompt user for confirmation to continue
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Spec sync status (synced / sync skipped / no delta specs)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs
All artifacts complete. All tasks complete.
```
**Output On Success (No Delta Specs)**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** No delta specs
All artifacts complete. All tasks complete.
```
**Output On Success With Warnings**
```
## Archive Complete (with warnings)
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** Sync skipped (user chose to skip)
**Warnings:**
- Archived with 2 incomplete artifacts
- Archived with 3 incomplete tasks
- Delta spec sync was skipped (user chose to skip)
Review the archive if this was not intentional.
```
**Output On Error (Archive Exists)**
```
## Archive Failed
**Change:** <change-name>
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
Target archive directory already exists.
**Options:**
1. Rename the existing archive
2. Delete the existing archive if it's a duplicate
3. Wait until a different date to archive
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,173 @@
---
name: "OPSX: Explore"
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
category: Workflow
tags: [workflow, explore, experimental, thinking]
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
- A vague idea: "real-time collaboration"
- A specific problem: "the auth system is getting unwieldy"
- A change name: "add-dark-mode" (to explore in context of that change)
- A comparison: "postgres vs sqlite for this"
- Nothing (just enter explore mode)
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
If the user mentioned a specific change name, read its artifacts for context.
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,106 @@
---
name: "OPSX: Propose"
description: Propose a new change - create it and generate all artifacts in one step
category: Workflow
tags: [workflow, artifacts, experimental]
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
**Steps**
1. **If no input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` to start implementing."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,152 @@
---
name: /opsx-apply
id: opsx-apply
category: Workflow
description: Implement tasks from an OpenSpec change (Experimental)
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue`
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! You can archive this change with `/opsx:archive`.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,157 @@
---
name: /opsx-archive
id: opsx-archive
category: Workflow
description: Archive a completed change in the experimental workflow
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Prompt user for confirmation to continue
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Prompt user for confirmation to continue
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Spec sync status (synced / sync skipped / no delta specs)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs
All artifacts complete. All tasks complete.
```
**Output On Success (No Delta Specs)**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** No delta specs
All artifacts complete. All tasks complete.
```
**Output On Success With Warnings**
```
## Archive Complete (with warnings)
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** Sync skipped (user chose to skip)
**Warnings:**
- Archived with 2 incomplete artifacts
- Archived with 3 incomplete tasks
- Delta spec sync was skipped (user chose to skip)
Review the archive if this was not intentional.
```
**Output On Error (Archive Exists)**
```
## Archive Failed
**Change:** <change-name>
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
Target archive directory already exists.
**Options:**
1. Rename the existing archive
2. Delete the existing archive if it's a duplicate
3. Wait until a different date to archive
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,173 @@
---
name: /opsx-explore
id: opsx-explore
category: Workflow
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
- A vague idea: "real-time collaboration"
- A specific problem: "the auth system is getting unwieldy"
- A change name: "add-dark-mode" (to explore in context of that change)
- A comparison: "postgres vs sqlite for this"
- Nothing (just enter explore mode)
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
If the user mentioned a specific change name, read its artifacts for context.
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,106 @@
---
name: /opsx-propose
id: opsx-propose
category: Workflow
description: Propose a new change - create it and generate all artifacts in one step
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
**Steps**
1. **If no input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` to start implementing."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -21,11 +21,11 @@ API_BOOTSTRAP_TOKEN=change-me-in-production
# =============================================================================
# API Service
API_LISTEN_ADDR=0.0.0.0:8080
API_BASE_URL=http://api:8080
API_LISTEN_ADDR=0.0.0.0:7080
API_BASE_URL=http://api:7080
# Indexer Service
INDEXER_LISTEN_ADDR=0.0.0.0:8081
INDEXER_LISTEN_ADDR=0.0.0.0:7081
INDEXER_SCAN_INTERVAL_SECONDS=5
# Meilisearch Search Engine
@@ -56,8 +56,8 @@ THUMBNAILS_HOST_PATH=../data/thumbnails
# Port Configuration
# =============================================================================
# To change ports, edit docker-compose.yml directly:
# - API: change "7080:8080" to "YOUR_PORT:8080"
# - Indexer: change "7081:8081" to "YOUR_PORT:8081"
# - Backoffice: change "7082:8082" to "YOUR_PORT:8082"
# - API: change "7080:7080" to "YOUR_PORT:7080"
# - Indexer: change "7081:7081" to "YOUR_PORT:7081"
# - Backoffice: change "7082:7082" to "YOUR_PORT:7082"
# - Meilisearch: change "7700:7700" to "YOUR_PORT:7700"
# - PostgreSQL: change "6432:5432" to "YOUR_PORT:5432"

View File

@@ -0,0 +1,149 @@
---
description: Implement tasks from an OpenSpec change (Experimental)
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name (e.g., `/opsx-apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx-continue`
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! You can archive this change with `/opsx-archive`.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,154 @@
---
description: Archive a completed change in the experimental workflow
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name after `/opsx-archive` (e.g., `/opsx-archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Prompt user for confirmation to continue
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Prompt user for confirmation to continue
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Spec sync status (synced / sync skipped / no delta specs)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs
All artifacts complete. All tasks complete.
```
**Output On Success (No Delta Specs)**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** No delta specs
All artifacts complete. All tasks complete.
```
**Output On Success With Warnings**
```
## Archive Complete (with warnings)
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** Sync skipped (user chose to skip)
**Warnings:**
- Archived with 2 incomplete artifacts
- Archived with 3 incomplete tasks
- Delta spec sync was skipped (user chose to skip)
Review the archive if this was not intentional.
```
**Output On Error (Archive Exists)**
```
## Archive Failed
**Change:** <change-name>
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
Target archive directory already exists.
**Options:**
1. Rename the existing archive
2. Delete the existing archive if it's a duplicate
3. Wait until a different date to archive
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,170 @@
---
description: Enter explore mode - think through ideas, investigate problems, clarify requirements
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
**Input**: The argument after `/opsx-explore` is whatever the user wants to think about. Could be:
- A vague idea: "real-time collaboration"
- A specific problem: "the auth system is getting unwieldy"
- A change name: "add-dark-mode" (to explore in context of that change)
- A comparison: "postgres vs sqlite for this"
- Nothing (just enter explore mode)
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
If the user mentioned a specific change name, read its artifacts for context.
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,103 @@
---
description: Propose a new change - create it and generate all artifacts in one step
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx-apply
---
**Input**: The argument after `/opsx-propose` is the change name (kebab-case), OR a description of what the user wants to build.
**Steps**
1. **If no input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx-apply` to start implementing."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read the files listed in `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|--------------|------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
┌───────────┐
│ Session │
└─────┬─────┘
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx-explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.2.0"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx-apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx-apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -73,12 +73,14 @@ sqlx migrate add -r migration_name
### Docker Development
`docker-compose.yml` est à la **racine** du projet (pas dans `infra/`).
```bash
# Start infrastructure only
cd infra && docker compose up -d postgres meilisearch
docker compose up -d postgres meilisearch
# Start full stack
cd infra && docker compose up -d
docker compose up -d
# View logs
docker compose logs -f api
@@ -226,24 +228,21 @@ pub struct BookItem {
```
stripstream-librarian/
├── apps/
│ ├── api/ # REST API (axum)
│ │ └── src/
│ ├── main.rs
│ │ ├── books.rs
│ ├── pages.rs
│ │ └── ...
│ ├── indexer/ # Background indexing service
│ │ └── src/
│ │ └── main.rs
│ └── backoffice/ # Next.js admin UI
│ ├── api/ # REST API (axum) — port 7080
│ │ └── src/ # books.rs, pages.rs, thumbnails.rs, state.rs, auth.rs...
├── indexer/ # Background indexing service — port 7081
│ │ └── src/ # worker.rs, scanner.rs, batch.rs, scheduler.rs, watcher.rs...
└── backoffice/ # Next.js admin UI — port 7082
├── crates/
│ ├── core/ # Shared config
│ ├── core/ # Shared config (env vars)
│ │ └── src/config.rs
│ └── parsers/ # Book parsing (CBZ, CBR, PDF)
├── infra/
── migrations/ # SQL migrations
│ └── docker-compose.yml
└── libraries/ # Book storage (mounted volume)
── migrations/ # SQL migrations (sqlx)
├── data/
│ └── thumbnails/ # Thumbnails générés par l'API
├── libraries/ # Book storage (mounted volume)
└── docker-compose.yml # À la racine (pas dans infra/)
```
### Key Files
@@ -251,8 +250,13 @@ stripstream-librarian/
| File | Purpose |
|------|---------|
| `apps/api/src/books.rs` | Book CRUD endpoints |
| `apps/api/src/pages.rs` | Page rendering & caching |
| `apps/indexer/src/main.rs` | Indexing logic, batch processing |
| `apps/api/src/pages.rs` | Page rendering & caching (LRU + disk) |
| `apps/api/src/thumbnails.rs` | Endpoints pour créer des jobs thumbnail (rebuild/regenerate) |
| `apps/api/src/state.rs` | AppState, Semaphore concurrent_renders |
| `apps/indexer/src/scanner.rs` | Phase 1 discovery : scan rapide sans I/O archive, skip dossiers inchangés |
| `apps/indexer/src/analyzer.rs` | Phase 2 analysis : `analyze_book` + génération thumbnails WebP |
| `apps/indexer/src/batch.rs` | Bulk DB ops via UNNEST |
| `apps/indexer/src/worker.rs` | Job loop, watcher, scheduler orchestration |
| `crates/parsers/src/lib.rs` | Format detection, metadata parsing |
| `crates/core/src/config.rs` | Configuration from environment |
| `infra/migrations/*.sql` | Database schema |
@@ -269,7 +273,7 @@ impl IndexerConfig {
pub fn from_env() -> Result<Self> {
Ok(Self {
listen_addr: std::env::var("INDEXER_LISTEN_ADDR")
.unwrap_or_else(|_| "0.0.0.0:8081".to_string()),
.unwrap_or_else(|_| "0.0.0.0:7081".to_string()),
database_url: std::env::var("DATABASE_URL")
.context("DATABASE_URL is required")?,
// ...
@@ -298,4 +302,6 @@ fn remap_libraries_path(path: &str) -> String {
- **Workspace**: This is a Cargo workspace. Always specify the package when building specific apps.
- **Dependencies**: External crates are defined in workspace `Cargo.toml`, not individual `Cargo.toml`.
- **Database**: PostgreSQL is required. Run migrations before starting services.
- **External Tools**: The indexer relies on `unar` (for CBR) and `pdftoppm` (for PDF) being installed on the system.
- **External Tools**: 4 system tools required — `unrar` (CBR page count), `unar` (CBR extraction), `pdfinfo` (PDF page count), `pdftoppm` (PDF page render). Note: `unrar` and `unar` are distinct tools.
- **Thumbnails**: generated by the **indexer** service (phase 2, `analyzer.rs`). The API only creates jobs in DB — it does not generate thumbnails directly.
- **Sub-AGENTS.md**: module-specific guidelines in `apps/api/`, `apps/indexer/`, `apps/backoffice/`, `crates/parsers/`.

72
CLAUDE.md Normal file
View File

@@ -0,0 +1,72 @@
# Stripstream Librarian
Gestionnaire de bibliothèque de bandes dessinées/ebooks. Workspace Cargo multi-crates avec backoffice Next.js.
## Architecture
| Service | Dossier | Port local |
|---------|---------|------------|
| API REST (axum) | `apps/api/` | 7080 |
| Indexer (background) | `apps/indexer/` | 7081 |
| Backoffice (Next.js) | `apps/backoffice/` | 7082 |
| PostgreSQL | infra | 6432 |
| Meilisearch | infra | 7700 |
Crates partagés : `crates/core` (config env), `crates/parsers` (CBZ/CBR/PDF).
## Commandes
```bash
# Build
cargo build # workspace entier
cargo build -p api # crate spécifique
cargo build --release # version optimisée
# Linting / format
cargo clippy
cargo fmt
# Tests
cargo test
cargo test -p parsers
# Infra (dépendances uniquement) — docker-compose.yml est à la racine
docker compose up -d postgres meilisearch
# Backoffice dev
cd apps/backoffice && npm install && npm run dev # http://localhost:7082
# Migrations
sqlx migrate run # DATABASE_URL doit être défini
```
## Environnement
```bash
cp .env.example .env # puis éditer les valeurs REQUIRED
```
Variables **requises** au démarrage : `DATABASE_URL`, `MEILI_URL`, `MEILI_MASTER_KEY`, `API_BOOTSTRAP_TOKEN`.
## Gotchas
- **Dépendances système** : 4 outils requis — `unrar` (CBR listing), `unar` (CBR extraction), `pdfinfo` (PDF page count), `pdftoppm` (PDF rendu). `unrar``unar`.
- **Port backoffice** : `npm run dev` écoute sur **7082**, pas 3000.
- **LIBRARIES_ROOT_PATH** : les chemins en DB commencent par `/libraries/` ; en dev local, définir cette variable pour remapper vers le dossier réel.
- **Thumbnails** : stockés dans `THUMBNAIL_DIRECTORY` (défaut `/data/thumbnails`), générés par **l'API** (pas l'indexer) — l'indexer déclenche un checkup via `POST /index/jobs/:id/thumbnails/checkup`.
- **Workspace Cargo** : les dépendances externes sont définies dans le `Cargo.toml` racine, pas dans les crates individuels.
- **Migrations** : dossier `infra/migrations/`, géré par sqlx. Toujours migrer avant de démarrer les services.
## Fichiers clés
| Fichier | Rôle |
|---------|------|
| `crates/core/src/config.rs` | Config depuis env (API, Indexer, AdminUI) |
| `crates/parsers/src/lib.rs` | Détection format, extraction métadonnées |
| `apps/api/src/books.rs` | Endpoints CRUD livres |
| `apps/api/src/pages.rs` | Rendu pages + cache LRU |
| `apps/indexer/src/scanner.rs` | Scan filesystem |
| `infra/migrations/*.sql` | Schéma DB |
> Voir `AGENTS.md` pour les conventions de code détaillées (error handling, patterns sqlx, async/tokio).
> Des `AGENTS.md` spécifiques existent dans `apps/api/`, `apps/indexer/`, `apps/backoffice/`, `crates/parsers/`.

3
Cargo.lock generated
View File

@@ -1146,6 +1146,8 @@ dependencies = [
"anyhow",
"axum",
"chrono",
"futures",
"image",
"notify",
"parsers",
"rand 0.8.5",
@@ -1161,6 +1163,7 @@ dependencies = [
"tracing-subscriber",
"uuid",
"walkdir",
"webp",
]
[[package]]

View File

@@ -33,5 +33,6 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
uuid = { version = "1.12", features = ["serde", "v4"] }
walkdir = "2.5"
webp = "0.3"
utoipa = "4.0"
utoipa-swagger-ui = "6.0"

View File

@@ -38,16 +38,16 @@ docker compose up -d
```
This will start:
- PostgreSQL (port 5432)
- PostgreSQL (port 6432)
- Meilisearch (port 7700)
- API service (port 8080)
- Indexer service (port 8081)
- Backoffice web UI (port 8082)
- API service (port 7080)
- Indexer service (port 7081)
- Backoffice web UI (port 7082)
### Accessing the Application
- **Backoffice**: http://localhost:8082
- **API**: http://localhost:8080
- **Backoffice**: http://localhost:7082
- **API**: http://localhost:7080
- **Meilisearch**: http://localhost:7700
### Default Credentials
@@ -113,9 +113,9 @@ The backoffice will be available at http://localhost:3000
| Variable | Description | Default |
|----------|-------------|---------|
| `API_LISTEN_ADDR` | API service bind address | `0.0.0.0:8080` |
| `INDEXER_LISTEN_ADDR` | Indexer service bind address | `0.0.0.0:8081` |
| `BACKOFFICE_PORT` | Backoffice web UI port | `8082` |
| `API_LISTEN_ADDR` | API service bind address | `0.0.0.0:7080` |
| `INDEXER_LISTEN_ADDR` | Indexer service bind address | `0.0.0.0:7081` |
| `BACKOFFICE_PORT` | Backoffice web UI port | `7082` |
| `DATABASE_URL` | PostgreSQL connection string | `postgres://stripstream:stripstream@postgres:5432/stripstream` |
| `MEILI_URL` | Meilisearch connection URL | `http://meilisearch:7700` |
| `MEILI_MASTER_KEY` | Meilisearch master key (required) | - |
@@ -128,7 +128,7 @@ The backoffice will be available at http://localhost:3000
The API is documented with OpenAPI/Swagger. When running locally, access the docs at:
```
http://localhost:8080/api-docs
http://localhost:7080/swagger-ui
```
## Project Structure

73
apps/api/AGENTS.md Normal file
View File

@@ -0,0 +1,73 @@
# apps/api — REST API (axum)
Service HTTP sur le port **7080**. Voir `AGENTS.md` racine pour les conventions globales.
## Structure des fichiers
| Fichier | Rôle |
|---------|------|
| `main.rs` | Routes, initialisation AppState, Semaphore concurrent_renders |
| `state.rs` | `AppState` (pool, caches, métriques), `load_concurrent_renders` |
| `auth.rs` | Middlewares `require_admin` / `require_read`, authentification tokens |
| `error.rs` | `ApiError` avec constructeurs `bad_request`, `not_found`, `internal`, etc. |
| `books.rs` | CRUD livres, thumbnails |
| `pages.rs` | Rendu page + double cache (mémoire LRU + disque) |
| `libraries.rs` | CRUD bibliothèques, déclenchement scans |
| `index_jobs.rs` | Suivi jobs, SSE streaming progression |
| `thumbnails.rs` | Rebuild/regénération thumbnails |
| `tokens.rs` | Gestion tokens API (create/revoke) |
| `settings.rs` | Paramètres applicatifs (stockés en DB, clé `limits`) |
| `openapi.rs` | Doc OpenAPI via utoipa, accessible sur `/swagger-ui` |
## Patterns clés
### Handler type
```rust
async fn my_handler(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<Json<MyDto>, ApiError> {
// ...
}
```
### Erreurs API
```rust
// Constructeurs disponibles dans error.rs
ApiError::bad_request("message")
ApiError::not_found("resource not found")
ApiError::internal("unexpected error")
ApiError::unauthorized("missing token")
ApiError::forbidden("admin required")
// Conversion auto depuis sqlx::Error et std::io::Error
```
### Authentification
- **Bootstrap token** : comparaison directe (`API_BOOTSTRAP_TOKEN`), scope Admin
- **Tokens DB** : format `stl_<prefix>_<secret>`, hash argon2 en DB, scope `admin` ou `read`
- Middleware `require_admin` → routes admin ; `require_read` → routes lecture
### OpenAPI (utoipa)
```rust
#[utoipa::path(get, path = "/books/{id}", ...)]
async fn get_book(...) { }
// Ajouter le handler dans openapi.rs (ApiDoc)
```
### Cache pages (`pages.rs`)
- **Cache mémoire** : LRU 512 entrées (`AppState.page_cache`)
- **Cache disque** : `IMAGE_CACHE_DIR` (défaut `/tmp/stripstream-image-cache`), clé SHA256
- Concurrence limitée par `AppState.page_render_limit` (Semaphore, configurable en DB)
- `spawn_blocking` pour le rendu image (CPU-bound)
### Paramètre concurrent_renders
Stocké en DB : `SELECT value FROM app_settings WHERE key = 'limits'` → JSON `{"concurrent_renders": N}`.
Chargé au démarrage dans `load_concurrent_renders`.
## Gotchas
- **LIBRARIES_ROOT_PATH** : les `abs_path` en DB commencent par `/libraries/`. Appeler `remap_libraries_path()` avant tout accès fichier.
- **Rate limit lecture** : middleware `read_rate_limit` sur les routes read (100 req/5s par défaut).
- **Métriques** : `/metrics` expose `requests_total`, `page_cache_hits`, `page_cache_misses` (atomics dans `AppState.metrics`).
- **Swagger** : accessible sur `/swagger-ui`, spec JSON sur `/openapi.json`.

View File

@@ -31,5 +31,5 @@ uuid.workspace = true
zip = { version = "2.2", default-features = false, features = ["deflate"] }
utoipa.workspace = true
utoipa-swagger-ui = { workspace = true, features = ["axum"] }
webp = "0.3"
webp.workspace = true
walkdir = "2"

View File

@@ -26,5 +26,5 @@ RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8
COPY --from=builder /app/target/release/api /usr/local/bin/api
EXPOSE 8080
EXPOSE 7080
CMD ["/usr/local/bin/api"]

View File

@@ -0,0 +1,42 @@
use axum::{
extract::State,
middleware::Next,
response::{IntoResponse, Response},
};
use std::time::Duration;
use std::sync::atomic::Ordering;
use crate::state::AppState;
pub async fn request_counter(
State(state): State<AppState>,
req: axum::extract::Request,
next: Next,
) -> Response {
state.metrics.requests_total.fetch_add(1, Ordering::Relaxed);
next.run(req).await
}
pub async fn read_rate_limit(
State(state): State<AppState>,
req: axum::extract::Request,
next: Next,
) -> Response {
let mut limiter = state.read_rate_limit.lock().await;
if limiter.window_started_at.elapsed() >= Duration::from_secs(1) {
limiter.window_started_at = std::time::Instant::now();
limiter.requests_in_window = 0;
}
if limiter.requests_in_window >= 120 {
return (
axum::http::StatusCode::TOO_MANY_REQUESTS,
"rate limit exceeded",
)
.into_response();
}
limiter.requests_in_window += 1;
drop(limiter);
next.run(req).await
}

View File

@@ -8,7 +8,7 @@ use axum::{
use chrono::Utc;
use sqlx::Row;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Clone, Debug)]
pub enum Scope {

View File

@@ -5,7 +5,7 @@ use sqlx::Row;
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)]
pub struct ListBooksQuery {

26
apps/api/src/handlers.rs Normal file
View File

@@ -0,0 +1,26 @@
use axum::{extract::State, Json};
use std::sync::atomic::Ordering;
use crate::{error::ApiError, state::AppState};
pub async fn health() -> &'static str {
"ok"
}
pub async fn docs_redirect() -> impl axum::response::IntoResponse {
axum::response::Redirect::to("/swagger-ui/")
}
pub async fn ready(State(state): State<AppState>) -> Result<Json<serde_json::Value>, ApiError> {
sqlx::query("SELECT 1").execute(&state.pool).await?;
Ok(Json(serde_json::json!({"status": "ready"})))
}
pub async fn metrics(State(state): State<AppState>) -> String {
format!(
"requests_total {}\npage_cache_hits {}\npage_cache_misses {}\n",
state.metrics.requests_total.load(Ordering::Relaxed),
state.metrics.page_cache_hits.load(Ordering::Relaxed),
state.metrics.page_cache_misses.load(Ordering::Relaxed),
)
}

View File

@@ -8,7 +8,7 @@ use tokio_stream::Stream;
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)]
pub struct RebuildRequest {
@@ -247,7 +247,7 @@ pub async fn list_folders(
}
let mut folders = Vec::new();
let depth = if params.get("path").is_some() {
let depth = if params.contains_key("path") {
canonical_target.strip_prefix(&canonical_base)
.map(|p| p.components().count())
.unwrap_or(0)

View File

@@ -6,7 +6,7 @@ use sqlx::Row;
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Serialize, ToSchema)]
pub struct LibraryResponse {

View File

@@ -1,70 +1,36 @@
mod auth;
mod books;
mod error;
mod handlers;
mod index_jobs;
mod libraries;
mod api_middleware;
mod openapi;
mod pages;
mod search;
mod settings;
mod state;
mod thumbnails;
mod tokens;
use std::{
num::NonZeroUsize,
sync::{
atomic::{AtomicU64, Ordering},
Arc,
},
time::{Duration, Instant},
};
use std::sync::Arc;
use std::time::Instant;
use axum::{
middleware,
response::IntoResponse,
routing::{delete, get},
Json, Router,
Router,
};
use utoipa::OpenApi;
use utoipa_swagger_ui::SwaggerUi;
use lru::LruCache;
use std::num::NonZeroUsize;
use stripstream_core::config::ApiConfig;
use sqlx::postgres::PgPoolOptions;
use tokio::sync::{Mutex, Semaphore};
use tracing::info;
#[derive(Clone)]
struct AppState {
pool: sqlx::PgPool,
bootstrap_token: Arc<str>,
meili_url: Arc<str>,
meili_master_key: Arc<str>,
page_cache: Arc<Mutex<LruCache<String, Arc<Vec<u8>>>>>,
page_render_limit: Arc<Semaphore>,
metrics: Arc<Metrics>,
read_rate_limit: Arc<Mutex<ReadRateLimit>>,
}
struct Metrics {
requests_total: AtomicU64,
page_cache_hits: AtomicU64,
page_cache_misses: AtomicU64,
}
struct ReadRateLimit {
window_started_at: Instant,
requests_in_window: u32,
}
impl Metrics {
fn new() -> Self {
Self {
requests_total: AtomicU64::new(0),
page_cache_hits: AtomicU64::new(0),
page_cache_misses: AtomicU64::new(0),
}
}
}
use crate::state::{load_concurrent_renders, AppState, Metrics, ReadRateLimit};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
@@ -80,13 +46,17 @@ async fn main() -> anyhow::Result<()> {
.connect(&config.database_url)
.await?;
// Load concurrent_renders from settings, default to 8
let concurrent_renders = load_concurrent_renders(&pool).await;
info!("Using concurrent_renders limit: {}", concurrent_renders);
let state = AppState {
pool,
bootstrap_token: Arc::from(config.api_bootstrap_token),
meili_url: Arc::from(config.meili_url),
meili_master_key: Arc::from(config.meili_master_key),
page_cache: Arc::new(Mutex::new(LruCache::new(NonZeroUsize::new(512).expect("non-zero")))),
page_render_limit: Arc::new(Semaphore::new(8)),
page_render_limit: Arc::new(Semaphore::new(concurrent_renders)),
metrics: Arc::new(Metrics::new()),
read_rate_limit: Arc::new(Mutex::new(ReadRateLimit {
window_started_at: Instant::now(),
@@ -106,7 +76,6 @@ async fn main() -> anyhow::Result<()> {
.route("/index/jobs/active", get(index_jobs::get_active_jobs))
.route("/index/jobs/:id", get(index_jobs::get_job_details))
.route("/index/jobs/:id/stream", get(index_jobs::stream_job_progress))
.route("/index/jobs/:id/thumbnails/checkup", axum::routing::post(thumbnails::start_checkup))
.route("/index/jobs/:id/errors", get(index_jobs::get_job_errors))
.route("/index/cancel/:id", axum::routing::post(index_jobs::cancel_job))
.route("/folders", get(index_jobs::list_folders))
@@ -125,21 +94,21 @@ async fn main() -> anyhow::Result<()> {
.route("/books/:id/pages/:n", get(pages::get_page))
.route("/libraries/:library_id/series", get(books::list_series))
.route("/search", get(search::search_books))
.route_layer(middleware::from_fn_with_state(state.clone(), read_rate_limit))
.route_layer(middleware::from_fn_with_state(state.clone(), api_middleware::read_rate_limit))
.route_layer(middleware::from_fn_with_state(
state.clone(),
auth::require_read,
));
let app = Router::new()
.route("/health", get(health))
.route("/ready", get(ready))
.route("/metrics", get(metrics))
.route("/docs", get(docs_redirect))
.route("/health", get(handlers::health))
.route("/ready", get(handlers::ready))
.route("/metrics", get(handlers::metrics))
.route("/docs", get(handlers::docs_redirect))
.merge(SwaggerUi::new("/swagger-ui").url("/openapi.json", openapi::ApiDoc::openapi()))
.merge(admin_routes)
.merge(read_routes)
.layer(middleware::from_fn_with_state(state.clone(), request_counter))
.layer(middleware::from_fn_with_state(state.clone(), api_middleware::request_counter))
.with_state(state);
let listener = tokio::net::TcpListener::bind(&config.listen_addr).await?;
@@ -148,57 +117,3 @@ async fn main() -> anyhow::Result<()> {
Ok(())
}
async fn health() -> &'static str {
"ok"
}
async fn docs_redirect() -> impl axum::response::IntoResponse {
axum::response::Redirect::to("/swagger-ui/")
}
async fn ready(axum::extract::State(state): axum::extract::State<AppState>) -> Result<Json<serde_json::Value>, error::ApiError> {
sqlx::query("SELECT 1").execute(&state.pool).await?;
Ok(Json(serde_json::json!({"status": "ready"})))
}
async fn metrics(axum::extract::State(state): axum::extract::State<AppState>) -> String {
format!(
"requests_total {}\npage_cache_hits {}\npage_cache_misses {}\n",
state.metrics.requests_total.load(Ordering::Relaxed),
state.metrics.page_cache_hits.load(Ordering::Relaxed),
state.metrics.page_cache_misses.load(Ordering::Relaxed),
)
}
async fn request_counter(
axum::extract::State(state): axum::extract::State<AppState>,
req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
state.metrics.requests_total.fetch_add(1, Ordering::Relaxed);
next.run(req).await
}
async fn read_rate_limit(
axum::extract::State(state): axum::extract::State<AppState>,
req: axum::extract::Request,
next: axum::middleware::Next,
) -> axum::response::Response {
let mut limiter = state.read_rate_limit.lock().await;
if limiter.window_started_at.elapsed() >= Duration::from_secs(1) {
limiter.window_started_at = Instant::now();
limiter.requests_in_window = 0;
}
if limiter.requests_in_window >= 120 {
return (
axum::http::StatusCode::TOO_MANY_REQUESTS,
"rate limit exceeded",
)
.into_response();
}
limiter.requests_in_window += 1;
drop(limiter);
next.run(req).await
}

View File

@@ -20,7 +20,7 @@ use tracing::{debug, error, info, instrument, warn};
use uuid::Uuid;
use walkdir::WalkDir;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
fn remap_libraries_path(path: &str) -> String {
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
@@ -550,12 +550,12 @@ fn transcode_image(input: &[u8], out_format: &OutputFormat, quality: u8, width:
}
fn format_matches(source: &ImageFormat, target: &OutputFormat) -> bool {
match (source, target) {
(ImageFormat::Jpeg, OutputFormat::Jpeg) => true,
(ImageFormat::Png, OutputFormat::Png) => true,
(ImageFormat::WebP, OutputFormat::Webp) => true,
_ => false,
}
matches!(
(source, target),
(ImageFormat::Jpeg, OutputFormat::Jpeg)
| (ImageFormat::Png, OutputFormat::Png)
| (ImageFormat::WebP, OutputFormat::Webp)
)
}
fn is_image_name(name: &str) -> bool {

View File

@@ -2,7 +2,7 @@ use axum::{extract::{Query, State}, Json};
use serde::{Deserialize, Serialize};
use utoipa::ToSchema;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)]
pub struct SearchQuery {

View File

@@ -7,7 +7,7 @@ use serde::{Deserialize, Serialize};
use serde_json::Value;
use sqlx::Row;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct UpdateSettingRequest {

61
apps/api/src/state.rs Normal file
View File

@@ -0,0 +1,61 @@
use std::sync::{
atomic::AtomicU64,
Arc,
};
use std::time::Instant;
use lru::LruCache;
use sqlx::{Pool, Postgres, Row};
use tokio::sync::{Mutex, Semaphore};
#[derive(Clone)]
pub struct AppState {
pub pool: sqlx::PgPool,
pub bootstrap_token: Arc<str>,
pub meili_url: Arc<str>,
pub meili_master_key: Arc<str>,
pub page_cache: Arc<Mutex<LruCache<String, Arc<Vec<u8>>>>>,
pub page_render_limit: Arc<Semaphore>,
pub metrics: Arc<Metrics>,
pub read_rate_limit: Arc<Mutex<ReadRateLimit>>,
}
pub struct Metrics {
pub requests_total: AtomicU64,
pub page_cache_hits: AtomicU64,
pub page_cache_misses: AtomicU64,
}
pub struct ReadRateLimit {
pub window_started_at: Instant,
pub requests_in_window: u32,
}
impl Metrics {
pub fn new() -> Self {
Self {
requests_total: AtomicU64::new(0),
page_cache_hits: AtomicU64::new(0),
page_cache_misses: AtomicU64::new(0),
}
}
}
pub async fn load_concurrent_renders(pool: &Pool<Postgres>) -> usize {
let default_concurrency = 8;
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
value
.get("concurrent_renders")
.and_then(|v: &serde_json::Value| v.as_u64())
.map(|v| v as usize)
.unwrap_or(default_concurrency)
}
_ => default_concurrency,
}
}

View File

@@ -1,203 +1,12 @@
use std::path::Path;
use anyhow::Context;
use axum::{
extract::{Path as AxumPath, State},
http::StatusCode,
extract::State,
Json,
};
use image::GenericImageView;
use serde::Deserialize;
use sqlx::Row;
use tracing::{info, warn};
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, index_jobs, pages, AppState};
#[derive(Clone)]
struct ThumbnailConfig {
enabled: bool,
width: u32,
height: u32,
quality: u8,
directory: String,
}
async fn load_thumbnail_config(pool: &sqlx::PgPool) -> ThumbnailConfig {
let fallback = ThumbnailConfig {
enabled: true,
width: 300,
height: 400,
quality: 80,
directory: "/data/thumbnails".to_string(),
};
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
ThumbnailConfig {
enabled: value
.get("enabled")
.and_then(|v| v.as_bool())
.unwrap_or(fallback.enabled),
width: value
.get("width")
.and_then(|v| v.as_u64())
.map(|v| v as u32)
.unwrap_or(fallback.width),
height: value
.get("height")
.and_then(|v| v.as_u64())
.map(|v| v as u32)
.unwrap_or(fallback.height),
quality: value
.get("quality")
.and_then(|v| v.as_u64())
.map(|v| v as u8)
.unwrap_or(fallback.quality),
directory: value
.get("directory")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| fallback.directory.clone()),
}
}
_ => fallback,
}
}
fn generate_thumbnail(image_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<Vec<u8>> {
let img = image::load_from_memory(image_bytes).context("failed to load image")?;
let (orig_w, orig_h) = img.dimensions();
let ratio_w = config.width as f32 / orig_w as f32;
let ratio_h = config.height as f32 / orig_h as f32;
let ratio = ratio_w.min(ratio_h);
let new_w = (orig_w as f32 * ratio) as u32;
let new_h = (orig_h as f32 * ratio) as u32;
let resized = img.resize(new_w, new_h, image::imageops::FilterType::Lanczos3);
let rgba = resized.to_rgba8();
let (w, h) = rgba.dimensions();
let rgb_data: Vec<u8> = rgba.pixels().flat_map(|p| [p[0], p[1], p[2]]).collect();
let quality = f32::max(config.quality as f32, 85.0);
let webp_data =
webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h).encode(quality);
Ok(webp_data.to_vec())
}
fn save_thumbnail(book_id: Uuid, thumbnail_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<String> {
let dir = Path::new(&config.directory);
std::fs::create_dir_all(dir)?;
let filename = format!("{}.webp", book_id);
let path = dir.join(&filename);
std::fs::write(&path, thumbnail_bytes)?;
Ok(path.to_string_lossy().to_string())
}
async fn run_checkup(state: AppState, job_id: Uuid) {
let pool = &state.pool;
let row = sqlx::query("SELECT library_id, type FROM index_jobs WHERE id = $1")
.bind(job_id)
.fetch_optional(pool)
.await;
let (library_id, job_type) = match row {
Ok(Some(r)) => (
r.get::<Option<Uuid>, _>("library_id"),
r.get::<String, _>("type"),
),
_ => {
warn!("thumbnails checkup: job {} not found", job_id);
return;
}
};
// Regenerate: clear existing thumbnails in scope so they get regenerated
if job_type == "thumbnail_regenerate" {
let cleared = sqlx::query(
r#"UPDATE books SET thumbnail_path = NULL WHERE (library_id = $1 OR $1 IS NULL)"#,
)
.bind(library_id)
.execute(pool)
.await;
if let Ok(res) = cleared {
info!("thumbnails regenerate: cleared {} books", res.rows_affected());
}
}
let book_ids: Vec<Uuid> = sqlx::query_scalar(
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL) AND thumbnail_path IS NULL"#,
)
.bind(library_id)
.fetch_all(pool)
.await
.unwrap_or_default();
let config = load_thumbnail_config(pool).await;
if !config.enabled || book_ids.is_empty() {
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(pool)
.await;
return;
}
let total = book_ids.len() as i32;
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'generating_thumbnails', total_files = $2, processed_files = 0, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.bind(total)
.execute(pool)
.await;
for (i, &book_id) in book_ids.iter().enumerate() {
match pages::render_book_page_1(&state, book_id, config.width, config.quality).await {
Ok(page_bytes) => {
match generate_thumbnail(&page_bytes, &config) {
Ok(thumb_bytes) => {
if let Ok(path) = save_thumbnail(book_id, &thumb_bytes, &config) {
if sqlx::query("UPDATE books SET thumbnail_path = $1 WHERE id = $2")
.bind(&path)
.bind(book_id)
.execute(pool)
.await
.is_ok()
{
let processed = (i + 1) as i32;
let percent = ((i + 1) as f64 / total as f64 * 100.0) as i32;
let _ = sqlx::query(
"UPDATE index_jobs SET processed_files = $2, progress_percent = $3 WHERE id = $1",
)
.bind(job_id)
.bind(processed)
.bind(percent)
.execute(pool)
.await;
}
}
}
Err(e) => warn!("thumbnail generate failed for book {}: {:?}", book_id, e),
}
}
Err(e) => warn!("render page 1 failed for book {}: {:?}", book_id, e),
}
}
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(pool)
.await;
info!("thumbnails checkup finished for job {} ({} books)", job_id, total);
}
use crate::{error::ApiError, index_jobs, state::AppState};
#[derive(Deserialize, ToSchema)]
pub struct ThumbnailsRebuildRequest {
@@ -205,7 +14,7 @@ pub struct ThumbnailsRebuildRequest {
pub library_id: Option<Uuid>,
}
/// POST /index/thumbnails/rebuild — create a job and generate thumbnails for books that don't have one (optional library scope).
/// POST /index/thumbnails/rebuild — create a job to generate thumbnails for books that don't have one.
#[utoipa::path(
post,
path = "/index/thumbnails/rebuild",
@@ -239,7 +48,7 @@ pub async fn start_thumbnails_rebuild(
Ok(Json(index_jobs::map_row(row)))
}
/// POST /index/thumbnails/regenerate — create a job and regenerate all thumbnails in scope (clears then regenerates).
/// POST /index/thumbnails/regenerate — create a job to regenerate all thumbnails (clears then regenerates).
#[utoipa::path(
post,
path = "/index/thumbnails/regenerate",
@@ -272,13 +81,3 @@ pub async fn start_thumbnails_regenerate(
Ok(Json(index_jobs::map_row(row)))
}
/// POST /index/jobs/:id/thumbnails/checkup — start thumbnail generation for books missing thumbnails (called by indexer at end of build).
pub async fn start_checkup(
State(state): State<AppState>,
AxumPath(job_id): AxumPath<Uuid>,
) -> Result<StatusCode, ApiError> {
let state = state.clone();
tokio::spawn(async move { run_checkup(state, job_id).await });
Ok(StatusCode::ACCEPTED)
}

View File

@@ -8,7 +8,7 @@ use sqlx::Row;
use uuid::Uuid;
use utoipa::ToSchema;
use crate::{error::ApiError, AppState};
use crate::{error::ApiError, state::AppState};
#[derive(Deserialize, ToSchema)]
pub struct CreateTokenRequest {

View File

@@ -1,4 +1,4 @@
API_BASE_URL=http://localhost:8080
API_BASE_URL=http://localhost:7080
API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token
NEXT_PUBLIC_API_BASE_URL=http://localhost:8080
NEXT_PUBLIC_API_BASE_URL=http://localhost:7080
NEXT_PUBLIC_API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token

66
apps/backoffice/AGENTS.md Normal file
View File

@@ -0,0 +1,66 @@
# apps/backoffice — Interface d'administration (Next.js)
App Next.js 16 avec React 19, Tailwind CSS v4, TypeScript. Port de dev : **7082** (`npm run dev`).
## Structure
```
app/
├── layout.tsx # Layout global (nav sticky glassmorphism, ThemeProvider)
├── page.tsx # Dashboard
├── books/ # Liste et détail des livres
├── libraries/ # Gestion bibliothèques
├── jobs/ # Monitoring jobs
├── tokens/ # Tokens API
├── settings/ # Paramètres
├── components/ # Composants métier
│ ├── ui/ # Composants génériques (Button, Card, Badge, Icon, Input, ProgressBar, StatBox...)
│ ├── BookCard.tsx
│ ├── JobProgress.tsx
│ ├── JobsList.tsx
│ ├── LibraryForm.tsx
│ ├── FolderBrowser.tsx / FolderPicker.tsx
│ └── ...
└── globals.css # Variables CSS, Tailwind base
lib/
└── api.ts # Client API : types DTO + fonctions fetch vers l'API Rust
```
## Client API (lib/api.ts)
Tous les appels vers l'API Rust passent par `lib/api.ts`. Les types DTO sont définis là :
- `LibraryDto`, `IndexJobDto`, `BookDto`, `TokenDto`, `FolderItem`
Ajouter les nouveaux endpoints et types dans ce fichier.
## Composants UI
Les composants génériques sont dans `app/components/ui/`. Utiliser ces composants plutôt que des éléments HTML bruts :
```tsx
import { Button, Card, Badge, Icon, Input, ProgressBar, StatBox } from "@/app/components/ui";
```
## Conventions
- **App Router** : toutes les pages sont des Server Components par défaut. Utiliser `"use client"` seulement pour l'interactivité.
- **Tailwind v4** : config dans `postcss.config.js` + `tailwind.config.js`. Variables CSS dans `globals.css`.
- **Thème** : `ThemeProvider` + `ThemeToggle` pour dark/light mode via `next-themes`.
- **Icônes** : composant `<Icon name="..." size="sm|md|lg" />` dans `ui/Icon.tsx` — pas de librairie externe.
- **Navigation** : routes typées dans `layout.tsx` (`"/" | "/books" | "/libraries" | "/jobs" | "/tokens" | "/settings"`).
## Commandes
```bash
npm install
npm run dev # http://localhost:7082
npm run build
npm run start # Production sur http://localhost:7082
```
## Gotchas
- **Port 7082** : pas le port Next.js par défaut (3000). Défini dans `package.json` scripts (`-p 7082`).
- **API_BASE_URL** : en prod, configuré via env. En dev local, l'API doit tourner sur `http://localhost:7080`.
- **React 19 + Next.js 16** : utiliser les nouvelles APIs (actions serveur, `use()` hook) si disponibles.
- **Pas de gestion d'état global** : fetch direct depuis les Server Components ou `useState`/`useEffect` dans les Client Components.

View File

@@ -12,11 +12,11 @@ RUN npm run build
FROM node:22-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
ENV PORT=8082
ENV PORT=7082
ENV HOST=0.0.0.0
RUN apk add --no-cache wget
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 8082
EXPOSE 7082
CMD ["node", "server.js"]

View File

@@ -1,35 +1,25 @@
import { NextRequest, NextResponse } from "next/server";
import { config } from "@/lib/api";
export async function GET(
request: NextRequest,
{ params }: { params: Promise<{ bookId: string; pageNum: string }> }
) {
const { bookId, pageNum } = await params;
// Récupérer les query params (format, width, quality)
const { searchParams } = new URL(request.url);
const format = searchParams.get("format") || "webp";
const width = searchParams.get("width") || "";
const quality = searchParams.get("quality") || "";
// Construire l'URL vers l'API backend
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiUrl = new URL(`${apiBaseUrl}/books/${bookId}/pages/${pageNum}`);
apiUrl.searchParams.set("format", format);
if (width) apiUrl.searchParams.set("width", width);
if (quality) apiUrl.searchParams.set("quality", quality);
// Faire la requête à l'API
const token = process.env.API_BOOTSTRAP_TOKEN;
if (!token) {
return new NextResponse("API token not configured", { status: 500 });
}
try {
const { baseUrl, token } = config();
const { searchParams } = new URL(request.url);
const format = searchParams.get("format") || "webp";
const width = searchParams.get("width") || "";
const quality = searchParams.get("quality") || "";
const apiUrl = new URL(`${baseUrl}/books/${bookId}/pages/${pageNum}`);
apiUrl.searchParams.set("format", format);
if (width) apiUrl.searchParams.set("width", width);
if (quality) apiUrl.searchParams.set("quality", quality);
const response = await fetch(apiUrl.toString(), {
headers: {
Authorization: `Bearer ${token}`,
},
headers: { Authorization: `Bearer ${token}` },
});
if (!response.ok) {

View File

@@ -1,4 +1,5 @@
import { NextRequest, NextResponse } from "next/server";
import { config } from "@/lib/api";
export async function GET(
request: NextRequest,
@@ -6,19 +7,10 @@ export async function GET(
) {
const { bookId } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiUrl = `${apiBaseUrl}/books/${bookId}/thumbnail`;
const token = process.env.API_BOOTSTRAP_TOKEN;
if (!token) {
return new NextResponse("API token not configured", { status: 500 });
}
try {
const response = await fetch(apiUrl, {
headers: {
Authorization: `Bearer ${token}`,
},
const { baseUrl, token } = config();
const response = await fetch(`${baseUrl}/books/${bookId}/thumbnail`, {
headers: { Authorization: `Bearer ${token}` },
});
if (!response.ok) {

View File

@@ -1,39 +1,13 @@
import { NextRequest, NextResponse } from "next/server";
import { listFolders } from "@/lib/api";
export async function GET(request: NextRequest) {
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try {
const { searchParams } = new URL(request.url);
const path = searchParams.get("path");
let apiUrl = `${apiBaseUrl}/folders`;
if (path) {
apiUrl += `?path=${encodeURIComponent(path)}`;
}
const response = await fetch(apiUrl, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
const path = searchParams.get("path") || undefined;
const data = await listFolders(path);
return NextResponse.json(data);
} catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to fetch folders" }, { status: 500 });
}
}

View File

@@ -1,36 +1,15 @@
import { NextRequest, NextResponse } from "next/server";
import { cancelJob } from "@/lib/api";
export async function POST(
request: NextRequest,
_request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const { id } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try {
const response = await fetch(`${apiBaseUrl}/index/cancel/${id}`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
const data = await cancelJob(id);
return NextResponse.json(data);
} catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to cancel job" }, { status: 500 });
}
}

View File

@@ -1,35 +1,15 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch, IndexJobDto } from "@/lib/api";
export async function GET(
request: NextRequest,
_request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const { id } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try {
const response = await fetch(`${apiBaseUrl}/index/jobs/${id}`, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
const data = await apiFetch<IndexJobDto>(`/index/jobs/${id}`);
return NextResponse.json(data);
} catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to fetch job" }, { status: 500 });
}
}

View File

@@ -1,19 +1,12 @@
import { NextRequest } from "next/server";
import { config } from "@/lib/api";
export async function GET(
request: NextRequest,
{ params }: { params: Promise<{ id: string }> }
) {
const { id } = await params;
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return new Response(
`data: ${JSON.stringify({ error: "API token not configured" })}\n\n`,
{ status: 500, headers: { "Content-Type": "text/event-stream" } }
);
}
const { baseUrl, token } = config();
const stream = new ReadableStream({
async start(controller) {
@@ -27,10 +20,8 @@ export async function GET(
if (!isActive) return;
try {
const response = await fetch(`${apiBaseUrl}/index/jobs/${id}`, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
const response = await fetch(`${baseUrl}/index/jobs/${id}`, {
headers: { Authorization: `Bearer ${token}` },
});
if (response.ok && isActive) {

View File

@@ -0,0 +1,11 @@
import { NextResponse } from "next/server";
import { apiFetch, IndexJobDto } from "@/lib/api";
export async function GET() {
try {
const data = await apiFetch<IndexJobDto[]>("/index/jobs/active");
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Failed to fetch active jobs" }, { status: 500 });
}
}

View File

@@ -1,31 +0,0 @@
import { NextRequest, NextResponse } from "next/server";
export async function GET(request: NextRequest) {
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
}
try {
const response = await fetch(`${apiBaseUrl}/index/status`, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
});
if (!response.ok) {
return NextResponse.json(
{ error: `API error: ${response.status}` },
{ status: response.status }
);
}
const data = await response.json();
return NextResponse.json(data);
} catch (error) {
console.error("Proxy error:", error);
return NextResponse.json({ error: "Failed to fetch jobs" }, { status: 500 });
}
}

View File

@@ -1,15 +1,8 @@
import { NextRequest } from "next/server";
import { config } from "@/lib/api";
export async function GET(request: NextRequest) {
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
if (!apiToken) {
return new Response(
`data: ${JSON.stringify({ error: "API token not configured" })}\n\n`,
{ status: 500, headers: { "Content-Type": "text/event-stream" } }
);
}
const { baseUrl, token } = config();
const stream = new ReadableStream({
async start(controller) {
@@ -22,10 +15,8 @@ export async function GET(request: NextRequest) {
if (!isActive) return;
try {
const response = await fetch(`${apiBaseUrl}/index/status`, {
headers: {
Authorization: `Bearer ${apiToken}`,
},
const response = await fetch(`${baseUrl}/index/status`, {
headers: { Authorization: `Bearer ${token}` },
});
if (response.ok && isActive) {

View File

@@ -1,29 +1,16 @@
import { NextRequest, NextResponse } from "next/server";
import { apiFetch, updateSetting } from "@/lib/api";
export async function GET(
request: NextRequest,
_request: NextRequest,
{ params }: { params: Promise<{ key: string }> }
) {
const { key } = await params;
try {
const { key } = await params;
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
const token = process.env.API_BOOTSTRAP_TOKEN;
const response = await fetch(`${baseUrl}/settings/${key}`, {
headers: {
Authorization: `Bearer ${token}`,
},
cache: "no-store"
});
if (!response.ok) {
return NextResponse.json({ error: "Failed to fetch setting" }, { status: response.status });
}
const data = await response.json();
const data = await apiFetch<unknown>(`/settings/${key}`);
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
return NextResponse.json({ error: "Failed to fetch setting" }, { status: 500 });
}
}
@@ -31,29 +18,12 @@ export async function POST(
request: NextRequest,
{ params }: { params: Promise<{ key: string }> }
) {
const { key } = await params;
try {
const { key } = await params;
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
const token = process.env.API_BOOTSTRAP_TOKEN;
const body = await request.json();
const response = await fetch(`${baseUrl}/settings/${key}`, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
cache: "no-store"
});
if (!response.ok) {
return NextResponse.json({ error: "Failed to update setting" }, { status: response.status });
}
const data = await response.json();
const { value } = await request.json();
const data = await updateSetting(key, value);
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
return NextResponse.json({ error: "Failed to update setting" }, { status: 500 });
}
}

View File

@@ -1,25 +1,11 @@
import { NextRequest, NextResponse } from "next/server";
import { NextResponse } from "next/server";
import { clearCache } from "@/lib/api";
export async function POST(request: NextRequest) {
export async function POST() {
try {
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
const token = process.env.API_BOOTSTRAP_TOKEN;
const response = await fetch(`${baseUrl}/settings/cache/clear`, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
},
cache: "no-store"
});
if (!response.ok) {
return NextResponse.json({ error: "Failed to clear cache" }, { status: response.status });
}
const data = await response.json();
const data = await clearCache();
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
return NextResponse.json({ error: "Failed to clear cache" }, { status: 500 });
}
}

View File

@@ -1,24 +1,11 @@
import { NextRequest, NextResponse } from "next/server";
import { NextResponse } from "next/server";
import { getCacheStats } from "@/lib/api";
export async function GET(request: NextRequest) {
export async function GET() {
try {
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
const token = process.env.API_BOOTSTRAP_TOKEN;
const response = await fetch(`${baseUrl}/settings/cache/stats`, {
headers: {
Authorization: `Bearer ${token}`,
},
cache: "no-store"
});
if (!response.ok) {
return NextResponse.json({ error: "Failed to fetch cache stats" }, { status: response.status });
}
const data = await response.json();
const data = await getCacheStats();
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
return NextResponse.json({ error: "Failed to fetch cache stats" }, { status: 500 });
}
}

View File

@@ -1,24 +0,0 @@
import { NextRequest, NextResponse } from "next/server";
export async function GET(request: NextRequest) {
try {
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
const token = process.env.API_BOOTSTRAP_TOKEN;
const response = await fetch(`${baseUrl}/settings`, {
headers: {
Authorization: `Bearer ${token}`,
},
cache: "no-store"
});
if (!response.ok) {
return NextResponse.json({ error: "Failed to fetch settings" }, { status: response.status });
}
const data = await response.json();
return NextResponse.json(data);
} catch (error) {
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
}
}

View File

@@ -1,4 +1,5 @@
import { fetchLibraries, getBookCoverUrl, BookDto, apiFetch } from "../../../lib/api";
import { BookPreview } from "../../components/BookPreview";
import Image from "next/image";
import Link from "next/link";
import { notFound } from "next/navigation";
@@ -157,6 +158,12 @@ export default async function BookDetailPage({
</div>
</div>
</div>
{book.page_count && book.page_count > 0 && (
<div className="mt-8">
<BookPreview bookId={book.id} pageCount={book.page_count} />
</div>
)}
</>
);
}

View File

@@ -0,0 +1,60 @@
"use client";
import { useState } from "react";
import Image from "next/image";
const PAGE_SIZE = 5;
export function BookPreview({ bookId, pageCount }: { bookId: string; pageCount: number }) {
const [offset, setOffset] = useState(0);
const pages = Array.from({ length: PAGE_SIZE }, (_, i) => offset + i + 1).filter(
(p) => p <= pageCount
);
return (
<div className="bg-card rounded-xl border border-border p-6">
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold text-foreground">
Preview
<span className="ml-2 text-sm font-normal text-muted-foreground">
pages {offset + 1}{Math.min(offset + PAGE_SIZE, pageCount)} / {pageCount}
</span>
</h2>
<div className="flex gap-2">
<button
onClick={() => setOffset((o) => Math.max(0, o - PAGE_SIZE))}
disabled={offset === 0}
className="px-3 py-1.5 text-sm rounded-lg border border-border bg-muted/50 text-foreground hover:bg-muted disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
>
Prev
</button>
<button
onClick={() => setOffset((o) => Math.min(o + PAGE_SIZE, pageCount - 1))}
disabled={offset + PAGE_SIZE >= pageCount}
className="px-3 py-1.5 text-sm rounded-lg border border-border bg-muted/50 text-foreground hover:bg-muted disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
>
Next
</button>
</div>
</div>
<div className="grid grid-cols-5 gap-3">
{pages.map((pageNum) => (
<div key={pageNum} className="flex flex-col items-center gap-1.5">
<div className="relative w-full aspect-[2/3] bg-muted rounded-lg overflow-hidden border border-border">
<Image
src={`/api/books/${bookId}/pages/${pageNum}?format=webp&width=600&quality=80`}
alt={`Page ${pageNum}`}
fill
className="object-contain"
unoptimized
/>
</div>
<span className="text-xs text-muted-foreground">{pageNum}</span>
</div>
))}
</div>
</div>
);
}

View File

@@ -247,7 +247,7 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
<Icon name="performance" size="md" />
Performance Limits
</CardTitle>
<CardDescription>Configure API performance and rate limiting</CardDescription>
<CardDescription>Configure API performance, rate limiting, and thumbnail generation concurrency</CardDescription>
</CardHeader>
<CardContent>
<div className="space-y-4">
@@ -266,6 +266,9 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
}}
onBlur={() => handleUpdateSetting("limits", settings.limits)}
/>
<p className="text-xs text-muted-foreground mt-1">
Maximum number of page renders and thumbnail generations running in parallel
</p>
</FormField>
<FormField className="flex-1">
<label className="text-sm font-medium text-muted-foreground mb-1 block">Timeout (seconds)</label>
@@ -299,7 +302,7 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
</FormField>
</FormRow>
<p className="text-sm text-muted-foreground">
Note: Changes to limits require a server restart to take effect.
Note: Changes to limits require a server restart to take effect. The "Concurrent Renders" setting controls both page rendering and thumbnail generation parallelism.
</p>
</div>
</CardContent>
@@ -424,7 +427,7 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
</div>
<p className="text-sm text-muted-foreground">
Note: Thumbnail settings are used during indexing. Existing thumbnails will not be regenerated automatically.
Note: Thumbnail settings are used during indexing. Existing thumbnails will not be regenerated automatically. The concurrency for thumbnail generation is controlled by the "Concurrent Renders" setting in Performance Limits above.
</p>
</div>
</CardContent>

View File

@@ -89,8 +89,8 @@ export type SeriesDto = {
first_book_id: string;
};
function config() {
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
export function config() {
const baseUrl = process.env.API_BASE_URL || "http://api:7080";
const token = process.env.API_BOOTSTRAP_TOKEN;
if (!token) {
throw new Error("API_BOOTSTRAP_TOKEN is required for backoffice");

View File

@@ -21,7 +21,10 @@
{
"name": "next"
}
]
],
"paths": {
"@/*": ["./*"]
}
},
"include": [
"next-env.d.ts",

104
apps/indexer/AGENTS.md Normal file
View File

@@ -0,0 +1,104 @@
# apps/indexer — Service d'indexation
Service background sur le port **7081**. Voir `AGENTS.md` racine pour les conventions globales.
## Structure des fichiers
| Fichier | Rôle |
|---------|------|
| `main.rs` | Point d'entrée, initialisation, lancement du worker |
| `lib.rs` | `AppState` (pool, meili_url, meili_master_key) |
| `worker.rs` | Boucle principale : claim job → process → cleanup stale |
| `job.rs` | `claim_next_job`, `process_job`, `fail_job`, `cleanup_stale_jobs` |
| `scanner.rs` | Phase 1 discovery : WalkDir + `parse_metadata_fast` (zéro I/O archive), skip dossiers inchangés via mtime, batching DB |
| `analyzer.rs` | Phase 2 analysis : ouvre chaque archive une fois (`analyze_book`), génère page_count + thumbnail WebP |
| `batch.rs` | `flush_all_batches` avec UNNEST, structures `BookInsert/Update/FileInsert/Update/ErrorInsert` |
| `scheduler.rs` | Auto-scan : vérifie toutes les 60s les bibliothèques à monitorer |
| `watcher.rs` | File watcher temps réel |
| `meili.rs` | Indexation/sync Meilisearch |
| `api.rs` | Endpoints HTTP de l'indexer (/health, /ready) |
| `utils.rs` | `remap_libraries_path`, `unmap_libraries_path`, `compute_fingerprint`, `kind_from_format` |
## Cycle de vie d'un job
```
claim_next_job (UPDATE ... RETURNING, status pending→running)
└─ process_job
├─ Phase 1 : scanner::scan_library_discovery
│ ├─ WalkDir + parse_metadata_fast (zéro I/O archive)
│ ├─ skip dossiers via directory_mtimes (table DB)
│ └─ INSERT books (page_count=NULL) → livres visibles immédiatement
├─ meili::sync_meili
├─ analyzer::cleanup_orphaned_thumbnails (full_rebuild uniquement)
└─ Phase 2 : analyzer::analyze_library_books
├─ SELECT books WHERE page_count IS NULL
├─ parsers::analyze_book → (page_count, first_page_bytes)
├─ generate_thumbnail (WebP, Lanczos3)
└─ UPDATE books SET page_count, thumbnail_path
Jobs spéciaux :
thumbnail_rebuild → analyze_library_books(thumbnail_only=true)
thumbnail_regenerate → regenerate_thumbnails (clear + re-analyze)
```
- Annulation : `is_job_cancelled` vérifié toutes les 10 fichiers ou 1s — retourne `Err("Job cancelled")`
- Jobs stale (running au redémarrage) → nettoyés par `cleanup_stale_jobs` au boot
## Pattern batch (batch.rs)
Toutes les opérations DB massives passent par `flush_all_batches` avec UNNEST :
```rust
// Accumuler dans des Vec<BookInsert>, Vec<FileInsert>, etc.
books_to_insert.push(BookInsert { ... });
// Flush quand plein ou en fin de scan
if books_to_insert.len() >= BATCH_SIZE {
flush_all_batches(&pool, &mut books_update, &mut files_update,
&mut books_insert, &mut files_insert, &mut errors_insert).await?;
}
```
Toutes les opérations du flush sont dans une seule transaction.
## Scan filesystem — architecture 2 phases
### Phase 1 : Discovery (`scanner.rs`)
Pipeline allégé — **zéro ouverture d'archive** :
1. Charger `directory_mtimes` depuis la DB
2. WalkDir : pour chaque dossier, comparer mtime filesystem vs mtime stocké → skip si inchangé
3. Pour chaque fichier : `parse_metadata_fast` (title/series/volume depuis filename uniquement)
4. INSERT/UPDATE avec `page_count = NULL` — les livres sont visibles immédiatement
5. Upsert `directory_mtimes` en fin de scan
Fingerprint = SHA256(taille + mtime + filename) pour détecter les changements sans relire le fichier.
### Phase 2 : Analysis (`analyzer.rs`)
Traitement progressif en background :
- Query `WHERE page_count IS NULL` (ou `thumbnail_path IS NULL` pour thumbnail jobs)
- Concurrence bornée (`futures::stream::for_each_concurrent`, défaut 4)
- Par livre : `parsers::analyze_book(path, format)``(page_count, first_page_bytes)`
- Génération thumbnail : resize Lanczos3 + encode WebP
- UPDATE `books SET page_count, thumbnail_path`
- Config lue depuis `app_settings` (clés `'thumbnail'` et `'limits'`)
## Path remapping
```rust
// abs_path en DB = chemin conteneur (/libraries/...)
// Sur l'hôte : LIBRARIES_ROOT_PATH remplace /libraries
utils::remap_libraries_path(&abs_path) // DB → filesystem local
utils::unmap_libraries_path(&local_path) // filesystem local → DB
```
## Gotchas
- **Thumbnails** : générés **directement par l'indexer** (phase 2, `analyzer.rs`). L'API ne gère plus la génération — elle crée juste les jobs en DB.
- **page_count = NULL** : après la phase discovery, tous les nouveaux livres ont `page_count = NULL`. La phase analysis les remplit progressivement. Ne pas confondre avec une erreur.
- **directory_mtimes** : table DB qui stocke le mtime de chaque dossier scanné. Vidée au full_rebuild, mise à jour après chaque scan. Permet de skipper les dossiers inchangés en scan incrémental.
- **full_rebuild** : supprime toutes les données puis re-insère. Ignore les fingerprints et les directory_mtimes.
- **Annulation** : vérifier `is_job_cancelled` régulièrement pour respecter les annulations utilisateur.
- **Watcher + scheduler** : tournent en tâches tokio séparées dans `worker.rs`, en parallèle de la boucle principale.
- **spawn_blocking** : l'ouverture d'archive (`analyze_book`) et la génération de thumbnail sont des opérations bloquantes — toujours les wrapper dans `tokio::task::spawn_blocking`.

View File

@@ -4,10 +4,14 @@ version.workspace = true
edition.workspace = true
license.workspace = true
[lib]
[dependencies]
anyhow.workspace = true
axum.workspace = true
chrono.workspace = true
futures = "0.3"
image.workspace = true
notify = "6.1"
parsers = { path = "../../crates/parsers" }
rand.workspace = true
@@ -23,3 +27,4 @@ tracing.workspace = true
tracing-subscriber.workspace = true
uuid.workspace = true
walkdir.workspace = true
webp.workspace = true

View File

@@ -21,7 +21,11 @@ RUN --mount=type=cache,target=/sccache \
cargo build --release -p indexer
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates wget unrar-free && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates wget \
unrar-free unar \
poppler-utils \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/indexer /usr/local/bin/indexer
EXPOSE 8081
EXPOSE 7081
CMD ["/usr/local/bin/indexer"]

View File

@@ -0,0 +1,442 @@
use anyhow::Result;
use futures::stream::{self, StreamExt};
use image::GenericImageView;
use parsers::{analyze_book, BookFormat};
use sqlx::Row;
use std::path::Path;
use std::sync::atomic::{AtomicI32, Ordering};
use std::sync::Arc;
use tracing::{info, warn};
use uuid::Uuid;
use crate::{utils, AppState};
#[derive(Clone)]
struct ThumbnailConfig {
enabled: bool,
width: u32,
height: u32,
quality: u8,
directory: String,
}
async fn load_thumbnail_config(pool: &sqlx::PgPool) -> ThumbnailConfig {
let fallback = ThumbnailConfig {
enabled: true,
width: 300,
height: 400,
quality: 80,
directory: "/data/thumbnails".to_string(),
};
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
ThumbnailConfig {
enabled: value
.get("enabled")
.and_then(|v| v.as_bool())
.unwrap_or(fallback.enabled),
width: value
.get("width")
.and_then(|v| v.as_u64())
.map(|v| v as u32)
.unwrap_or(fallback.width),
height: value
.get("height")
.and_then(|v| v.as_u64())
.map(|v| v as u32)
.unwrap_or(fallback.height),
quality: value
.get("quality")
.and_then(|v| v.as_u64())
.map(|v| v as u8)
.unwrap_or(fallback.quality),
directory: value
.get("directory")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| fallback.directory.clone()),
}
}
_ => fallback,
}
}
async fn load_thumbnail_concurrency(pool: &sqlx::PgPool) -> usize {
let default_concurrency = 4;
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
.fetch_optional(pool)
.await;
match row {
Ok(Some(row)) => {
let value: serde_json::Value = row.get("value");
value
.get("concurrent_renders")
.and_then(|v| v.as_u64())
.map(|v| v as usize)
.unwrap_or(default_concurrency)
}
_ => default_concurrency,
}
}
fn generate_thumbnail(image_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<Vec<u8>> {
let img = image::load_from_memory(image_bytes)
.map_err(|e| anyhow::anyhow!("failed to load image: {}", e))?;
let (orig_w, orig_h) = img.dimensions();
let ratio_w = config.width as f32 / orig_w as f32;
let ratio_h = config.height as f32 / orig_h as f32;
let ratio = ratio_w.min(ratio_h);
let new_w = (orig_w as f32 * ratio) as u32;
let new_h = (orig_h as f32 * ratio) as u32;
let resized = img.resize(new_w, new_h, image::imageops::FilterType::Lanczos3);
let rgba = resized.to_rgba8();
let (w, h) = rgba.dimensions();
let rgb_data: Vec<u8> = rgba.pixels().flat_map(|p| [p[0], p[1], p[2]]).collect();
let quality = f32::max(config.quality as f32, 85.0);
let webp_data = webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h).encode(quality);
Ok(webp_data.to_vec())
}
fn save_thumbnail(
book_id: Uuid,
thumbnail_bytes: &[u8],
config: &ThumbnailConfig,
) -> anyhow::Result<String> {
let dir = Path::new(&config.directory);
std::fs::create_dir_all(dir)?;
let filename = format!("{}.webp", book_id);
let path = dir.join(&filename);
std::fs::write(&path, thumbnail_bytes)?;
Ok(path.to_string_lossy().to_string())
}
fn book_format_from_str(s: &str) -> Option<BookFormat> {
match s {
"cbz" => Some(BookFormat::Cbz),
"cbr" => Some(BookFormat::Cbr),
"pdf" => Some(BookFormat::Pdf),
_ => None,
}
}
/// Phase 2 — Analysis: open each unanalyzed archive once, extract page_count + thumbnail.
/// `thumbnail_only` = true: only process books missing thumbnail (page_count may already be set).
/// `thumbnail_only` = false: process books missing page_count.
pub async fn analyze_library_books(
state: &AppState,
job_id: Uuid,
library_id: Option<Uuid>,
thumbnail_only: bool,
) -> Result<()> {
let config = load_thumbnail_config(&state.pool).await;
if !config.enabled {
info!("[ANALYZER] Thumbnails disabled, skipping analysis phase");
return Ok(());
}
let concurrency = load_thumbnail_concurrency(&state.pool).await;
// Query books that need analysis
let query_filter = if thumbnail_only {
"b.thumbnail_path IS NULL"
} else {
"b.page_count IS NULL"
};
let sql = format!(
r#"
SELECT b.id AS book_id, bf.abs_path, bf.format
FROM books b
JOIN book_files bf ON bf.book_id = b.id
WHERE (b.library_id = $1 OR $1 IS NULL)
AND {}
"#,
query_filter
);
let rows = sqlx::query(&sql)
.bind(library_id)
.fetch_all(&state.pool)
.await?;
if rows.is_empty() {
info!("[ANALYZER] No books to analyze");
return Ok(());
}
let total = rows.len() as i32;
info!(
"[ANALYZER] Analyzing {} books (thumbnail_only={}, concurrency={})",
total, thumbnail_only, concurrency
);
// Update job status
let _ = sqlx::query(
"UPDATE index_jobs SET status = 'generating_thumbnails', total_files = $2, processed_files = 0, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.bind(total)
.execute(&state.pool)
.await;
let processed_count = Arc::new(AtomicI32::new(0));
struct BookTask {
book_id: Uuid,
abs_path: String,
format: String,
}
let tasks: Vec<BookTask> = rows
.into_iter()
.map(|row| BookTask {
book_id: row.get("book_id"),
abs_path: row.get("abs_path"),
format: row.get("format"),
})
.collect();
stream::iter(tasks)
.for_each_concurrent(concurrency, |task| {
let processed_count = processed_count.clone();
let pool = state.pool.clone();
let config = config.clone();
async move {
let local_path = utils::remap_libraries_path(&task.abs_path);
let path = Path::new(&local_path);
let format = match book_format_from_str(&task.format) {
Some(f) => f,
None => {
warn!("[ANALYZER] Unknown format '{}' for book {}", task.format, task.book_id);
return;
}
};
// Run blocking archive I/O on a thread pool
let book_id = task.book_id;
let path_owned = path.to_path_buf();
let analyze_result = tokio::task::spawn_blocking(move || {
analyze_book(&path_owned, format)
})
.await;
let (page_count, image_bytes) = match analyze_result {
Ok(Ok(result)) => result,
Ok(Err(e)) => {
warn!("[ANALYZER] analyze_book failed for book {}: {}", book_id, e);
// Mark parse_status = error in book_files
let _ = sqlx::query(
"UPDATE book_files SET parse_status = 'error', parse_error_opt = $2 WHERE book_id = $1",
)
.bind(book_id)
.bind(e.to_string())
.execute(&pool)
.await;
return;
}
Err(e) => {
warn!("[ANALYZER] spawn_blocking error for book {}: {}", book_id, e);
return;
}
};
// Generate thumbnail
let thumb_result = tokio::task::spawn_blocking({
let config = config.clone();
move || generate_thumbnail(&image_bytes, &config)
})
.await;
let thumb_bytes = match thumb_result {
Ok(Ok(b)) => b,
Ok(Err(e)) => {
warn!("[ANALYZER] thumbnail generation failed for book {}: {}", book_id, e);
// Still update page_count even if thumbnail fails
let _ = sqlx::query(
"UPDATE books SET page_count = $1 WHERE id = $2",
)
.bind(page_count)
.bind(book_id)
.execute(&pool)
.await;
return;
}
Err(e) => {
warn!("[ANALYZER] spawn_blocking thumbnail error for book {}: {}", book_id, e);
return;
}
};
// Save thumbnail file
let save_result = {
let config = config.clone();
tokio::task::spawn_blocking(move || save_thumbnail(book_id, &thumb_bytes, &config))
.await
};
let thumb_path = match save_result {
Ok(Ok(p)) => p,
Ok(Err(e)) => {
warn!("[ANALYZER] save_thumbnail failed for book {}: {}", book_id, e);
let _ = sqlx::query("UPDATE books SET page_count = $1 WHERE id = $2")
.bind(page_count)
.bind(book_id)
.execute(&pool)
.await;
return;
}
Err(e) => {
warn!("[ANALYZER] spawn_blocking save error for book {}: {}", book_id, e);
return;
}
};
// Update DB
if let Err(e) = sqlx::query(
"UPDATE books SET page_count = $1, thumbnail_path = $2 WHERE id = $3",
)
.bind(page_count)
.bind(&thumb_path)
.bind(book_id)
.execute(&pool)
.await
{
warn!("[ANALYZER] DB update failed for book {}: {}", book_id, e);
return;
}
let processed = processed_count.fetch_add(1, Ordering::Relaxed) + 1;
let percent = (processed as f64 / total as f64 * 100.0) as i32;
let _ = sqlx::query(
"UPDATE index_jobs SET processed_files = $2, progress_percent = $3 WHERE id = $1",
)
.bind(job_id)
.bind(processed)
.bind(percent)
.execute(&pool)
.await;
}
})
.await;
let final_count = processed_count.load(Ordering::Relaxed);
info!(
"[ANALYZER] Analysis complete: {}/{} books processed",
final_count, total
);
Ok(())
}
/// Clear thumbnail files and DB references for books in scope, then re-analyze.
pub async fn regenerate_thumbnails(
state: &AppState,
job_id: Uuid,
library_id: Option<Uuid>,
) -> Result<()> {
let config = load_thumbnail_config(&state.pool).await;
// Delete thumbnail files for all books in scope
let book_ids_to_clear: Vec<Uuid> = sqlx::query_scalar(
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL) AND thumbnail_path IS NOT NULL"#,
)
.bind(library_id)
.fetch_all(&state.pool)
.await
.unwrap_or_default();
let mut deleted_count = 0usize;
for book_id in &book_ids_to_clear {
let filename = format!("{}.webp", book_id);
let thumbnail_path = Path::new(&config.directory).join(&filename);
if thumbnail_path.exists() {
if let Err(e) = std::fs::remove_file(&thumbnail_path) {
warn!(
"[ANALYZER] Failed to delete thumbnail {}: {}",
thumbnail_path.display(),
e
);
} else {
deleted_count += 1;
}
}
}
info!(
"[ANALYZER] Deleted {} thumbnail files for regeneration",
deleted_count
);
// Clear thumbnail_path in DB
sqlx::query(
r#"UPDATE books SET thumbnail_path = NULL WHERE (library_id = $1 OR $1 IS NULL)"#,
)
.bind(library_id)
.execute(&state.pool)
.await?;
// Re-analyze all books (now thumbnail_path IS NULL for all)
analyze_library_books(state, job_id, library_id, true).await
}
/// Delete orphaned thumbnail files (books deleted in full_rebuild get new UUIDs).
pub async fn cleanup_orphaned_thumbnails(
state: &AppState,
library_id: Option<Uuid>,
) -> Result<()> {
let config = load_thumbnail_config(&state.pool).await;
let existing_book_ids: std::collections::HashSet<Uuid> = sqlx::query_scalar(
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL)"#,
)
.bind(library_id)
.fetch_all(&state.pool)
.await
.unwrap_or_default()
.into_iter()
.collect();
let thumbnail_dir = Path::new(&config.directory);
if !thumbnail_dir.exists() {
return Ok(());
}
let mut deleted_count = 0usize;
if let Ok(entries) = std::fs::read_dir(thumbnail_dir) {
for entry in entries.flatten() {
if let Some(file_name) = entry.file_name().to_str() {
if file_name.ends_with(".webp") {
if let Some(book_id_str) = file_name.strip_suffix(".webp") {
if let Ok(book_id) = Uuid::parse_str(book_id_str) {
if !existing_book_ids.contains(&book_id) {
if let Err(e) = std::fs::remove_file(entry.path()) {
warn!(
"Failed to delete orphaned thumbnail {}: {}",
entry.path().display(),
e
);
} else {
deleted_count += 1;
}
}
}
}
}
}
}
}
info!(
"[ANALYZER] Deleted {} orphaned thumbnail files",
deleted_count
);
Ok(())
}

16
apps/indexer/src/api.rs Normal file
View File

@@ -0,0 +1,16 @@
use axum::{extract::State, http::StatusCode, Json};
use serde_json;
use crate::AppState;
pub async fn health() -> &'static str {
"ok"
}
pub async fn ready(State(state): State<AppState>) -> Result<Json<serde_json::Value>, StatusCode> {
sqlx::query("SELECT 1")
.execute(&state.pool)
.await
.map_err(|_| StatusCode::SERVICE_UNAVAILABLE)?;
Ok(Json(serde_json::json!({"status": "ready"})))
}

233
apps/indexer/src/batch.rs Normal file
View File

@@ -0,0 +1,233 @@
use anyhow::Result;
use chrono::{DateTime, Utc};
use sqlx::PgPool;
use uuid::Uuid;
// Batched update data structures
pub struct BookUpdate {
pub book_id: Uuid,
pub title: String,
pub kind: String,
pub series: Option<String>,
pub volume: Option<i32>,
pub page_count: Option<i32>,
}
pub struct FileUpdate {
pub file_id: Uuid,
pub format: String,
pub size_bytes: i64,
pub mtime: DateTime<Utc>,
pub fingerprint: String,
}
pub struct BookInsert {
pub book_id: Uuid,
pub library_id: Uuid,
pub kind: String,
pub title: String,
pub series: Option<String>,
pub volume: Option<i32>,
pub page_count: Option<i32>,
pub thumbnail_path: Option<String>,
}
pub struct FileInsert {
pub file_id: Uuid,
pub book_id: Uuid,
pub format: String,
pub abs_path: String,
pub size_bytes: i64,
pub mtime: DateTime<Utc>,
pub fingerprint: String,
pub parse_status: String,
pub parse_error: Option<String>,
}
pub struct ErrorInsert {
pub job_id: Uuid,
pub file_path: String,
pub error_message: String,
}
pub async fn flush_all_batches(
pool: &PgPool,
books_update: &mut Vec<BookUpdate>,
files_update: &mut Vec<FileUpdate>,
books_insert: &mut Vec<BookInsert>,
files_insert: &mut Vec<FileInsert>,
errors_insert: &mut Vec<ErrorInsert>,
) -> Result<()> {
if books_update.is_empty() && files_update.is_empty() && books_insert.is_empty() && files_insert.is_empty() && errors_insert.is_empty() {
return Ok(());
}
let start = std::time::Instant::now();
let mut tx = pool.begin().await?;
// Batch update books using UNNEST
if !books_update.is_empty() {
let book_ids: Vec<Uuid> = books_update.iter().map(|b| b.book_id).collect();
let titles: Vec<String> = books_update.iter().map(|b| b.title.clone()).collect();
let kinds: Vec<String> = books_update.iter().map(|b| b.kind.clone()).collect();
let series: Vec<Option<String>> = books_update.iter().map(|b| b.series.clone()).collect();
let volumes: Vec<Option<i32>> = books_update.iter().map(|b| b.volume).collect();
let page_counts: Vec<Option<i32>> = books_update.iter().map(|b| b.page_count).collect();
sqlx::query(
r#"
UPDATE books SET
title = data.title,
kind = data.kind,
series = data.series,
volume = data.volume,
page_count = data.page_count,
updated_at = NOW()
FROM (
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::text[], $4::text[], $5::int[], $6::int[])
AS t(book_id, title, kind, series, volume, page_count)
) AS data
WHERE books.id = data.book_id
"#
)
.bind(&book_ids)
.bind(&titles)
.bind(&kinds)
.bind(&series)
.bind(&volumes)
.bind(&page_counts)
.execute(&mut *tx)
.await?;
books_update.clear();
}
// Batch update files using UNNEST
if !files_update.is_empty() {
let file_ids: Vec<Uuid> = files_update.iter().map(|f| f.file_id).collect();
let formats: Vec<String> = files_update.iter().map(|f| f.format.clone()).collect();
let sizes: Vec<i64> = files_update.iter().map(|f| f.size_bytes).collect();
let mtimes: Vec<DateTime<Utc>> = files_update.iter().map(|f| f.mtime).collect();
let fingerprints: Vec<String> = files_update.iter().map(|f| f.fingerprint.clone()).collect();
sqlx::query(
r#"
UPDATE book_files SET
format = data.format,
size_bytes = data.size,
mtime = data.mtime,
fingerprint = data.fp,
parse_status = 'ok',
parse_error_opt = NULL,
updated_at = NOW()
FROM (
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::bigint[], $4::timestamptz[], $5::text[])
AS t(file_id, format, size, mtime, fp)
) AS data
WHERE book_files.id = data.file_id
"#
)
.bind(&file_ids)
.bind(&formats)
.bind(&sizes)
.bind(&mtimes)
.bind(&fingerprints)
.execute(&mut *tx)
.await?;
files_update.clear();
}
// Batch insert books using UNNEST
if !books_insert.is_empty() {
let book_ids: Vec<Uuid> = books_insert.iter().map(|b| b.book_id).collect();
let library_ids: Vec<Uuid> = books_insert.iter().map(|b| b.library_id).collect();
let kinds: Vec<String> = books_insert.iter().map(|b| b.kind.clone()).collect();
let titles: Vec<String> = books_insert.iter().map(|b| b.title.clone()).collect();
let series: Vec<Option<String>> = books_insert.iter().map(|b| b.series.clone()).collect();
let volumes: Vec<Option<i32>> = books_insert.iter().map(|b| b.volume).collect();
let page_counts: Vec<Option<i32>> = books_insert.iter().map(|b| b.page_count).collect();
let thumbnail_paths: Vec<Option<String>> = books_insert.iter().map(|b| b.thumbnail_path.clone()).collect();
sqlx::query(
r#"
INSERT INTO books (id, library_id, kind, title, series, volume, page_count, thumbnail_path)
SELECT * FROM UNNEST($1::uuid[], $2::uuid[], $3::text[], $4::text[], $5::text[], $6::int[], $7::int[], $8::text[])
AS t(id, library_id, kind, title, series, volume, page_count, thumbnail_path)
"#
)
.bind(&book_ids)
.bind(&library_ids)
.bind(&kinds)
.bind(&titles)
.bind(&series)
.bind(&volumes)
.bind(&page_counts)
.bind(&thumbnail_paths)
.execute(&mut *tx)
.await?;
books_insert.clear();
}
// Batch insert files using UNNEST
if !files_insert.is_empty() {
let file_ids: Vec<Uuid> = files_insert.iter().map(|f| f.file_id).collect();
let book_ids: Vec<Uuid> = files_insert.iter().map(|f| f.book_id).collect();
let formats: Vec<String> = files_insert.iter().map(|f| f.format.clone()).collect();
let abs_paths: Vec<String> = files_insert.iter().map(|f| f.abs_path.clone()).collect();
let sizes: Vec<i64> = files_insert.iter().map(|f| f.size_bytes).collect();
let mtimes: Vec<DateTime<Utc>> = files_insert.iter().map(|f| f.mtime).collect();
let fingerprints: Vec<String> = files_insert.iter().map(|f| f.fingerprint.clone()).collect();
let statuses: Vec<String> = files_insert.iter().map(|f| f.parse_status.clone()).collect();
let errors: Vec<Option<String>> = files_insert.iter().map(|f| f.parse_error.clone()).collect();
sqlx::query(
r#"
INSERT INTO book_files (id, book_id, format, abs_path, size_bytes, mtime, fingerprint, parse_status, parse_error_opt)
SELECT * FROM UNNEST($1::uuid[], $2::uuid[], $3::text[], $4::text[], $5::bigint[], $6::timestamptz[], $7::text[], $8::text[], $9::text[])
AS t(id, book_id, format, abs_path, size_bytes, mtime, fingerprint, parse_status, parse_error_opt)
"#
)
.bind(&file_ids)
.bind(&book_ids)
.bind(&formats)
.bind(&abs_paths)
.bind(&sizes)
.bind(&mtimes)
.bind(&fingerprints)
.bind(&statuses)
.bind(&errors)
.execute(&mut *tx)
.await?;
files_insert.clear();
}
// Batch insert errors using UNNEST
if !errors_insert.is_empty() {
let job_ids: Vec<Uuid> = errors_insert.iter().map(|e| e.job_id).collect();
let file_paths: Vec<String> = errors_insert.iter().map(|e| e.file_path.clone()).collect();
let messages: Vec<String> = errors_insert.iter().map(|e| e.error_message.clone()).collect();
sqlx::query(
r#"
INSERT INTO index_job_errors (job_id, file_path, error_message)
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::text[])
AS t(job_id, file_path, error_message)
"#
)
.bind(&job_ids)
.bind(&file_paths)
.bind(&messages)
.execute(&mut *tx)
.await?;
errors_insert.clear();
}
tx.commit().await?;
tracing::info!("[BATCH] Flushed all batches in {:?}", start.elapsed());
Ok(())
}

329
apps/indexer/src/job.rs Normal file
View File

@@ -0,0 +1,329 @@
use anyhow::Result;
use rayon::prelude::*;
use sqlx::{PgPool, Row};
use tracing::{error, info};
use uuid::Uuid;
use crate::{analyzer, meili, scanner, AppState};
pub async fn cleanup_stale_jobs(pool: &PgPool) -> Result<()> {
let result = sqlx::query(
r#"
UPDATE index_jobs
SET status = 'failed',
finished_at = NOW(),
error_opt = 'Job interrupted by indexer restart'
WHERE status = 'running'
AND started_at < NOW() - INTERVAL '5 minutes'
RETURNING id
"#,
)
.fetch_all(pool)
.await?;
if !result.is_empty() {
let count = result.len();
let ids: Vec<String> = result
.iter()
.map(|row| row.get::<Uuid, _>("id").to_string())
.collect();
info!(
"[CLEANUP] Marked {} stale job(s) as failed: {}",
count,
ids.join(", ")
);
}
Ok(())
}
pub async fn claim_next_job(pool: &PgPool) -> Result<Option<(Uuid, Option<Uuid>)>> {
let mut tx = pool.begin().await?;
let row = sqlx::query(
r#"
SELECT j.id, j.type, j.library_id
FROM index_jobs j
WHERE j.status = 'pending'
AND (
(j.type IN ('rebuild', 'full_rebuild') AND NOT EXISTS (
SELECT 1 FROM index_jobs
WHERE status = 'running'
AND type IN ('rebuild', 'full_rebuild')
))
OR
j.type NOT IN ('rebuild', 'full_rebuild')
)
ORDER BY
CASE j.type
WHEN 'full_rebuild' THEN 1
WHEN 'rebuild' THEN 2
ELSE 3
END,
j.created_at ASC
FOR UPDATE SKIP LOCKED
LIMIT 1
"#,
)
.fetch_optional(&mut *tx)
.await?;
let Some(row) = row else {
tx.commit().await?;
return Ok(None);
};
let id: Uuid = row.get("id");
let job_type: String = row.get("type");
let library_id: Option<Uuid> = row.get("library_id");
if job_type == "rebuild" || job_type == "full_rebuild" {
let has_running_rebuild: bool = sqlx::query_scalar(
r#"
SELECT EXISTS(
SELECT 1 FROM index_jobs
WHERE status = 'running'
AND type IN ('rebuild', 'full_rebuild')
AND id != $1
)
"#,
)
.bind(id)
.fetch_one(&mut *tx)
.await?;
if has_running_rebuild {
tx.rollback().await?;
return Ok(None);
}
}
sqlx::query(
"UPDATE index_jobs SET status = 'running', started_at = NOW(), error_opt = NULL WHERE id = $1",
)
.bind(id)
.execute(&mut *tx)
.await?;
tx.commit().await?;
Ok(Some((id, library_id)))
}
pub async fn fail_job(pool: &PgPool, job_id: Uuid, error_message: &str) -> Result<()> {
sqlx::query(
"UPDATE index_jobs SET status = 'failed', finished_at = NOW(), error_opt = $2 WHERE id = $1",
)
.bind(job_id)
.bind(error_message)
.execute(pool)
.await?;
Ok(())
}
pub async fn is_job_cancelled(pool: &PgPool, job_id: Uuid) -> Result<bool> {
let status: Option<String> =
sqlx::query_scalar("SELECT status FROM index_jobs WHERE id = $1")
.bind(job_id)
.fetch_optional(pool)
.await?;
Ok(status.as_deref() == Some("cancelled"))
}
pub async fn process_job(
state: &AppState,
job_id: Uuid,
target_library_id: Option<Uuid>,
) -> Result<()> {
info!("[JOB] Processing {} library={:?}", job_id, target_library_id);
let job_type: String = sqlx::query_scalar("SELECT type FROM index_jobs WHERE id = $1")
.bind(job_id)
.fetch_one(&state.pool)
.await?;
// Thumbnail rebuild: generate thumbnails for books missing them
if job_type == "thumbnail_rebuild" {
sqlx::query(
"UPDATE index_jobs SET status = 'generating_thumbnails', started_at = NOW() WHERE id = $1",
)
.bind(job_id)
.execute(&state.pool)
.await?;
analyzer::analyze_library_books(state, job_id, target_library_id, true).await?;
sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(&state.pool)
.await?;
return Ok(());
}
// Thumbnail regenerate: clear all thumbnails then re-generate
if job_type == "thumbnail_regenerate" {
sqlx::query(
"UPDATE index_jobs SET status = 'generating_thumbnails', started_at = NOW() WHERE id = $1",
)
.bind(job_id)
.execute(&state.pool)
.await?;
analyzer::regenerate_thumbnails(state, job_id, target_library_id).await?;
sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(&state.pool)
.await?;
return Ok(());
}
let is_full_rebuild = job_type == "full_rebuild";
info!(
"[JOB] {} type={} full_rebuild={}",
job_id, job_type, is_full_rebuild
);
// Full rebuild: delete existing data first
if is_full_rebuild {
info!("[JOB] Full rebuild: deleting existing data");
if let Some(library_id) = target_library_id {
sqlx::query(
"DELETE FROM book_files WHERE book_id IN (SELECT id FROM books WHERE library_id = $1)",
)
.bind(library_id)
.execute(&state.pool)
.await?;
sqlx::query("DELETE FROM books WHERE library_id = $1")
.bind(library_id)
.execute(&state.pool)
.await?;
info!("[JOB] Deleted existing data for library {}", library_id);
} else {
sqlx::query("DELETE FROM book_files")
.execute(&state.pool)
.await?;
sqlx::query("DELETE FROM books").execute(&state.pool).await?;
info!("[JOB] Deleted all existing data");
}
}
let libraries = if let Some(library_id) = target_library_id {
sqlx::query("SELECT id, root_path FROM libraries WHERE id = $1 AND enabled = TRUE")
.bind(library_id)
.fetch_all(&state.pool)
.await?
} else {
sqlx::query("SELECT id, root_path FROM libraries WHERE enabled = TRUE")
.fetch_all(&state.pool)
.await?
};
// Count total files for progress estimation
let library_paths: Vec<String> = libraries
.iter()
.map(|library| {
crate::utils::remap_libraries_path(&library.get::<String, _>("root_path"))
})
.collect();
let total_files: usize = library_paths
.par_iter()
.map(|root_path| {
walkdir::WalkDir::new(root_path)
.into_iter()
.filter_map(Result::ok)
.filter(|entry| {
entry.file_type().is_file()
&& parsers::detect_format(entry.path()).is_some()
})
.count()
})
.sum();
info!(
"[JOB] Found {} libraries, {} total files to index",
libraries.len(),
total_files
);
sqlx::query("UPDATE index_jobs SET total_files = $2 WHERE id = $1")
.bind(job_id)
.bind(total_files as i32)
.execute(&state.pool)
.await?;
let mut stats = scanner::JobStats {
scanned_files: 0,
indexed_files: 0,
removed_files: 0,
errors: 0,
};
let mut total_processed_count = 0i32;
// Phase 1: Discovery
for library in &libraries {
let library_id: Uuid = library.get("id");
let root_path: String = library.get("root_path");
let root_path = crate::utils::remap_libraries_path(&root_path);
match scanner::scan_library_discovery(
state,
job_id,
library_id,
std::path::Path::new(&root_path),
&mut stats,
&mut total_processed_count,
total_files,
is_full_rebuild,
)
.await
{
Ok(()) => {}
Err(err) => {
let err_str = err.to_string();
if err_str.contains("cancelled") || err_str.contains("Cancelled") {
return Err(err);
}
stats.errors += 1;
error!(library_id = %library_id, error = %err, "library scan failed");
}
}
}
// Sync search index after discovery (books are visible immediately)
meili::sync_meili(&state.pool, &state.meili_url, &state.meili_master_key).await?;
// For full rebuild: clean up orphaned thumbnail files (old UUIDs)
if is_full_rebuild {
analyzer::cleanup_orphaned_thumbnails(state, target_library_id).await?;
}
// Phase 2: Analysis (extract page_count + thumbnails for new/updated books)
sqlx::query(
"UPDATE index_jobs SET status = 'generating_thumbnails', stats_json = $2, current_file = NULL, processed_files = $3 WHERE id = $1",
)
.bind(job_id)
.bind(serde_json::to_value(&stats)?)
.bind(total_processed_count)
.execute(&state.pool)
.await?;
analyzer::analyze_library_books(state, job_id, target_library_id, false).await?;
sqlx::query(
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
)
.bind(job_id)
.execute(&state.pool)
.await?;
Ok(())
}

19
apps/indexer/src/lib.rs Normal file
View File

@@ -0,0 +1,19 @@
pub mod analyzer;
pub mod api;
pub mod batch;
pub mod job;
pub mod meili;
pub mod scheduler;
pub mod scanner;
pub mod utils;
pub mod watcher;
pub mod worker;
use sqlx::PgPool;
#[derive(Clone)]
pub struct AppState {
pub pool: PgPool,
pub meili_url: String,
pub meili_master_key: String,
}

File diff suppressed because it is too large Load Diff

180
apps/indexer/src/meili.rs Normal file
View File

@@ -0,0 +1,180 @@
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
use reqwest::Client;
use serde::Serialize;
use sqlx::{PgPool, Row};
use tracing::info;
use uuid::Uuid;
#[derive(Serialize)]
struct SearchDoc {
id: String,
library_id: String,
kind: String,
title: String,
author: Option<String>,
series: Option<String>,
volume: Option<i32>,
language: Option<String>,
}
pub async fn sync_meili(pool: &PgPool, meili_url: &str, meili_master_key: &str) -> Result<()> {
let client = Client::new();
let base = meili_url.trim_end_matches('/');
// Ensure index exists and has proper settings
let _ = client
.post(format!("{base}/indexes"))
.header("Authorization", format!("Bearer {meili_master_key}"))
.json(&serde_json::json!({"uid": "books", "primaryKey": "id"}))
.send()
.await;
let _ = client
.patch(format!("{base}/indexes/books/settings/filterable-attributes"))
.header("Authorization", format!("Bearer {meili_master_key}"))
.json(&serde_json::json!(["library_id", "kind"]))
.send()
.await;
// Get last sync timestamp
let last_sync: Option<DateTime<Utc>> = sqlx::query_scalar(
"SELECT last_meili_sync FROM sync_metadata WHERE id = 1 AND last_meili_sync IS NOT NULL"
)
.fetch_optional(pool)
.await?;
// If no previous sync, do a full sync
let is_full_sync = last_sync.is_none();
// Get books to sync: all if full sync, only modified since last sync otherwise
let rows = if is_full_sync {
info!("[MEILI] Performing full sync");
sqlx::query(
"SELECT id, library_id, kind, title, author, series, volume, language, updated_at FROM books",
)
.fetch_all(pool)
.await?
} else {
let since = last_sync.unwrap();
info!("[MEILI] Performing incremental sync since {}", since);
// Also get deleted book IDs to remove from MeiliSearch
// For now, we'll do a diff approach: get all book IDs from DB and compare with Meili
sqlx::query(
"SELECT id, library_id, kind, title, author, series, volume, language, updated_at FROM books WHERE updated_at > $1",
)
.bind(since)
.fetch_all(pool)
.await?
};
if rows.is_empty() && !is_full_sync {
info!("[MEILI] No changes to sync");
// Still update the timestamp
sqlx::query(
"INSERT INTO sync_metadata (id, last_meili_sync) VALUES (1, NOW()) ON CONFLICT (id) DO UPDATE SET last_meili_sync = NOW()"
)
.execute(pool)
.await?;
return Ok(());
}
let docs: Vec<SearchDoc> = rows
.into_iter()
.map(|row| SearchDoc {
id: row.get::<Uuid, _>("id").to_string(),
library_id: row.get::<Uuid, _>("library_id").to_string(),
kind: row.get("kind"),
title: row.get("title"),
author: row.get("author"),
series: row.get("series"),
volume: row.get("volume"),
language: row.get("language"),
})
.collect();
let doc_count = docs.len();
// Send documents to MeiliSearch in batches of 1000
const MEILI_BATCH_SIZE: usize = 1000;
for (i, chunk) in docs.chunks(MEILI_BATCH_SIZE).enumerate() {
let batch_num = i + 1;
info!("[MEILI] Sending batch {}/{} ({} docs)", batch_num, doc_count.div_ceil(MEILI_BATCH_SIZE), chunk.len());
let response = client
.post(format!("{base}/indexes/books/documents"))
.header("Authorization", format!("Bearer {meili_master_key}"))
.json(&chunk)
.send()
.await
.context("failed to send docs to meili")?;
if !response.status().is_success() {
let status = response.status();
let body = response.text().await.unwrap_or_default();
return Err(anyhow::anyhow!("MeiliSearch error {}: {}", status, body));
}
}
// Handle deletions: get all book IDs from DB and remove from MeiliSearch any that don't exist
// This is expensive, so we only do it periodically (every 10 syncs) or on full syncs
if is_full_sync || rand::random::<u8>() < 26 { // ~10% chance
info!("[MEILI] Checking for documents to delete");
// Get all book IDs from database
let db_ids: Vec<String> = sqlx::query_scalar("SELECT id::text FROM books")
.fetch_all(pool)
.await?;
// Get all document IDs from MeiliSearch (this requires fetching all documents)
// For efficiency, we'll just delete by query for documents that might be stale
// A better approach would be to track deletions in a separate table
// For now, we'll do a simple approach: fetch all Meili docs and compare
// Note: This could be slow for large collections
let meili_response = client
.post(format!("{base}/indexes/books/documents/fetch"))
.header("Authorization", format!("Bearer {meili_master_key}"))
.json(&serde_json::json!({
"fields": ["id"],
"limit": 100000
}))
.send()
.await;
if let Ok(response) = meili_response {
if response.status().is_success() {
if let Ok(meili_docs) = response.json::<Vec<serde_json::Value>>().await {
let meili_ids: std::collections::HashSet<String> = meili_docs
.into_iter()
.filter_map(|doc| doc.get("id").and_then(|id| id.as_str()).map(|s| s.to_string()))
.collect();
let db_ids_set: std::collections::HashSet<String> = db_ids.into_iter().collect();
let to_delete: Vec<String> = meili_ids.difference(&db_ids_set).cloned().collect();
if !to_delete.is_empty() {
info!("[MEILI] Deleting {} stale documents", to_delete.len());
let _ = client
.post(format!("{base}/indexes/books/documents/delete-batch"))
.header("Authorization", format!("Bearer {meili_master_key}"))
.json(&to_delete)
.send()
.await;
}
}
}
}
}
// Update last sync timestamp
sqlx::query(
"INSERT INTO sync_metadata (id, last_meili_sync) VALUES (1, NOW()) ON CONFLICT (id) DO UPDATE SET last_meili_sync = NOW()"
)
.execute(pool)
.await?;
info!("[MEILI] Sync completed: {} documents indexed", doc_count);
Ok(())
}

443
apps/indexer/src/scanner.rs Normal file
View File

@@ -0,0 +1,443 @@
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
use parsers::{detect_format, parse_metadata_fast};
use serde::Serialize;
use sqlx::Row;
use std::{collections::HashMap, path::Path, time::Duration};
use tracing::{error, info, trace, warn};
use uuid::Uuid;
use walkdir::WalkDir;
use crate::{
batch::{flush_all_batches, BookInsert, BookUpdate, ErrorInsert, FileInsert, FileUpdate},
job::is_job_cancelled,
utils,
AppState,
};
#[derive(Serialize)]
pub struct JobStats {
pub scanned_files: usize,
pub indexed_files: usize,
pub removed_files: usize,
pub errors: usize,
}
const BATCH_SIZE: usize = 100;
/// Phase 1 — Discovery: walk filesystem, extract metadata from filenames only (no archive I/O).
/// New books are inserted with page_count = NULL so the analyzer phase can fill them in.
/// Updated books (fingerprint changed) get page_count/thumbnail reset.
#[allow(clippy::too_many_arguments)]
pub async fn scan_library_discovery(
state: &AppState,
job_id: Uuid,
library_id: Uuid,
root: &Path,
stats: &mut JobStats,
total_processed_count: &mut i32,
total_files: usize,
is_full_rebuild: bool,
) -> Result<()> {
info!(
"[SCAN] Starting discovery scan of library {} at path: {} (full_rebuild={})",
library_id,
root.display(),
is_full_rebuild
);
// Load existing files from DB
let existing_rows = sqlx::query(
r#"
SELECT bf.id AS file_id, bf.book_id, bf.abs_path, bf.fingerprint
FROM book_files bf
JOIN books b ON b.id = bf.book_id
WHERE b.library_id = $1
"#,
)
.bind(library_id)
.fetch_all(&state.pool)
.await?;
let mut existing: HashMap<String, (Uuid, Uuid, String)> = HashMap::new();
if !is_full_rebuild {
for row in existing_rows {
let abs_path: String = row.get("abs_path");
let remapped_path = utils::remap_libraries_path(&abs_path);
existing.insert(
remapped_path,
(row.get("file_id"), row.get("book_id"), row.get("fingerprint")),
);
}
info!(
"[SCAN] Found {} existing files in database for library {}",
existing.len(),
library_id
);
} else {
info!("[SCAN] Full rebuild: skipping existing files lookup");
// Delete stale directory mtime records for full rebuild
let _ = sqlx::query("DELETE FROM directory_mtimes WHERE library_id = $1")
.bind(library_id)
.execute(&state.pool)
.await;
}
// Load stored directory mtimes for incremental skip
let dir_mtimes: HashMap<String, DateTime<Utc>> = if !is_full_rebuild {
let rows = sqlx::query(
"SELECT dir_path, mtime FROM directory_mtimes WHERE library_id = $1",
)
.bind(library_id)
.fetch_all(&state.pool)
.await
.unwrap_or_default();
rows.into_iter()
.map(|row| {
let db_path: String = row.get("dir_path");
let local_path = utils::remap_libraries_path(&db_path);
let mtime: DateTime<Utc> = row.get("mtime");
(local_path, mtime)
})
.collect()
} else {
HashMap::new()
};
let mut seen: HashMap<String, bool> = HashMap::new();
let mut library_processed_count = 0i32;
let mut last_progress_update = std::time::Instant::now();
// Batching buffers
let mut books_to_update: Vec<BookUpdate> = Vec::with_capacity(BATCH_SIZE);
let mut files_to_update: Vec<FileUpdate> = Vec::with_capacity(BATCH_SIZE);
let mut books_to_insert: Vec<BookInsert> = Vec::with_capacity(BATCH_SIZE);
let mut files_to_insert: Vec<FileInsert> = Vec::with_capacity(BATCH_SIZE);
let mut errors_to_insert: Vec<ErrorInsert> = Vec::with_capacity(BATCH_SIZE);
// Track discovered directory mtimes for upsert after scan
let mut new_dir_mtimes: Vec<(String, DateTime<Utc>)> = Vec::new();
// Prefixes (with trailing "/") of directories whose mtime hasn't changed.
// Files under these prefixes are added to `seen` but not reprocessed.
let mut skipped_dir_prefixes: Vec<String> = Vec::new();
for entry in WalkDir::new(root).into_iter().filter_map(Result::ok) {
let path = entry.path().to_path_buf();
let local_path = path.to_string_lossy().to_string();
if entry.file_type().is_dir() {
if entry.depth() == 0 {
continue; // skip root itself
}
// Check if parent dir is already skipped (propagate skip to subdirs)
let already_under_skipped = skipped_dir_prefixes
.iter()
.any(|p| local_path.starts_with(p.as_str()));
if let Ok(meta) = entry.metadata() {
if let Ok(sys_mtime) = meta.modified() {
let mtime_utc: DateTime<Utc> = DateTime::from(sys_mtime);
// Only record mtimes for non-skipped dirs (to avoid polluting DB)
if !already_under_skipped {
new_dir_mtimes.push((local_path.clone(), mtime_utc));
}
// Skip if mtime unchanged (incremental only, not already skipped subtree)
if !is_full_rebuild && !already_under_skipped {
if let Some(&stored_mtime) = dir_mtimes.get(&local_path) {
if mtime_utc <= stored_mtime {
trace!("[SCAN] Skipping unchanged dir: {}", local_path);
// Add trailing slash so starts_with check is exact per-segment
skipped_dir_prefixes.push(format!("{}/", local_path));
}
}
}
}
}
continue;
}
if !entry.file_type().is_file() {
continue;
}
// Check if this file is under a skipped dir
let under_skipped = skipped_dir_prefixes
.iter()
.any(|p| local_path.starts_with(p.as_str()));
if under_skipped {
// Dir unchanged — just mark file as seen so it's not deleted
let abs_path_local = local_path.clone();
let abs_path = utils::unmap_libraries_path(&abs_path_local);
let lookup_path = utils::remap_libraries_path(&abs_path);
seen.insert(lookup_path, true);
continue;
}
let Some(format) = detect_format(&path) else {
trace!("[SCAN] Skipping non-book file: {}", path.display());
continue;
};
info!(
"[SCAN] Found book file: {} (format: {:?})",
path.display(),
format
);
stats.scanned_files += 1;
let abs_path_local = path.to_string_lossy().to_string();
let abs_path = utils::unmap_libraries_path(&abs_path_local);
let file_name = path
.file_name()
.map(|s| s.to_string_lossy().to_string())
.unwrap_or_else(|| abs_path.clone());
let metadata = std::fs::metadata(&path)
.with_context(|| format!("cannot stat {}", path.display()))?;
let mtime: DateTime<Utc> = metadata
.modified()
.map(DateTime::<Utc>::from)
.unwrap_or_else(|_| Utc::now());
let fingerprint = utils::compute_fingerprint(&path, metadata.len(), &mtime)?;
let lookup_path = utils::remap_libraries_path(&abs_path);
library_processed_count += 1;
*total_processed_count += 1;
// Progress update
let should_update_progress = last_progress_update.elapsed() > Duration::from_secs(1)
|| library_processed_count % 10 == 0;
if should_update_progress {
let progress_percent = if total_files > 0 {
((*total_processed_count as f64 / total_files as f64) * 100.0) as i32
} else {
0
};
sqlx::query(
"UPDATE index_jobs SET current_file = $2, processed_files = $3, progress_percent = $4 WHERE id = $1",
)
.bind(job_id)
.bind(&file_name)
.bind(*total_processed_count)
.bind(progress_percent)
.execute(&state.pool)
.await
.map_err(|e| {
error!("[BDD] Failed to update progress for job {}: {}", job_id, e);
e
})?;
last_progress_update = std::time::Instant::now();
if is_job_cancelled(&state.pool, job_id).await? {
info!("[JOB] Job {} cancelled by user, stopping...", job_id);
flush_all_batches(
&state.pool,
&mut books_to_update,
&mut files_to_update,
&mut books_to_insert,
&mut files_to_insert,
&mut errors_to_insert,
)
.await?;
return Err(anyhow::anyhow!("Job cancelled by user"));
}
}
seen.insert(lookup_path.clone(), true);
// Fast metadata extraction — no archive I/O
let parsed = parse_metadata_fast(&path, format, root);
if let Some((file_id, book_id, old_fingerprint)) =
existing.get(&lookup_path).cloned()
{
if !is_full_rebuild && old_fingerprint == fingerprint {
trace!("[PROCESS] Skipping unchanged file: {}", file_name);
continue;
}
info!(
"[PROCESS] Updating existing file: {} (fingerprint_changed={})",
file_name,
old_fingerprint != fingerprint
);
books_to_update.push(BookUpdate {
book_id,
title: parsed.title,
kind: utils::kind_from_format(format).to_string(),
series: parsed.series,
volume: parsed.volume,
// Reset page_count so analyzer re-processes this book
page_count: None,
});
files_to_update.push(FileUpdate {
file_id,
format: format.as_str().to_string(),
size_bytes: metadata.len() as i64,
mtime,
fingerprint,
});
// Also clear thumbnail so it gets regenerated
if let Err(e) = sqlx::query(
"UPDATE books SET thumbnail_path = NULL WHERE id = $1",
)
.bind(book_id)
.execute(&state.pool)
.await
{
warn!(
"[BDD] Failed to clear thumbnail for book {}: {}",
book_id, e
);
}
stats.indexed_files += 1;
if books_to_update.len() >= BATCH_SIZE || files_to_update.len() >= BATCH_SIZE {
flush_all_batches(
&state.pool,
&mut books_to_update,
&mut files_to_update,
&mut books_to_insert,
&mut files_to_insert,
&mut errors_to_insert,
)
.await?;
}
continue;
}
// New file — insert with page_count = NULL (analyzer fills it in)
info!("[PROCESS] Inserting new file: {}", file_name);
let book_id = Uuid::new_v4();
let file_id = Uuid::new_v4();
books_to_insert.push(BookInsert {
book_id,
library_id,
kind: utils::kind_from_format(format).to_string(),
title: parsed.title,
series: parsed.series,
volume: parsed.volume,
page_count: None,
thumbnail_path: None,
});
files_to_insert.push(FileInsert {
file_id,
book_id,
format: format.as_str().to_string(),
abs_path: abs_path.clone(),
size_bytes: metadata.len() as i64,
mtime,
fingerprint,
parse_status: "ok".to_string(),
parse_error: None,
});
stats.indexed_files += 1;
if books_to_insert.len() >= BATCH_SIZE || files_to_insert.len() >= BATCH_SIZE {
flush_all_batches(
&state.pool,
&mut books_to_update,
&mut files_to_update,
&mut books_to_insert,
&mut files_to_insert,
&mut errors_to_insert,
)
.await?;
}
}
// Flush remaining batches
flush_all_batches(
&state.pool,
&mut books_to_update,
&mut files_to_update,
&mut books_to_insert,
&mut files_to_insert,
&mut errors_to_insert,
)
.await?;
if !skipped_dir_prefixes.is_empty() {
info!(
"[SCAN] Skipped {} unchanged directories",
skipped_dir_prefixes.len()
);
}
info!(
"[SCAN] Library {} discovery complete: {} files scanned, {} indexed, {} errors",
library_id, library_processed_count, stats.indexed_files, stats.errors
);
// Handle deletions
let mut removed_count = 0usize;
for (abs_path, (file_id, book_id, _)) in &existing {
if seen.contains_key(abs_path) {
continue;
}
sqlx::query("DELETE FROM book_files WHERE id = $1")
.bind(file_id)
.execute(&state.pool)
.await?;
sqlx::query(
"DELETE FROM books WHERE id = $1 AND NOT EXISTS (SELECT 1 FROM book_files WHERE book_id = $1)",
)
.bind(book_id)
.execute(&state.pool)
.await?;
stats.removed_files += 1;
removed_count += 1;
}
if removed_count > 0 {
info!(
"[SCAN] Removed {} stale files from database",
removed_count
);
}
// Upsert directory mtimes for next incremental scan
if !new_dir_mtimes.is_empty() {
let dir_paths_db: Vec<String> = new_dir_mtimes
.iter()
.map(|(local, _)| utils::unmap_libraries_path(local))
.collect();
let mtimes: Vec<DateTime<Utc>> = new_dir_mtimes.iter().map(|(_, m)| *m).collect();
let library_ids: Vec<Uuid> = vec![library_id; new_dir_mtimes.len()];
if let Err(e) = sqlx::query(
r#"
INSERT INTO directory_mtimes (library_id, dir_path, mtime)
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::timestamptz[])
AS t(library_id, dir_path, mtime)
ON CONFLICT (library_id, dir_path) DO UPDATE SET mtime = EXCLUDED.mtime
"#,
)
.bind(&library_ids)
.bind(&dir_paths_db)
.bind(&mtimes)
.execute(&state.pool)
.await
{
warn!("[SCAN] Failed to upsert directory mtimes: {}", e);
}
}
Ok(())
}

View File

@@ -0,0 +1,67 @@
use anyhow::Result;
use sqlx::{PgPool, Row};
use tracing::info;
use uuid::Uuid;
pub async fn check_and_schedule_auto_scans(pool: &PgPool) -> Result<()> {
let libraries = sqlx::query(
r#"
SELECT id, scan_mode, last_scan_at
FROM libraries
WHERE monitor_enabled = TRUE
AND (
next_scan_at IS NULL
OR next_scan_at <= NOW()
)
AND NOT EXISTS (
SELECT 1 FROM index_jobs
WHERE library_id = libraries.id
AND status IN ('pending', 'running')
)
"#
)
.fetch_all(pool)
.await?;
for row in libraries {
let library_id: Uuid = row.get("id");
let scan_mode: String = row.get("scan_mode");
info!("[SCHEDULER] Auto-scanning library {} (mode: {})", library_id, scan_mode);
let job_id = Uuid::new_v4();
let job_type = match scan_mode.as_str() {
"full" => "full_rebuild",
_ => "rebuild",
};
sqlx::query(
"INSERT INTO index_jobs (id, library_id, type, status) VALUES ($1, $2, $3, 'pending')"
)
.bind(job_id)
.bind(library_id)
.bind(job_type)
.execute(pool)
.await?;
// Update next_scan_at
let interval_minutes = match scan_mode.as_str() {
"hourly" => 60,
"daily" => 1440,
"weekly" => 10080,
_ => 1440, // default daily
};
sqlx::query(
"UPDATE libraries SET last_scan_at = NOW(), next_scan_at = NOW() + INTERVAL '1 minute' * $2 WHERE id = $1"
)
.bind(library_id)
.bind(interval_minutes)
.execute(pool)
.await?;
info!("[SCHEDULER] Created job {} for library {}", job_id, library_id);
}
Ok(())
}

52
apps/indexer/src/utils.rs Normal file
View File

@@ -0,0 +1,52 @@
use anyhow::Result;
use chrono::DateTime;
use parsers::BookFormat;
use sha2::{Digest, Sha256};
use std::path::Path;
use chrono::Utc;
pub fn remap_libraries_path(path: &str) -> String {
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
if path.starts_with("/libraries/") {
return path.replacen("/libraries", &root, 1);
}
}
path.to_string()
}
pub fn unmap_libraries_path(path: &str) -> String {
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
if path.starts_with(&root) {
return path.replacen(&root, "/libraries", 1);
}
}
path.to_string()
}
pub fn compute_fingerprint(path: &Path, size: u64, mtime: &DateTime<Utc>) -> Result<String> {
// Optimized: only use size + mtime + first bytes of filename for fast fingerprinting
// This is 100x faster than reading file content while still being reliable for change detection
let mut hasher = Sha256::new();
hasher.update(size.to_le_bytes());
hasher.update(mtime.timestamp().to_le_bytes());
// Add filename for extra uniqueness (in case of rapid changes with same size+mtime)
if let Some(filename) = path.file_name() {
hasher.update(filename.as_encoded_bytes());
}
Ok(format!("{:x}", hasher.finalize()))
}
pub fn kind_from_format(format: BookFormat) -> &'static str {
match format {
BookFormat::Pdf => "ebook",
BookFormat::Cbz | BookFormat::Cbr => "comic",
}
}
pub fn file_display_name(path: &Path) -> String {
path.file_stem()
.map(|s| s.to_string_lossy().to_string())
.unwrap_or_else(|| "Untitled".to_string())
}

147
apps/indexer/src/watcher.rs Normal file
View File

@@ -0,0 +1,147 @@
use anyhow::Result;
use notify::{Event, RecommendedWatcher, RecursiveMode, Watcher};
use sqlx::Row;
use std::collections::HashMap;
use std::time::Duration;
use tokio::sync::mpsc;
use tracing::{error, info, trace};
use uuid::Uuid;
use crate::utils;
use crate::AppState;
pub async fn run_file_watcher(state: AppState) -> Result<()> {
let (tx, mut rx) = mpsc::channel::<(Uuid, String)>(100);
// Start watcher refresh loop
let refresh_interval = Duration::from_secs(30);
let pool = state.pool.clone();
tokio::spawn(async move {
let mut watched_libraries: HashMap<Uuid, String> = HashMap::new();
loop {
// Get libraries with watcher enabled
match sqlx::query(
"SELECT id, root_path FROM libraries WHERE watcher_enabled = TRUE AND enabled = TRUE"
)
.fetch_all(&pool)
.await
{
Ok(rows) => {
let current_libraries: HashMap<Uuid, String> = rows
.into_iter()
.map(|row| {
let id: Uuid = row.get("id");
let root_path: String = row.get("root_path");
let local_path = utils::remap_libraries_path(&root_path);
(id, local_path)
})
.collect();
// Check if we need to recreate watcher
let needs_restart = watched_libraries.len() != current_libraries.len()
|| watched_libraries.iter().any(|(id, path)| {
current_libraries.get(id) != Some(path)
});
if needs_restart {
info!("[WATCHER] Restarting watcher for {} libraries", current_libraries.len());
if !current_libraries.is_empty() {
let tx_clone = tx.clone();
let libraries_clone = current_libraries.clone();
match setup_watcher(libraries_clone, tx_clone) {
Ok(_new_watcher) => {
watched_libraries = current_libraries;
info!("[WATCHER] Watching {} libraries", watched_libraries.len());
}
Err(err) => {
error!("[WATCHER] Failed to setup watcher: {}", err);
}
}
}
}
}
Err(err) => {
error!("[WATCHER] Failed to fetch libraries: {}", err);
}
}
tokio::time::sleep(refresh_interval).await;
}
});
// Process watcher events
while let Some((library_id, file_path)) = rx.recv().await {
info!("[WATCHER] File changed in library {}: {}", library_id, file_path);
// Check if there's already a pending job for this library
match sqlx::query_scalar::<_, bool>(
"SELECT EXISTS(SELECT 1 FROM index_jobs WHERE library_id = $1 AND status IN ('pending', 'running'))"
)
.bind(library_id)
.fetch_one(&state.pool)
.await
{
Ok(exists) => {
if !exists {
// Create a quick scan job
let job_id = Uuid::new_v4();
match sqlx::query(
"INSERT INTO index_jobs (id, library_id, type, status) VALUES ($1, $2, 'rebuild', 'pending')"
)
.bind(job_id)
.bind(library_id)
.execute(&state.pool)
.await
{
Ok(_) => info!("[WATCHER] Created job {} for library {}", job_id, library_id),
Err(err) => error!("[WATCHER] Failed to create job: {}", err),
}
} else {
trace!("[WATCHER] Job already pending for library {}, skipping", library_id);
}
}
Err(err) => error!("[WATCHER] Failed to check existing jobs: {}", err),
}
}
Ok(())
}
fn setup_watcher(
libraries: HashMap<Uuid, String>,
tx: mpsc::Sender<(Uuid, String)>,
) -> Result<RecommendedWatcher> {
let libraries_for_closure = libraries.clone();
let mut watcher = notify::recommended_watcher(move |res: Result<Event, notify::Error>| {
match res {
Ok(event) => {
if event.kind.is_modify() || event.kind.is_create() || event.kind.is_remove() {
for path in event.paths {
if let Some((library_id, _)) = libraries_for_closure.iter().find(|(_, root)| {
path.starts_with(root)
}) {
let path_str = path.to_string_lossy().to_string();
if parsers::detect_format(&path).is_some() {
let _ = tx.try_send((*library_id, path_str));
}
}
}
}
}
Err(err) => error!("[WATCHER] Event error: {}", err),
}
})?;
// Actually watch the library directories
for root_path in libraries.values() {
info!("[WATCHER] Watching directory: {}", root_path);
watcher.watch(std::path::Path::new(root_path), RecursiveMode::Recursive)?;
}
Ok(watcher)
}

View File

@@ -0,0 +1,61 @@
use std::time::Duration;
use tracing::{error, info, trace};
use crate::{job, scheduler, watcher, AppState};
pub async fn run_worker(state: AppState, interval_seconds: u64) {
let wait = Duration::from_secs(interval_seconds.max(1));
// Cleanup stale jobs from previous runs
if let Err(err) = job::cleanup_stale_jobs(&state.pool).await {
error!("[CLEANUP] Failed to cleanup stale jobs: {}", err);
}
// Start file watcher task
let watcher_state = state.clone();
let _watcher_handle = tokio::spawn(async move {
info!("[WATCHER] Starting file watcher service");
if let Err(err) = watcher::run_file_watcher(watcher_state).await {
error!("[WATCHER] Error: {}", err);
}
});
// Start scheduler task for auto-monitoring
let scheduler_state = state.clone();
let _scheduler_handle = tokio::spawn(async move {
let scheduler_wait = Duration::from_secs(60); // Check every minute
loop {
if let Err(err) = scheduler::check_and_schedule_auto_scans(&scheduler_state.pool).await {
error!("[SCHEDULER] Error: {}", err);
}
tokio::time::sleep(scheduler_wait).await;
}
});
loop {
match job::claim_next_job(&state.pool).await {
Ok(Some((job_id, library_id))) => {
info!("[INDEXER] Starting job {} library={:?}", job_id, library_id);
if let Err(err) = job::process_job(&state, job_id, library_id).await {
let err_str = err.to_string();
if err_str.contains("cancelled") || err_str.contains("Cancelled") {
info!("[INDEXER] Job {} was cancelled by user", job_id);
// Status is already 'cancelled' in DB, don't change it
} else {
error!("[INDEXER] Job {} failed: {}", job_id, err);
let _ = job::fail_job(&state.pool, job_id, &err_str).await;
}
} else {
info!("[INDEXER] Job {} completed", job_id);
}
}
Ok(None) => {
trace!("[INDEXER] No pending jobs, waiting...");
tokio::time::sleep(wait).await;
}
Err(err) => {
error!("[INDEXER] Worker error: {}", err);
tokio::time::sleep(wait).await;
}
}
}
}

View File

@@ -13,7 +13,7 @@ impl ApiConfig {
pub fn from_env() -> Result<Self> {
Ok(Self {
listen_addr: std::env::var("API_LISTEN_ADDR")
.unwrap_or_else(|_| "0.0.0.0:8080".to_string()),
.unwrap_or_else(|_| "0.0.0.0:7080".to_string()),
database_url: std::env::var("DATABASE_URL").context("DATABASE_URL is required")?,
meili_url: std::env::var("MEILI_URL").context("MEILI_URL is required")?,
meili_master_key: std::env::var("MEILI_MASTER_KEY")
@@ -32,10 +32,6 @@ pub struct IndexerConfig {
pub meili_master_key: String,
pub scan_interval_seconds: u64,
pub thumbnail_config: ThumbnailConfig,
/// API base URL for thumbnail checkup at end of build (e.g. http://api:8080)
pub api_base_url: String,
/// Token to call API (e.g. API_BOOTSTRAP_TOKEN)
pub api_bootstrap_token: String,
}
#[derive(Debug, Clone)]
@@ -87,7 +83,7 @@ impl IndexerConfig {
Ok(Self {
listen_addr: std::env::var("INDEXER_LISTEN_ADDR")
.unwrap_or_else(|_| "0.0.0.0:8081".to_string()),
.unwrap_or_else(|_| "0.0.0.0:7081".to_string()),
database_url: std::env::var("DATABASE_URL").context("DATABASE_URL is required")?,
meili_url: std::env::var("MEILI_URL").context("MEILI_URL is required")?,
meili_master_key: std::env::var("MEILI_MASTER_KEY")
@@ -97,10 +93,6 @@ impl IndexerConfig {
.and_then(|v| v.parse::<u64>().ok())
.unwrap_or(5),
thumbnail_config,
api_base_url: std::env::var("API_BASE_URL")
.unwrap_or_else(|_| "http://api:8080".to_string()),
api_bootstrap_token: std::env::var("API_BOOTSTRAP_TOKEN")
.context("API_BOOTSTRAP_TOKEN is required for thumbnail checkup")?,
})
}
}
@@ -116,9 +108,9 @@ impl AdminUiConfig {
pub fn from_env() -> Result<Self> {
Ok(Self {
listen_addr: std::env::var("ADMIN_UI_LISTEN_ADDR")
.unwrap_or_else(|_| "0.0.0.0:8082".to_string()),
.unwrap_or_else(|_| "0.0.0.0:7082".to_string()),
api_base_url: std::env::var("API_BASE_URL")
.unwrap_or_else(|_| "http://api:8080".to_string()),
.unwrap_or_else(|_| "http://api:7080".to_string()),
api_token: std::env::var("API_BOOTSTRAP_TOKEN")
.context("API_BOOTSTRAP_TOKEN is required")?,
})

76
crates/parsers/AGENTS.md Normal file
View File

@@ -0,0 +1,76 @@
# crates/parsers — Parsing de livres (CBZ, CBR, PDF)
Crate utilitaire sans état, utilisée par `apps/api` et `apps/indexer`.
## API publique (lib.rs)
```rust
// Détection du format par extension
pub fn detect_format(path: &Path) -> Option<BookFormat> // .cbz | .cbr | .pdf
// Extraction des métadonnées
pub fn parse_metadata(path: &Path, format: BookFormat, library_root: &Path) -> Result<ParsedMetadata>
// Extraction de la première page (pour thumbnails)
pub fn extract_first_page(path: &Path, format: BookFormat) -> Result<Vec<u8>>
pub enum BookFormat { Cbz, Cbr, Pdf }
pub struct ParsedMetadata {
pub title: String, // = nom de fichier (sans extension)
pub series: Option<String>, // = premier dossier relatif à library_root
pub volume: Option<i32>, // extrait du nom de fichier
pub page_count: Option<i32>,
}
```
## Logique de parsing
### Titre
Nom de fichier sans extension, conservé tel quel (pas de nettoyage).
### Série
Premier composant du chemin relatif entre `library_root` et le fichier :
- `/libraries/One Piece/T01.cbz` → série = `"One Piece"`
- `/libraries/one-shot.cbz` → série = `None`
### Volume (`extract_volume`)
Patterns reconnus dans le nom de fichier (dans l'ordre de priorité) :
- `T01`, `T1` (manga/comics français)
- `Vol. 1`, `Vol 1`, `Volume 1`
- `#1`, `#01`
- `- 1`, `- 01` (en fin de nom)
### Nombre de pages
| Format | Outil |
|--------|-------|
| CBZ | `zip::ZipArchive` — compte les entrées image (jpg/jpeg/png/webp/avif) |
| CBR | `unrar lb <path>` — liste les fichiers, filtre les images |
| PDF | `pdfinfo <path>` — lit la ligne `Pages:` |
## Dépendances système requises
| Outil | Utilisé pour | Installation |
|-------|-------------|-------------|
| `unrar` | CBR page count | `brew install rar` / `apt install unrar` |
| `unar` | CBR first page extraction | `brew install unar` / `apt install unar` |
| `pdfinfo` | PDF page count | inclus dans `poppler-utils` |
| `pdftoppm` | PDF first page render | inclus dans `poppler-utils` |
**Important** : `unrar` (pour le listing) et `unar` (pour l'extraction) sont deux outils différents.
## Extraction première page
- **CBZ** : `zip::ZipArchive`, trie les noms d'images, lit la première
- **CBR** : `unar -o <tmp_dir>`, `WalkDir` récursif, trie, lit la première — nettoie `tmp_dir` ensuite
- **PDF** : `pdftoppm -f 1 -singlefile -png -scale-to 800` → fichier PNG temporaire — nettoie `tmp_dir` ensuite
Répertoire temp : `std::env::temp_dir()/stripstream-{cbr|pdf}-thumb-<uuid>`.
## Gotchas
- `clean_title()` existe mais est marqué `#[allow(dead_code)]` — le titre n'est **pas** nettoyé (décision volontaire).
- Les CBR peuvent avoir des sous-dossiers internes → WalkDir nécessaire (pas de listing plat).
- La détection du format est **uniquement par extension** (pas de magic bytes).
- `pdfinfo` et `pdftoppm` doivent être du paquet `poppler-utils` (pas `poppler` seul).
- En cas d'échec de parsing, l'appelant (indexer/api) stocke `parse_status = 'error'` en DB mais continue.

View File

@@ -2,6 +2,7 @@ use anyhow::{Context, Result};
use std::io::Read;
use std::path::Path;
use std::process::Command;
use std::sync::OnceLock;
use uuid::Uuid;
use walkdir::WalkDir;
@@ -40,38 +41,52 @@ pub fn detect_format(path: &Path) -> Option<BookFormat> {
}
}
pub fn parse_metadata(
path: &Path,
format: BookFormat,
library_root: &Path,
) -> Result<ParsedMetadata> {
let filename = path
.file_stem()
.map(|s| s.to_string_lossy().to_string())
.unwrap_or_else(|| "Untitled".to_string());
// Cache compiled regex patterns — compiled once on first use
static VOLUME_PATTERNS: OnceLock<Vec<(regex::Regex, usize)>> = OnceLock::new();
// Extract volume from filename (patterns: T01, T02, Vol 1, Volume 1, #1, - 01, etc.)
let volume = extract_volume(&filename);
fn get_volume_patterns() -> &'static Vec<(regex::Regex, usize)> {
VOLUME_PATTERNS.get_or_init(|| {
[
// T01, T02 pattern (most common for manga/comics)
(r"(?i)T(\d+)", 1usize),
// Vol 1, Vol. 1, Volume 1
(r"(?i)Vol\.?\s*(\d+)", 1),
(r"(?i)Volume\s*(\d+)", 1),
// #1, #01
(r"#(\d+)", 1),
// - 1, - 01 at the end
(r"-\s*(\d+)\s*$", 1),
]
.iter()
.filter_map(|(pattern, group)| {
regex::Regex::new(pattern).ok().map(|re| (re, *group))
})
.collect()
})
}
// Keep original filename as title (don't clean it)
let title = filename;
fn extract_volume(filename: &str) -> Option<i32> {
for (re, group) in get_volume_patterns() {
if let Some(caps) = re.captures(filename) {
if let Some(mat) = caps.get(*group) {
return mat.as_str().parse::<i32>().ok();
}
}
}
None
}
// Determine series from parent folder relative to library root
let series = path.parent().and_then(|parent| {
// Normalize paths for comparison (handle different separators, etc.)
fn extract_series(path: &Path, library_root: &Path) -> Option<String> {
path.parent().and_then(|parent| {
let parent_str = parent.to_string_lossy().to_string();
let root_str = library_root.to_string_lossy().to_string();
// Try to find the library root in the parent path
let relative = if let Some(idx) = parent_str.find(&root_str) {
// Found root in parent, extract what comes after
let after_root = &parent_str[idx + root_str.len()..];
Path::new(after_root)
} else if let Some(relative) = parent.strip_prefix(library_root).ok() {
// Standard approach works
} else if let Ok(relative) = parent.strip_prefix(library_root) {
relative
} else {
// Log for diagnostic on server
eprintln!(
"[PARSER] Cannot determine series: parent '{}' doesn't start with root '{}'",
parent.display(),
@@ -80,16 +95,14 @@ pub fn parse_metadata(
return None;
};
// Remove leading separators
let relative_str = relative.to_string_lossy().to_string();
let relative_clean = relative_str.trim_start_matches(|c| c == '/' || c == '\\');
let relative_clean = relative_str.trim_start_matches(['/', '\\']);
if relative_clean.is_empty() {
return None;
}
// Get first component as series
let first_sep = relative_clean.find(|c| c == '/' || c == '\\');
let first_sep = relative_clean.find(['/', '\\']);
let series_name = match first_sep {
Some(idx) => &relative_clean[..idx],
None => relative_clean,
@@ -100,80 +113,178 @@ pub fn parse_metadata(
} else {
Some(series_name.to_string())
}
});
})
}
let page_count = match format {
/// Fast metadata extraction from filename only — no archive I/O. Always succeeds.
pub fn parse_metadata_fast(path: &Path, _format: BookFormat, library_root: &Path) -> ParsedMetadata {
let filename = path
.file_stem()
.map(|s| s.to_string_lossy().to_string())
.unwrap_or_else(|| "Untitled".to_string());
let volume = extract_volume(&filename);
let title = filename;
let series = extract_series(path, library_root);
ParsedMetadata {
title,
series,
volume,
page_count: None,
}
}
pub fn parse_metadata(
path: &Path,
format: BookFormat,
library_root: &Path,
) -> Result<ParsedMetadata> {
let mut meta = parse_metadata_fast(path, format, library_root);
meta.page_count = match format {
BookFormat::Cbz => parse_cbz_page_count(path).ok(),
BookFormat::Cbr => parse_cbr_page_count(path).ok(),
BookFormat::Pdf => parse_pdf_page_count(path).ok(),
};
Ok(ParsedMetadata {
title,
series,
volume,
page_count,
})
Ok(meta)
}
fn extract_volume(filename: &str) -> Option<i32> {
// Common volume patterns: T01, T02, T1, T2, Vol 1, Vol. 1, Volume 1, #1, #01, - 1, - 01
let patterns = [
// T01, T02 pattern (most common for manga/comics)
(r"(?i)T(\d+)", 1),
// Vol 1, Vol. 1, Volume 1
(r"(?i)Vol\.?\s*(\d+)", 1),
(r"(?i)Volume\s*(\d+)", 1),
// #1, #01
(r"#(\d+)", 1),
// - 1, - 01 at the end
(r"-\s*(\d+)\s*$", 1),
];
/// Open an archive once and return (page_count, first_page_bytes).
/// This is more efficient than calling parse_metadata + extract_first_page separately.
pub fn analyze_book(path: &Path, format: BookFormat) -> Result<(i32, Vec<u8>)> {
match format {
BookFormat::Cbz => analyze_cbz(path),
BookFormat::Cbr => analyze_cbr(path),
BookFormat::Pdf => analyze_pdf(path),
}
}
for (pattern, group) in &patterns {
if let Ok(re) = regex::Regex::new(pattern) {
if let Some(caps) = re.captures(filename) {
if let Some(mat) = caps.get(*group) {
// Parse as integer to remove leading zeros
return mat.as_str().parse::<i32>().ok();
}
}
fn analyze_cbz(path: &Path) -> Result<(i32, Vec<u8>)> {
let file = std::fs::File::open(path)
.with_context(|| format!("cannot open cbz: {}", path.display()))?;
let mut archive = zip::ZipArchive::new(file).context("invalid cbz archive")?;
let mut image_names: Vec<String> = Vec::new();
for i in 0..archive.len() {
let entry = archive.by_index(i).context("cannot read cbz entry")?;
let name = entry.name().to_ascii_lowercase();
if is_image_name(&name) {
image_names.push(entry.name().to_string());
}
}
image_names.sort();
let count = image_names.len() as i32;
let first_image = image_names.first().context("no images found in cbz")?;
let mut entry = archive
.by_name(first_image)
.context("cannot read first image")?;
let mut buf = Vec::new();
entry.read_to_end(&mut buf)?;
Ok((count, buf))
}
fn list_cbr_images(path: &Path) -> Result<Vec<String>> {
// Try unrar lb first (fast)
let output = std::process::Command::new("unrar")
.arg("lb")
.arg(path)
.output()
.with_context(|| format!("failed to execute unrar lb for {}", path.display()))?;
if output.status.success() {
let stdout = String::from_utf8_lossy(&output.stdout);
let images: Vec<String> = stdout
.lines()
.filter(|line| is_image_name(&line.to_ascii_lowercase()))
.map(|l| l.to_string())
.collect();
if !images.is_empty() {
return Ok(images);
}
}
None
// Fallback: lsar (from unar package) handles UTF-16BE encoded filenames
let lsar_output = std::process::Command::new("lsar")
.arg(path)
.output()
.with_context(|| format!("failed to execute lsar for {}", path.display()))?;
if !lsar_output.status.success() {
return Err(anyhow::anyhow!(
"both unrar lb and lsar failed for {}",
path.display()
));
}
let stdout = String::from_utf8_lossy(&lsar_output.stdout);
// lsar output: first line is archive info, then one file per line (indented)
let images: Vec<String> = stdout
.lines()
.skip(1) // skip the archive header line
.map(|l| l.trim().to_string())
.filter(|line| is_image_name(&line.to_ascii_lowercase()))
.collect();
Ok(images)
}
#[allow(dead_code)]
fn clean_title(filename: &str) -> String {
// Remove volume patterns from title to clean it up
let cleaned = regex::Regex::new(r"(?i)\s*T\d+\s*")
.ok()
.and_then(|re| Some(re.replace_all(filename, " ").to_string()))
.unwrap_or_else(|| filename.to_string());
fn analyze_cbr(path: &Path) -> Result<(i32, Vec<u8>)> {
let mut image_names = list_cbr_images(path)?;
image_names.sort();
let cleaned = regex::Regex::new(r"(?i)\s*Vol\.?\s*\d+\s*")
.ok()
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
.unwrap_or_else(|| cleaned);
let count = image_names.len() as i32;
if count == 0 {
return Err(anyhow::anyhow!("no images found in cbr: {}", path.display()));
}
let cleaned = regex::Regex::new(r"(?i)\s*Volume\s*\d+\s*")
.ok()
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
.unwrap_or_else(|| cleaned);
let first_name = &image_names[0];
let cleaned = regex::Regex::new(r"#\d+")
.ok()
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
.unwrap_or_else(|| cleaned);
// Try unrar p to extract first image to stdout (faster — no temp dir)
let p_output = std::process::Command::new("unrar")
.args(["p", "-inul"])
.arg(path)
.arg(first_name)
.output();
let cleaned = regex::Regex::new(r"-\s*\d+\s*$")
.ok()
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
.unwrap_or_else(|| cleaned);
match p_output {
Ok(out) if out.status.success() && looks_like_image(&out.stdout) => Ok((count, out.stdout)),
_ => {
// Fallback: full extraction with unar (handles special chars, encoding issues)
let image_bytes = extract_cbr_first_page(path)?;
Ok((count, image_bytes))
}
}
}
// Clean up extra spaces
cleaned.split_whitespace().collect::<Vec<_>>().join(" ")
/// Check image magic bytes to validate that bytes are a real image before decoding.
fn looks_like_image(bytes: &[u8]) -> bool {
if bytes.len() < 12 {
return false;
}
// JPEG: FF D8 FF
if bytes.starts_with(&[0xFF, 0xD8, 0xFF]) {
return true;
}
// PNG: 89 50 4E 47 0D 0A 1A 0A
if bytes.starts_with(&[0x89, 0x50, 0x4E, 0x47]) {
return true;
}
// WebP: RIFF....WEBP
if &bytes[0..4] == b"RIFF" && &bytes[8..12] == b"WEBP" {
return true;
}
false
}
fn analyze_pdf(path: &Path) -> Result<(i32, Vec<u8>)> {
let count = parse_pdf_page_count(path)?;
let image_bytes = extract_pdf_first_page(path)?;
Ok((count, image_bytes))
}
fn parse_cbz_page_count(path: &Path) -> Result<i32> {
@@ -192,26 +303,11 @@ fn parse_cbz_page_count(path: &Path) -> Result<i32> {
}
fn parse_cbr_page_count(path: &Path) -> Result<i32> {
let output = std::process::Command::new("unrar")
.arg("lb")
.arg(path)
.output()
.with_context(|| format!("failed to execute unrar for {}", path.display()))?;
if !output.status.success() {
return Err(anyhow::anyhow!("unrar failed for {}", path.display()));
}
let stdout = String::from_utf8_lossy(&output.stdout);
let count = stdout
.lines()
.filter(|line| is_image_name(&line.to_ascii_lowercase()))
.count() as i32;
Ok(count)
let images = list_cbr_images(path)?;
Ok(images.len() as i32)
}
fn parse_pdf_page_count(path: &Path) -> Result<i32> {
// Use pdfinfo command line tool instead of lopdf for better performance
let output = std::process::Command::new("pdfinfo")
.arg(path)
.output()
@@ -238,6 +334,10 @@ fn parse_pdf_page_count(path: &Path) -> Result<i32> {
}
fn is_image_name(name: &str) -> bool {
// Skip macOS metadata entries (__MACOSX/ prefix or AppleDouble ._* files)
if name.starts_with("__macosx/") || name.contains("/._") || name.starts_with("._") {
return false;
}
name.ends_with(".jpg")
|| name.ends_with(".jpeg")
|| name.ends_with(".png")
@@ -282,7 +382,6 @@ fn extract_cbr_first_page(path: &Path) -> Result<Vec<u8>> {
let tmp_dir = std::env::temp_dir().join(format!("stripstream-cbr-thumb-{}", Uuid::new_v4()));
std::fs::create_dir_all(&tmp_dir).context("cannot create temp dir")?;
// Use env command like the API does
let output = std::process::Command::new("env")
.args(["LC_ALL=en_US.UTF-8", "LANG=en_US.UTF-8", "unar", "-o"])
.arg(&tmp_dir)
@@ -298,7 +397,6 @@ fn extract_cbr_first_page(path: &Path) -> Result<Vec<u8>> {
));
}
// Use WalkDir for recursive search (CBR can have subdirectories)
let mut image_files: Vec<_> = WalkDir::new(&tmp_dir)
.into_iter()
.filter_map(|e| e.ok())
@@ -346,3 +444,33 @@ fn extract_pdf_first_page(path: &Path) -> Result<Vec<u8>> {
let _ = std::fs::remove_dir_all(&tmp_dir);
Ok(data)
}
#[allow(dead_code)]
fn clean_title(filename: &str) -> String {
let cleaned = regex::Regex::new(r"(?i)\s*T\d+\s*")
.ok()
.map(|re| re.replace_all(filename, " ").to_string())
.unwrap_or_else(|| filename.to_string());
let cleaned = regex::Regex::new(r"(?i)\s*Vol\.?\s*\d+\s*")
.ok()
.map(|re| re.replace_all(&cleaned, " ").to_string())
.unwrap_or(cleaned);
let cleaned = regex::Regex::new(r"(?i)\s*Volume\s*\d+\s*")
.ok()
.map(|re| re.replace_all(&cleaned, " ").to_string())
.unwrap_or(cleaned);
let cleaned = regex::Regex::new(r"#\d+")
.ok()
.map(|re| re.replace_all(&cleaned, " ").to_string())
.unwrap_or(cleaned);
let cleaned = regex::Regex::new(r"-\s*\d+\s*$")
.ok()
.map(|re| re.replace_all(&cleaned, " ").to_string())
.unwrap_or(cleaned);
cleaned.split_whitespace().collect::<Vec<_>>().join(" ")
}

View File

@@ -18,7 +18,7 @@ services:
meilisearch:
image: getmeili/meilisearch:v1.12
env_file:
- ../.env
- .env
ports:
- "7700:7700"
volumes:
@@ -39,7 +39,7 @@ services:
POSTGRES_PASSWORD: stripstream
POSTGRES_DB: stripstream
volumes:
- ./migrations:/migrations:ro
- ./infra/migrations:/migrations:ro
command:
[
"sh",
@@ -49,15 +49,15 @@ services:
api:
build:
context: ..
context: .
dockerfile: apps/api/Dockerfile
env_file:
- ../.env
- .env
ports:
- "7080:8080"
- "7080:7080"
volumes:
- ${LIBRARIES_HOST_PATH:-../libraries}:/libraries
- ${THUMBNAILS_HOST_PATH:-../data/thumbnails}:/data/thumbnails
- ${LIBRARIES_HOST_PATH:-./libraries}:/libraries
- ${THUMBNAILS_HOST_PATH:-./data/thumbnails}:/data/thumbnails
depends_on:
migrate:
condition: service_completed_successfully
@@ -66,22 +66,22 @@ services:
meilisearch:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8080/health"]
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:7080/health"]
interval: 10s
timeout: 5s
retries: 5
indexer:
build:
context: ..
context: .
dockerfile: apps/indexer/Dockerfile
env_file:
- ../.env
- .env
ports:
- "7081:8081"
- "7081:7081"
volumes:
- ${LIBRARIES_HOST_PATH:-../libraries}:/libraries
- ${THUMBNAILS_HOST_PATH:-../data/thumbnails}:/data/thumbnails
- ${LIBRARIES_HOST_PATH:-./libraries}:/libraries
- ${THUMBNAILS_HOST_PATH:-./data/thumbnails}:/data/thumbnails
depends_on:
migrate:
condition: service_completed_successfully
@@ -90,27 +90,27 @@ services:
meilisearch:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8081/health"]
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:7081/health"]
interval: 10s
timeout: 5s
retries: 5
backoffice:
build:
context: ..
context: .
dockerfile: apps/backoffice/Dockerfile
env_file:
- ../.env
- .env
environment:
- PORT=8082
- PORT=7082
- HOST=0.0.0.0
ports:
- "7082:8082"
- "7082:7082"
depends_on:
api:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8082/health"]
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:7082/health"]
interval: 10s
timeout: 5s
retries: 5

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
BASE_API="${BASE_API:-http://127.0.0.1:8080}"
BASE_API="${BASE_API:-http://127.0.0.1:7080}"
TOKEN="${API_TOKEN:-stripstream-dev-bootstrap-token}"
measure() {

View File

@@ -0,0 +1,8 @@
CREATE TABLE directory_mtimes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
library_id UUID NOT NULL REFERENCES libraries(id) ON DELETE CASCADE,
dir_path TEXT NOT NULL,
mtime TIMESTAMPTZ NOT NULL,
UNIQUE(library_id, dir_path)
);
CREATE INDEX idx_directory_mtimes_library ON directory_mtimes(library_id);

View File

@@ -1,41 +1,129 @@
#!/usr/bin/env bash
set -euo pipefail
BASE_API="${BASE_API:-http://127.0.0.1:8080}"
BASE_INDEXER="${BASE_INDEXER:-http://127.0.0.1:8081}"
BASE_BACKOFFICE="${BASE_BACKOFFICE:-${BASE_ADMIN:-http://127.0.0.1:8082}}"
BASE_API="${BASE_API:-http://127.0.0.1:7080}"
BASE_INDEXER="${BASE_INDEXER:-http://127.0.0.1:7081}"
BASE_BACKOFFICE="${BASE_BACKOFFICE:-${BASE_ADMIN:-http://127.0.0.1:7082}}"
TOKEN="${API_TOKEN:-stripstream-dev-bootstrap-token}"
# Max seconds to wait for a job to finish
JOB_TIMEOUT="${JOB_TIMEOUT:-120}"
# ─── helpers ────────────────────────────────────────────────────────────────
auth() { curl -fsS -H "Authorization: Bearer $TOKEN" "$@"; }
# Wait for a job (by id) to reach status success or failed.
wait_job() {
local job_id="$1"
local label="${2:-job}"
local waited=0
while true; do
local status
status="$(auth "$BASE_API/index/jobs/$job_id" | python3 -c "import sys,json; print(json.load(sys.stdin).get('status',''))")"
case "$status" in
success) echo "[smoke] $label finished: success"; return 0 ;;
failed) echo "[smoke] $label finished: FAILED"; return 1 ;;
cancelled) echo "[smoke] $label finished: cancelled"; return 1 ;;
esac
if [ "$waited" -ge "$JOB_TIMEOUT" ]; then
echo "[smoke] $label timed out after ${JOB_TIMEOUT}s (last status: $status)"; return 1
fi
sleep 2; waited=$((waited + 2))
done
}
# ─── health ──────────────────────────────────────────────────────────────────
echo "[smoke] health checks"
curl -fsS "$BASE_API/health" >/dev/null
curl -fsS "$BASE_API/ready" >/dev/null
curl -fsS "$BASE_API/health" >/dev/null
curl -fsS "$BASE_API/ready" >/dev/null
curl -fsS "$BASE_INDEXER/health" >/dev/null
curl -fsS "$BASE_INDEXER/ready" >/dev/null
curl -fsS "$BASE_BACKOFFICE/health" >/dev/null
# ─── libraries ───────────────────────────────────────────────────────────────
echo "[smoke] list libraries"
curl -fsS -H "Authorization: Bearer $TOKEN" "$BASE_API/libraries" >/dev/null
auth "$BASE_API/libraries" >/dev/null
echo "[smoke] queue rebuild"
curl -fsS -X POST -H "Authorization: Bearer $TOKEN" "$BASE_API/index/rebuild" >/dev/null
sleep 2
# ─── full rebuild (2-phase: discovery + analysis) ────────────────────────────
echo "[smoke] list books and optional page fetch"
BOOKS_JSON="$(curl -fsS -H "Authorization: Bearer $TOKEN" "$BASE_API/books")"
BOOK_ID="$(BOOKS_JSON="$BOOKS_JSON" python3 - <<'PY'
import json
import os
echo "[smoke] queue full rebuild"
REBUILD_JOB_ID="$(auth -X POST "$BASE_API/index/rebuild" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")"
echo "[smoke] rebuild job id: $REBUILD_JOB_ID"
wait_job "$REBUILD_JOB_ID" "rebuild"
# ─── verify books have page_count + thumbnail after analysis phase ────────────
echo "[smoke] verify books metadata (page_count + thumbnail)"
BOOKS_JSON="$(auth "$BASE_API/books")"
export BOOKS_JSON
python3 - <<'PY'
import json, os, sys
payload = json.loads(os.environ.get("BOOKS_JSON", "{}"))
items = payload.get("items") or []
if not items:
print("[smoke] no books found — skipping metadata check")
sys.exit(0)
missing_page_count = [b["id"] for b in items if not b.get("page_count")]
missing_thumbnail = [b["id"] for b in items if not b.get("thumbnail_url")]
if missing_page_count:
print(f"[smoke] WARN: {len(missing_page_count)} book(s) still missing page_count")
if missing_thumbnail:
print(f"[smoke] WARN: {len(missing_thumbnail)} book(s) still missing thumbnail")
print(f"[smoke] {len(items)} books, {len(items)-len(missing_page_count)} with page_count, {len(items)-len(missing_thumbnail)} with thumbnail")
PY
# ─── page fetch ──────────────────────────────────────────────────────────────
BOOK_ID="$(python3 - <<'PY'
import json, os
items = json.loads(os.environ.get("BOOKS_JSON", "{}")).get("items") or []
print(items[0]["id"] if items else "")
PY
)"
if [ -n "$BOOK_ID" ]; then
curl -fsS -H "Authorization: Bearer $TOKEN" "$BASE_API/books/$BOOK_ID/pages/1?format=webp&quality=80&width=1080" >/dev/null
echo "[smoke] fetch page 1 for book $BOOK_ID"
auth "$BASE_API/books/$BOOK_ID/pages/1?format=webp&quality=80&width=1080" >/dev/null
echo "[smoke] fetch thumbnail for book $BOOK_ID"
auth "$BASE_API/books/$BOOK_ID/thumbnail" >/dev/null
fi
# ─── thumbnail rebuild (handled by indexer, not API) ─────────────────────────
echo "[smoke] thumbnail rebuild job"
THUMB_REBUILD_ID="$(auth -X POST -H "Content-Type: application/json" -d '{}' "$BASE_API/index/thumbnails/rebuild" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")"
echo "[smoke] thumbnail rebuild job id: $THUMB_REBUILD_ID"
wait_job "$THUMB_REBUILD_ID" "thumbnail_rebuild"
# ─── thumbnail regenerate ────────────────────────────────────────────────────
echo "[smoke] thumbnail regenerate job"
THUMB_REGEN_ID="$(auth -X POST -H "Content-Type: application/json" -d '{}' "$BASE_API/index/thumbnails/regenerate" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")"
echo "[smoke] thumbnail regenerate job id: $THUMB_REGEN_ID"
wait_job "$THUMB_REGEN_ID" "thumbnail_regenerate"
# ─── route checkup supprimée (doit retourner 404) ────────────────────────────
echo "[smoke] /index/jobs/:id/thumbnails/checkup must be gone (404)"
HTTP_CODE="$(curl -s -o /dev/null -w "%{http_code}" -X POST \
-H "Authorization: Bearer $TOKEN" \
"$BASE_API/index/jobs/$REBUILD_JOB_ID/thumbnails/checkup")"
if [ "$HTTP_CODE" = "404" ]; then
echo "[smoke] checkup route correctly returns 404"
else
echo "[smoke] FAIL: checkup route returned $HTTP_CODE (expected 404)"
exit 1
fi
# ─── metrics ─────────────────────────────────────────────────────────────────
echo "[smoke] metrics"
curl -fsS "$BASE_API/metrics" >/dev/null

20
openspec/config.yaml Normal file
View File

@@ -0,0 +1,20 @@
schema: spec-driven
# Project context (optional)
# This is shown to AI when creating artifacts.
# Add your tech stack, conventions, style guides, domain knowledge, etc.
# Example:
# context: |
# Tech stack: TypeScript, React, Node.js
# We use conventional commits
# Domain: e-commerce platform
# Per-artifact rules (optional)
# Add custom rules for specific artifacts.
# Example:
# rules:
# proposal:
# - Keep proposals under 500 words
# - Always include a "Non-goals" section
# tasks:
# - Break tasks into chunks of max 2 hours