Compare commits
10 Commits
feat/thumb
...
473e849dfa
| Author | SHA1 | Date | |
|---|---|---|---|
| 473e849dfa | |||
| cfc896e92f | |||
| 36af34443e | |||
| 85cad1a7e7 | |||
| 0f5094575a | |||
| 131c50b1a1 | |||
| 6d4c400017 | |||
| 539dc77d57 | |||
| 9c7120c3dc | |||
| b1844a4f01 |
152
.claude/commands/opsx/apply.md
Normal file
152
.claude/commands/opsx/apply.md
Normal file
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: "OPSX: Apply"
|
||||
description: Implement tasks from an OpenSpec change (Experimental)
|
||||
category: Workflow
|
||||
tags: [workflow, artifacts, experimental]
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue`
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! You can archive this change with `/opsx:archive`.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
157
.claude/commands/opsx/archive.md
Normal file
157
.claude/commands/opsx/archive.md
Normal file
@@ -0,0 +1,157 @@
|
||||
---
|
||||
name: "OPSX: Archive"
|
||||
description: Archive a completed change in the experimental workflow
|
||||
category: Workflow
|
||||
tags: [workflow, archive, experimental]
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Spec sync status (synced / sync skipped / no delta specs)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success (No Delta Specs)**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** No delta specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success With Warnings**
|
||||
|
||||
```
|
||||
## Archive Complete (with warnings)
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** Sync skipped (user chose to skip)
|
||||
|
||||
**Warnings:**
|
||||
- Archived with 2 incomplete artifacts
|
||||
- Archived with 3 incomplete tasks
|
||||
- Delta spec sync was skipped (user chose to skip)
|
||||
|
||||
Review the archive if this was not intentional.
|
||||
```
|
||||
|
||||
**Output On Error (Archive Exists)**
|
||||
|
||||
```
|
||||
## Archive Failed
|
||||
|
||||
**Change:** <change-name>
|
||||
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
|
||||
Target archive directory already exists.
|
||||
|
||||
**Options:**
|
||||
1. Rename the existing archive
|
||||
2. Delete the existing archive if it's a duplicate
|
||||
3. Wait until a different date to archive
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
173
.claude/commands/opsx/explore.md
Normal file
173
.claude/commands/opsx/explore.md
Normal file
@@ -0,0 +1,173 @@
|
||||
---
|
||||
name: "OPSX: Explore"
|
||||
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
|
||||
category: Workflow
|
||||
tags: [workflow, explore, experimental, thinking]
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
|
||||
- A vague idea: "real-time collaboration"
|
||||
- A specific problem: "the auth system is getting unwieldy"
|
||||
- A change name: "add-dark-mode" (to explore in context of that change)
|
||||
- A comparison: "postgres vs sqlite for this"
|
||||
- Nothing (just enter explore mode)
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
If the user mentioned a specific change name, read its artifacts for context.
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
106
.claude/commands/opsx/propose.md
Normal file
106
.claude/commands/opsx/propose.md
Normal file
@@ -0,0 +1,106 @@
|
||||
---
|
||||
name: "OPSX: Propose"
|
||||
description: Propose a new change - create it and generate all artifacts in one step
|
||||
category: Workflow
|
||||
tags: [workflow, artifacts, experimental]
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx:apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx:apply` to start implementing."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
156
.claude/skills/openspec-apply-change/SKILL.md
Normal file
156
.claude/skills/openspec-apply-change/SKILL.md
Normal file
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: openspec-apply-change
|
||||
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! Ready to archive this change.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
114
.claude/skills/openspec-archive-change/SKILL.md
Normal file
114
.claude/skills/openspec-archive-change/SKILL.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
name: openspec-archive-change
|
||||
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Whether specs were synced (if applicable)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use openspec-sync-specs approach (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
288
.claude/skills/openspec-explore/SKILL.md
Normal file
288
.claude/skills/openspec-explore/SKILL.md
Normal file
@@ -0,0 +1,288 @@
|
||||
---
|
||||
name: openspec-explore
|
||||
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Handling Different Entry Points
|
||||
|
||||
**User brings a vague idea:**
|
||||
```
|
||||
User: I'm thinking about adding real-time collaboration
|
||||
|
||||
You: Real-time collab is a big space. Let me think about this...
|
||||
|
||||
COLLABORATION SPECTRUM
|
||||
════════════════════════════════════════════
|
||||
|
||||
Awareness Coordination Sync
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌────────┐ ┌────────┐
|
||||
│Presence│ │Cursors │ │ CRDT │
|
||||
│ "3 │ │ Multi │ │Conflict│
|
||||
│online" │ │ select │ │ free │
|
||||
└────────┘ └────────┘ └────────┘
|
||||
│ │ │
|
||||
trivial moderate complex
|
||||
|
||||
Where's your head at?
|
||||
```
|
||||
|
||||
**User brings a specific problem:**
|
||||
```
|
||||
User: The auth system is a mess
|
||||
|
||||
You: [reads codebase]
|
||||
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ CURRENT AUTH FLOW │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Google │ │ GitHub │ │ Email │
|
||||
│ OAuth │ │ OAuth │ │ Magic │
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
└─────────────┼─────────────┘
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Session │
|
||||
└─────┬─────┘
|
||||
│
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Perms │
|
||||
└───────────┘
|
||||
|
||||
I see three tangles. Which one's burning?
|
||||
```
|
||||
|
||||
**User is stuck mid-implementation:**
|
||||
```
|
||||
User: /opsx:explore add-auth-system
|
||||
The OAuth integration is more complex than expected
|
||||
|
||||
You: [reads change artifacts]
|
||||
|
||||
You're on task 4: "Implement OAuth flow"
|
||||
|
||||
Let me trace what's involved...
|
||||
|
||||
[draws diagram, explores options, suggests paths]
|
||||
|
||||
Want to update the design to reflect this?
|
||||
Or add a spike task to investigate?
|
||||
```
|
||||
|
||||
**User wants to compare options:**
|
||||
```
|
||||
User: Should we use Postgres or SQLite?
|
||||
|
||||
You: Generic answer is boring. What's the context?
|
||||
|
||||
User: A CLI tool that tracks local dev environments
|
||||
|
||||
You: That changes everything.
|
||||
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ CLI TOOL DATA STORAGE │
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
||||
Key constraints:
|
||||
• No daemon running
|
||||
• Must work offline
|
||||
• Single user
|
||||
|
||||
SQLite Postgres
|
||||
Deployment embedded ✓ needs server ✗
|
||||
Offline yes ✓ no ✗
|
||||
Single file yes ✓ no ✗
|
||||
|
||||
SQLite. Not even close.
|
||||
|
||||
Unless... is there a sync component?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When it feels like things are crystallizing, you might summarize:
|
||||
|
||||
```
|
||||
## What We Figured Out
|
||||
|
||||
**The problem**: [crystallized understanding]
|
||||
|
||||
**The approach**: [if one emerged]
|
||||
|
||||
**Open questions**: [if any remain]
|
||||
|
||||
**Next steps** (if ready):
|
||||
- Create a change proposal
|
||||
- Keep exploring: just keep talking
|
||||
```
|
||||
|
||||
But this summary is optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
110
.claude/skills/openspec-propose/SKILL.md
Normal file
110
.claude/skills/openspec-propose/SKILL.md
Normal file
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: openspec-propose
|
||||
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx:apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no clear input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
156
.codex/skills/openspec-apply-change/SKILL.md
Normal file
156
.codex/skills/openspec-apply-change/SKILL.md
Normal file
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: openspec-apply-change
|
||||
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! Ready to archive this change.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
114
.codex/skills/openspec-archive-change/SKILL.md
Normal file
114
.codex/skills/openspec-archive-change/SKILL.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
name: openspec-archive-change
|
||||
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Whether specs were synced (if applicable)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use openspec-sync-specs approach (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
288
.codex/skills/openspec-explore/SKILL.md
Normal file
288
.codex/skills/openspec-explore/SKILL.md
Normal file
@@ -0,0 +1,288 @@
|
||||
---
|
||||
name: openspec-explore
|
||||
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Handling Different Entry Points
|
||||
|
||||
**User brings a vague idea:**
|
||||
```
|
||||
User: I'm thinking about adding real-time collaboration
|
||||
|
||||
You: Real-time collab is a big space. Let me think about this...
|
||||
|
||||
COLLABORATION SPECTRUM
|
||||
════════════════════════════════════════════
|
||||
|
||||
Awareness Coordination Sync
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌────────┐ ┌────────┐
|
||||
│Presence│ │Cursors │ │ CRDT │
|
||||
│ "3 │ │ Multi │ │Conflict│
|
||||
│online" │ │ select │ │ free │
|
||||
└────────┘ └────────┘ └────────┘
|
||||
│ │ │
|
||||
trivial moderate complex
|
||||
|
||||
Where's your head at?
|
||||
```
|
||||
|
||||
**User brings a specific problem:**
|
||||
```
|
||||
User: The auth system is a mess
|
||||
|
||||
You: [reads codebase]
|
||||
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ CURRENT AUTH FLOW │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Google │ │ GitHub │ │ Email │
|
||||
│ OAuth │ │ OAuth │ │ Magic │
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
└─────────────┼─────────────┘
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Session │
|
||||
└─────┬─────┘
|
||||
│
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Perms │
|
||||
└───────────┘
|
||||
|
||||
I see three tangles. Which one's burning?
|
||||
```
|
||||
|
||||
**User is stuck mid-implementation:**
|
||||
```
|
||||
User: /opsx:explore add-auth-system
|
||||
The OAuth integration is more complex than expected
|
||||
|
||||
You: [reads change artifacts]
|
||||
|
||||
You're on task 4: "Implement OAuth flow"
|
||||
|
||||
Let me trace what's involved...
|
||||
|
||||
[draws diagram, explores options, suggests paths]
|
||||
|
||||
Want to update the design to reflect this?
|
||||
Or add a spike task to investigate?
|
||||
```
|
||||
|
||||
**User wants to compare options:**
|
||||
```
|
||||
User: Should we use Postgres or SQLite?
|
||||
|
||||
You: Generic answer is boring. What's the context?
|
||||
|
||||
User: A CLI tool that tracks local dev environments
|
||||
|
||||
You: That changes everything.
|
||||
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ CLI TOOL DATA STORAGE │
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
||||
Key constraints:
|
||||
• No daemon running
|
||||
• Must work offline
|
||||
• Single user
|
||||
|
||||
SQLite Postgres
|
||||
Deployment embedded ✓ needs server ✗
|
||||
Offline yes ✓ no ✗
|
||||
Single file yes ✓ no ✗
|
||||
|
||||
SQLite. Not even close.
|
||||
|
||||
Unless... is there a sync component?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When it feels like things are crystallizing, you might summarize:
|
||||
|
||||
```
|
||||
## What We Figured Out
|
||||
|
||||
**The problem**: [crystallized understanding]
|
||||
|
||||
**The approach**: [if one emerged]
|
||||
|
||||
**Open questions**: [if any remain]
|
||||
|
||||
**Next steps** (if ready):
|
||||
- Create a change proposal
|
||||
- Keep exploring: just keep talking
|
||||
```
|
||||
|
||||
But this summary is optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
110
.codex/skills/openspec-propose/SKILL.md
Normal file
110
.codex/skills/openspec-propose/SKILL.md
Normal file
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: openspec-propose
|
||||
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx:apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no clear input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
152
.cursor/commands/opsx-apply.md
Normal file
152
.cursor/commands/opsx-apply.md
Normal file
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: /opsx-apply
|
||||
id: opsx-apply
|
||||
category: Workflow
|
||||
description: Implement tasks from an OpenSpec change (Experimental)
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue`
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! You can archive this change with `/opsx:archive`.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
157
.cursor/commands/opsx-archive.md
Normal file
157
.cursor/commands/opsx-archive.md
Normal file
@@ -0,0 +1,157 @@
|
||||
---
|
||||
name: /opsx-archive
|
||||
id: opsx-archive
|
||||
category: Workflow
|
||||
description: Archive a completed change in the experimental workflow
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Spec sync status (synced / sync skipped / no delta specs)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success (No Delta Specs)**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** No delta specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success With Warnings**
|
||||
|
||||
```
|
||||
## Archive Complete (with warnings)
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** Sync skipped (user chose to skip)
|
||||
|
||||
**Warnings:**
|
||||
- Archived with 2 incomplete artifacts
|
||||
- Archived with 3 incomplete tasks
|
||||
- Delta spec sync was skipped (user chose to skip)
|
||||
|
||||
Review the archive if this was not intentional.
|
||||
```
|
||||
|
||||
**Output On Error (Archive Exists)**
|
||||
|
||||
```
|
||||
## Archive Failed
|
||||
|
||||
**Change:** <change-name>
|
||||
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
|
||||
Target archive directory already exists.
|
||||
|
||||
**Options:**
|
||||
1. Rename the existing archive
|
||||
2. Delete the existing archive if it's a duplicate
|
||||
3. Wait until a different date to archive
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
173
.cursor/commands/opsx-explore.md
Normal file
173
.cursor/commands/opsx-explore.md
Normal file
@@ -0,0 +1,173 @@
|
||||
---
|
||||
name: /opsx-explore
|
||||
id: opsx-explore
|
||||
category: Workflow
|
||||
description: "Enter explore mode - think through ideas, investigate problems, clarify requirements"
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be:
|
||||
- A vague idea: "real-time collaboration"
|
||||
- A specific problem: "the auth system is getting unwieldy"
|
||||
- A change name: "add-dark-mode" (to explore in context of that change)
|
||||
- A comparison: "postgres vs sqlite for this"
|
||||
- Nothing (just enter explore mode)
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
If the user mentioned a specific change name, read its artifacts for context.
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
106
.cursor/commands/opsx-propose.md
Normal file
106
.cursor/commands/opsx-propose.md
Normal file
@@ -0,0 +1,106 @@
|
||||
---
|
||||
name: /opsx-propose
|
||||
id: opsx-propose
|
||||
category: Workflow
|
||||
description: Propose a new change - create it and generate all artifacts in one step
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx:apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx:apply` to start implementing."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
156
.cursor/skills/openspec-apply-change/SKILL.md
Normal file
156
.cursor/skills/openspec-apply-change/SKILL.md
Normal file
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: openspec-apply-change
|
||||
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! Ready to archive this change.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
114
.cursor/skills/openspec-archive-change/SKILL.md
Normal file
114
.cursor/skills/openspec-archive-change/SKILL.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
name: openspec-archive-change
|
||||
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Whether specs were synced (if applicable)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use openspec-sync-specs approach (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
288
.cursor/skills/openspec-explore/SKILL.md
Normal file
288
.cursor/skills/openspec-explore/SKILL.md
Normal file
@@ -0,0 +1,288 @@
|
||||
---
|
||||
name: openspec-explore
|
||||
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Handling Different Entry Points
|
||||
|
||||
**User brings a vague idea:**
|
||||
```
|
||||
User: I'm thinking about adding real-time collaboration
|
||||
|
||||
You: Real-time collab is a big space. Let me think about this...
|
||||
|
||||
COLLABORATION SPECTRUM
|
||||
════════════════════════════════════════════
|
||||
|
||||
Awareness Coordination Sync
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌────────┐ ┌────────┐
|
||||
│Presence│ │Cursors │ │ CRDT │
|
||||
│ "3 │ │ Multi │ │Conflict│
|
||||
│online" │ │ select │ │ free │
|
||||
└────────┘ └────────┘ └────────┘
|
||||
│ │ │
|
||||
trivial moderate complex
|
||||
|
||||
Where's your head at?
|
||||
```
|
||||
|
||||
**User brings a specific problem:**
|
||||
```
|
||||
User: The auth system is a mess
|
||||
|
||||
You: [reads codebase]
|
||||
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ CURRENT AUTH FLOW │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Google │ │ GitHub │ │ Email │
|
||||
│ OAuth │ │ OAuth │ │ Magic │
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
└─────────────┼─────────────┘
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Session │
|
||||
└─────┬─────┘
|
||||
│
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Perms │
|
||||
└───────────┘
|
||||
|
||||
I see three tangles. Which one's burning?
|
||||
```
|
||||
|
||||
**User is stuck mid-implementation:**
|
||||
```
|
||||
User: /opsx:explore add-auth-system
|
||||
The OAuth integration is more complex than expected
|
||||
|
||||
You: [reads change artifacts]
|
||||
|
||||
You're on task 4: "Implement OAuth flow"
|
||||
|
||||
Let me trace what's involved...
|
||||
|
||||
[draws diagram, explores options, suggests paths]
|
||||
|
||||
Want to update the design to reflect this?
|
||||
Or add a spike task to investigate?
|
||||
```
|
||||
|
||||
**User wants to compare options:**
|
||||
```
|
||||
User: Should we use Postgres or SQLite?
|
||||
|
||||
You: Generic answer is boring. What's the context?
|
||||
|
||||
User: A CLI tool that tracks local dev environments
|
||||
|
||||
You: That changes everything.
|
||||
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ CLI TOOL DATA STORAGE │
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
||||
Key constraints:
|
||||
• No daemon running
|
||||
• Must work offline
|
||||
• Single user
|
||||
|
||||
SQLite Postgres
|
||||
Deployment embedded ✓ needs server ✗
|
||||
Offline yes ✓ no ✗
|
||||
Single file yes ✓ no ✗
|
||||
|
||||
SQLite. Not even close.
|
||||
|
||||
Unless... is there a sync component?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When it feels like things are crystallizing, you might summarize:
|
||||
|
||||
```
|
||||
## What We Figured Out
|
||||
|
||||
**The problem**: [crystallized understanding]
|
||||
|
||||
**The approach**: [if one emerged]
|
||||
|
||||
**Open questions**: [if any remain]
|
||||
|
||||
**Next steps** (if ready):
|
||||
- Create a change proposal
|
||||
- Keep exploring: just keep talking
|
||||
```
|
||||
|
||||
But this summary is optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
110
.cursor/skills/openspec-propose/SKILL.md
Normal file
110
.cursor/skills/openspec-propose/SKILL.md
Normal file
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: openspec-propose
|
||||
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx:apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no clear input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
12
.env.example
12
.env.example
@@ -21,11 +21,11 @@ API_BOOTSTRAP_TOKEN=change-me-in-production
|
||||
# =============================================================================
|
||||
|
||||
# API Service
|
||||
API_LISTEN_ADDR=0.0.0.0:8080
|
||||
API_BASE_URL=http://api:8080
|
||||
API_LISTEN_ADDR=0.0.0.0:7080
|
||||
API_BASE_URL=http://api:7080
|
||||
|
||||
# Indexer Service
|
||||
INDEXER_LISTEN_ADDR=0.0.0.0:8081
|
||||
INDEXER_LISTEN_ADDR=0.0.0.0:7081
|
||||
INDEXER_SCAN_INTERVAL_SECONDS=5
|
||||
|
||||
# Meilisearch Search Engine
|
||||
@@ -56,8 +56,8 @@ THUMBNAILS_HOST_PATH=../data/thumbnails
|
||||
# Port Configuration
|
||||
# =============================================================================
|
||||
# To change ports, edit docker-compose.yml directly:
|
||||
# - API: change "7080:8080" to "YOUR_PORT:8080"
|
||||
# - Indexer: change "7081:8081" to "YOUR_PORT:8081"
|
||||
# - Backoffice: change "7082:8082" to "YOUR_PORT:8082"
|
||||
# - API: change "7080:7080" to "YOUR_PORT:7080"
|
||||
# - Indexer: change "7081:7081" to "YOUR_PORT:7081"
|
||||
# - Backoffice: change "7082:7082" to "YOUR_PORT:7082"
|
||||
# - Meilisearch: change "7700:7700" to "YOUR_PORT:7700"
|
||||
# - PostgreSQL: change "6432:5432" to "YOUR_PORT:5432"
|
||||
|
||||
149
.opencode/command/opsx-apply.md
Normal file
149
.opencode/command/opsx-apply.md
Normal file
@@ -0,0 +1,149 @@
|
||||
---
|
||||
description: Implement tasks from an OpenSpec change (Experimental)
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name (e.g., `/opsx-apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx-continue`
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! You can archive this change with `/opsx-archive`.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
154
.opencode/command/opsx-archive.md
Normal file
154
.opencode/command/opsx-archive.md
Normal file
@@ -0,0 +1,154 @@
|
||||
---
|
||||
description: Archive a completed change in the experimental workflow
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name after `/opsx-archive` (e.g., `/opsx-archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Prompt user for confirmation to continue
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Spec sync status (synced / sync skipped / no delta specs)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success (No Delta Specs)**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** No delta specs
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Output On Success With Warnings**
|
||||
|
||||
```
|
||||
## Archive Complete (with warnings)
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** Sync skipped (user chose to skip)
|
||||
|
||||
**Warnings:**
|
||||
- Archived with 2 incomplete artifacts
|
||||
- Archived with 3 incomplete tasks
|
||||
- Delta spec sync was skipped (user chose to skip)
|
||||
|
||||
Review the archive if this was not intentional.
|
||||
```
|
||||
|
||||
**Output On Error (Archive Exists)**
|
||||
|
||||
```
|
||||
## Archive Failed
|
||||
|
||||
**Change:** <change-name>
|
||||
**Target:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
|
||||
Target archive directory already exists.
|
||||
|
||||
**Options:**
|
||||
1. Rename the existing archive
|
||||
2. Delete the existing archive if it's a duplicate
|
||||
3. Wait until a different date to archive
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use the Skill tool to invoke `openspec-sync-specs` (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
170
.opencode/command/opsx-explore.md
Normal file
170
.opencode/command/opsx-explore.md
Normal file
@@ -0,0 +1,170 @@
|
||||
---
|
||||
description: Enter explore mode - think through ideas, investigate problems, clarify requirements
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
**Input**: The argument after `/opsx-explore` is whatever the user wants to think about. Could be:
|
||||
- A vague idea: "real-time collaboration"
|
||||
- A specific problem: "the auth system is getting unwieldy"
|
||||
- A change name: "add-dark-mode" (to explore in context of that change)
|
||||
- A comparison: "postgres vs sqlite for this"
|
||||
- Nothing (just enter explore mode)
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
If the user mentioned a specific change name, read its artifacts for context.
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
103
.opencode/command/opsx-propose.md
Normal file
103
.opencode/command/opsx-propose.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
description: Propose a new change - create it and generate all artifacts in one step
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx-apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The argument after `/opsx-propose` is the change name (kebab-case), OR a description of what the user wants to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx-apply` to start implementing."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
156
.opencode/skills/openspec-apply-change/SKILL.md
Normal file
156
.opencode/skills/openspec-apply-change/SKILL.md
Normal file
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: openspec-apply-change
|
||||
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Implement tasks from an OpenSpec change.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **Select the change**
|
||||
|
||||
If a name is provided, use it. Otherwise:
|
||||
- Infer from conversation context if the user mentioned a change
|
||||
- Auto-select if only one active change exists
|
||||
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
|
||||
|
||||
Always announce: "Using change: <name>" and how to override (e.g., `/opsx-apply <other>`).
|
||||
|
||||
2. **Check status to understand the schema**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used (e.g., "spec-driven")
|
||||
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
|
||||
|
||||
3. **Get apply instructions**
|
||||
|
||||
```bash
|
||||
openspec instructions apply --change "<name>" --json
|
||||
```
|
||||
|
||||
This returns:
|
||||
- Context file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
|
||||
- Progress (total, complete, remaining)
|
||||
- Task list with status
|
||||
- Dynamic instruction based on current state
|
||||
|
||||
**Handle states:**
|
||||
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
|
||||
- If `state: "all_done"`: congratulate, suggest archive
|
||||
- Otherwise: proceed to implementation
|
||||
|
||||
4. **Read context files**
|
||||
|
||||
Read the files listed in `contextFiles` from the apply instructions output.
|
||||
The files depend on the schema being used:
|
||||
- **spec-driven**: proposal, specs, design, tasks
|
||||
- Other schemas: follow the contextFiles from CLI output
|
||||
|
||||
5. **Show current progress**
|
||||
|
||||
Display:
|
||||
- Schema being used
|
||||
- Progress: "N/M tasks complete"
|
||||
- Remaining tasks overview
|
||||
- Dynamic instruction from CLI
|
||||
|
||||
6. **Implement tasks (loop until done or blocked)**
|
||||
|
||||
For each pending task:
|
||||
- Show which task is being worked on
|
||||
- Make the code changes required
|
||||
- Keep changes minimal and focused
|
||||
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
|
||||
- Continue to next task
|
||||
|
||||
**Pause if:**
|
||||
- Task is unclear → ask for clarification
|
||||
- Implementation reveals a design issue → suggest updating artifacts
|
||||
- Error or blocker encountered → report and wait for guidance
|
||||
- User interrupts
|
||||
|
||||
7. **On completion or pause, show status**
|
||||
|
||||
Display:
|
||||
- Tasks completed this session
|
||||
- Overall progress: "N/M tasks complete"
|
||||
- If all done: suggest archive
|
||||
- If paused: explain why and wait for guidance
|
||||
|
||||
**Output During Implementation**
|
||||
|
||||
```
|
||||
## Implementing: <change-name> (schema: <schema-name>)
|
||||
|
||||
Working on task 3/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
|
||||
Working on task 4/7: <task description>
|
||||
[...implementation happening...]
|
||||
✓ Task complete
|
||||
```
|
||||
|
||||
**Output On Completion**
|
||||
|
||||
```
|
||||
## Implementation Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 7/7 tasks complete ✓
|
||||
|
||||
### Completed This Session
|
||||
- [x] Task 1
|
||||
- [x] Task 2
|
||||
...
|
||||
|
||||
All tasks complete! Ready to archive this change.
|
||||
```
|
||||
|
||||
**Output On Pause (Issue Encountered)**
|
||||
|
||||
```
|
||||
## Implementation Paused
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Progress:** 4/7 tasks complete
|
||||
|
||||
### Issue Encountered
|
||||
<description of the issue>
|
||||
|
||||
**Options:**
|
||||
1. <option 1>
|
||||
2. <option 2>
|
||||
3. Other approach
|
||||
|
||||
What would you like to do?
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Keep going through tasks until done or blocked
|
||||
- Always read context files before starting (from the apply instructions output)
|
||||
- If task is ambiguous, pause and ask before implementing
|
||||
- If implementation reveals issues, pause and suggest artifact updates
|
||||
- Keep code changes minimal and scoped to each task
|
||||
- Update task checkbox immediately after completing each task
|
||||
- Pause on errors, blockers, or unclear requirements - don't guess
|
||||
- Use contextFiles from CLI output, don't assume specific file names
|
||||
|
||||
**Fluid Workflow Integration**
|
||||
|
||||
This skill supports the "actions on a change" model:
|
||||
|
||||
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
|
||||
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly
|
||||
114
.opencode/skills/openspec-archive-change/SKILL.md
Normal file
114
.opencode/skills/openspec-archive-change/SKILL.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
name: openspec-archive-change
|
||||
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Archive a completed change in the experimental workflow.
|
||||
|
||||
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no change name provided, prompt for selection**
|
||||
|
||||
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
|
||||
|
||||
Show only active changes (not already archived).
|
||||
Include the schema used for each change if available.
|
||||
|
||||
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
|
||||
|
||||
2. **Check artifact completion status**
|
||||
|
||||
Run `openspec status --change "<name>" --json` to check artifact completion.
|
||||
|
||||
Parse the JSON to understand:
|
||||
- `schemaName`: The workflow being used
|
||||
- `artifacts`: List of artifacts with their status (`done` or other)
|
||||
|
||||
**If any artifacts are not `done`:**
|
||||
- Display warning listing incomplete artifacts
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
3. **Check task completion status**
|
||||
|
||||
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
|
||||
|
||||
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
|
||||
|
||||
**If incomplete tasks found:**
|
||||
- Display warning showing count of incomplete tasks
|
||||
- Use **AskUserQuestion tool** to confirm user wants to proceed
|
||||
- Proceed if user confirms
|
||||
|
||||
**If no tasks file exists:** Proceed without task-related warning.
|
||||
|
||||
4. **Assess delta spec sync state**
|
||||
|
||||
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
|
||||
|
||||
**If delta specs exist:**
|
||||
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
|
||||
- Determine what changes would be applied (adds, modifications, removals, renames)
|
||||
- Show a combined summary before prompting
|
||||
|
||||
**Prompt options:**
|
||||
- If changes needed: "Sync now (recommended)", "Archive without syncing"
|
||||
- If already synced: "Archive now", "Sync anyway", "Cancel"
|
||||
|
||||
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
|
||||
|
||||
5. **Perform the archive**
|
||||
|
||||
Create the archive directory if it doesn't exist:
|
||||
```bash
|
||||
mkdir -p openspec/changes/archive
|
||||
```
|
||||
|
||||
Generate target name using current date: `YYYY-MM-DD-<change-name>`
|
||||
|
||||
**Check if target already exists:**
|
||||
- If yes: Fail with error, suggest renaming existing archive or using different date
|
||||
- If no: Move the change directory to archive
|
||||
|
||||
```bash
|
||||
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
|
||||
```
|
||||
|
||||
6. **Display summary**
|
||||
|
||||
Show archive completion summary including:
|
||||
- Change name
|
||||
- Schema that was used
|
||||
- Archive location
|
||||
- Whether specs were synced (if applicable)
|
||||
- Note about any warnings (incomplete artifacts/tasks)
|
||||
|
||||
**Output On Success**
|
||||
|
||||
```
|
||||
## Archive Complete
|
||||
|
||||
**Change:** <change-name>
|
||||
**Schema:** <schema-name>
|
||||
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
|
||||
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
|
||||
|
||||
All artifacts complete. All tasks complete.
|
||||
```
|
||||
|
||||
**Guardrails**
|
||||
- Always prompt for change selection if not provided
|
||||
- Use artifact graph (openspec status --json) for completion checking
|
||||
- Don't block archive on warnings - just inform and confirm
|
||||
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
|
||||
- Show clear summary of what happened
|
||||
- If sync is requested, use openspec-sync-specs approach (agent-driven)
|
||||
- If delta specs exist, always run the sync assessment and show the combined summary before prompting
|
||||
288
.opencode/skills/openspec-explore/SKILL.md
Normal file
288
.opencode/skills/openspec-explore/SKILL.md
Normal file
@@ -0,0 +1,288 @@
|
||||
---
|
||||
name: openspec-explore
|
||||
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
|
||||
|
||||
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
|
||||
|
||||
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
|
||||
|
||||
---
|
||||
|
||||
## The Stance
|
||||
|
||||
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
|
||||
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
|
||||
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
|
||||
- **Adaptive** - Follow interesting threads, pivot when new information emerges
|
||||
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
|
||||
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
|
||||
|
||||
---
|
||||
|
||||
## What You Might Do
|
||||
|
||||
Depending on what the user brings, you might:
|
||||
|
||||
**Explore the problem space**
|
||||
- Ask clarifying questions that emerge from what they said
|
||||
- Challenge assumptions
|
||||
- Reframe the problem
|
||||
- Find analogies
|
||||
|
||||
**Investigate the codebase**
|
||||
- Map existing architecture relevant to the discussion
|
||||
- Find integration points
|
||||
- Identify patterns already in use
|
||||
- Surface hidden complexity
|
||||
|
||||
**Compare options**
|
||||
- Brainstorm multiple approaches
|
||||
- Build comparison tables
|
||||
- Sketch tradeoffs
|
||||
- Recommend a path (if asked)
|
||||
|
||||
**Visualize**
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Use ASCII diagrams liberally │
|
||||
├─────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌────────┐ ┌────────┐ │
|
||||
│ │ State │────────▶│ State │ │
|
||||
│ │ A │ │ B │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
│ │
|
||||
│ System diagrams, state machines, │
|
||||
│ data flows, architecture sketches, │
|
||||
│ dependency graphs, comparison tables │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Surface risks and unknowns**
|
||||
- Identify what could go wrong
|
||||
- Find gaps in understanding
|
||||
- Suggest spikes or investigations
|
||||
|
||||
---
|
||||
|
||||
## OpenSpec Awareness
|
||||
|
||||
You have full context of the OpenSpec system. Use it naturally, don't force it.
|
||||
|
||||
### Check for context
|
||||
|
||||
At the start, quickly check what exists:
|
||||
```bash
|
||||
openspec list --json
|
||||
```
|
||||
|
||||
This tells you:
|
||||
- If there are active changes
|
||||
- Their names, schemas, and status
|
||||
- What the user might be working on
|
||||
|
||||
### When no change exists
|
||||
|
||||
Think freely. When insights crystallize, you might offer:
|
||||
|
||||
- "This feels solid enough to start a change. Want me to create a proposal?"
|
||||
- Or keep exploring - no pressure to formalize
|
||||
|
||||
### When a change exists
|
||||
|
||||
If the user mentions a change or you detect one is relevant:
|
||||
|
||||
1. **Read existing artifacts for context**
|
||||
- `openspec/changes/<name>/proposal.md`
|
||||
- `openspec/changes/<name>/design.md`
|
||||
- `openspec/changes/<name>/tasks.md`
|
||||
- etc.
|
||||
|
||||
2. **Reference them naturally in conversation**
|
||||
- "Your design mentions using Redis, but we just realized SQLite fits better..."
|
||||
- "The proposal scopes this to premium users, but we're now thinking everyone..."
|
||||
|
||||
3. **Offer to capture when decisions are made**
|
||||
|
||||
| Insight Type | Where to Capture |
|
||||
|--------------|------------------|
|
||||
| New requirement discovered | `specs/<capability>/spec.md` |
|
||||
| Requirement changed | `specs/<capability>/spec.md` |
|
||||
| Design decision made | `design.md` |
|
||||
| Scope changed | `proposal.md` |
|
||||
| New work identified | `tasks.md` |
|
||||
| Assumption invalidated | Relevant artifact |
|
||||
|
||||
Example offers:
|
||||
- "That's a design decision. Capture it in design.md?"
|
||||
- "This is a new requirement. Add it to specs?"
|
||||
- "This changes scope. Update the proposal?"
|
||||
|
||||
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
|
||||
|
||||
---
|
||||
|
||||
## What You Don't Have To Do
|
||||
|
||||
- Follow a script
|
||||
- Ask the same questions every time
|
||||
- Produce a specific artifact
|
||||
- Reach a conclusion
|
||||
- Stay on topic if a tangent is valuable
|
||||
- Be brief (this is thinking time)
|
||||
|
||||
---
|
||||
|
||||
## Handling Different Entry Points
|
||||
|
||||
**User brings a vague idea:**
|
||||
```
|
||||
User: I'm thinking about adding real-time collaboration
|
||||
|
||||
You: Real-time collab is a big space. Let me think about this...
|
||||
|
||||
COLLABORATION SPECTRUM
|
||||
════════════════════════════════════════════
|
||||
|
||||
Awareness Coordination Sync
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌────────┐ ┌────────┐
|
||||
│Presence│ │Cursors │ │ CRDT │
|
||||
│ "3 │ │ Multi │ │Conflict│
|
||||
│online" │ │ select │ │ free │
|
||||
└────────┘ └────────┘ └────────┘
|
||||
│ │ │
|
||||
trivial moderate complex
|
||||
|
||||
Where's your head at?
|
||||
```
|
||||
|
||||
**User brings a specific problem:**
|
||||
```
|
||||
User: The auth system is a mess
|
||||
|
||||
You: [reads codebase]
|
||||
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ CURRENT AUTH FLOW │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Google │ │ GitHub │ │ Email │
|
||||
│ OAuth │ │ OAuth │ │ Magic │
|
||||
└────┬────┘ └────┬────┘ └────┬────┘
|
||||
│ │ │
|
||||
└─────────────┼─────────────┘
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Session │
|
||||
└─────┬─────┘
|
||||
│
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Perms │
|
||||
└───────────┘
|
||||
|
||||
I see three tangles. Which one's burning?
|
||||
```
|
||||
|
||||
**User is stuck mid-implementation:**
|
||||
```
|
||||
User: /opsx-explore add-auth-system
|
||||
The OAuth integration is more complex than expected
|
||||
|
||||
You: [reads change artifacts]
|
||||
|
||||
You're on task 4: "Implement OAuth flow"
|
||||
|
||||
Let me trace what's involved...
|
||||
|
||||
[draws diagram, explores options, suggests paths]
|
||||
|
||||
Want to update the design to reflect this?
|
||||
Or add a spike task to investigate?
|
||||
```
|
||||
|
||||
**User wants to compare options:**
|
||||
```
|
||||
User: Should we use Postgres or SQLite?
|
||||
|
||||
You: Generic answer is boring. What's the context?
|
||||
|
||||
User: A CLI tool that tracks local dev environments
|
||||
|
||||
You: That changes everything.
|
||||
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ CLI TOOL DATA STORAGE │
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
||||
Key constraints:
|
||||
• No daemon running
|
||||
• Must work offline
|
||||
• Single user
|
||||
|
||||
SQLite Postgres
|
||||
Deployment embedded ✓ needs server ✗
|
||||
Offline yes ✓ no ✗
|
||||
Single file yes ✓ no ✗
|
||||
|
||||
SQLite. Not even close.
|
||||
|
||||
Unless... is there a sync component?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ending Discovery
|
||||
|
||||
There's no required ending. Discovery might:
|
||||
|
||||
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
|
||||
- **Result in artifact updates**: "Updated design.md with these decisions"
|
||||
- **Just provide clarity**: User has what they need, moves on
|
||||
- **Continue later**: "We can pick this up anytime"
|
||||
|
||||
When it feels like things are crystallizing, you might summarize:
|
||||
|
||||
```
|
||||
## What We Figured Out
|
||||
|
||||
**The problem**: [crystallized understanding]
|
||||
|
||||
**The approach**: [if one emerged]
|
||||
|
||||
**Open questions**: [if any remain]
|
||||
|
||||
**Next steps** (if ready):
|
||||
- Create a change proposal
|
||||
- Keep exploring: just keep talking
|
||||
```
|
||||
|
||||
But this summary is optional. Sometimes the thinking IS the value.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
|
||||
- **Don't fake understanding** - If something is unclear, dig deeper
|
||||
- **Don't rush** - Discovery is thinking time, not task time
|
||||
- **Don't force structure** - Let patterns emerge naturally
|
||||
- **Don't auto-capture** - Offer to save insights, don't just do it
|
||||
- **Do visualize** - A good diagram is worth many paragraphs
|
||||
- **Do explore the codebase** - Ground discussions in reality
|
||||
- **Do question assumptions** - Including the user's and your own
|
||||
110
.opencode/skills/openspec-propose/SKILL.md
Normal file
110
.opencode/skills/openspec-propose/SKILL.md
Normal file
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: openspec-propose
|
||||
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
|
||||
license: MIT
|
||||
compatibility: Requires openspec CLI.
|
||||
metadata:
|
||||
author: openspec
|
||||
version: "1.0"
|
||||
generatedBy: "1.2.0"
|
||||
---
|
||||
|
||||
Propose a new change - create the change and generate all artifacts in one step.
|
||||
|
||||
I'll create a change with artifacts:
|
||||
- proposal.md (what & why)
|
||||
- design.md (how)
|
||||
- tasks.md (implementation steps)
|
||||
|
||||
When ready to implement, run /opsx-apply
|
||||
|
||||
---
|
||||
|
||||
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
|
||||
|
||||
**Steps**
|
||||
|
||||
1. **If no clear input provided, ask what they want to build**
|
||||
|
||||
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
|
||||
> "What change do you want to work on? Describe what you want to build or fix."
|
||||
|
||||
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
|
||||
|
||||
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
|
||||
|
||||
2. **Create the change directory**
|
||||
```bash
|
||||
openspec new change "<name>"
|
||||
```
|
||||
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
|
||||
|
||||
3. **Get the artifact build order**
|
||||
```bash
|
||||
openspec status --change "<name>" --json
|
||||
```
|
||||
Parse the JSON to get:
|
||||
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
|
||||
- `artifacts`: list of all artifacts with their status and dependencies
|
||||
|
||||
4. **Create artifacts in sequence until apply-ready**
|
||||
|
||||
Use the **TodoWrite tool** to track progress through the artifacts.
|
||||
|
||||
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
|
||||
|
||||
a. **For each artifact that is `ready` (dependencies satisfied)**:
|
||||
- Get instructions:
|
||||
```bash
|
||||
openspec instructions <artifact-id> --change "<name>" --json
|
||||
```
|
||||
- The instructions JSON includes:
|
||||
- `context`: Project background (constraints for you - do NOT include in output)
|
||||
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
|
||||
- `template`: The structure to use for your output file
|
||||
- `instruction`: Schema-specific guidance for this artifact type
|
||||
- `outputPath`: Where to write the artifact
|
||||
- `dependencies`: Completed artifacts to read for context
|
||||
- Read any completed dependency files for context
|
||||
- Create the artifact file using `template` as the structure
|
||||
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
|
||||
- Show brief progress: "Created <artifact-id>"
|
||||
|
||||
b. **Continue until all `applyRequires` artifacts are complete**
|
||||
- After creating each artifact, re-run `openspec status --change "<name>" --json`
|
||||
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
|
||||
- Stop when all `applyRequires` artifacts are done
|
||||
|
||||
c. **If an artifact requires user input** (unclear context):
|
||||
- Use **AskUserQuestion tool** to clarify
|
||||
- Then continue with creation
|
||||
|
||||
5. **Show final status**
|
||||
```bash
|
||||
openspec status --change "<name>"
|
||||
```
|
||||
|
||||
**Output**
|
||||
|
||||
After completing all artifacts, summarize:
|
||||
- Change name and location
|
||||
- List of artifacts created with brief descriptions
|
||||
- What's ready: "All artifacts created! Ready for implementation."
|
||||
- Prompt: "Run `/opsx-apply` or ask me to implement to start working on the tasks."
|
||||
|
||||
**Artifact Creation Guidelines**
|
||||
|
||||
- Follow the `instruction` field from `openspec instructions` for each artifact type
|
||||
- The schema defines what each artifact should contain - follow it
|
||||
- Read dependency artifacts for context before creating new ones
|
||||
- Use `template` as the structure for your output file - fill in its sections
|
||||
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
|
||||
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
|
||||
- These guide what you write, but should never appear in the output
|
||||
|
||||
**Guardrails**
|
||||
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
|
||||
- Always read dependency artifacts before creating a new one
|
||||
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
|
||||
- If a change with that name already exists, ask if user wants to continue it or create a new one
|
||||
- Verify each artifact file exists after writing before proceeding to next
|
||||
46
AGENTS.md
46
AGENTS.md
@@ -73,12 +73,14 @@ sqlx migrate add -r migration_name
|
||||
|
||||
### Docker Development
|
||||
|
||||
`docker-compose.yml` est à la **racine** du projet (pas dans `infra/`).
|
||||
|
||||
```bash
|
||||
# Start infrastructure only
|
||||
cd infra && docker compose up -d postgres meilisearch
|
||||
docker compose up -d postgres meilisearch
|
||||
|
||||
# Start full stack
|
||||
cd infra && docker compose up -d
|
||||
docker compose up -d
|
||||
|
||||
# View logs
|
||||
docker compose logs -f api
|
||||
@@ -226,24 +228,21 @@ pub struct BookItem {
|
||||
```
|
||||
stripstream-librarian/
|
||||
├── apps/
|
||||
│ ├── api/ # REST API (axum)
|
||||
│ │ └── src/
|
||||
│ │ ├── main.rs
|
||||
│ │ ├── books.rs
|
||||
│ │ ├── pages.rs
|
||||
│ │ └── ...
|
||||
│ ├── indexer/ # Background indexing service
|
||||
│ │ └── src/
|
||||
│ │ └── main.rs
|
||||
│ └── backoffice/ # Next.js admin UI
|
||||
│ ├── api/ # REST API (axum) — port 7080
|
||||
│ │ └── src/ # books.rs, pages.rs, thumbnails.rs, state.rs, auth.rs...
|
||||
│ ├── indexer/ # Background indexing service — port 7081
|
||||
│ │ └── src/ # worker.rs, scanner.rs, batch.rs, scheduler.rs, watcher.rs...
|
||||
│ └── backoffice/ # Next.js admin UI — port 7082
|
||||
├── crates/
|
||||
│ ├── core/ # Shared config
|
||||
│ ├── core/ # Shared config (env vars)
|
||||
│ │ └── src/config.rs
|
||||
│ └── parsers/ # Book parsing (CBZ, CBR, PDF)
|
||||
├── infra/
|
||||
│ ├── migrations/ # SQL migrations
|
||||
│ └── docker-compose.yml
|
||||
└── libraries/ # Book storage (mounted volume)
|
||||
│ └── migrations/ # SQL migrations (sqlx)
|
||||
├── data/
|
||||
│ └── thumbnails/ # Thumbnails générés par l'API
|
||||
├── libraries/ # Book storage (mounted volume)
|
||||
└── docker-compose.yml # À la racine (pas dans infra/)
|
||||
```
|
||||
|
||||
### Key Files
|
||||
@@ -251,8 +250,13 @@ stripstream-librarian/
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `apps/api/src/books.rs` | Book CRUD endpoints |
|
||||
| `apps/api/src/pages.rs` | Page rendering & caching |
|
||||
| `apps/indexer/src/main.rs` | Indexing logic, batch processing |
|
||||
| `apps/api/src/pages.rs` | Page rendering & caching (LRU + disk) |
|
||||
| `apps/api/src/thumbnails.rs` | Endpoints pour créer des jobs thumbnail (rebuild/regenerate) |
|
||||
| `apps/api/src/state.rs` | AppState, Semaphore concurrent_renders |
|
||||
| `apps/indexer/src/scanner.rs` | Phase 1 discovery : scan rapide sans I/O archive, skip dossiers inchangés |
|
||||
| `apps/indexer/src/analyzer.rs` | Phase 2 analysis : `analyze_book` + génération thumbnails WebP |
|
||||
| `apps/indexer/src/batch.rs` | Bulk DB ops via UNNEST |
|
||||
| `apps/indexer/src/worker.rs` | Job loop, watcher, scheduler orchestration |
|
||||
| `crates/parsers/src/lib.rs` | Format detection, metadata parsing |
|
||||
| `crates/core/src/config.rs` | Configuration from environment |
|
||||
| `infra/migrations/*.sql` | Database schema |
|
||||
@@ -269,7 +273,7 @@ impl IndexerConfig {
|
||||
pub fn from_env() -> Result<Self> {
|
||||
Ok(Self {
|
||||
listen_addr: std::env::var("INDEXER_LISTEN_ADDR")
|
||||
.unwrap_or_else(|_| "0.0.0.0:8081".to_string()),
|
||||
.unwrap_or_else(|_| "0.0.0.0:7081".to_string()),
|
||||
database_url: std::env::var("DATABASE_URL")
|
||||
.context("DATABASE_URL is required")?,
|
||||
// ...
|
||||
@@ -298,4 +302,6 @@ fn remap_libraries_path(path: &str) -> String {
|
||||
- **Workspace**: This is a Cargo workspace. Always specify the package when building specific apps.
|
||||
- **Dependencies**: External crates are defined in workspace `Cargo.toml`, not individual `Cargo.toml`.
|
||||
- **Database**: PostgreSQL is required. Run migrations before starting services.
|
||||
- **External Tools**: The indexer relies on `unar` (for CBR) and `pdftoppm` (for PDF) being installed on the system.
|
||||
- **External Tools**: 4 system tools required — `unrar` (CBR page count), `unar` (CBR extraction), `pdfinfo` (PDF page count), `pdftoppm` (PDF page render). Note: `unrar` and `unar` are distinct tools.
|
||||
- **Thumbnails**: generated by the **indexer** service (phase 2, `analyzer.rs`). The API only creates jobs in DB — it does not generate thumbnails directly.
|
||||
- **Sub-AGENTS.md**: module-specific guidelines in `apps/api/`, `apps/indexer/`, `apps/backoffice/`, `crates/parsers/`.
|
||||
|
||||
72
CLAUDE.md
Normal file
72
CLAUDE.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Stripstream Librarian
|
||||
|
||||
Gestionnaire de bibliothèque de bandes dessinées/ebooks. Workspace Cargo multi-crates avec backoffice Next.js.
|
||||
|
||||
## Architecture
|
||||
|
||||
| Service | Dossier | Port local |
|
||||
|---------|---------|------------|
|
||||
| API REST (axum) | `apps/api/` | 7080 |
|
||||
| Indexer (background) | `apps/indexer/` | 7081 |
|
||||
| Backoffice (Next.js) | `apps/backoffice/` | 7082 |
|
||||
| PostgreSQL | infra | 6432 |
|
||||
| Meilisearch | infra | 7700 |
|
||||
|
||||
Crates partagés : `crates/core` (config env), `crates/parsers` (CBZ/CBR/PDF).
|
||||
|
||||
## Commandes
|
||||
|
||||
```bash
|
||||
# Build
|
||||
cargo build # workspace entier
|
||||
cargo build -p api # crate spécifique
|
||||
cargo build --release # version optimisée
|
||||
|
||||
# Linting / format
|
||||
cargo clippy
|
||||
cargo fmt
|
||||
|
||||
# Tests
|
||||
cargo test
|
||||
cargo test -p parsers
|
||||
|
||||
# Infra (dépendances uniquement) — docker-compose.yml est à la racine
|
||||
docker compose up -d postgres meilisearch
|
||||
|
||||
# Backoffice dev
|
||||
cd apps/backoffice && npm install && npm run dev # http://localhost:7082
|
||||
|
||||
# Migrations
|
||||
sqlx migrate run # DATABASE_URL doit être défini
|
||||
```
|
||||
|
||||
## Environnement
|
||||
|
||||
```bash
|
||||
cp .env.example .env # puis éditer les valeurs REQUIRED
|
||||
```
|
||||
|
||||
Variables **requises** au démarrage : `DATABASE_URL`, `MEILI_URL`, `MEILI_MASTER_KEY`, `API_BOOTSTRAP_TOKEN`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Dépendances système** : 4 outils requis — `unrar` (CBR listing), `unar` (CBR extraction), `pdfinfo` (PDF page count), `pdftoppm` (PDF rendu). `unrar` ≠ `unar`.
|
||||
- **Port backoffice** : `npm run dev` écoute sur **7082**, pas 3000.
|
||||
- **LIBRARIES_ROOT_PATH** : les chemins en DB commencent par `/libraries/` ; en dev local, définir cette variable pour remapper vers le dossier réel.
|
||||
- **Thumbnails** : stockés dans `THUMBNAIL_DIRECTORY` (défaut `/data/thumbnails`), générés par **l'API** (pas l'indexer) — l'indexer déclenche un checkup via `POST /index/jobs/:id/thumbnails/checkup`.
|
||||
- **Workspace Cargo** : les dépendances externes sont définies dans le `Cargo.toml` racine, pas dans les crates individuels.
|
||||
- **Migrations** : dossier `infra/migrations/`, géré par sqlx. Toujours migrer avant de démarrer les services.
|
||||
|
||||
## Fichiers clés
|
||||
|
||||
| Fichier | Rôle |
|
||||
|---------|------|
|
||||
| `crates/core/src/config.rs` | Config depuis env (API, Indexer, AdminUI) |
|
||||
| `crates/parsers/src/lib.rs` | Détection format, extraction métadonnées |
|
||||
| `apps/api/src/books.rs` | Endpoints CRUD livres |
|
||||
| `apps/api/src/pages.rs` | Rendu pages + cache LRU |
|
||||
| `apps/indexer/src/scanner.rs` | Scan filesystem |
|
||||
| `infra/migrations/*.sql` | Schéma DB |
|
||||
|
||||
> Voir `AGENTS.md` pour les conventions de code détaillées (error handling, patterns sqlx, async/tokio).
|
||||
> Des `AGENTS.md` spécifiques existent dans `apps/api/`, `apps/indexer/`, `apps/backoffice/`, `crates/parsers/`.
|
||||
3
Cargo.lock
generated
3
Cargo.lock
generated
@@ -1146,6 +1146,8 @@ dependencies = [
|
||||
"anyhow",
|
||||
"axum",
|
||||
"chrono",
|
||||
"futures",
|
||||
"image",
|
||||
"notify",
|
||||
"parsers",
|
||||
"rand 0.8.5",
|
||||
@@ -1161,6 +1163,7 @@ dependencies = [
|
||||
"tracing-subscriber",
|
||||
"uuid",
|
||||
"walkdir",
|
||||
"webp",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
@@ -33,5 +33,6 @@ tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
||||
uuid = { version = "1.12", features = ["serde", "v4"] }
|
||||
walkdir = "2.5"
|
||||
webp = "0.3"
|
||||
utoipa = "4.0"
|
||||
utoipa-swagger-ui = "6.0"
|
||||
|
||||
20
README.md
20
README.md
@@ -38,16 +38,16 @@ docker compose up -d
|
||||
```
|
||||
|
||||
This will start:
|
||||
- PostgreSQL (port 5432)
|
||||
- PostgreSQL (port 6432)
|
||||
- Meilisearch (port 7700)
|
||||
- API service (port 8080)
|
||||
- Indexer service (port 8081)
|
||||
- Backoffice web UI (port 8082)
|
||||
- API service (port 7080)
|
||||
- Indexer service (port 7081)
|
||||
- Backoffice web UI (port 7082)
|
||||
|
||||
### Accessing the Application
|
||||
|
||||
- **Backoffice**: http://localhost:8082
|
||||
- **API**: http://localhost:8080
|
||||
- **Backoffice**: http://localhost:7082
|
||||
- **API**: http://localhost:7080
|
||||
- **Meilisearch**: http://localhost:7700
|
||||
|
||||
### Default Credentials
|
||||
@@ -113,9 +113,9 @@ The backoffice will be available at http://localhost:3000
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `API_LISTEN_ADDR` | API service bind address | `0.0.0.0:8080` |
|
||||
| `INDEXER_LISTEN_ADDR` | Indexer service bind address | `0.0.0.0:8081` |
|
||||
| `BACKOFFICE_PORT` | Backoffice web UI port | `8082` |
|
||||
| `API_LISTEN_ADDR` | API service bind address | `0.0.0.0:7080` |
|
||||
| `INDEXER_LISTEN_ADDR` | Indexer service bind address | `0.0.0.0:7081` |
|
||||
| `BACKOFFICE_PORT` | Backoffice web UI port | `7082` |
|
||||
| `DATABASE_URL` | PostgreSQL connection string | `postgres://stripstream:stripstream@postgres:5432/stripstream` |
|
||||
| `MEILI_URL` | Meilisearch connection URL | `http://meilisearch:7700` |
|
||||
| `MEILI_MASTER_KEY` | Meilisearch master key (required) | - |
|
||||
@@ -128,7 +128,7 @@ The backoffice will be available at http://localhost:3000
|
||||
The API is documented with OpenAPI/Swagger. When running locally, access the docs at:
|
||||
|
||||
```
|
||||
http://localhost:8080/api-docs
|
||||
http://localhost:7080/swagger-ui
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
73
apps/api/AGENTS.md
Normal file
73
apps/api/AGENTS.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# apps/api — REST API (axum)
|
||||
|
||||
Service HTTP sur le port **7080**. Voir `AGENTS.md` racine pour les conventions globales.
|
||||
|
||||
## Structure des fichiers
|
||||
|
||||
| Fichier | Rôle |
|
||||
|---------|------|
|
||||
| `main.rs` | Routes, initialisation AppState, Semaphore concurrent_renders |
|
||||
| `state.rs` | `AppState` (pool, caches, métriques), `load_concurrent_renders` |
|
||||
| `auth.rs` | Middlewares `require_admin` / `require_read`, authentification tokens |
|
||||
| `error.rs` | `ApiError` avec constructeurs `bad_request`, `not_found`, `internal`, etc. |
|
||||
| `books.rs` | CRUD livres, thumbnails |
|
||||
| `pages.rs` | Rendu page + double cache (mémoire LRU + disque) |
|
||||
| `libraries.rs` | CRUD bibliothèques, déclenchement scans |
|
||||
| `index_jobs.rs` | Suivi jobs, SSE streaming progression |
|
||||
| `thumbnails.rs` | Rebuild/regénération thumbnails |
|
||||
| `tokens.rs` | Gestion tokens API (create/revoke) |
|
||||
| `settings.rs` | Paramètres applicatifs (stockés en DB, clé `limits`) |
|
||||
| `openapi.rs` | Doc OpenAPI via utoipa, accessible sur `/swagger-ui` |
|
||||
|
||||
## Patterns clés
|
||||
|
||||
### Handler type
|
||||
```rust
|
||||
async fn my_handler(
|
||||
State(state): State<AppState>,
|
||||
Path(id): Path<Uuid>,
|
||||
) -> Result<Json<MyDto>, ApiError> {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Erreurs API
|
||||
```rust
|
||||
// Constructeurs disponibles dans error.rs
|
||||
ApiError::bad_request("message")
|
||||
ApiError::not_found("resource not found")
|
||||
ApiError::internal("unexpected error")
|
||||
ApiError::unauthorized("missing token")
|
||||
ApiError::forbidden("admin required")
|
||||
|
||||
// Conversion auto depuis sqlx::Error et std::io::Error
|
||||
```
|
||||
|
||||
### Authentification
|
||||
- **Bootstrap token** : comparaison directe (`API_BOOTSTRAP_TOKEN`), scope Admin
|
||||
- **Tokens DB** : format `stl_<prefix>_<secret>`, hash argon2 en DB, scope `admin` ou `read`
|
||||
- Middleware `require_admin` → routes admin ; `require_read` → routes lecture
|
||||
|
||||
### OpenAPI (utoipa)
|
||||
```rust
|
||||
#[utoipa::path(get, path = "/books/{id}", ...)]
|
||||
async fn get_book(...) { }
|
||||
// Ajouter le handler dans openapi.rs (ApiDoc)
|
||||
```
|
||||
|
||||
### Cache pages (`pages.rs`)
|
||||
- **Cache mémoire** : LRU 512 entrées (`AppState.page_cache`)
|
||||
- **Cache disque** : `IMAGE_CACHE_DIR` (défaut `/tmp/stripstream-image-cache`), clé SHA256
|
||||
- Concurrence limitée par `AppState.page_render_limit` (Semaphore, configurable en DB)
|
||||
- `spawn_blocking` pour le rendu image (CPU-bound)
|
||||
|
||||
### Paramètre concurrent_renders
|
||||
Stocké en DB : `SELECT value FROM app_settings WHERE key = 'limits'` → JSON `{"concurrent_renders": N}`.
|
||||
Chargé au démarrage dans `load_concurrent_renders`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **LIBRARIES_ROOT_PATH** : les `abs_path` en DB commencent par `/libraries/`. Appeler `remap_libraries_path()` avant tout accès fichier.
|
||||
- **Rate limit lecture** : middleware `read_rate_limit` sur les routes read (100 req/5s par défaut).
|
||||
- **Métriques** : `/metrics` expose `requests_total`, `page_cache_hits`, `page_cache_misses` (atomics dans `AppState.metrics`).
|
||||
- **Swagger** : accessible sur `/swagger-ui`, spec JSON sur `/openapi.json`.
|
||||
@@ -31,5 +31,5 @@ uuid.workspace = true
|
||||
zip = { version = "2.2", default-features = false, features = ["deflate"] }
|
||||
utoipa.workspace = true
|
||||
utoipa-swagger-ui = { workspace = true, features = ["axum"] }
|
||||
webp = "0.3"
|
||||
webp.workspace = true
|
||||
walkdir = "2"
|
||||
|
||||
@@ -26,5 +26,5 @@ RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
|
||||
ENV LANG=en_US.UTF-8
|
||||
ENV LC_ALL=en_US.UTF-8
|
||||
COPY --from=builder /app/target/release/api /usr/local/bin/api
|
||||
EXPOSE 8080
|
||||
EXPOSE 7080
|
||||
CMD ["/usr/local/bin/api"]
|
||||
|
||||
42
apps/api/src/api_middleware.rs
Normal file
42
apps/api/src/api_middleware.rs
Normal file
@@ -0,0 +1,42 @@
|
||||
use axum::{
|
||||
extract::State,
|
||||
middleware::Next,
|
||||
response::{IntoResponse, Response},
|
||||
};
|
||||
use std::time::Duration;
|
||||
use std::sync::atomic::Ordering;
|
||||
|
||||
use crate::state::AppState;
|
||||
|
||||
pub async fn request_counter(
|
||||
State(state): State<AppState>,
|
||||
req: axum::extract::Request,
|
||||
next: Next,
|
||||
) -> Response {
|
||||
state.metrics.requests_total.fetch_add(1, Ordering::Relaxed);
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
pub async fn read_rate_limit(
|
||||
State(state): State<AppState>,
|
||||
req: axum::extract::Request,
|
||||
next: Next,
|
||||
) -> Response {
|
||||
let mut limiter = state.read_rate_limit.lock().await;
|
||||
if limiter.window_started_at.elapsed() >= Duration::from_secs(1) {
|
||||
limiter.window_started_at = std::time::Instant::now();
|
||||
limiter.requests_in_window = 0;
|
||||
}
|
||||
|
||||
if limiter.requests_in_window >= 120 {
|
||||
return (
|
||||
axum::http::StatusCode::TOO_MANY_REQUESTS,
|
||||
"rate limit exceeded",
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
|
||||
limiter.requests_in_window += 1;
|
||||
drop(limiter);
|
||||
next.run(req).await
|
||||
}
|
||||
@@ -8,7 +8,7 @@ use axum::{
|
||||
use chrono::Utc;
|
||||
use sqlx::Row;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum Scope {
|
||||
|
||||
@@ -5,7 +5,7 @@ use sqlx::Row;
|
||||
use uuid::Uuid;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct ListBooksQuery {
|
||||
|
||||
26
apps/api/src/handlers.rs
Normal file
26
apps/api/src/handlers.rs
Normal file
@@ -0,0 +1,26 @@
|
||||
use axum::{extract::State, Json};
|
||||
use std::sync::atomic::Ordering;
|
||||
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
pub async fn health() -> &'static str {
|
||||
"ok"
|
||||
}
|
||||
|
||||
pub async fn docs_redirect() -> impl axum::response::IntoResponse {
|
||||
axum::response::Redirect::to("/swagger-ui/")
|
||||
}
|
||||
|
||||
pub async fn ready(State(state): State<AppState>) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
sqlx::query("SELECT 1").execute(&state.pool).await?;
|
||||
Ok(Json(serde_json::json!({"status": "ready"})))
|
||||
}
|
||||
|
||||
pub async fn metrics(State(state): State<AppState>) -> String {
|
||||
format!(
|
||||
"requests_total {}\npage_cache_hits {}\npage_cache_misses {}\n",
|
||||
state.metrics.requests_total.load(Ordering::Relaxed),
|
||||
state.metrics.page_cache_hits.load(Ordering::Relaxed),
|
||||
state.metrics.page_cache_misses.load(Ordering::Relaxed),
|
||||
)
|
||||
}
|
||||
@@ -8,7 +8,7 @@ use tokio_stream::Stream;
|
||||
use uuid::Uuid;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct RebuildRequest {
|
||||
@@ -247,7 +247,7 @@ pub async fn list_folders(
|
||||
}
|
||||
|
||||
let mut folders = Vec::new();
|
||||
let depth = if params.get("path").is_some() {
|
||||
let depth = if params.contains_key("path") {
|
||||
canonical_target.strip_prefix(&canonical_base)
|
||||
.map(|p| p.components().count())
|
||||
.unwrap_or(0)
|
||||
|
||||
@@ -6,7 +6,7 @@ use sqlx::Row;
|
||||
use uuid::Uuid;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Serialize, ToSchema)]
|
||||
pub struct LibraryResponse {
|
||||
|
||||
@@ -1,70 +1,36 @@
|
||||
mod auth;
|
||||
mod books;
|
||||
mod error;
|
||||
mod handlers;
|
||||
mod index_jobs;
|
||||
mod libraries;
|
||||
mod api_middleware;
|
||||
mod openapi;
|
||||
mod pages;
|
||||
mod search;
|
||||
mod settings;
|
||||
mod state;
|
||||
mod thumbnails;
|
||||
mod tokens;
|
||||
|
||||
use std::{
|
||||
num::NonZeroUsize,
|
||||
sync::{
|
||||
atomic::{AtomicU64, Ordering},
|
||||
Arc,
|
||||
},
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
use std::sync::Arc;
|
||||
use std::time::Instant;
|
||||
|
||||
use axum::{
|
||||
middleware,
|
||||
response::IntoResponse,
|
||||
routing::{delete, get},
|
||||
Json, Router,
|
||||
Router,
|
||||
};
|
||||
use utoipa::OpenApi;
|
||||
use utoipa_swagger_ui::SwaggerUi;
|
||||
use lru::LruCache;
|
||||
use std::num::NonZeroUsize;
|
||||
use stripstream_core::config::ApiConfig;
|
||||
use sqlx::postgres::PgPoolOptions;
|
||||
use tokio::sync::{Mutex, Semaphore};
|
||||
use tracing::info;
|
||||
|
||||
#[derive(Clone)]
|
||||
struct AppState {
|
||||
pool: sqlx::PgPool,
|
||||
bootstrap_token: Arc<str>,
|
||||
meili_url: Arc<str>,
|
||||
meili_master_key: Arc<str>,
|
||||
page_cache: Arc<Mutex<LruCache<String, Arc<Vec<u8>>>>>,
|
||||
page_render_limit: Arc<Semaphore>,
|
||||
metrics: Arc<Metrics>,
|
||||
read_rate_limit: Arc<Mutex<ReadRateLimit>>,
|
||||
}
|
||||
|
||||
struct Metrics {
|
||||
requests_total: AtomicU64,
|
||||
page_cache_hits: AtomicU64,
|
||||
page_cache_misses: AtomicU64,
|
||||
}
|
||||
|
||||
struct ReadRateLimit {
|
||||
window_started_at: Instant,
|
||||
requests_in_window: u32,
|
||||
}
|
||||
|
||||
impl Metrics {
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
requests_total: AtomicU64::new(0),
|
||||
page_cache_hits: AtomicU64::new(0),
|
||||
page_cache_misses: AtomicU64::new(0),
|
||||
}
|
||||
}
|
||||
}
|
||||
use crate::state::{load_concurrent_renders, AppState, Metrics, ReadRateLimit};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
@@ -80,13 +46,17 @@ async fn main() -> anyhow::Result<()> {
|
||||
.connect(&config.database_url)
|
||||
.await?;
|
||||
|
||||
// Load concurrent_renders from settings, default to 8
|
||||
let concurrent_renders = load_concurrent_renders(&pool).await;
|
||||
info!("Using concurrent_renders limit: {}", concurrent_renders);
|
||||
|
||||
let state = AppState {
|
||||
pool,
|
||||
bootstrap_token: Arc::from(config.api_bootstrap_token),
|
||||
meili_url: Arc::from(config.meili_url),
|
||||
meili_master_key: Arc::from(config.meili_master_key),
|
||||
page_cache: Arc::new(Mutex::new(LruCache::new(NonZeroUsize::new(512).expect("non-zero")))),
|
||||
page_render_limit: Arc::new(Semaphore::new(8)),
|
||||
page_render_limit: Arc::new(Semaphore::new(concurrent_renders)),
|
||||
metrics: Arc::new(Metrics::new()),
|
||||
read_rate_limit: Arc::new(Mutex::new(ReadRateLimit {
|
||||
window_started_at: Instant::now(),
|
||||
@@ -106,7 +76,6 @@ async fn main() -> anyhow::Result<()> {
|
||||
.route("/index/jobs/active", get(index_jobs::get_active_jobs))
|
||||
.route("/index/jobs/:id", get(index_jobs::get_job_details))
|
||||
.route("/index/jobs/:id/stream", get(index_jobs::stream_job_progress))
|
||||
.route("/index/jobs/:id/thumbnails/checkup", axum::routing::post(thumbnails::start_checkup))
|
||||
.route("/index/jobs/:id/errors", get(index_jobs::get_job_errors))
|
||||
.route("/index/cancel/:id", axum::routing::post(index_jobs::cancel_job))
|
||||
.route("/folders", get(index_jobs::list_folders))
|
||||
@@ -125,21 +94,21 @@ async fn main() -> anyhow::Result<()> {
|
||||
.route("/books/:id/pages/:n", get(pages::get_page))
|
||||
.route("/libraries/:library_id/series", get(books::list_series))
|
||||
.route("/search", get(search::search_books))
|
||||
.route_layer(middleware::from_fn_with_state(state.clone(), read_rate_limit))
|
||||
.route_layer(middleware::from_fn_with_state(state.clone(), api_middleware::read_rate_limit))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
auth::require_read,
|
||||
));
|
||||
|
||||
let app = Router::new()
|
||||
.route("/health", get(health))
|
||||
.route("/ready", get(ready))
|
||||
.route("/metrics", get(metrics))
|
||||
.route("/docs", get(docs_redirect))
|
||||
.route("/health", get(handlers::health))
|
||||
.route("/ready", get(handlers::ready))
|
||||
.route("/metrics", get(handlers::metrics))
|
||||
.route("/docs", get(handlers::docs_redirect))
|
||||
.merge(SwaggerUi::new("/swagger-ui").url("/openapi.json", openapi::ApiDoc::openapi()))
|
||||
.merge(admin_routes)
|
||||
.merge(read_routes)
|
||||
.layer(middleware::from_fn_with_state(state.clone(), request_counter))
|
||||
.layer(middleware::from_fn_with_state(state.clone(), api_middleware::request_counter))
|
||||
.with_state(state);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(&config.listen_addr).await?;
|
||||
@@ -148,57 +117,3 @@ async fn main() -> anyhow::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn health() -> &'static str {
|
||||
"ok"
|
||||
}
|
||||
|
||||
async fn docs_redirect() -> impl axum::response::IntoResponse {
|
||||
axum::response::Redirect::to("/swagger-ui/")
|
||||
}
|
||||
|
||||
async fn ready(axum::extract::State(state): axum::extract::State<AppState>) -> Result<Json<serde_json::Value>, error::ApiError> {
|
||||
sqlx::query("SELECT 1").execute(&state.pool).await?;
|
||||
Ok(Json(serde_json::json!({"status": "ready"})))
|
||||
}
|
||||
|
||||
async fn metrics(axum::extract::State(state): axum::extract::State<AppState>) -> String {
|
||||
format!(
|
||||
"requests_total {}\npage_cache_hits {}\npage_cache_misses {}\n",
|
||||
state.metrics.requests_total.load(Ordering::Relaxed),
|
||||
state.metrics.page_cache_hits.load(Ordering::Relaxed),
|
||||
state.metrics.page_cache_misses.load(Ordering::Relaxed),
|
||||
)
|
||||
}
|
||||
|
||||
async fn request_counter(
|
||||
axum::extract::State(state): axum::extract::State<AppState>,
|
||||
req: axum::extract::Request,
|
||||
next: axum::middleware::Next,
|
||||
) -> axum::response::Response {
|
||||
state.metrics.requests_total.fetch_add(1, Ordering::Relaxed);
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
async fn read_rate_limit(
|
||||
axum::extract::State(state): axum::extract::State<AppState>,
|
||||
req: axum::extract::Request,
|
||||
next: axum::middleware::Next,
|
||||
) -> axum::response::Response {
|
||||
let mut limiter = state.read_rate_limit.lock().await;
|
||||
if limiter.window_started_at.elapsed() >= Duration::from_secs(1) {
|
||||
limiter.window_started_at = Instant::now();
|
||||
limiter.requests_in_window = 0;
|
||||
}
|
||||
|
||||
if limiter.requests_in_window >= 120 {
|
||||
return (
|
||||
axum::http::StatusCode::TOO_MANY_REQUESTS,
|
||||
"rate limit exceeded",
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
|
||||
limiter.requests_in_window += 1;
|
||||
drop(limiter);
|
||||
next.run(req).await
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ use tracing::{debug, error, info, instrument, warn};
|
||||
use uuid::Uuid;
|
||||
use walkdir::WalkDir;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
fn remap_libraries_path(path: &str) -> String {
|
||||
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
|
||||
@@ -550,12 +550,12 @@ fn transcode_image(input: &[u8], out_format: &OutputFormat, quality: u8, width:
|
||||
}
|
||||
|
||||
fn format_matches(source: &ImageFormat, target: &OutputFormat) -> bool {
|
||||
match (source, target) {
|
||||
(ImageFormat::Jpeg, OutputFormat::Jpeg) => true,
|
||||
(ImageFormat::Png, OutputFormat::Png) => true,
|
||||
(ImageFormat::WebP, OutputFormat::Webp) => true,
|
||||
_ => false,
|
||||
}
|
||||
matches!(
|
||||
(source, target),
|
||||
(ImageFormat::Jpeg, OutputFormat::Jpeg)
|
||||
| (ImageFormat::Png, OutputFormat::Png)
|
||||
| (ImageFormat::WebP, OutputFormat::Webp)
|
||||
)
|
||||
}
|
||||
|
||||
fn is_image_name(name: &str) -> bool {
|
||||
|
||||
@@ -2,7 +2,7 @@ use axum::{extract::{Query, State}, Json};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct SearchQuery {
|
||||
|
||||
@@ -7,7 +7,7 @@ use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
use sqlx::Row;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct UpdateSettingRequest {
|
||||
|
||||
61
apps/api/src/state.rs
Normal file
61
apps/api/src/state.rs
Normal file
@@ -0,0 +1,61 @@
|
||||
use std::sync::{
|
||||
atomic::AtomicU64,
|
||||
Arc,
|
||||
};
|
||||
use std::time::Instant;
|
||||
|
||||
use lru::LruCache;
|
||||
use sqlx::{Pool, Postgres, Row};
|
||||
use tokio::sync::{Mutex, Semaphore};
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct AppState {
|
||||
pub pool: sqlx::PgPool,
|
||||
pub bootstrap_token: Arc<str>,
|
||||
pub meili_url: Arc<str>,
|
||||
pub meili_master_key: Arc<str>,
|
||||
pub page_cache: Arc<Mutex<LruCache<String, Arc<Vec<u8>>>>>,
|
||||
pub page_render_limit: Arc<Semaphore>,
|
||||
pub metrics: Arc<Metrics>,
|
||||
pub read_rate_limit: Arc<Mutex<ReadRateLimit>>,
|
||||
}
|
||||
|
||||
pub struct Metrics {
|
||||
pub requests_total: AtomicU64,
|
||||
pub page_cache_hits: AtomicU64,
|
||||
pub page_cache_misses: AtomicU64,
|
||||
}
|
||||
|
||||
pub struct ReadRateLimit {
|
||||
pub window_started_at: Instant,
|
||||
pub requests_in_window: u32,
|
||||
}
|
||||
|
||||
impl Metrics {
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
requests_total: AtomicU64::new(0),
|
||||
page_cache_hits: AtomicU64::new(0),
|
||||
page_cache_misses: AtomicU64::new(0),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn load_concurrent_renders(pool: &Pool<Postgres>) -> usize {
|
||||
let default_concurrency = 8;
|
||||
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
|
||||
match row {
|
||||
Ok(Some(row)) => {
|
||||
let value: serde_json::Value = row.get("value");
|
||||
value
|
||||
.get("concurrent_renders")
|
||||
.and_then(|v: &serde_json::Value| v.as_u64())
|
||||
.map(|v| v as usize)
|
||||
.unwrap_or(default_concurrency)
|
||||
}
|
||||
_ => default_concurrency,
|
||||
}
|
||||
}
|
||||
@@ -1,203 +1,12 @@
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::Context;
|
||||
use axum::{
|
||||
extract::{Path as AxumPath, State},
|
||||
http::StatusCode,
|
||||
extract::State,
|
||||
Json,
|
||||
};
|
||||
use image::GenericImageView;
|
||||
use serde::Deserialize;
|
||||
use sqlx::Row;
|
||||
use tracing::{info, warn};
|
||||
use uuid::Uuid;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::{error::ApiError, index_jobs, pages, AppState};
|
||||
|
||||
#[derive(Clone)]
|
||||
struct ThumbnailConfig {
|
||||
enabled: bool,
|
||||
width: u32,
|
||||
height: u32,
|
||||
quality: u8,
|
||||
directory: String,
|
||||
}
|
||||
|
||||
async fn load_thumbnail_config(pool: &sqlx::PgPool) -> ThumbnailConfig {
|
||||
let fallback = ThumbnailConfig {
|
||||
enabled: true,
|
||||
width: 300,
|
||||
height: 400,
|
||||
quality: 80,
|
||||
directory: "/data/thumbnails".to_string(),
|
||||
};
|
||||
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
|
||||
match row {
|
||||
Ok(Some(row)) => {
|
||||
let value: serde_json::Value = row.get("value");
|
||||
ThumbnailConfig {
|
||||
enabled: value
|
||||
.get("enabled")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(fallback.enabled),
|
||||
width: value
|
||||
.get("width")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as u32)
|
||||
.unwrap_or(fallback.width),
|
||||
height: value
|
||||
.get("height")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as u32)
|
||||
.unwrap_or(fallback.height),
|
||||
quality: value
|
||||
.get("quality")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as u8)
|
||||
.unwrap_or(fallback.quality),
|
||||
directory: value
|
||||
.get("directory")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| s.to_string())
|
||||
.unwrap_or_else(|| fallback.directory.clone()),
|
||||
}
|
||||
}
|
||||
_ => fallback,
|
||||
}
|
||||
}
|
||||
|
||||
fn generate_thumbnail(image_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<Vec<u8>> {
|
||||
let img = image::load_from_memory(image_bytes).context("failed to load image")?;
|
||||
let (orig_w, orig_h) = img.dimensions();
|
||||
let ratio_w = config.width as f32 / orig_w as f32;
|
||||
let ratio_h = config.height as f32 / orig_h as f32;
|
||||
let ratio = ratio_w.min(ratio_h);
|
||||
let new_w = (orig_w as f32 * ratio) as u32;
|
||||
let new_h = (orig_h as f32 * ratio) as u32;
|
||||
let resized = img.resize(new_w, new_h, image::imageops::FilterType::Lanczos3);
|
||||
let rgba = resized.to_rgba8();
|
||||
let (w, h) = rgba.dimensions();
|
||||
let rgb_data: Vec<u8> = rgba.pixels().flat_map(|p| [p[0], p[1], p[2]]).collect();
|
||||
let quality = f32::max(config.quality as f32, 85.0);
|
||||
let webp_data =
|
||||
webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h).encode(quality);
|
||||
Ok(webp_data.to_vec())
|
||||
}
|
||||
|
||||
fn save_thumbnail(book_id: Uuid, thumbnail_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<String> {
|
||||
let dir = Path::new(&config.directory);
|
||||
std::fs::create_dir_all(dir)?;
|
||||
let filename = format!("{}.webp", book_id);
|
||||
let path = dir.join(&filename);
|
||||
std::fs::write(&path, thumbnail_bytes)?;
|
||||
Ok(path.to_string_lossy().to_string())
|
||||
}
|
||||
|
||||
async fn run_checkup(state: AppState, job_id: Uuid) {
|
||||
let pool = &state.pool;
|
||||
let row = sqlx::query("SELECT library_id, type FROM index_jobs WHERE id = $1")
|
||||
.bind(job_id)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
|
||||
let (library_id, job_type) = match row {
|
||||
Ok(Some(r)) => (
|
||||
r.get::<Option<Uuid>, _>("library_id"),
|
||||
r.get::<String, _>("type"),
|
||||
),
|
||||
_ => {
|
||||
warn!("thumbnails checkup: job {} not found", job_id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Regenerate: clear existing thumbnails in scope so they get regenerated
|
||||
if job_type == "thumbnail_regenerate" {
|
||||
let cleared = sqlx::query(
|
||||
r#"UPDATE books SET thumbnail_path = NULL WHERE (library_id = $1 OR $1 IS NULL)"#,
|
||||
)
|
||||
.bind(library_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
if let Ok(res) = cleared {
|
||||
info!("thumbnails regenerate: cleared {} books", res.rows_affected());
|
||||
}
|
||||
}
|
||||
|
||||
let book_ids: Vec<Uuid> = sqlx::query_scalar(
|
||||
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL) AND thumbnail_path IS NULL"#,
|
||||
)
|
||||
.bind(library_id)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
|
||||
let config = load_thumbnail_config(pool).await;
|
||||
if !config.enabled || book_ids.is_empty() {
|
||||
let _ = sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
|
||||
let total = book_ids.len() as i32;
|
||||
let _ = sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'generating_thumbnails', total_files = $2, processed_files = 0, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(total)
|
||||
.execute(pool)
|
||||
.await;
|
||||
|
||||
for (i, &book_id) in book_ids.iter().enumerate() {
|
||||
match pages::render_book_page_1(&state, book_id, config.width, config.quality).await {
|
||||
Ok(page_bytes) => {
|
||||
match generate_thumbnail(&page_bytes, &config) {
|
||||
Ok(thumb_bytes) => {
|
||||
if let Ok(path) = save_thumbnail(book_id, &thumb_bytes, &config) {
|
||||
if sqlx::query("UPDATE books SET thumbnail_path = $1 WHERE id = $2")
|
||||
.bind(&path)
|
||||
.bind(book_id)
|
||||
.execute(pool)
|
||||
.await
|
||||
.is_ok()
|
||||
{
|
||||
let processed = (i + 1) as i32;
|
||||
let percent = ((i + 1) as f64 / total as f64 * 100.0) as i32;
|
||||
let _ = sqlx::query(
|
||||
"UPDATE index_jobs SET processed_files = $2, progress_percent = $3 WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(processed)
|
||||
.bind(percent)
|
||||
.execute(pool)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => warn!("thumbnail generate failed for book {}: {:?}", book_id, e),
|
||||
}
|
||||
}
|
||||
Err(e) => warn!("render page 1 failed for book {}: {:?}", book_id, e),
|
||||
}
|
||||
}
|
||||
|
||||
let _ = sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(pool)
|
||||
.await;
|
||||
|
||||
info!("thumbnails checkup finished for job {} ({} books)", job_id, total);
|
||||
}
|
||||
use crate::{error::ApiError, index_jobs, state::AppState};
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct ThumbnailsRebuildRequest {
|
||||
@@ -205,7 +14,7 @@ pub struct ThumbnailsRebuildRequest {
|
||||
pub library_id: Option<Uuid>,
|
||||
}
|
||||
|
||||
/// POST /index/thumbnails/rebuild — create a job and generate thumbnails for books that don't have one (optional library scope).
|
||||
/// POST /index/thumbnails/rebuild — create a job to generate thumbnails for books that don't have one.
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/index/thumbnails/rebuild",
|
||||
@@ -239,7 +48,7 @@ pub async fn start_thumbnails_rebuild(
|
||||
Ok(Json(index_jobs::map_row(row)))
|
||||
}
|
||||
|
||||
/// POST /index/thumbnails/regenerate — create a job and regenerate all thumbnails in scope (clears then regenerates).
|
||||
/// POST /index/thumbnails/regenerate — create a job to regenerate all thumbnails (clears then regenerates).
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/index/thumbnails/regenerate",
|
||||
@@ -272,13 +81,3 @@ pub async fn start_thumbnails_regenerate(
|
||||
|
||||
Ok(Json(index_jobs::map_row(row)))
|
||||
}
|
||||
|
||||
/// POST /index/jobs/:id/thumbnails/checkup — start thumbnail generation for books missing thumbnails (called by indexer at end of build).
|
||||
pub async fn start_checkup(
|
||||
State(state): State<AppState>,
|
||||
AxumPath(job_id): AxumPath<Uuid>,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
let state = state.clone();
|
||||
tokio::spawn(async move { run_checkup(state, job_id).await });
|
||||
Ok(StatusCode::ACCEPTED)
|
||||
}
|
||||
|
||||
@@ -8,7 +8,7 @@ use sqlx::Row;
|
||||
use uuid::Uuid;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::{error::ApiError, AppState};
|
||||
use crate::{error::ApiError, state::AppState};
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct CreateTokenRequest {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
API_BASE_URL=http://localhost:8080
|
||||
API_BASE_URL=http://localhost:7080
|
||||
API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token
|
||||
NEXT_PUBLIC_API_BASE_URL=http://localhost:8080
|
||||
NEXT_PUBLIC_API_BASE_URL=http://localhost:7080
|
||||
NEXT_PUBLIC_API_BOOTSTRAP_TOKEN=stripstream-dev-bootstrap-token
|
||||
|
||||
66
apps/backoffice/AGENTS.md
Normal file
66
apps/backoffice/AGENTS.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# apps/backoffice — Interface d'administration (Next.js)
|
||||
|
||||
App Next.js 16 avec React 19, Tailwind CSS v4, TypeScript. Port de dev : **7082** (`npm run dev`).
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
app/
|
||||
├── layout.tsx # Layout global (nav sticky glassmorphism, ThemeProvider)
|
||||
├── page.tsx # Dashboard
|
||||
├── books/ # Liste et détail des livres
|
||||
├── libraries/ # Gestion bibliothèques
|
||||
├── jobs/ # Monitoring jobs
|
||||
├── tokens/ # Tokens API
|
||||
├── settings/ # Paramètres
|
||||
├── components/ # Composants métier
|
||||
│ ├── ui/ # Composants génériques (Button, Card, Badge, Icon, Input, ProgressBar, StatBox...)
|
||||
│ ├── BookCard.tsx
|
||||
│ ├── JobProgress.tsx
|
||||
│ ├── JobsList.tsx
|
||||
│ ├── LibraryForm.tsx
|
||||
│ ├── FolderBrowser.tsx / FolderPicker.tsx
|
||||
│ └── ...
|
||||
└── globals.css # Variables CSS, Tailwind base
|
||||
lib/
|
||||
└── api.ts # Client API : types DTO + fonctions fetch vers l'API Rust
|
||||
```
|
||||
|
||||
## Client API (lib/api.ts)
|
||||
|
||||
Tous les appels vers l'API Rust passent par `lib/api.ts`. Les types DTO sont définis là :
|
||||
- `LibraryDto`, `IndexJobDto`, `BookDto`, `TokenDto`, `FolderItem`
|
||||
|
||||
Ajouter les nouveaux endpoints et types dans ce fichier.
|
||||
|
||||
## Composants UI
|
||||
|
||||
Les composants génériques sont dans `app/components/ui/`. Utiliser ces composants plutôt que des éléments HTML bruts :
|
||||
|
||||
```tsx
|
||||
import { Button, Card, Badge, Icon, Input, ProgressBar, StatBox } from "@/app/components/ui";
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **App Router** : toutes les pages sont des Server Components par défaut. Utiliser `"use client"` seulement pour l'interactivité.
|
||||
- **Tailwind v4** : config dans `postcss.config.js` + `tailwind.config.js`. Variables CSS dans `globals.css`.
|
||||
- **Thème** : `ThemeProvider` + `ThemeToggle` pour dark/light mode via `next-themes`.
|
||||
- **Icônes** : composant `<Icon name="..." size="sm|md|lg" />` dans `ui/Icon.tsx` — pas de librairie externe.
|
||||
- **Navigation** : routes typées dans `layout.tsx` (`"/" | "/books" | "/libraries" | "/jobs" | "/tokens" | "/settings"`).
|
||||
|
||||
## Commandes
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run dev # http://localhost:7082
|
||||
npm run build
|
||||
npm run start # Production sur http://localhost:7082
|
||||
```
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Port 7082** : pas le port Next.js par défaut (3000). Défini dans `package.json` scripts (`-p 7082`).
|
||||
- **API_BASE_URL** : en prod, configuré via env. En dev local, l'API doit tourner sur `http://localhost:7080`.
|
||||
- **React 19 + Next.js 16** : utiliser les nouvelles APIs (actions serveur, `use()` hook) si disponibles.
|
||||
- **Pas de gestion d'état global** : fetch direct depuis les Server Components ou `useState`/`useEffect` dans les Client Components.
|
||||
@@ -12,11 +12,11 @@ RUN npm run build
|
||||
FROM node:22-alpine AS runner
|
||||
WORKDIR /app
|
||||
ENV NODE_ENV=production
|
||||
ENV PORT=8082
|
||||
ENV PORT=7082
|
||||
ENV HOST=0.0.0.0
|
||||
RUN apk add --no-cache wget
|
||||
COPY --from=builder /app/.next/standalone ./
|
||||
COPY --from=builder /app/.next/static ./.next/static
|
||||
COPY --from=builder /app/public ./public
|
||||
EXPOSE 8082
|
||||
EXPOSE 7082
|
||||
CMD ["node", "server.js"]
|
||||
|
||||
@@ -1,35 +1,25 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { config } from "@/lib/api";
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
{ params }: { params: Promise<{ bookId: string; pageNum: string }> }
|
||||
) {
|
||||
const { bookId, pageNum } = await params;
|
||||
|
||||
// Récupérer les query params (format, width, quality)
|
||||
try {
|
||||
const { baseUrl, token } = config();
|
||||
const { searchParams } = new URL(request.url);
|
||||
const format = searchParams.get("format") || "webp";
|
||||
const width = searchParams.get("width") || "";
|
||||
const quality = searchParams.get("quality") || "";
|
||||
|
||||
// Construire l'URL vers l'API backend
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiUrl = new URL(`${apiBaseUrl}/books/${bookId}/pages/${pageNum}`);
|
||||
const apiUrl = new URL(`${baseUrl}/books/${bookId}/pages/${pageNum}`);
|
||||
apiUrl.searchParams.set("format", format);
|
||||
if (width) apiUrl.searchParams.set("width", width);
|
||||
if (quality) apiUrl.searchParams.set("quality", quality);
|
||||
|
||||
// Faire la requête à l'API
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
if (!token) {
|
||||
return new NextResponse("API token not configured", { status: 500 });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(apiUrl.toString(), {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { config } from "@/lib/api";
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
@@ -6,19 +7,10 @@ export async function GET(
|
||||
) {
|
||||
const { bookId } = await params;
|
||||
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiUrl = `${apiBaseUrl}/books/${bookId}/thumbnail`;
|
||||
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
if (!token) {
|
||||
return new NextResponse("API token not configured", { status: 500 });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(apiUrl, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
const { baseUrl, token } = config();
|
||||
const response = await fetch(`${baseUrl}/books/${bookId}/thumbnail`, {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
|
||||
@@ -1,39 +1,13 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { listFolders } from "@/lib/api";
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
if (!apiToken) {
|
||||
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
|
||||
}
|
||||
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const path = searchParams.get("path");
|
||||
|
||||
let apiUrl = `${apiBaseUrl}/folders`;
|
||||
if (path) {
|
||||
apiUrl += `?path=${encodeURIComponent(path)}`;
|
||||
}
|
||||
|
||||
const response = await fetch(apiUrl, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiToken}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: `API error: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const path = searchParams.get("path") || undefined;
|
||||
const data = await listFolders(path);
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error("Proxy error:", error);
|
||||
return NextResponse.json({ error: "Failed to fetch folders" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,36 +1,15 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { cancelJob } from "@/lib/api";
|
||||
|
||||
export async function POST(
|
||||
request: NextRequest,
|
||||
_request: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> }
|
||||
) {
|
||||
const { id } = await params;
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
if (!apiToken) {
|
||||
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(`${apiBaseUrl}/index/cancel/${id}`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiToken}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: `API error: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const data = await cancelJob(id);
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error("Proxy error:", error);
|
||||
return NextResponse.json({ error: "Failed to cancel job" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,35 +1,15 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { apiFetch, IndexJobDto } from "@/lib/api";
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
_request: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> }
|
||||
) {
|
||||
const { id } = await params;
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
if (!apiToken) {
|
||||
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(`${apiBaseUrl}/index/jobs/${id}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiToken}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: `API error: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const data = await apiFetch<IndexJobDto>(`/index/jobs/${id}`);
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error("Proxy error:", error);
|
||||
return NextResponse.json({ error: "Failed to fetch job" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,19 +1,12 @@
|
||||
import { NextRequest } from "next/server";
|
||||
import { config } from "@/lib/api";
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> }
|
||||
) {
|
||||
const { id } = await params;
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
if (!apiToken) {
|
||||
return new Response(
|
||||
`data: ${JSON.stringify({ error: "API token not configured" })}\n\n`,
|
||||
{ status: 500, headers: { "Content-Type": "text/event-stream" } }
|
||||
);
|
||||
}
|
||||
const { baseUrl, token } = config();
|
||||
|
||||
const stream = new ReadableStream({
|
||||
async start(controller) {
|
||||
@@ -27,10 +20,8 @@ export async function GET(
|
||||
if (!isActive) return;
|
||||
|
||||
try {
|
||||
const response = await fetch(`${apiBaseUrl}/index/jobs/${id}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiToken}`,
|
||||
},
|
||||
const response = await fetch(`${baseUrl}/index/jobs/${id}`, {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
});
|
||||
|
||||
if (response.ok && isActive) {
|
||||
|
||||
11
apps/backoffice/app/api/jobs/active/route.ts
Normal file
11
apps/backoffice/app/api/jobs/active/route.ts
Normal file
@@ -0,0 +1,11 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { apiFetch, IndexJobDto } from "@/lib/api";
|
||||
|
||||
export async function GET() {
|
||||
try {
|
||||
const data = await apiFetch<IndexJobDto[]>("/index/jobs/active");
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: "Failed to fetch active jobs" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
if (!apiToken) {
|
||||
return NextResponse.json({ error: "API token not configured" }, { status: 500 });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(`${apiBaseUrl}/index/status`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiToken}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json(
|
||||
{ error: `API error: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
console.error("Proxy error:", error);
|
||||
return NextResponse.json({ error: "Failed to fetch jobs" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -1,15 +1,8 @@
|
||||
import { NextRequest } from "next/server";
|
||||
import { config } from "@/lib/api";
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
const apiBaseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const apiToken = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
if (!apiToken) {
|
||||
return new Response(
|
||||
`data: ${JSON.stringify({ error: "API token not configured" })}\n\n`,
|
||||
{ status: 500, headers: { "Content-Type": "text/event-stream" } }
|
||||
);
|
||||
}
|
||||
const { baseUrl, token } = config();
|
||||
|
||||
const stream = new ReadableStream({
|
||||
async start(controller) {
|
||||
@@ -22,10 +15,8 @@ export async function GET(request: NextRequest) {
|
||||
if (!isActive) return;
|
||||
|
||||
try {
|
||||
const response = await fetch(`${apiBaseUrl}/index/status`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiToken}`,
|
||||
},
|
||||
const response = await fetch(`${baseUrl}/index/status`, {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
});
|
||||
|
||||
if (response.ok && isActive) {
|
||||
|
||||
@@ -1,29 +1,16 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { apiFetch, updateSetting } from "@/lib/api";
|
||||
|
||||
export async function GET(
|
||||
request: NextRequest,
|
||||
_request: NextRequest,
|
||||
{ params }: { params: Promise<{ key: string }> }
|
||||
) {
|
||||
try {
|
||||
const { key } = await params;
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
const response = await fetch(`${baseUrl}/settings/${key}`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
cache: "no-store"
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json({ error: "Failed to fetch setting" }, { status: response.status });
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
try {
|
||||
const data = await apiFetch<unknown>(`/settings/${key}`);
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
|
||||
return NextResponse.json({ error: "Failed to fetch setting" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -31,29 +18,12 @@ export async function POST(
|
||||
request: NextRequest,
|
||||
{ params }: { params: Promise<{ key: string }> }
|
||||
) {
|
||||
try {
|
||||
const { key } = await params;
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
const body = await request.json();
|
||||
|
||||
const response = await fetch(`${baseUrl}/settings/${key}`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
cache: "no-store"
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json({ error: "Failed to update setting" }, { status: response.status });
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
try {
|
||||
const { value } = await request.json();
|
||||
const data = await updateSetting(key, value);
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
|
||||
return NextResponse.json({ error: "Failed to update setting" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,25 +1,11 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { NextResponse } from "next/server";
|
||||
import { clearCache } from "@/lib/api";
|
||||
|
||||
export async function POST(request: NextRequest) {
|
||||
export async function POST() {
|
||||
try {
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
const response = await fetch(`${baseUrl}/settings/cache/clear`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
cache: "no-store"
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json({ error: "Failed to clear cache" }, { status: response.status });
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const data = await clearCache();
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
|
||||
return NextResponse.json({ error: "Failed to clear cache" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,24 +1,11 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { NextResponse } from "next/server";
|
||||
import { getCacheStats } from "@/lib/api";
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
export async function GET() {
|
||||
try {
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
const response = await fetch(`${baseUrl}/settings/cache/stats`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
cache: "no-store"
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json({ error: "Failed to fetch cache stats" }, { status: response.status });
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const data = await getCacheStats();
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
|
||||
return NextResponse.json({ error: "Failed to fetch cache stats" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
try {
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
|
||||
const response = await fetch(`${baseUrl}/settings`, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
cache: "no-store"
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return NextResponse.json({ error: "Failed to fetch settings" }, { status: response.status });
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return NextResponse.json(data);
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: "Internal server error" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import { fetchLibraries, getBookCoverUrl, BookDto, apiFetch } from "../../../lib/api";
|
||||
import { BookPreview } from "../../components/BookPreview";
|
||||
import Image from "next/image";
|
||||
import Link from "next/link";
|
||||
import { notFound } from "next/navigation";
|
||||
@@ -157,6 +158,12 @@ export default async function BookDetailPage({
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{book.page_count && book.page_count > 0 && (
|
||||
<div className="mt-8">
|
||||
<BookPreview bookId={book.id} pageCount={book.page_count} />
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
60
apps/backoffice/app/components/BookPreview.tsx
Normal file
60
apps/backoffice/app/components/BookPreview.tsx
Normal file
@@ -0,0 +1,60 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import Image from "next/image";
|
||||
|
||||
const PAGE_SIZE = 5;
|
||||
|
||||
export function BookPreview({ bookId, pageCount }: { bookId: string; pageCount: number }) {
|
||||
const [offset, setOffset] = useState(0);
|
||||
|
||||
const pages = Array.from({ length: PAGE_SIZE }, (_, i) => offset + i + 1).filter(
|
||||
(p) => p <= pageCount
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="bg-card rounded-xl border border-border p-6">
|
||||
<div className="flex items-center justify-between mb-4">
|
||||
<h2 className="text-lg font-semibold text-foreground">
|
||||
Preview
|
||||
<span className="ml-2 text-sm font-normal text-muted-foreground">
|
||||
pages {offset + 1}–{Math.min(offset + PAGE_SIZE, pageCount)} / {pageCount}
|
||||
</span>
|
||||
</h2>
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
onClick={() => setOffset((o) => Math.max(0, o - PAGE_SIZE))}
|
||||
disabled={offset === 0}
|
||||
className="px-3 py-1.5 text-sm rounded-lg border border-border bg-muted/50 text-foreground hover:bg-muted disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
|
||||
>
|
||||
← Prev
|
||||
</button>
|
||||
<button
|
||||
onClick={() => setOffset((o) => Math.min(o + PAGE_SIZE, pageCount - 1))}
|
||||
disabled={offset + PAGE_SIZE >= pageCount}
|
||||
className="px-3 py-1.5 text-sm rounded-lg border border-border bg-muted/50 text-foreground hover:bg-muted disabled:opacity-40 disabled:cursor-not-allowed transition-colors"
|
||||
>
|
||||
Next →
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-5 gap-3">
|
||||
{pages.map((pageNum) => (
|
||||
<div key={pageNum} className="flex flex-col items-center gap-1.5">
|
||||
<div className="relative w-full aspect-[2/3] bg-muted rounded-lg overflow-hidden border border-border">
|
||||
<Image
|
||||
src={`/api/books/${bookId}/pages/${pageNum}?format=webp&width=600&quality=80`}
|
||||
alt={`Page ${pageNum}`}
|
||||
fill
|
||||
className="object-contain"
|
||||
unoptimized
|
||||
/>
|
||||
</div>
|
||||
<span className="text-xs text-muted-foreground">{pageNum}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -247,7 +247,7 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
|
||||
<Icon name="performance" size="md" />
|
||||
Performance Limits
|
||||
</CardTitle>
|
||||
<CardDescription>Configure API performance and rate limiting</CardDescription>
|
||||
<CardDescription>Configure API performance, rate limiting, and thumbnail generation concurrency</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="space-y-4">
|
||||
@@ -266,6 +266,9 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
|
||||
}}
|
||||
onBlur={() => handleUpdateSetting("limits", settings.limits)}
|
||||
/>
|
||||
<p className="text-xs text-muted-foreground mt-1">
|
||||
Maximum number of page renders and thumbnail generations running in parallel
|
||||
</p>
|
||||
</FormField>
|
||||
<FormField className="flex-1">
|
||||
<label className="text-sm font-medium text-muted-foreground mb-1 block">Timeout (seconds)</label>
|
||||
@@ -299,7 +302,7 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
|
||||
</FormField>
|
||||
</FormRow>
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Note: Changes to limits require a server restart to take effect.
|
||||
Note: Changes to limits require a server restart to take effect. The "Concurrent Renders" setting controls both page rendering and thumbnail generation parallelism.
|
||||
</p>
|
||||
</div>
|
||||
</CardContent>
|
||||
@@ -424,7 +427,7 @@ export default function SettingsPage({ initialSettings, initialCacheStats, initi
|
||||
</div>
|
||||
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Note: Thumbnail settings are used during indexing. Existing thumbnails will not be regenerated automatically.
|
||||
Note: Thumbnail settings are used during indexing. Existing thumbnails will not be regenerated automatically. The concurrency for thumbnail generation is controlled by the "Concurrent Renders" setting in Performance Limits above.
|
||||
</p>
|
||||
</div>
|
||||
</CardContent>
|
||||
|
||||
@@ -89,8 +89,8 @@ export type SeriesDto = {
|
||||
first_book_id: string;
|
||||
};
|
||||
|
||||
function config() {
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:8080";
|
||||
export function config() {
|
||||
const baseUrl = process.env.API_BASE_URL || "http://api:7080";
|
||||
const token = process.env.API_BOOTSTRAP_TOKEN;
|
||||
if (!token) {
|
||||
throw new Error("API_BOOTSTRAP_TOKEN is required for backoffice");
|
||||
|
||||
@@ -21,7 +21,10 @@
|
||||
{
|
||||
"name": "next"
|
||||
}
|
||||
]
|
||||
],
|
||||
"paths": {
|
||||
"@/*": ["./*"]
|
||||
}
|
||||
},
|
||||
"include": [
|
||||
"next-env.d.ts",
|
||||
|
||||
104
apps/indexer/AGENTS.md
Normal file
104
apps/indexer/AGENTS.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# apps/indexer — Service d'indexation
|
||||
|
||||
Service background sur le port **7081**. Voir `AGENTS.md` racine pour les conventions globales.
|
||||
|
||||
## Structure des fichiers
|
||||
|
||||
| Fichier | Rôle |
|
||||
|---------|------|
|
||||
| `main.rs` | Point d'entrée, initialisation, lancement du worker |
|
||||
| `lib.rs` | `AppState` (pool, meili_url, meili_master_key) |
|
||||
| `worker.rs` | Boucle principale : claim job → process → cleanup stale |
|
||||
| `job.rs` | `claim_next_job`, `process_job`, `fail_job`, `cleanup_stale_jobs` |
|
||||
| `scanner.rs` | Phase 1 discovery : WalkDir + `parse_metadata_fast` (zéro I/O archive), skip dossiers inchangés via mtime, batching DB |
|
||||
| `analyzer.rs` | Phase 2 analysis : ouvre chaque archive une fois (`analyze_book`), génère page_count + thumbnail WebP |
|
||||
| `batch.rs` | `flush_all_batches` avec UNNEST, structures `BookInsert/Update/FileInsert/Update/ErrorInsert` |
|
||||
| `scheduler.rs` | Auto-scan : vérifie toutes les 60s les bibliothèques à monitorer |
|
||||
| `watcher.rs` | File watcher temps réel |
|
||||
| `meili.rs` | Indexation/sync Meilisearch |
|
||||
| `api.rs` | Endpoints HTTP de l'indexer (/health, /ready) |
|
||||
| `utils.rs` | `remap_libraries_path`, `unmap_libraries_path`, `compute_fingerprint`, `kind_from_format` |
|
||||
|
||||
## Cycle de vie d'un job
|
||||
|
||||
```
|
||||
claim_next_job (UPDATE ... RETURNING, status pending→running)
|
||||
└─ process_job
|
||||
├─ Phase 1 : scanner::scan_library_discovery
|
||||
│ ├─ WalkDir + parse_metadata_fast (zéro I/O archive)
|
||||
│ ├─ skip dossiers via directory_mtimes (table DB)
|
||||
│ └─ INSERT books (page_count=NULL) → livres visibles immédiatement
|
||||
├─ meili::sync_meili
|
||||
├─ analyzer::cleanup_orphaned_thumbnails (full_rebuild uniquement)
|
||||
└─ Phase 2 : analyzer::analyze_library_books
|
||||
├─ SELECT books WHERE page_count IS NULL
|
||||
├─ parsers::analyze_book → (page_count, first_page_bytes)
|
||||
├─ generate_thumbnail (WebP, Lanczos3)
|
||||
└─ UPDATE books SET page_count, thumbnail_path
|
||||
|
||||
Jobs spéciaux :
|
||||
thumbnail_rebuild → analyze_library_books(thumbnail_only=true)
|
||||
thumbnail_regenerate → regenerate_thumbnails (clear + re-analyze)
|
||||
```
|
||||
|
||||
- Annulation : `is_job_cancelled` vérifié toutes les 10 fichiers ou 1s — retourne `Err("Job cancelled")`
|
||||
- Jobs stale (running au redémarrage) → nettoyés par `cleanup_stale_jobs` au boot
|
||||
|
||||
## Pattern batch (batch.rs)
|
||||
|
||||
Toutes les opérations DB massives passent par `flush_all_batches` avec UNNEST :
|
||||
|
||||
```rust
|
||||
// Accumuler dans des Vec<BookInsert>, Vec<FileInsert>, etc.
|
||||
books_to_insert.push(BookInsert { ... });
|
||||
|
||||
// Flush quand plein ou en fin de scan
|
||||
if books_to_insert.len() >= BATCH_SIZE {
|
||||
flush_all_batches(&pool, &mut books_update, &mut files_update,
|
||||
&mut books_insert, &mut files_insert, &mut errors_insert).await?;
|
||||
}
|
||||
```
|
||||
|
||||
Toutes les opérations du flush sont dans une seule transaction.
|
||||
|
||||
## Scan filesystem — architecture 2 phases
|
||||
|
||||
### Phase 1 : Discovery (`scanner.rs`)
|
||||
|
||||
Pipeline allégé — **zéro ouverture d'archive** :
|
||||
1. Charger `directory_mtimes` depuis la DB
|
||||
2. WalkDir : pour chaque dossier, comparer mtime filesystem vs mtime stocké → skip si inchangé
|
||||
3. Pour chaque fichier : `parse_metadata_fast` (title/series/volume depuis filename uniquement)
|
||||
4. INSERT/UPDATE avec `page_count = NULL` — les livres sont visibles immédiatement
|
||||
5. Upsert `directory_mtimes` en fin de scan
|
||||
|
||||
Fingerprint = SHA256(taille + mtime + filename) pour détecter les changements sans relire le fichier.
|
||||
|
||||
### Phase 2 : Analysis (`analyzer.rs`)
|
||||
|
||||
Traitement progressif en background :
|
||||
- Query `WHERE page_count IS NULL` (ou `thumbnail_path IS NULL` pour thumbnail jobs)
|
||||
- Concurrence bornée (`futures::stream::for_each_concurrent`, défaut 4)
|
||||
- Par livre : `parsers::analyze_book(path, format)` → `(page_count, first_page_bytes)`
|
||||
- Génération thumbnail : resize Lanczos3 + encode WebP
|
||||
- UPDATE `books SET page_count, thumbnail_path`
|
||||
- Config lue depuis `app_settings` (clés `'thumbnail'` et `'limits'`)
|
||||
|
||||
## Path remapping
|
||||
|
||||
```rust
|
||||
// abs_path en DB = chemin conteneur (/libraries/...)
|
||||
// Sur l'hôte : LIBRARIES_ROOT_PATH remplace /libraries
|
||||
utils::remap_libraries_path(&abs_path) // DB → filesystem local
|
||||
utils::unmap_libraries_path(&local_path) // filesystem local → DB
|
||||
```
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Thumbnails** : générés **directement par l'indexer** (phase 2, `analyzer.rs`). L'API ne gère plus la génération — elle crée juste les jobs en DB.
|
||||
- **page_count = NULL** : après la phase discovery, tous les nouveaux livres ont `page_count = NULL`. La phase analysis les remplit progressivement. Ne pas confondre avec une erreur.
|
||||
- **directory_mtimes** : table DB qui stocke le mtime de chaque dossier scanné. Vidée au full_rebuild, mise à jour après chaque scan. Permet de skipper les dossiers inchangés en scan incrémental.
|
||||
- **full_rebuild** : supprime toutes les données puis re-insère. Ignore les fingerprints et les directory_mtimes.
|
||||
- **Annulation** : vérifier `is_job_cancelled` régulièrement pour respecter les annulations utilisateur.
|
||||
- **Watcher + scheduler** : tournent en tâches tokio séparées dans `worker.rs`, en parallèle de la boucle principale.
|
||||
- **spawn_blocking** : l'ouverture d'archive (`analyze_book`) et la génération de thumbnail sont des opérations bloquantes — toujours les wrapper dans `tokio::task::spawn_blocking`.
|
||||
@@ -4,10 +4,14 @@ version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[lib]
|
||||
|
||||
[dependencies]
|
||||
anyhow.workspace = true
|
||||
axum.workspace = true
|
||||
chrono.workspace = true
|
||||
futures = "0.3"
|
||||
image.workspace = true
|
||||
notify = "6.1"
|
||||
parsers = { path = "../../crates/parsers" }
|
||||
rand.workspace = true
|
||||
@@ -23,3 +27,4 @@ tracing.workspace = true
|
||||
tracing-subscriber.workspace = true
|
||||
uuid.workspace = true
|
||||
walkdir.workspace = true
|
||||
webp.workspace = true
|
||||
|
||||
@@ -21,7 +21,11 @@ RUN --mount=type=cache,target=/sccache \
|
||||
cargo build --release -p indexer
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates wget unrar-free && rm -rf /var/lib/apt/lists/*
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates wget \
|
||||
unrar-free unar \
|
||||
poppler-utils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY --from=builder /app/target/release/indexer /usr/local/bin/indexer
|
||||
EXPOSE 8081
|
||||
EXPOSE 7081
|
||||
CMD ["/usr/local/bin/indexer"]
|
||||
|
||||
442
apps/indexer/src/analyzer.rs
Normal file
442
apps/indexer/src/analyzer.rs
Normal file
@@ -0,0 +1,442 @@
|
||||
use anyhow::Result;
|
||||
use futures::stream::{self, StreamExt};
|
||||
use image::GenericImageView;
|
||||
use parsers::{analyze_book, BookFormat};
|
||||
use sqlx::Row;
|
||||
use std::path::Path;
|
||||
use std::sync::atomic::{AtomicI32, Ordering};
|
||||
use std::sync::Arc;
|
||||
use tracing::{info, warn};
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::{utils, AppState};
|
||||
|
||||
#[derive(Clone)]
|
||||
struct ThumbnailConfig {
|
||||
enabled: bool,
|
||||
width: u32,
|
||||
height: u32,
|
||||
quality: u8,
|
||||
directory: String,
|
||||
}
|
||||
|
||||
async fn load_thumbnail_config(pool: &sqlx::PgPool) -> ThumbnailConfig {
|
||||
let fallback = ThumbnailConfig {
|
||||
enabled: true,
|
||||
width: 300,
|
||||
height: 400,
|
||||
quality: 80,
|
||||
directory: "/data/thumbnails".to_string(),
|
||||
};
|
||||
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'thumbnail'"#)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
|
||||
match row {
|
||||
Ok(Some(row)) => {
|
||||
let value: serde_json::Value = row.get("value");
|
||||
ThumbnailConfig {
|
||||
enabled: value
|
||||
.get("enabled")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(fallback.enabled),
|
||||
width: value
|
||||
.get("width")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as u32)
|
||||
.unwrap_or(fallback.width),
|
||||
height: value
|
||||
.get("height")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as u32)
|
||||
.unwrap_or(fallback.height),
|
||||
quality: value
|
||||
.get("quality")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as u8)
|
||||
.unwrap_or(fallback.quality),
|
||||
directory: value
|
||||
.get("directory")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|s| s.to_string())
|
||||
.unwrap_or_else(|| fallback.directory.clone()),
|
||||
}
|
||||
}
|
||||
_ => fallback,
|
||||
}
|
||||
}
|
||||
|
||||
async fn load_thumbnail_concurrency(pool: &sqlx::PgPool) -> usize {
|
||||
let default_concurrency = 4;
|
||||
let row = sqlx::query(r#"SELECT value FROM app_settings WHERE key = 'limits'"#)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
|
||||
match row {
|
||||
Ok(Some(row)) => {
|
||||
let value: serde_json::Value = row.get("value");
|
||||
value
|
||||
.get("concurrent_renders")
|
||||
.and_then(|v| v.as_u64())
|
||||
.map(|v| v as usize)
|
||||
.unwrap_or(default_concurrency)
|
||||
}
|
||||
_ => default_concurrency,
|
||||
}
|
||||
}
|
||||
|
||||
fn generate_thumbnail(image_bytes: &[u8], config: &ThumbnailConfig) -> anyhow::Result<Vec<u8>> {
|
||||
let img = image::load_from_memory(image_bytes)
|
||||
.map_err(|e| anyhow::anyhow!("failed to load image: {}", e))?;
|
||||
let (orig_w, orig_h) = img.dimensions();
|
||||
let ratio_w = config.width as f32 / orig_w as f32;
|
||||
let ratio_h = config.height as f32 / orig_h as f32;
|
||||
let ratio = ratio_w.min(ratio_h);
|
||||
let new_w = (orig_w as f32 * ratio) as u32;
|
||||
let new_h = (orig_h as f32 * ratio) as u32;
|
||||
let resized = img.resize(new_w, new_h, image::imageops::FilterType::Lanczos3);
|
||||
let rgba = resized.to_rgba8();
|
||||
let (w, h) = rgba.dimensions();
|
||||
let rgb_data: Vec<u8> = rgba.pixels().flat_map(|p| [p[0], p[1], p[2]]).collect();
|
||||
let quality = f32::max(config.quality as f32, 85.0);
|
||||
let webp_data = webp::Encoder::new(&rgb_data, webp::PixelLayout::Rgb, w, h).encode(quality);
|
||||
Ok(webp_data.to_vec())
|
||||
}
|
||||
|
||||
fn save_thumbnail(
|
||||
book_id: Uuid,
|
||||
thumbnail_bytes: &[u8],
|
||||
config: &ThumbnailConfig,
|
||||
) -> anyhow::Result<String> {
|
||||
let dir = Path::new(&config.directory);
|
||||
std::fs::create_dir_all(dir)?;
|
||||
let filename = format!("{}.webp", book_id);
|
||||
let path = dir.join(&filename);
|
||||
std::fs::write(&path, thumbnail_bytes)?;
|
||||
Ok(path.to_string_lossy().to_string())
|
||||
}
|
||||
|
||||
fn book_format_from_str(s: &str) -> Option<BookFormat> {
|
||||
match s {
|
||||
"cbz" => Some(BookFormat::Cbz),
|
||||
"cbr" => Some(BookFormat::Cbr),
|
||||
"pdf" => Some(BookFormat::Pdf),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Phase 2 — Analysis: open each unanalyzed archive once, extract page_count + thumbnail.
|
||||
/// `thumbnail_only` = true: only process books missing thumbnail (page_count may already be set).
|
||||
/// `thumbnail_only` = false: process books missing page_count.
|
||||
pub async fn analyze_library_books(
|
||||
state: &AppState,
|
||||
job_id: Uuid,
|
||||
library_id: Option<Uuid>,
|
||||
thumbnail_only: bool,
|
||||
) -> Result<()> {
|
||||
let config = load_thumbnail_config(&state.pool).await;
|
||||
|
||||
if !config.enabled {
|
||||
info!("[ANALYZER] Thumbnails disabled, skipping analysis phase");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let concurrency = load_thumbnail_concurrency(&state.pool).await;
|
||||
|
||||
// Query books that need analysis
|
||||
let query_filter = if thumbnail_only {
|
||||
"b.thumbnail_path IS NULL"
|
||||
} else {
|
||||
"b.page_count IS NULL"
|
||||
};
|
||||
|
||||
let sql = format!(
|
||||
r#"
|
||||
SELECT b.id AS book_id, bf.abs_path, bf.format
|
||||
FROM books b
|
||||
JOIN book_files bf ON bf.book_id = b.id
|
||||
WHERE (b.library_id = $1 OR $1 IS NULL)
|
||||
AND {}
|
||||
"#,
|
||||
query_filter
|
||||
);
|
||||
|
||||
let rows = sqlx::query(&sql)
|
||||
.bind(library_id)
|
||||
.fetch_all(&state.pool)
|
||||
.await?;
|
||||
|
||||
if rows.is_empty() {
|
||||
info!("[ANALYZER] No books to analyze");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let total = rows.len() as i32;
|
||||
info!(
|
||||
"[ANALYZER] Analyzing {} books (thumbnail_only={}, concurrency={})",
|
||||
total, thumbnail_only, concurrency
|
||||
);
|
||||
|
||||
// Update job status
|
||||
let _ = sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'generating_thumbnails', total_files = $2, processed_files = 0, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(total)
|
||||
.execute(&state.pool)
|
||||
.await;
|
||||
|
||||
let processed_count = Arc::new(AtomicI32::new(0));
|
||||
|
||||
struct BookTask {
|
||||
book_id: Uuid,
|
||||
abs_path: String,
|
||||
format: String,
|
||||
}
|
||||
|
||||
let tasks: Vec<BookTask> = rows
|
||||
.into_iter()
|
||||
.map(|row| BookTask {
|
||||
book_id: row.get("book_id"),
|
||||
abs_path: row.get("abs_path"),
|
||||
format: row.get("format"),
|
||||
})
|
||||
.collect();
|
||||
|
||||
stream::iter(tasks)
|
||||
.for_each_concurrent(concurrency, |task| {
|
||||
let processed_count = processed_count.clone();
|
||||
let pool = state.pool.clone();
|
||||
let config = config.clone();
|
||||
|
||||
async move {
|
||||
let local_path = utils::remap_libraries_path(&task.abs_path);
|
||||
let path = Path::new(&local_path);
|
||||
|
||||
let format = match book_format_from_str(&task.format) {
|
||||
Some(f) => f,
|
||||
None => {
|
||||
warn!("[ANALYZER] Unknown format '{}' for book {}", task.format, task.book_id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Run blocking archive I/O on a thread pool
|
||||
let book_id = task.book_id;
|
||||
let path_owned = path.to_path_buf();
|
||||
let analyze_result = tokio::task::spawn_blocking(move || {
|
||||
analyze_book(&path_owned, format)
|
||||
})
|
||||
.await;
|
||||
|
||||
let (page_count, image_bytes) = match analyze_result {
|
||||
Ok(Ok(result)) => result,
|
||||
Ok(Err(e)) => {
|
||||
warn!("[ANALYZER] analyze_book failed for book {}: {}", book_id, e);
|
||||
// Mark parse_status = error in book_files
|
||||
let _ = sqlx::query(
|
||||
"UPDATE book_files SET parse_status = 'error', parse_error_opt = $2 WHERE book_id = $1",
|
||||
)
|
||||
.bind(book_id)
|
||||
.bind(e.to_string())
|
||||
.execute(&pool)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
warn!("[ANALYZER] spawn_blocking error for book {}: {}", book_id, e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Generate thumbnail
|
||||
let thumb_result = tokio::task::spawn_blocking({
|
||||
let config = config.clone();
|
||||
move || generate_thumbnail(&image_bytes, &config)
|
||||
})
|
||||
.await;
|
||||
|
||||
let thumb_bytes = match thumb_result {
|
||||
Ok(Ok(b)) => b,
|
||||
Ok(Err(e)) => {
|
||||
warn!("[ANALYZER] thumbnail generation failed for book {}: {}", book_id, e);
|
||||
// Still update page_count even if thumbnail fails
|
||||
let _ = sqlx::query(
|
||||
"UPDATE books SET page_count = $1 WHERE id = $2",
|
||||
)
|
||||
.bind(page_count)
|
||||
.bind(book_id)
|
||||
.execute(&pool)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
warn!("[ANALYZER] spawn_blocking thumbnail error for book {}: {}", book_id, e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Save thumbnail file
|
||||
let save_result = {
|
||||
let config = config.clone();
|
||||
tokio::task::spawn_blocking(move || save_thumbnail(book_id, &thumb_bytes, &config))
|
||||
.await
|
||||
};
|
||||
|
||||
let thumb_path = match save_result {
|
||||
Ok(Ok(p)) => p,
|
||||
Ok(Err(e)) => {
|
||||
warn!("[ANALYZER] save_thumbnail failed for book {}: {}", book_id, e);
|
||||
let _ = sqlx::query("UPDATE books SET page_count = $1 WHERE id = $2")
|
||||
.bind(page_count)
|
||||
.bind(book_id)
|
||||
.execute(&pool)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
warn!("[ANALYZER] spawn_blocking save error for book {}: {}", book_id, e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Update DB
|
||||
if let Err(e) = sqlx::query(
|
||||
"UPDATE books SET page_count = $1, thumbnail_path = $2 WHERE id = $3",
|
||||
)
|
||||
.bind(page_count)
|
||||
.bind(&thumb_path)
|
||||
.bind(book_id)
|
||||
.execute(&pool)
|
||||
.await
|
||||
{
|
||||
warn!("[ANALYZER] DB update failed for book {}: {}", book_id, e);
|
||||
return;
|
||||
}
|
||||
|
||||
let processed = processed_count.fetch_add(1, Ordering::Relaxed) + 1;
|
||||
let percent = (processed as f64 / total as f64 * 100.0) as i32;
|
||||
let _ = sqlx::query(
|
||||
"UPDATE index_jobs SET processed_files = $2, progress_percent = $3 WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(processed)
|
||||
.bind(percent)
|
||||
.execute(&pool)
|
||||
.await;
|
||||
}
|
||||
})
|
||||
.await;
|
||||
|
||||
let final_count = processed_count.load(Ordering::Relaxed);
|
||||
info!(
|
||||
"[ANALYZER] Analysis complete: {}/{} books processed",
|
||||
final_count, total
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Clear thumbnail files and DB references for books in scope, then re-analyze.
|
||||
pub async fn regenerate_thumbnails(
|
||||
state: &AppState,
|
||||
job_id: Uuid,
|
||||
library_id: Option<Uuid>,
|
||||
) -> Result<()> {
|
||||
let config = load_thumbnail_config(&state.pool).await;
|
||||
|
||||
// Delete thumbnail files for all books in scope
|
||||
let book_ids_to_clear: Vec<Uuid> = sqlx::query_scalar(
|
||||
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL) AND thumbnail_path IS NOT NULL"#,
|
||||
)
|
||||
.bind(library_id)
|
||||
.fetch_all(&state.pool)
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
|
||||
let mut deleted_count = 0usize;
|
||||
for book_id in &book_ids_to_clear {
|
||||
let filename = format!("{}.webp", book_id);
|
||||
let thumbnail_path = Path::new(&config.directory).join(&filename);
|
||||
if thumbnail_path.exists() {
|
||||
if let Err(e) = std::fs::remove_file(&thumbnail_path) {
|
||||
warn!(
|
||||
"[ANALYZER] Failed to delete thumbnail {}: {}",
|
||||
thumbnail_path.display(),
|
||||
e
|
||||
);
|
||||
} else {
|
||||
deleted_count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
info!(
|
||||
"[ANALYZER] Deleted {} thumbnail files for regeneration",
|
||||
deleted_count
|
||||
);
|
||||
|
||||
// Clear thumbnail_path in DB
|
||||
sqlx::query(
|
||||
r#"UPDATE books SET thumbnail_path = NULL WHERE (library_id = $1 OR $1 IS NULL)"#,
|
||||
)
|
||||
.bind(library_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
// Re-analyze all books (now thumbnail_path IS NULL for all)
|
||||
analyze_library_books(state, job_id, library_id, true).await
|
||||
}
|
||||
|
||||
/// Delete orphaned thumbnail files (books deleted in full_rebuild get new UUIDs).
|
||||
pub async fn cleanup_orphaned_thumbnails(
|
||||
state: &AppState,
|
||||
library_id: Option<Uuid>,
|
||||
) -> Result<()> {
|
||||
let config = load_thumbnail_config(&state.pool).await;
|
||||
|
||||
let existing_book_ids: std::collections::HashSet<Uuid> = sqlx::query_scalar(
|
||||
r#"SELECT id FROM books WHERE (library_id = $1 OR $1 IS NULL)"#,
|
||||
)
|
||||
.bind(library_id)
|
||||
.fetch_all(&state.pool)
|
||||
.await
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.collect();
|
||||
|
||||
let thumbnail_dir = Path::new(&config.directory);
|
||||
if !thumbnail_dir.exists() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let mut deleted_count = 0usize;
|
||||
if let Ok(entries) = std::fs::read_dir(thumbnail_dir) {
|
||||
for entry in entries.flatten() {
|
||||
if let Some(file_name) = entry.file_name().to_str() {
|
||||
if file_name.ends_with(".webp") {
|
||||
if let Some(book_id_str) = file_name.strip_suffix(".webp") {
|
||||
if let Ok(book_id) = Uuid::parse_str(book_id_str) {
|
||||
if !existing_book_ids.contains(&book_id) {
|
||||
if let Err(e) = std::fs::remove_file(entry.path()) {
|
||||
warn!(
|
||||
"Failed to delete orphaned thumbnail {}: {}",
|
||||
entry.path().display(),
|
||||
e
|
||||
);
|
||||
} else {
|
||||
deleted_count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info!(
|
||||
"[ANALYZER] Deleted {} orphaned thumbnail files",
|
||||
deleted_count
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
16
apps/indexer/src/api.rs
Normal file
16
apps/indexer/src/api.rs
Normal file
@@ -0,0 +1,16 @@
|
||||
use axum::{extract::State, http::StatusCode, Json};
|
||||
use serde_json;
|
||||
|
||||
use crate::AppState;
|
||||
|
||||
pub async fn health() -> &'static str {
|
||||
"ok"
|
||||
}
|
||||
|
||||
pub async fn ready(State(state): State<AppState>) -> Result<Json<serde_json::Value>, StatusCode> {
|
||||
sqlx::query("SELECT 1")
|
||||
.execute(&state.pool)
|
||||
.await
|
||||
.map_err(|_| StatusCode::SERVICE_UNAVAILABLE)?;
|
||||
Ok(Json(serde_json::json!({"status": "ready"})))
|
||||
}
|
||||
233
apps/indexer/src/batch.rs
Normal file
233
apps/indexer/src/batch.rs
Normal file
@@ -0,0 +1,233 @@
|
||||
use anyhow::Result;
|
||||
use chrono::{DateTime, Utc};
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
// Batched update data structures
|
||||
pub struct BookUpdate {
|
||||
pub book_id: Uuid,
|
||||
pub title: String,
|
||||
pub kind: String,
|
||||
pub series: Option<String>,
|
||||
pub volume: Option<i32>,
|
||||
pub page_count: Option<i32>,
|
||||
}
|
||||
|
||||
pub struct FileUpdate {
|
||||
pub file_id: Uuid,
|
||||
pub format: String,
|
||||
pub size_bytes: i64,
|
||||
pub mtime: DateTime<Utc>,
|
||||
pub fingerprint: String,
|
||||
}
|
||||
|
||||
pub struct BookInsert {
|
||||
pub book_id: Uuid,
|
||||
pub library_id: Uuid,
|
||||
pub kind: String,
|
||||
pub title: String,
|
||||
pub series: Option<String>,
|
||||
pub volume: Option<i32>,
|
||||
pub page_count: Option<i32>,
|
||||
pub thumbnail_path: Option<String>,
|
||||
}
|
||||
|
||||
pub struct FileInsert {
|
||||
pub file_id: Uuid,
|
||||
pub book_id: Uuid,
|
||||
pub format: String,
|
||||
pub abs_path: String,
|
||||
pub size_bytes: i64,
|
||||
pub mtime: DateTime<Utc>,
|
||||
pub fingerprint: String,
|
||||
pub parse_status: String,
|
||||
pub parse_error: Option<String>,
|
||||
}
|
||||
|
||||
pub struct ErrorInsert {
|
||||
pub job_id: Uuid,
|
||||
pub file_path: String,
|
||||
pub error_message: String,
|
||||
}
|
||||
|
||||
pub async fn flush_all_batches(
|
||||
pool: &PgPool,
|
||||
books_update: &mut Vec<BookUpdate>,
|
||||
files_update: &mut Vec<FileUpdate>,
|
||||
books_insert: &mut Vec<BookInsert>,
|
||||
files_insert: &mut Vec<FileInsert>,
|
||||
errors_insert: &mut Vec<ErrorInsert>,
|
||||
) -> Result<()> {
|
||||
if books_update.is_empty() && files_update.is_empty() && books_insert.is_empty() && files_insert.is_empty() && errors_insert.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let start = std::time::Instant::now();
|
||||
let mut tx = pool.begin().await?;
|
||||
|
||||
// Batch update books using UNNEST
|
||||
if !books_update.is_empty() {
|
||||
let book_ids: Vec<Uuid> = books_update.iter().map(|b| b.book_id).collect();
|
||||
let titles: Vec<String> = books_update.iter().map(|b| b.title.clone()).collect();
|
||||
let kinds: Vec<String> = books_update.iter().map(|b| b.kind.clone()).collect();
|
||||
let series: Vec<Option<String>> = books_update.iter().map(|b| b.series.clone()).collect();
|
||||
let volumes: Vec<Option<i32>> = books_update.iter().map(|b| b.volume).collect();
|
||||
let page_counts: Vec<Option<i32>> = books_update.iter().map(|b| b.page_count).collect();
|
||||
|
||||
sqlx::query(
|
||||
r#"
|
||||
UPDATE books SET
|
||||
title = data.title,
|
||||
kind = data.kind,
|
||||
series = data.series,
|
||||
volume = data.volume,
|
||||
page_count = data.page_count,
|
||||
updated_at = NOW()
|
||||
FROM (
|
||||
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::text[], $4::text[], $5::int[], $6::int[])
|
||||
AS t(book_id, title, kind, series, volume, page_count)
|
||||
) AS data
|
||||
WHERE books.id = data.book_id
|
||||
"#
|
||||
)
|
||||
.bind(&book_ids)
|
||||
.bind(&titles)
|
||||
.bind(&kinds)
|
||||
.bind(&series)
|
||||
.bind(&volumes)
|
||||
.bind(&page_counts)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
books_update.clear();
|
||||
}
|
||||
|
||||
// Batch update files using UNNEST
|
||||
if !files_update.is_empty() {
|
||||
let file_ids: Vec<Uuid> = files_update.iter().map(|f| f.file_id).collect();
|
||||
let formats: Vec<String> = files_update.iter().map(|f| f.format.clone()).collect();
|
||||
let sizes: Vec<i64> = files_update.iter().map(|f| f.size_bytes).collect();
|
||||
let mtimes: Vec<DateTime<Utc>> = files_update.iter().map(|f| f.mtime).collect();
|
||||
let fingerprints: Vec<String> = files_update.iter().map(|f| f.fingerprint.clone()).collect();
|
||||
|
||||
sqlx::query(
|
||||
r#"
|
||||
UPDATE book_files SET
|
||||
format = data.format,
|
||||
size_bytes = data.size,
|
||||
mtime = data.mtime,
|
||||
fingerprint = data.fp,
|
||||
parse_status = 'ok',
|
||||
parse_error_opt = NULL,
|
||||
updated_at = NOW()
|
||||
FROM (
|
||||
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::bigint[], $4::timestamptz[], $5::text[])
|
||||
AS t(file_id, format, size, mtime, fp)
|
||||
) AS data
|
||||
WHERE book_files.id = data.file_id
|
||||
"#
|
||||
)
|
||||
.bind(&file_ids)
|
||||
.bind(&formats)
|
||||
.bind(&sizes)
|
||||
.bind(&mtimes)
|
||||
.bind(&fingerprints)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
files_update.clear();
|
||||
}
|
||||
|
||||
// Batch insert books using UNNEST
|
||||
if !books_insert.is_empty() {
|
||||
let book_ids: Vec<Uuid> = books_insert.iter().map(|b| b.book_id).collect();
|
||||
let library_ids: Vec<Uuid> = books_insert.iter().map(|b| b.library_id).collect();
|
||||
let kinds: Vec<String> = books_insert.iter().map(|b| b.kind.clone()).collect();
|
||||
let titles: Vec<String> = books_insert.iter().map(|b| b.title.clone()).collect();
|
||||
let series: Vec<Option<String>> = books_insert.iter().map(|b| b.series.clone()).collect();
|
||||
let volumes: Vec<Option<i32>> = books_insert.iter().map(|b| b.volume).collect();
|
||||
let page_counts: Vec<Option<i32>> = books_insert.iter().map(|b| b.page_count).collect();
|
||||
let thumbnail_paths: Vec<Option<String>> = books_insert.iter().map(|b| b.thumbnail_path.clone()).collect();
|
||||
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO books (id, library_id, kind, title, series, volume, page_count, thumbnail_path)
|
||||
SELECT * FROM UNNEST($1::uuid[], $2::uuid[], $3::text[], $4::text[], $5::text[], $6::int[], $7::int[], $8::text[])
|
||||
AS t(id, library_id, kind, title, series, volume, page_count, thumbnail_path)
|
||||
"#
|
||||
)
|
||||
.bind(&book_ids)
|
||||
.bind(&library_ids)
|
||||
.bind(&kinds)
|
||||
.bind(&titles)
|
||||
.bind(&series)
|
||||
.bind(&volumes)
|
||||
.bind(&page_counts)
|
||||
.bind(&thumbnail_paths)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
books_insert.clear();
|
||||
}
|
||||
|
||||
// Batch insert files using UNNEST
|
||||
if !files_insert.is_empty() {
|
||||
let file_ids: Vec<Uuid> = files_insert.iter().map(|f| f.file_id).collect();
|
||||
let book_ids: Vec<Uuid> = files_insert.iter().map(|f| f.book_id).collect();
|
||||
let formats: Vec<String> = files_insert.iter().map(|f| f.format.clone()).collect();
|
||||
let abs_paths: Vec<String> = files_insert.iter().map(|f| f.abs_path.clone()).collect();
|
||||
let sizes: Vec<i64> = files_insert.iter().map(|f| f.size_bytes).collect();
|
||||
let mtimes: Vec<DateTime<Utc>> = files_insert.iter().map(|f| f.mtime).collect();
|
||||
let fingerprints: Vec<String> = files_insert.iter().map(|f| f.fingerprint.clone()).collect();
|
||||
let statuses: Vec<String> = files_insert.iter().map(|f| f.parse_status.clone()).collect();
|
||||
let errors: Vec<Option<String>> = files_insert.iter().map(|f| f.parse_error.clone()).collect();
|
||||
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO book_files (id, book_id, format, abs_path, size_bytes, mtime, fingerprint, parse_status, parse_error_opt)
|
||||
SELECT * FROM UNNEST($1::uuid[], $2::uuid[], $3::text[], $4::text[], $5::bigint[], $6::timestamptz[], $7::text[], $8::text[], $9::text[])
|
||||
AS t(id, book_id, format, abs_path, size_bytes, mtime, fingerprint, parse_status, parse_error_opt)
|
||||
"#
|
||||
)
|
||||
.bind(&file_ids)
|
||||
.bind(&book_ids)
|
||||
.bind(&formats)
|
||||
.bind(&abs_paths)
|
||||
.bind(&sizes)
|
||||
.bind(&mtimes)
|
||||
.bind(&fingerprints)
|
||||
.bind(&statuses)
|
||||
.bind(&errors)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
files_insert.clear();
|
||||
}
|
||||
|
||||
// Batch insert errors using UNNEST
|
||||
if !errors_insert.is_empty() {
|
||||
let job_ids: Vec<Uuid> = errors_insert.iter().map(|e| e.job_id).collect();
|
||||
let file_paths: Vec<String> = errors_insert.iter().map(|e| e.file_path.clone()).collect();
|
||||
let messages: Vec<String> = errors_insert.iter().map(|e| e.error_message.clone()).collect();
|
||||
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO index_job_errors (job_id, file_path, error_message)
|
||||
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::text[])
|
||||
AS t(job_id, file_path, error_message)
|
||||
"#
|
||||
)
|
||||
.bind(&job_ids)
|
||||
.bind(&file_paths)
|
||||
.bind(&messages)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
errors_insert.clear();
|
||||
}
|
||||
|
||||
tx.commit().await?;
|
||||
tracing::info!("[BATCH] Flushed all batches in {:?}", start.elapsed());
|
||||
|
||||
Ok(())
|
||||
}
|
||||
329
apps/indexer/src/job.rs
Normal file
329
apps/indexer/src/job.rs
Normal file
@@ -0,0 +1,329 @@
|
||||
use anyhow::Result;
|
||||
use rayon::prelude::*;
|
||||
use sqlx::{PgPool, Row};
|
||||
use tracing::{error, info};
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::{analyzer, meili, scanner, AppState};
|
||||
|
||||
pub async fn cleanup_stale_jobs(pool: &PgPool) -> Result<()> {
|
||||
let result = sqlx::query(
|
||||
r#"
|
||||
UPDATE index_jobs
|
||||
SET status = 'failed',
|
||||
finished_at = NOW(),
|
||||
error_opt = 'Job interrupted by indexer restart'
|
||||
WHERE status = 'running'
|
||||
AND started_at < NOW() - INTERVAL '5 minutes'
|
||||
RETURNING id
|
||||
"#,
|
||||
)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
|
||||
if !result.is_empty() {
|
||||
let count = result.len();
|
||||
let ids: Vec<String> = result
|
||||
.iter()
|
||||
.map(|row| row.get::<Uuid, _>("id").to_string())
|
||||
.collect();
|
||||
info!(
|
||||
"[CLEANUP] Marked {} stale job(s) as failed: {}",
|
||||
count,
|
||||
ids.join(", ")
|
||||
);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn claim_next_job(pool: &PgPool) -> Result<Option<(Uuid, Option<Uuid>)>> {
|
||||
let mut tx = pool.begin().await?;
|
||||
|
||||
let row = sqlx::query(
|
||||
r#"
|
||||
SELECT j.id, j.type, j.library_id
|
||||
FROM index_jobs j
|
||||
WHERE j.status = 'pending'
|
||||
AND (
|
||||
(j.type IN ('rebuild', 'full_rebuild') AND NOT EXISTS (
|
||||
SELECT 1 FROM index_jobs
|
||||
WHERE status = 'running'
|
||||
AND type IN ('rebuild', 'full_rebuild')
|
||||
))
|
||||
OR
|
||||
j.type NOT IN ('rebuild', 'full_rebuild')
|
||||
)
|
||||
ORDER BY
|
||||
CASE j.type
|
||||
WHEN 'full_rebuild' THEN 1
|
||||
WHEN 'rebuild' THEN 2
|
||||
ELSE 3
|
||||
END,
|
||||
j.created_at ASC
|
||||
FOR UPDATE SKIP LOCKED
|
||||
LIMIT 1
|
||||
"#,
|
||||
)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?;
|
||||
|
||||
let Some(row) = row else {
|
||||
tx.commit().await?;
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
let id: Uuid = row.get("id");
|
||||
let job_type: String = row.get("type");
|
||||
let library_id: Option<Uuid> = row.get("library_id");
|
||||
|
||||
if job_type == "rebuild" || job_type == "full_rebuild" {
|
||||
let has_running_rebuild: bool = sqlx::query_scalar(
|
||||
r#"
|
||||
SELECT EXISTS(
|
||||
SELECT 1 FROM index_jobs
|
||||
WHERE status = 'running'
|
||||
AND type IN ('rebuild', 'full_rebuild')
|
||||
AND id != $1
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.bind(id)
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
|
||||
if has_running_rebuild {
|
||||
tx.rollback().await?;
|
||||
return Ok(None);
|
||||
}
|
||||
}
|
||||
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'running', started_at = NOW(), error_opt = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(id)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
Ok(Some((id, library_id)))
|
||||
}
|
||||
|
||||
pub async fn fail_job(pool: &PgPool, job_id: Uuid, error_message: &str) -> Result<()> {
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'failed', finished_at = NOW(), error_opt = $2 WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(error_message)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn is_job_cancelled(pool: &PgPool, job_id: Uuid) -> Result<bool> {
|
||||
let status: Option<String> =
|
||||
sqlx::query_scalar("SELECT status FROM index_jobs WHERE id = $1")
|
||||
.bind(job_id)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
|
||||
Ok(status.as_deref() == Some("cancelled"))
|
||||
}
|
||||
|
||||
pub async fn process_job(
|
||||
state: &AppState,
|
||||
job_id: Uuid,
|
||||
target_library_id: Option<Uuid>,
|
||||
) -> Result<()> {
|
||||
info!("[JOB] Processing {} library={:?}", job_id, target_library_id);
|
||||
|
||||
let job_type: String = sqlx::query_scalar("SELECT type FROM index_jobs WHERE id = $1")
|
||||
.bind(job_id)
|
||||
.fetch_one(&state.pool)
|
||||
.await?;
|
||||
|
||||
// Thumbnail rebuild: generate thumbnails for books missing them
|
||||
if job_type == "thumbnail_rebuild" {
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'generating_thumbnails', started_at = NOW() WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
analyzer::analyze_library_books(state, job_id, target_library_id, true).await?;
|
||||
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Thumbnail regenerate: clear all thumbnails then re-generate
|
||||
if job_type == "thumbnail_regenerate" {
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'generating_thumbnails', started_at = NOW() WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
analyzer::regenerate_thumbnails(state, job_id, target_library_id).await?;
|
||||
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let is_full_rebuild = job_type == "full_rebuild";
|
||||
info!(
|
||||
"[JOB] {} type={} full_rebuild={}",
|
||||
job_id, job_type, is_full_rebuild
|
||||
);
|
||||
|
||||
// Full rebuild: delete existing data first
|
||||
if is_full_rebuild {
|
||||
info!("[JOB] Full rebuild: deleting existing data");
|
||||
|
||||
if let Some(library_id) = target_library_id {
|
||||
sqlx::query(
|
||||
"DELETE FROM book_files WHERE book_id IN (SELECT id FROM books WHERE library_id = $1)",
|
||||
)
|
||||
.bind(library_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
sqlx::query("DELETE FROM books WHERE library_id = $1")
|
||||
.bind(library_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
info!("[JOB] Deleted existing data for library {}", library_id);
|
||||
} else {
|
||||
sqlx::query("DELETE FROM book_files")
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
sqlx::query("DELETE FROM books").execute(&state.pool).await?;
|
||||
info!("[JOB] Deleted all existing data");
|
||||
}
|
||||
}
|
||||
|
||||
let libraries = if let Some(library_id) = target_library_id {
|
||||
sqlx::query("SELECT id, root_path FROM libraries WHERE id = $1 AND enabled = TRUE")
|
||||
.bind(library_id)
|
||||
.fetch_all(&state.pool)
|
||||
.await?
|
||||
} else {
|
||||
sqlx::query("SELECT id, root_path FROM libraries WHERE enabled = TRUE")
|
||||
.fetch_all(&state.pool)
|
||||
.await?
|
||||
};
|
||||
|
||||
// Count total files for progress estimation
|
||||
let library_paths: Vec<String> = libraries
|
||||
.iter()
|
||||
.map(|library| {
|
||||
crate::utils::remap_libraries_path(&library.get::<String, _>("root_path"))
|
||||
})
|
||||
.collect();
|
||||
|
||||
let total_files: usize = library_paths
|
||||
.par_iter()
|
||||
.map(|root_path| {
|
||||
walkdir::WalkDir::new(root_path)
|
||||
.into_iter()
|
||||
.filter_map(Result::ok)
|
||||
.filter(|entry| {
|
||||
entry.file_type().is_file()
|
||||
&& parsers::detect_format(entry.path()).is_some()
|
||||
})
|
||||
.count()
|
||||
})
|
||||
.sum();
|
||||
|
||||
info!(
|
||||
"[JOB] Found {} libraries, {} total files to index",
|
||||
libraries.len(),
|
||||
total_files
|
||||
);
|
||||
|
||||
sqlx::query("UPDATE index_jobs SET total_files = $2 WHERE id = $1")
|
||||
.bind(job_id)
|
||||
.bind(total_files as i32)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
let mut stats = scanner::JobStats {
|
||||
scanned_files: 0,
|
||||
indexed_files: 0,
|
||||
removed_files: 0,
|
||||
errors: 0,
|
||||
};
|
||||
|
||||
let mut total_processed_count = 0i32;
|
||||
|
||||
// Phase 1: Discovery
|
||||
for library in &libraries {
|
||||
let library_id: Uuid = library.get("id");
|
||||
let root_path: String = library.get("root_path");
|
||||
let root_path = crate::utils::remap_libraries_path(&root_path);
|
||||
match scanner::scan_library_discovery(
|
||||
state,
|
||||
job_id,
|
||||
library_id,
|
||||
std::path::Path::new(&root_path),
|
||||
&mut stats,
|
||||
&mut total_processed_count,
|
||||
total_files,
|
||||
is_full_rebuild,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(()) => {}
|
||||
Err(err) => {
|
||||
let err_str = err.to_string();
|
||||
if err_str.contains("cancelled") || err_str.contains("Cancelled") {
|
||||
return Err(err);
|
||||
}
|
||||
stats.errors += 1;
|
||||
error!(library_id = %library_id, error = %err, "library scan failed");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sync search index after discovery (books are visible immediately)
|
||||
meili::sync_meili(&state.pool, &state.meili_url, &state.meili_master_key).await?;
|
||||
|
||||
// For full rebuild: clean up orphaned thumbnail files (old UUIDs)
|
||||
if is_full_rebuild {
|
||||
analyzer::cleanup_orphaned_thumbnails(state, target_library_id).await?;
|
||||
}
|
||||
|
||||
// Phase 2: Analysis (extract page_count + thumbnails for new/updated books)
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'generating_thumbnails', stats_json = $2, current_file = NULL, processed_files = $3 WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(serde_json::to_value(&stats)?)
|
||||
.bind(total_processed_count)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
analyzer::analyze_library_books(state, job_id, target_library_id, false).await?;
|
||||
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET status = 'success', finished_at = NOW(), progress_percent = 100, current_file = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
19
apps/indexer/src/lib.rs
Normal file
19
apps/indexer/src/lib.rs
Normal file
@@ -0,0 +1,19 @@
|
||||
pub mod analyzer;
|
||||
pub mod api;
|
||||
pub mod batch;
|
||||
pub mod job;
|
||||
pub mod meili;
|
||||
pub mod scheduler;
|
||||
pub mod scanner;
|
||||
pub mod utils;
|
||||
pub mod watcher;
|
||||
pub mod worker;
|
||||
|
||||
use sqlx::PgPool;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct AppState {
|
||||
pub pool: PgPool,
|
||||
pub meili_url: String,
|
||||
pub meili_master_key: String,
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
180
apps/indexer/src/meili.rs
Normal file
180
apps/indexer/src/meili.rs
Normal file
@@ -0,0 +1,180 @@
|
||||
use anyhow::{Context, Result};
|
||||
use chrono::{DateTime, Utc};
|
||||
use reqwest::Client;
|
||||
use serde::Serialize;
|
||||
use sqlx::{PgPool, Row};
|
||||
use tracing::info;
|
||||
use uuid::Uuid;
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct SearchDoc {
|
||||
id: String,
|
||||
library_id: String,
|
||||
kind: String,
|
||||
title: String,
|
||||
author: Option<String>,
|
||||
series: Option<String>,
|
||||
volume: Option<i32>,
|
||||
language: Option<String>,
|
||||
}
|
||||
|
||||
pub async fn sync_meili(pool: &PgPool, meili_url: &str, meili_master_key: &str) -> Result<()> {
|
||||
let client = Client::new();
|
||||
let base = meili_url.trim_end_matches('/');
|
||||
|
||||
// Ensure index exists and has proper settings
|
||||
let _ = client
|
||||
.post(format!("{base}/indexes"))
|
||||
.header("Authorization", format!("Bearer {meili_master_key}"))
|
||||
.json(&serde_json::json!({"uid": "books", "primaryKey": "id"}))
|
||||
.send()
|
||||
.await;
|
||||
|
||||
let _ = client
|
||||
.patch(format!("{base}/indexes/books/settings/filterable-attributes"))
|
||||
.header("Authorization", format!("Bearer {meili_master_key}"))
|
||||
.json(&serde_json::json!(["library_id", "kind"]))
|
||||
.send()
|
||||
.await;
|
||||
|
||||
// Get last sync timestamp
|
||||
let last_sync: Option<DateTime<Utc>> = sqlx::query_scalar(
|
||||
"SELECT last_meili_sync FROM sync_metadata WHERE id = 1 AND last_meili_sync IS NOT NULL"
|
||||
)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
|
||||
// If no previous sync, do a full sync
|
||||
let is_full_sync = last_sync.is_none();
|
||||
|
||||
// Get books to sync: all if full sync, only modified since last sync otherwise
|
||||
let rows = if is_full_sync {
|
||||
info!("[MEILI] Performing full sync");
|
||||
sqlx::query(
|
||||
"SELECT id, library_id, kind, title, author, series, volume, language, updated_at FROM books",
|
||||
)
|
||||
.fetch_all(pool)
|
||||
.await?
|
||||
} else {
|
||||
let since = last_sync.unwrap();
|
||||
info!("[MEILI] Performing incremental sync since {}", since);
|
||||
|
||||
// Also get deleted book IDs to remove from MeiliSearch
|
||||
// For now, we'll do a diff approach: get all book IDs from DB and compare with Meili
|
||||
sqlx::query(
|
||||
"SELECT id, library_id, kind, title, author, series, volume, language, updated_at FROM books WHERE updated_at > $1",
|
||||
)
|
||||
.bind(since)
|
||||
.fetch_all(pool)
|
||||
.await?
|
||||
};
|
||||
|
||||
if rows.is_empty() && !is_full_sync {
|
||||
info!("[MEILI] No changes to sync");
|
||||
// Still update the timestamp
|
||||
sqlx::query(
|
||||
"INSERT INTO sync_metadata (id, last_meili_sync) VALUES (1, NOW()) ON CONFLICT (id) DO UPDATE SET last_meili_sync = NOW()"
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let docs: Vec<SearchDoc> = rows
|
||||
.into_iter()
|
||||
.map(|row| SearchDoc {
|
||||
id: row.get::<Uuid, _>("id").to_string(),
|
||||
library_id: row.get::<Uuid, _>("library_id").to_string(),
|
||||
kind: row.get("kind"),
|
||||
title: row.get("title"),
|
||||
author: row.get("author"),
|
||||
series: row.get("series"),
|
||||
volume: row.get("volume"),
|
||||
language: row.get("language"),
|
||||
})
|
||||
.collect();
|
||||
|
||||
let doc_count = docs.len();
|
||||
|
||||
// Send documents to MeiliSearch in batches of 1000
|
||||
const MEILI_BATCH_SIZE: usize = 1000;
|
||||
for (i, chunk) in docs.chunks(MEILI_BATCH_SIZE).enumerate() {
|
||||
let batch_num = i + 1;
|
||||
info!("[MEILI] Sending batch {}/{} ({} docs)", batch_num, doc_count.div_ceil(MEILI_BATCH_SIZE), chunk.len());
|
||||
|
||||
let response = client
|
||||
.post(format!("{base}/indexes/books/documents"))
|
||||
.header("Authorization", format!("Bearer {meili_master_key}"))
|
||||
.json(&chunk)
|
||||
.send()
|
||||
.await
|
||||
.context("failed to send docs to meili")?;
|
||||
|
||||
if !response.status().is_success() {
|
||||
let status = response.status();
|
||||
let body = response.text().await.unwrap_or_default();
|
||||
return Err(anyhow::anyhow!("MeiliSearch error {}: {}", status, body));
|
||||
}
|
||||
}
|
||||
|
||||
// Handle deletions: get all book IDs from DB and remove from MeiliSearch any that don't exist
|
||||
// This is expensive, so we only do it periodically (every 10 syncs) or on full syncs
|
||||
if is_full_sync || rand::random::<u8>() < 26 { // ~10% chance
|
||||
info!("[MEILI] Checking for documents to delete");
|
||||
|
||||
// Get all book IDs from database
|
||||
let db_ids: Vec<String> = sqlx::query_scalar("SELECT id::text FROM books")
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
|
||||
// Get all document IDs from MeiliSearch (this requires fetching all documents)
|
||||
// For efficiency, we'll just delete by query for documents that might be stale
|
||||
// A better approach would be to track deletions in a separate table
|
||||
|
||||
// For now, we'll do a simple approach: fetch all Meili docs and compare
|
||||
// Note: This could be slow for large collections
|
||||
let meili_response = client
|
||||
.post(format!("{base}/indexes/books/documents/fetch"))
|
||||
.header("Authorization", format!("Bearer {meili_master_key}"))
|
||||
.json(&serde_json::json!({
|
||||
"fields": ["id"],
|
||||
"limit": 100000
|
||||
}))
|
||||
.send()
|
||||
.await;
|
||||
|
||||
if let Ok(response) = meili_response {
|
||||
if response.status().is_success() {
|
||||
if let Ok(meili_docs) = response.json::<Vec<serde_json::Value>>().await {
|
||||
let meili_ids: std::collections::HashSet<String> = meili_docs
|
||||
.into_iter()
|
||||
.filter_map(|doc| doc.get("id").and_then(|id| id.as_str()).map(|s| s.to_string()))
|
||||
.collect();
|
||||
|
||||
let db_ids_set: std::collections::HashSet<String> = db_ids.into_iter().collect();
|
||||
let to_delete: Vec<String> = meili_ids.difference(&db_ids_set).cloned().collect();
|
||||
|
||||
if !to_delete.is_empty() {
|
||||
info!("[MEILI] Deleting {} stale documents", to_delete.len());
|
||||
let _ = client
|
||||
.post(format!("{base}/indexes/books/documents/delete-batch"))
|
||||
.header("Authorization", format!("Bearer {meili_master_key}"))
|
||||
.json(&to_delete)
|
||||
.send()
|
||||
.await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Update last sync timestamp
|
||||
sqlx::query(
|
||||
"INSERT INTO sync_metadata (id, last_meili_sync) VALUES (1, NOW()) ON CONFLICT (id) DO UPDATE SET last_meili_sync = NOW()"
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
info!("[MEILI] Sync completed: {} documents indexed", doc_count);
|
||||
Ok(())
|
||||
}
|
||||
443
apps/indexer/src/scanner.rs
Normal file
443
apps/indexer/src/scanner.rs
Normal file
@@ -0,0 +1,443 @@
|
||||
use anyhow::{Context, Result};
|
||||
use chrono::{DateTime, Utc};
|
||||
use parsers::{detect_format, parse_metadata_fast};
|
||||
use serde::Serialize;
|
||||
use sqlx::Row;
|
||||
use std::{collections::HashMap, path::Path, time::Duration};
|
||||
use tracing::{error, info, trace, warn};
|
||||
use uuid::Uuid;
|
||||
use walkdir::WalkDir;
|
||||
|
||||
use crate::{
|
||||
batch::{flush_all_batches, BookInsert, BookUpdate, ErrorInsert, FileInsert, FileUpdate},
|
||||
job::is_job_cancelled,
|
||||
utils,
|
||||
AppState,
|
||||
};
|
||||
|
||||
#[derive(Serialize)]
|
||||
pub struct JobStats {
|
||||
pub scanned_files: usize,
|
||||
pub indexed_files: usize,
|
||||
pub removed_files: usize,
|
||||
pub errors: usize,
|
||||
}
|
||||
|
||||
const BATCH_SIZE: usize = 100;
|
||||
|
||||
/// Phase 1 — Discovery: walk filesystem, extract metadata from filenames only (no archive I/O).
|
||||
/// New books are inserted with page_count = NULL so the analyzer phase can fill them in.
|
||||
/// Updated books (fingerprint changed) get page_count/thumbnail reset.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn scan_library_discovery(
|
||||
state: &AppState,
|
||||
job_id: Uuid,
|
||||
library_id: Uuid,
|
||||
root: &Path,
|
||||
stats: &mut JobStats,
|
||||
total_processed_count: &mut i32,
|
||||
total_files: usize,
|
||||
is_full_rebuild: bool,
|
||||
) -> Result<()> {
|
||||
info!(
|
||||
"[SCAN] Starting discovery scan of library {} at path: {} (full_rebuild={})",
|
||||
library_id,
|
||||
root.display(),
|
||||
is_full_rebuild
|
||||
);
|
||||
|
||||
// Load existing files from DB
|
||||
let existing_rows = sqlx::query(
|
||||
r#"
|
||||
SELECT bf.id AS file_id, bf.book_id, bf.abs_path, bf.fingerprint
|
||||
FROM book_files bf
|
||||
JOIN books b ON b.id = bf.book_id
|
||||
WHERE b.library_id = $1
|
||||
"#,
|
||||
)
|
||||
.bind(library_id)
|
||||
.fetch_all(&state.pool)
|
||||
.await?;
|
||||
|
||||
let mut existing: HashMap<String, (Uuid, Uuid, String)> = HashMap::new();
|
||||
if !is_full_rebuild {
|
||||
for row in existing_rows {
|
||||
let abs_path: String = row.get("abs_path");
|
||||
let remapped_path = utils::remap_libraries_path(&abs_path);
|
||||
existing.insert(
|
||||
remapped_path,
|
||||
(row.get("file_id"), row.get("book_id"), row.get("fingerprint")),
|
||||
);
|
||||
}
|
||||
info!(
|
||||
"[SCAN] Found {} existing files in database for library {}",
|
||||
existing.len(),
|
||||
library_id
|
||||
);
|
||||
} else {
|
||||
info!("[SCAN] Full rebuild: skipping existing files lookup");
|
||||
// Delete stale directory mtime records for full rebuild
|
||||
let _ = sqlx::query("DELETE FROM directory_mtimes WHERE library_id = $1")
|
||||
.bind(library_id)
|
||||
.execute(&state.pool)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Load stored directory mtimes for incremental skip
|
||||
let dir_mtimes: HashMap<String, DateTime<Utc>> = if !is_full_rebuild {
|
||||
let rows = sqlx::query(
|
||||
"SELECT dir_path, mtime FROM directory_mtimes WHERE library_id = $1",
|
||||
)
|
||||
.bind(library_id)
|
||||
.fetch_all(&state.pool)
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
|
||||
rows.into_iter()
|
||||
.map(|row| {
|
||||
let db_path: String = row.get("dir_path");
|
||||
let local_path = utils::remap_libraries_path(&db_path);
|
||||
let mtime: DateTime<Utc> = row.get("mtime");
|
||||
(local_path, mtime)
|
||||
})
|
||||
.collect()
|
||||
} else {
|
||||
HashMap::new()
|
||||
};
|
||||
|
||||
let mut seen: HashMap<String, bool> = HashMap::new();
|
||||
let mut library_processed_count = 0i32;
|
||||
let mut last_progress_update = std::time::Instant::now();
|
||||
|
||||
// Batching buffers
|
||||
let mut books_to_update: Vec<BookUpdate> = Vec::with_capacity(BATCH_SIZE);
|
||||
let mut files_to_update: Vec<FileUpdate> = Vec::with_capacity(BATCH_SIZE);
|
||||
let mut books_to_insert: Vec<BookInsert> = Vec::with_capacity(BATCH_SIZE);
|
||||
let mut files_to_insert: Vec<FileInsert> = Vec::with_capacity(BATCH_SIZE);
|
||||
let mut errors_to_insert: Vec<ErrorInsert> = Vec::with_capacity(BATCH_SIZE);
|
||||
|
||||
// Track discovered directory mtimes for upsert after scan
|
||||
let mut new_dir_mtimes: Vec<(String, DateTime<Utc>)> = Vec::new();
|
||||
|
||||
// Prefixes (with trailing "/") of directories whose mtime hasn't changed.
|
||||
// Files under these prefixes are added to `seen` but not reprocessed.
|
||||
let mut skipped_dir_prefixes: Vec<String> = Vec::new();
|
||||
|
||||
for entry in WalkDir::new(root).into_iter().filter_map(Result::ok) {
|
||||
let path = entry.path().to_path_buf();
|
||||
let local_path = path.to_string_lossy().to_string();
|
||||
|
||||
if entry.file_type().is_dir() {
|
||||
if entry.depth() == 0 {
|
||||
continue; // skip root itself
|
||||
}
|
||||
|
||||
// Check if parent dir is already skipped (propagate skip to subdirs)
|
||||
let already_under_skipped = skipped_dir_prefixes
|
||||
.iter()
|
||||
.any(|p| local_path.starts_with(p.as_str()));
|
||||
|
||||
if let Ok(meta) = entry.metadata() {
|
||||
if let Ok(sys_mtime) = meta.modified() {
|
||||
let mtime_utc: DateTime<Utc> = DateTime::from(sys_mtime);
|
||||
|
||||
// Only record mtimes for non-skipped dirs (to avoid polluting DB)
|
||||
if !already_under_skipped {
|
||||
new_dir_mtimes.push((local_path.clone(), mtime_utc));
|
||||
}
|
||||
|
||||
// Skip if mtime unchanged (incremental only, not already skipped subtree)
|
||||
if !is_full_rebuild && !already_under_skipped {
|
||||
if let Some(&stored_mtime) = dir_mtimes.get(&local_path) {
|
||||
if mtime_utc <= stored_mtime {
|
||||
trace!("[SCAN] Skipping unchanged dir: {}", local_path);
|
||||
// Add trailing slash so starts_with check is exact per-segment
|
||||
skipped_dir_prefixes.push(format!("{}/", local_path));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if !entry.file_type().is_file() {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check if this file is under a skipped dir
|
||||
let under_skipped = skipped_dir_prefixes
|
||||
.iter()
|
||||
.any(|p| local_path.starts_with(p.as_str()));
|
||||
|
||||
if under_skipped {
|
||||
// Dir unchanged — just mark file as seen so it's not deleted
|
||||
let abs_path_local = local_path.clone();
|
||||
let abs_path = utils::unmap_libraries_path(&abs_path_local);
|
||||
let lookup_path = utils::remap_libraries_path(&abs_path);
|
||||
seen.insert(lookup_path, true);
|
||||
continue;
|
||||
}
|
||||
|
||||
let Some(format) = detect_format(&path) else {
|
||||
trace!("[SCAN] Skipping non-book file: {}", path.display());
|
||||
continue;
|
||||
};
|
||||
|
||||
info!(
|
||||
"[SCAN] Found book file: {} (format: {:?})",
|
||||
path.display(),
|
||||
format
|
||||
);
|
||||
stats.scanned_files += 1;
|
||||
|
||||
let abs_path_local = path.to_string_lossy().to_string();
|
||||
let abs_path = utils::unmap_libraries_path(&abs_path_local);
|
||||
let file_name = path
|
||||
.file_name()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| abs_path.clone());
|
||||
|
||||
let metadata = std::fs::metadata(&path)
|
||||
.with_context(|| format!("cannot stat {}", path.display()))?;
|
||||
let mtime: DateTime<Utc> = metadata
|
||||
.modified()
|
||||
.map(DateTime::<Utc>::from)
|
||||
.unwrap_or_else(|_| Utc::now());
|
||||
let fingerprint = utils::compute_fingerprint(&path, metadata.len(), &mtime)?;
|
||||
let lookup_path = utils::remap_libraries_path(&abs_path);
|
||||
|
||||
library_processed_count += 1;
|
||||
*total_processed_count += 1;
|
||||
|
||||
// Progress update
|
||||
let should_update_progress = last_progress_update.elapsed() > Duration::from_secs(1)
|
||||
|| library_processed_count % 10 == 0;
|
||||
if should_update_progress {
|
||||
let progress_percent = if total_files > 0 {
|
||||
((*total_processed_count as f64 / total_files as f64) * 100.0) as i32
|
||||
} else {
|
||||
0
|
||||
};
|
||||
|
||||
sqlx::query(
|
||||
"UPDATE index_jobs SET current_file = $2, processed_files = $3, progress_percent = $4 WHERE id = $1",
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(&file_name)
|
||||
.bind(*total_processed_count)
|
||||
.bind(progress_percent)
|
||||
.execute(&state.pool)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
error!("[BDD] Failed to update progress for job {}: {}", job_id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
last_progress_update = std::time::Instant::now();
|
||||
|
||||
if is_job_cancelled(&state.pool, job_id).await? {
|
||||
info!("[JOB] Job {} cancelled by user, stopping...", job_id);
|
||||
flush_all_batches(
|
||||
&state.pool,
|
||||
&mut books_to_update,
|
||||
&mut files_to_update,
|
||||
&mut books_to_insert,
|
||||
&mut files_to_insert,
|
||||
&mut errors_to_insert,
|
||||
)
|
||||
.await?;
|
||||
return Err(anyhow::anyhow!("Job cancelled by user"));
|
||||
}
|
||||
}
|
||||
|
||||
seen.insert(lookup_path.clone(), true);
|
||||
|
||||
// Fast metadata extraction — no archive I/O
|
||||
let parsed = parse_metadata_fast(&path, format, root);
|
||||
|
||||
if let Some((file_id, book_id, old_fingerprint)) =
|
||||
existing.get(&lookup_path).cloned()
|
||||
{
|
||||
if !is_full_rebuild && old_fingerprint == fingerprint {
|
||||
trace!("[PROCESS] Skipping unchanged file: {}", file_name);
|
||||
continue;
|
||||
}
|
||||
|
||||
info!(
|
||||
"[PROCESS] Updating existing file: {} (fingerprint_changed={})",
|
||||
file_name,
|
||||
old_fingerprint != fingerprint
|
||||
);
|
||||
|
||||
books_to_update.push(BookUpdate {
|
||||
book_id,
|
||||
title: parsed.title,
|
||||
kind: utils::kind_from_format(format).to_string(),
|
||||
series: parsed.series,
|
||||
volume: parsed.volume,
|
||||
// Reset page_count so analyzer re-processes this book
|
||||
page_count: None,
|
||||
});
|
||||
|
||||
files_to_update.push(FileUpdate {
|
||||
file_id,
|
||||
format: format.as_str().to_string(),
|
||||
size_bytes: metadata.len() as i64,
|
||||
mtime,
|
||||
fingerprint,
|
||||
});
|
||||
|
||||
// Also clear thumbnail so it gets regenerated
|
||||
if let Err(e) = sqlx::query(
|
||||
"UPDATE books SET thumbnail_path = NULL WHERE id = $1",
|
||||
)
|
||||
.bind(book_id)
|
||||
.execute(&state.pool)
|
||||
.await
|
||||
{
|
||||
warn!(
|
||||
"[BDD] Failed to clear thumbnail for book {}: {}",
|
||||
book_id, e
|
||||
);
|
||||
}
|
||||
|
||||
stats.indexed_files += 1;
|
||||
|
||||
if books_to_update.len() >= BATCH_SIZE || files_to_update.len() >= BATCH_SIZE {
|
||||
flush_all_batches(
|
||||
&state.pool,
|
||||
&mut books_to_update,
|
||||
&mut files_to_update,
|
||||
&mut books_to_insert,
|
||||
&mut files_to_insert,
|
||||
&mut errors_to_insert,
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
// New file — insert with page_count = NULL (analyzer fills it in)
|
||||
info!("[PROCESS] Inserting new file: {}", file_name);
|
||||
let book_id = Uuid::new_v4();
|
||||
let file_id = Uuid::new_v4();
|
||||
|
||||
books_to_insert.push(BookInsert {
|
||||
book_id,
|
||||
library_id,
|
||||
kind: utils::kind_from_format(format).to_string(),
|
||||
title: parsed.title,
|
||||
series: parsed.series,
|
||||
volume: parsed.volume,
|
||||
page_count: None,
|
||||
thumbnail_path: None,
|
||||
});
|
||||
|
||||
files_to_insert.push(FileInsert {
|
||||
file_id,
|
||||
book_id,
|
||||
format: format.as_str().to_string(),
|
||||
abs_path: abs_path.clone(),
|
||||
size_bytes: metadata.len() as i64,
|
||||
mtime,
|
||||
fingerprint,
|
||||
parse_status: "ok".to_string(),
|
||||
parse_error: None,
|
||||
});
|
||||
|
||||
stats.indexed_files += 1;
|
||||
|
||||
if books_to_insert.len() >= BATCH_SIZE || files_to_insert.len() >= BATCH_SIZE {
|
||||
flush_all_batches(
|
||||
&state.pool,
|
||||
&mut books_to_update,
|
||||
&mut files_to_update,
|
||||
&mut books_to_insert,
|
||||
&mut files_to_insert,
|
||||
&mut errors_to_insert,
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
}
|
||||
|
||||
// Flush remaining batches
|
||||
flush_all_batches(
|
||||
&state.pool,
|
||||
&mut books_to_update,
|
||||
&mut files_to_update,
|
||||
&mut books_to_insert,
|
||||
&mut files_to_insert,
|
||||
&mut errors_to_insert,
|
||||
)
|
||||
.await?;
|
||||
|
||||
if !skipped_dir_prefixes.is_empty() {
|
||||
info!(
|
||||
"[SCAN] Skipped {} unchanged directories",
|
||||
skipped_dir_prefixes.len()
|
||||
);
|
||||
}
|
||||
|
||||
info!(
|
||||
"[SCAN] Library {} discovery complete: {} files scanned, {} indexed, {} errors",
|
||||
library_id, library_processed_count, stats.indexed_files, stats.errors
|
||||
);
|
||||
|
||||
// Handle deletions
|
||||
let mut removed_count = 0usize;
|
||||
for (abs_path, (file_id, book_id, _)) in &existing {
|
||||
if seen.contains_key(abs_path) {
|
||||
continue;
|
||||
}
|
||||
sqlx::query("DELETE FROM book_files WHERE id = $1")
|
||||
.bind(file_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
sqlx::query(
|
||||
"DELETE FROM books WHERE id = $1 AND NOT EXISTS (SELECT 1 FROM book_files WHERE book_id = $1)",
|
||||
)
|
||||
.bind(book_id)
|
||||
.execute(&state.pool)
|
||||
.await?;
|
||||
stats.removed_files += 1;
|
||||
removed_count += 1;
|
||||
}
|
||||
|
||||
if removed_count > 0 {
|
||||
info!(
|
||||
"[SCAN] Removed {} stale files from database",
|
||||
removed_count
|
||||
);
|
||||
}
|
||||
|
||||
// Upsert directory mtimes for next incremental scan
|
||||
if !new_dir_mtimes.is_empty() {
|
||||
let dir_paths_db: Vec<String> = new_dir_mtimes
|
||||
.iter()
|
||||
.map(|(local, _)| utils::unmap_libraries_path(local))
|
||||
.collect();
|
||||
let mtimes: Vec<DateTime<Utc>> = new_dir_mtimes.iter().map(|(_, m)| *m).collect();
|
||||
let library_ids: Vec<Uuid> = vec![library_id; new_dir_mtimes.len()];
|
||||
|
||||
if let Err(e) = sqlx::query(
|
||||
r#"
|
||||
INSERT INTO directory_mtimes (library_id, dir_path, mtime)
|
||||
SELECT * FROM UNNEST($1::uuid[], $2::text[], $3::timestamptz[])
|
||||
AS t(library_id, dir_path, mtime)
|
||||
ON CONFLICT (library_id, dir_path) DO UPDATE SET mtime = EXCLUDED.mtime
|
||||
"#,
|
||||
)
|
||||
.bind(&library_ids)
|
||||
.bind(&dir_paths_db)
|
||||
.bind(&mtimes)
|
||||
.execute(&state.pool)
|
||||
.await
|
||||
{
|
||||
warn!("[SCAN] Failed to upsert directory mtimes: {}", e);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
67
apps/indexer/src/scheduler.rs
Normal file
67
apps/indexer/src/scheduler.rs
Normal file
@@ -0,0 +1,67 @@
|
||||
use anyhow::Result;
|
||||
use sqlx::{PgPool, Row};
|
||||
use tracing::info;
|
||||
use uuid::Uuid;
|
||||
|
||||
pub async fn check_and_schedule_auto_scans(pool: &PgPool) -> Result<()> {
|
||||
let libraries = sqlx::query(
|
||||
r#"
|
||||
SELECT id, scan_mode, last_scan_at
|
||||
FROM libraries
|
||||
WHERE monitor_enabled = TRUE
|
||||
AND (
|
||||
next_scan_at IS NULL
|
||||
OR next_scan_at <= NOW()
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM index_jobs
|
||||
WHERE library_id = libraries.id
|
||||
AND status IN ('pending', 'running')
|
||||
)
|
||||
"#
|
||||
)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
|
||||
for row in libraries {
|
||||
let library_id: Uuid = row.get("id");
|
||||
let scan_mode: String = row.get("scan_mode");
|
||||
|
||||
info!("[SCHEDULER] Auto-scanning library {} (mode: {})", library_id, scan_mode);
|
||||
|
||||
let job_id = Uuid::new_v4();
|
||||
let job_type = match scan_mode.as_str() {
|
||||
"full" => "full_rebuild",
|
||||
_ => "rebuild",
|
||||
};
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO index_jobs (id, library_id, type, status) VALUES ($1, $2, $3, 'pending')"
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(library_id)
|
||||
.bind(job_type)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Update next_scan_at
|
||||
let interval_minutes = match scan_mode.as_str() {
|
||||
"hourly" => 60,
|
||||
"daily" => 1440,
|
||||
"weekly" => 10080,
|
||||
_ => 1440, // default daily
|
||||
};
|
||||
|
||||
sqlx::query(
|
||||
"UPDATE libraries SET last_scan_at = NOW(), next_scan_at = NOW() + INTERVAL '1 minute' * $2 WHERE id = $1"
|
||||
)
|
||||
.bind(library_id)
|
||||
.bind(interval_minutes)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
info!("[SCHEDULER] Created job {} for library {}", job_id, library_id);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
52
apps/indexer/src/utils.rs
Normal file
52
apps/indexer/src/utils.rs
Normal file
@@ -0,0 +1,52 @@
|
||||
use anyhow::Result;
|
||||
use chrono::DateTime;
|
||||
use parsers::BookFormat;
|
||||
use sha2::{Digest, Sha256};
|
||||
use std::path::Path;
|
||||
use chrono::Utc;
|
||||
|
||||
pub fn remap_libraries_path(path: &str) -> String {
|
||||
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
|
||||
if path.starts_with("/libraries/") {
|
||||
return path.replacen("/libraries", &root, 1);
|
||||
}
|
||||
}
|
||||
path.to_string()
|
||||
}
|
||||
|
||||
pub fn unmap_libraries_path(path: &str) -> String {
|
||||
if let Ok(root) = std::env::var("LIBRARIES_ROOT_PATH") {
|
||||
if path.starts_with(&root) {
|
||||
return path.replacen(&root, "/libraries", 1);
|
||||
}
|
||||
}
|
||||
path.to_string()
|
||||
}
|
||||
|
||||
pub fn compute_fingerprint(path: &Path, size: u64, mtime: &DateTime<Utc>) -> Result<String> {
|
||||
// Optimized: only use size + mtime + first bytes of filename for fast fingerprinting
|
||||
// This is 100x faster than reading file content while still being reliable for change detection
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(size.to_le_bytes());
|
||||
hasher.update(mtime.timestamp().to_le_bytes());
|
||||
|
||||
// Add filename for extra uniqueness (in case of rapid changes with same size+mtime)
|
||||
if let Some(filename) = path.file_name() {
|
||||
hasher.update(filename.as_encoded_bytes());
|
||||
}
|
||||
|
||||
Ok(format!("{:x}", hasher.finalize()))
|
||||
}
|
||||
|
||||
pub fn kind_from_format(format: BookFormat) -> &'static str {
|
||||
match format {
|
||||
BookFormat::Pdf => "ebook",
|
||||
BookFormat::Cbz | BookFormat::Cbr => "comic",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn file_display_name(path: &Path) -> String {
|
||||
path.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "Untitled".to_string())
|
||||
}
|
||||
147
apps/indexer/src/watcher.rs
Normal file
147
apps/indexer/src/watcher.rs
Normal file
@@ -0,0 +1,147 @@
|
||||
use anyhow::Result;
|
||||
use notify::{Event, RecommendedWatcher, RecursiveMode, Watcher};
|
||||
use sqlx::Row;
|
||||
use std::collections::HashMap;
|
||||
use std::time::Duration;
|
||||
use tokio::sync::mpsc;
|
||||
use tracing::{error, info, trace};
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::utils;
|
||||
use crate::AppState;
|
||||
|
||||
pub async fn run_file_watcher(state: AppState) -> Result<()> {
|
||||
let (tx, mut rx) = mpsc::channel::<(Uuid, String)>(100);
|
||||
|
||||
// Start watcher refresh loop
|
||||
let refresh_interval = Duration::from_secs(30);
|
||||
let pool = state.pool.clone();
|
||||
|
||||
tokio::spawn(async move {
|
||||
let mut watched_libraries: HashMap<Uuid, String> = HashMap::new();
|
||||
|
||||
loop {
|
||||
// Get libraries with watcher enabled
|
||||
match sqlx::query(
|
||||
"SELECT id, root_path FROM libraries WHERE watcher_enabled = TRUE AND enabled = TRUE"
|
||||
)
|
||||
.fetch_all(&pool)
|
||||
.await
|
||||
{
|
||||
Ok(rows) => {
|
||||
let current_libraries: HashMap<Uuid, String> = rows
|
||||
.into_iter()
|
||||
.map(|row| {
|
||||
let id: Uuid = row.get("id");
|
||||
let root_path: String = row.get("root_path");
|
||||
let local_path = utils::remap_libraries_path(&root_path);
|
||||
(id, local_path)
|
||||
})
|
||||
.collect();
|
||||
|
||||
// Check if we need to recreate watcher
|
||||
let needs_restart = watched_libraries.len() != current_libraries.len()
|
||||
|| watched_libraries.iter().any(|(id, path)| {
|
||||
current_libraries.get(id) != Some(path)
|
||||
});
|
||||
|
||||
if needs_restart {
|
||||
info!("[WATCHER] Restarting watcher for {} libraries", current_libraries.len());
|
||||
|
||||
if !current_libraries.is_empty() {
|
||||
let tx_clone = tx.clone();
|
||||
let libraries_clone = current_libraries.clone();
|
||||
|
||||
match setup_watcher(libraries_clone, tx_clone) {
|
||||
Ok(_new_watcher) => {
|
||||
watched_libraries = current_libraries;
|
||||
info!("[WATCHER] Watching {} libraries", watched_libraries.len());
|
||||
}
|
||||
Err(err) => {
|
||||
error!("[WATCHER] Failed to setup watcher: {}", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(err) => {
|
||||
error!("[WATCHER] Failed to fetch libraries: {}", err);
|
||||
}
|
||||
}
|
||||
|
||||
tokio::time::sleep(refresh_interval).await;
|
||||
}
|
||||
});
|
||||
|
||||
// Process watcher events
|
||||
while let Some((library_id, file_path)) = rx.recv().await {
|
||||
info!("[WATCHER] File changed in library {}: {}", library_id, file_path);
|
||||
|
||||
// Check if there's already a pending job for this library
|
||||
match sqlx::query_scalar::<_, bool>(
|
||||
"SELECT EXISTS(SELECT 1 FROM index_jobs WHERE library_id = $1 AND status IN ('pending', 'running'))"
|
||||
)
|
||||
.bind(library_id)
|
||||
.fetch_one(&state.pool)
|
||||
.await
|
||||
{
|
||||
Ok(exists) => {
|
||||
if !exists {
|
||||
// Create a quick scan job
|
||||
let job_id = Uuid::new_v4();
|
||||
match sqlx::query(
|
||||
"INSERT INTO index_jobs (id, library_id, type, status) VALUES ($1, $2, 'rebuild', 'pending')"
|
||||
)
|
||||
.bind(job_id)
|
||||
.bind(library_id)
|
||||
.execute(&state.pool)
|
||||
.await
|
||||
{
|
||||
Ok(_) => info!("[WATCHER] Created job {} for library {}", job_id, library_id),
|
||||
Err(err) => error!("[WATCHER] Failed to create job: {}", err),
|
||||
}
|
||||
} else {
|
||||
trace!("[WATCHER] Job already pending for library {}, skipping", library_id);
|
||||
}
|
||||
}
|
||||
Err(err) => error!("[WATCHER] Failed to check existing jobs: {}", err),
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn setup_watcher(
|
||||
libraries: HashMap<Uuid, String>,
|
||||
tx: mpsc::Sender<(Uuid, String)>,
|
||||
) -> Result<RecommendedWatcher> {
|
||||
let libraries_for_closure = libraries.clone();
|
||||
|
||||
let mut watcher = notify::recommended_watcher(move |res: Result<Event, notify::Error>| {
|
||||
match res {
|
||||
Ok(event) => {
|
||||
if event.kind.is_modify() || event.kind.is_create() || event.kind.is_remove() {
|
||||
for path in event.paths {
|
||||
if let Some((library_id, _)) = libraries_for_closure.iter().find(|(_, root)| {
|
||||
path.starts_with(root)
|
||||
}) {
|
||||
let path_str = path.to_string_lossy().to_string();
|
||||
if parsers::detect_format(&path).is_some() {
|
||||
let _ = tx.try_send((*library_id, path_str));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(err) => error!("[WATCHER] Event error: {}", err),
|
||||
}
|
||||
})?;
|
||||
|
||||
// Actually watch the library directories
|
||||
for root_path in libraries.values() {
|
||||
info!("[WATCHER] Watching directory: {}", root_path);
|
||||
watcher.watch(std::path::Path::new(root_path), RecursiveMode::Recursive)?;
|
||||
}
|
||||
|
||||
Ok(watcher)
|
||||
}
|
||||
61
apps/indexer/src/worker.rs
Normal file
61
apps/indexer/src/worker.rs
Normal file
@@ -0,0 +1,61 @@
|
||||
use std::time::Duration;
|
||||
use tracing::{error, info, trace};
|
||||
use crate::{job, scheduler, watcher, AppState};
|
||||
|
||||
pub async fn run_worker(state: AppState, interval_seconds: u64) {
|
||||
let wait = Duration::from_secs(interval_seconds.max(1));
|
||||
|
||||
// Cleanup stale jobs from previous runs
|
||||
if let Err(err) = job::cleanup_stale_jobs(&state.pool).await {
|
||||
error!("[CLEANUP] Failed to cleanup stale jobs: {}", err);
|
||||
}
|
||||
|
||||
// Start file watcher task
|
||||
let watcher_state = state.clone();
|
||||
let _watcher_handle = tokio::spawn(async move {
|
||||
info!("[WATCHER] Starting file watcher service");
|
||||
if let Err(err) = watcher::run_file_watcher(watcher_state).await {
|
||||
error!("[WATCHER] Error: {}", err);
|
||||
}
|
||||
});
|
||||
|
||||
// Start scheduler task for auto-monitoring
|
||||
let scheduler_state = state.clone();
|
||||
let _scheduler_handle = tokio::spawn(async move {
|
||||
let scheduler_wait = Duration::from_secs(60); // Check every minute
|
||||
loop {
|
||||
if let Err(err) = scheduler::check_and_schedule_auto_scans(&scheduler_state.pool).await {
|
||||
error!("[SCHEDULER] Error: {}", err);
|
||||
}
|
||||
tokio::time::sleep(scheduler_wait).await;
|
||||
}
|
||||
});
|
||||
|
||||
loop {
|
||||
match job::claim_next_job(&state.pool).await {
|
||||
Ok(Some((job_id, library_id))) => {
|
||||
info!("[INDEXER] Starting job {} library={:?}", job_id, library_id);
|
||||
if let Err(err) = job::process_job(&state, job_id, library_id).await {
|
||||
let err_str = err.to_string();
|
||||
if err_str.contains("cancelled") || err_str.contains("Cancelled") {
|
||||
info!("[INDEXER] Job {} was cancelled by user", job_id);
|
||||
// Status is already 'cancelled' in DB, don't change it
|
||||
} else {
|
||||
error!("[INDEXER] Job {} failed: {}", job_id, err);
|
||||
let _ = job::fail_job(&state.pool, job_id, &err_str).await;
|
||||
}
|
||||
} else {
|
||||
info!("[INDEXER] Job {} completed", job_id);
|
||||
}
|
||||
}
|
||||
Ok(None) => {
|
||||
trace!("[INDEXER] No pending jobs, waiting...");
|
||||
tokio::time::sleep(wait).await;
|
||||
}
|
||||
Err(err) => {
|
||||
error!("[INDEXER] Worker error: {}", err);
|
||||
tokio::time::sleep(wait).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,7 @@ impl ApiConfig {
|
||||
pub fn from_env() -> Result<Self> {
|
||||
Ok(Self {
|
||||
listen_addr: std::env::var("API_LISTEN_ADDR")
|
||||
.unwrap_or_else(|_| "0.0.0.0:8080".to_string()),
|
||||
.unwrap_or_else(|_| "0.0.0.0:7080".to_string()),
|
||||
database_url: std::env::var("DATABASE_URL").context("DATABASE_URL is required")?,
|
||||
meili_url: std::env::var("MEILI_URL").context("MEILI_URL is required")?,
|
||||
meili_master_key: std::env::var("MEILI_MASTER_KEY")
|
||||
@@ -32,10 +32,6 @@ pub struct IndexerConfig {
|
||||
pub meili_master_key: String,
|
||||
pub scan_interval_seconds: u64,
|
||||
pub thumbnail_config: ThumbnailConfig,
|
||||
/// API base URL for thumbnail checkup at end of build (e.g. http://api:8080)
|
||||
pub api_base_url: String,
|
||||
/// Token to call API (e.g. API_BOOTSTRAP_TOKEN)
|
||||
pub api_bootstrap_token: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
@@ -87,7 +83,7 @@ impl IndexerConfig {
|
||||
|
||||
Ok(Self {
|
||||
listen_addr: std::env::var("INDEXER_LISTEN_ADDR")
|
||||
.unwrap_or_else(|_| "0.0.0.0:8081".to_string()),
|
||||
.unwrap_or_else(|_| "0.0.0.0:7081".to_string()),
|
||||
database_url: std::env::var("DATABASE_URL").context("DATABASE_URL is required")?,
|
||||
meili_url: std::env::var("MEILI_URL").context("MEILI_URL is required")?,
|
||||
meili_master_key: std::env::var("MEILI_MASTER_KEY")
|
||||
@@ -97,10 +93,6 @@ impl IndexerConfig {
|
||||
.and_then(|v| v.parse::<u64>().ok())
|
||||
.unwrap_or(5),
|
||||
thumbnail_config,
|
||||
api_base_url: std::env::var("API_BASE_URL")
|
||||
.unwrap_or_else(|_| "http://api:8080".to_string()),
|
||||
api_bootstrap_token: std::env::var("API_BOOTSTRAP_TOKEN")
|
||||
.context("API_BOOTSTRAP_TOKEN is required for thumbnail checkup")?,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -116,9 +108,9 @@ impl AdminUiConfig {
|
||||
pub fn from_env() -> Result<Self> {
|
||||
Ok(Self {
|
||||
listen_addr: std::env::var("ADMIN_UI_LISTEN_ADDR")
|
||||
.unwrap_or_else(|_| "0.0.0.0:8082".to_string()),
|
||||
.unwrap_or_else(|_| "0.0.0.0:7082".to_string()),
|
||||
api_base_url: std::env::var("API_BASE_URL")
|
||||
.unwrap_or_else(|_| "http://api:8080".to_string()),
|
||||
.unwrap_or_else(|_| "http://api:7080".to_string()),
|
||||
api_token: std::env::var("API_BOOTSTRAP_TOKEN")
|
||||
.context("API_BOOTSTRAP_TOKEN is required")?,
|
||||
})
|
||||
|
||||
76
crates/parsers/AGENTS.md
Normal file
76
crates/parsers/AGENTS.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# crates/parsers — Parsing de livres (CBZ, CBR, PDF)
|
||||
|
||||
Crate utilitaire sans état, utilisée par `apps/api` et `apps/indexer`.
|
||||
|
||||
## API publique (lib.rs)
|
||||
|
||||
```rust
|
||||
// Détection du format par extension
|
||||
pub fn detect_format(path: &Path) -> Option<BookFormat> // .cbz | .cbr | .pdf
|
||||
|
||||
// Extraction des métadonnées
|
||||
pub fn parse_metadata(path: &Path, format: BookFormat, library_root: &Path) -> Result<ParsedMetadata>
|
||||
|
||||
// Extraction de la première page (pour thumbnails)
|
||||
pub fn extract_first_page(path: &Path, format: BookFormat) -> Result<Vec<u8>>
|
||||
|
||||
pub enum BookFormat { Cbz, Cbr, Pdf }
|
||||
|
||||
pub struct ParsedMetadata {
|
||||
pub title: String, // = nom de fichier (sans extension)
|
||||
pub series: Option<String>, // = premier dossier relatif à library_root
|
||||
pub volume: Option<i32>, // extrait du nom de fichier
|
||||
pub page_count: Option<i32>,
|
||||
}
|
||||
```
|
||||
|
||||
## Logique de parsing
|
||||
|
||||
### Titre
|
||||
Nom de fichier sans extension, conservé tel quel (pas de nettoyage).
|
||||
|
||||
### Série
|
||||
Premier composant du chemin relatif entre `library_root` et le fichier :
|
||||
- `/libraries/One Piece/T01.cbz` → série = `"One Piece"`
|
||||
- `/libraries/one-shot.cbz` → série = `None`
|
||||
|
||||
### Volume (`extract_volume`)
|
||||
Patterns reconnus dans le nom de fichier (dans l'ordre de priorité) :
|
||||
- `T01`, `T1` (manga/comics français)
|
||||
- `Vol. 1`, `Vol 1`, `Volume 1`
|
||||
- `#1`, `#01`
|
||||
- `- 1`, `- 01` (en fin de nom)
|
||||
|
||||
### Nombre de pages
|
||||
| Format | Outil |
|
||||
|--------|-------|
|
||||
| CBZ | `zip::ZipArchive` — compte les entrées image (jpg/jpeg/png/webp/avif) |
|
||||
| CBR | `unrar lb <path>` — liste les fichiers, filtre les images |
|
||||
| PDF | `pdfinfo <path>` — lit la ligne `Pages:` |
|
||||
|
||||
## Dépendances système requises
|
||||
|
||||
| Outil | Utilisé pour | Installation |
|
||||
|-------|-------------|-------------|
|
||||
| `unrar` | CBR page count | `brew install rar` / `apt install unrar` |
|
||||
| `unar` | CBR first page extraction | `brew install unar` / `apt install unar` |
|
||||
| `pdfinfo` | PDF page count | inclus dans `poppler-utils` |
|
||||
| `pdftoppm` | PDF first page render | inclus dans `poppler-utils` |
|
||||
|
||||
**Important** : `unrar` (pour le listing) et `unar` (pour l'extraction) sont deux outils différents.
|
||||
|
||||
## Extraction première page
|
||||
|
||||
- **CBZ** : `zip::ZipArchive`, trie les noms d'images, lit la première
|
||||
- **CBR** : `unar -o <tmp_dir>`, `WalkDir` récursif, trie, lit la première — nettoie `tmp_dir` ensuite
|
||||
- **PDF** : `pdftoppm -f 1 -singlefile -png -scale-to 800` → fichier PNG temporaire — nettoie `tmp_dir` ensuite
|
||||
|
||||
Répertoire temp : `std::env::temp_dir()/stripstream-{cbr|pdf}-thumb-<uuid>`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- `clean_title()` existe mais est marqué `#[allow(dead_code)]` — le titre n'est **pas** nettoyé (décision volontaire).
|
||||
- Les CBR peuvent avoir des sous-dossiers internes → WalkDir nécessaire (pas de listing plat).
|
||||
- La détection du format est **uniquement par extension** (pas de magic bytes).
|
||||
- `pdfinfo` et `pdftoppm` doivent être du paquet `poppler-utils` (pas `poppler` seul).
|
||||
- En cas d'échec de parsing, l'appelant (indexer/api) stocke `parse_status = 'error'` en DB mais continue.
|
||||
@@ -2,6 +2,7 @@ use anyhow::{Context, Result};
|
||||
use std::io::Read;
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
use std::sync::OnceLock;
|
||||
use uuid::Uuid;
|
||||
use walkdir::WalkDir;
|
||||
|
||||
@@ -40,38 +41,52 @@ pub fn detect_format(path: &Path) -> Option<BookFormat> {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse_metadata(
|
||||
path: &Path,
|
||||
format: BookFormat,
|
||||
library_root: &Path,
|
||||
) -> Result<ParsedMetadata> {
|
||||
let filename = path
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "Untitled".to_string());
|
||||
// Cache compiled regex patterns — compiled once on first use
|
||||
static VOLUME_PATTERNS: OnceLock<Vec<(regex::Regex, usize)>> = OnceLock::new();
|
||||
|
||||
// Extract volume from filename (patterns: T01, T02, Vol 1, Volume 1, #1, - 01, etc.)
|
||||
let volume = extract_volume(&filename);
|
||||
fn get_volume_patterns() -> &'static Vec<(regex::Regex, usize)> {
|
||||
VOLUME_PATTERNS.get_or_init(|| {
|
||||
[
|
||||
// T01, T02 pattern (most common for manga/comics)
|
||||
(r"(?i)T(\d+)", 1usize),
|
||||
// Vol 1, Vol. 1, Volume 1
|
||||
(r"(?i)Vol\.?\s*(\d+)", 1),
|
||||
(r"(?i)Volume\s*(\d+)", 1),
|
||||
// #1, #01
|
||||
(r"#(\d+)", 1),
|
||||
// - 1, - 01 at the end
|
||||
(r"-\s*(\d+)\s*$", 1),
|
||||
]
|
||||
.iter()
|
||||
.filter_map(|(pattern, group)| {
|
||||
regex::Regex::new(pattern).ok().map(|re| (re, *group))
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
}
|
||||
|
||||
// Keep original filename as title (don't clean it)
|
||||
let title = filename;
|
||||
fn extract_volume(filename: &str) -> Option<i32> {
|
||||
for (re, group) in get_volume_patterns() {
|
||||
if let Some(caps) = re.captures(filename) {
|
||||
if let Some(mat) = caps.get(*group) {
|
||||
return mat.as_str().parse::<i32>().ok();
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
// Determine series from parent folder relative to library root
|
||||
let series = path.parent().and_then(|parent| {
|
||||
// Normalize paths for comparison (handle different separators, etc.)
|
||||
fn extract_series(path: &Path, library_root: &Path) -> Option<String> {
|
||||
path.parent().and_then(|parent| {
|
||||
let parent_str = parent.to_string_lossy().to_string();
|
||||
let root_str = library_root.to_string_lossy().to_string();
|
||||
|
||||
// Try to find the library root in the parent path
|
||||
let relative = if let Some(idx) = parent_str.find(&root_str) {
|
||||
// Found root in parent, extract what comes after
|
||||
let after_root = &parent_str[idx + root_str.len()..];
|
||||
Path::new(after_root)
|
||||
} else if let Some(relative) = parent.strip_prefix(library_root).ok() {
|
||||
// Standard approach works
|
||||
} else if let Ok(relative) = parent.strip_prefix(library_root) {
|
||||
relative
|
||||
} else {
|
||||
// Log for diagnostic on server
|
||||
eprintln!(
|
||||
"[PARSER] Cannot determine series: parent '{}' doesn't start with root '{}'",
|
||||
parent.display(),
|
||||
@@ -80,16 +95,14 @@ pub fn parse_metadata(
|
||||
return None;
|
||||
};
|
||||
|
||||
// Remove leading separators
|
||||
let relative_str = relative.to_string_lossy().to_string();
|
||||
let relative_clean = relative_str.trim_start_matches(|c| c == '/' || c == '\\');
|
||||
let relative_clean = relative_str.trim_start_matches(['/', '\\']);
|
||||
|
||||
if relative_clean.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
// Get first component as series
|
||||
let first_sep = relative_clean.find(|c| c == '/' || c == '\\');
|
||||
let first_sep = relative_clean.find(['/', '\\']);
|
||||
let series_name = match first_sep {
|
||||
Some(idx) => &relative_clean[..idx],
|
||||
None => relative_clean,
|
||||
@@ -100,80 +113,178 @@ pub fn parse_metadata(
|
||||
} else {
|
||||
Some(series_name.to_string())
|
||||
}
|
||||
});
|
||||
})
|
||||
}
|
||||
|
||||
let page_count = match format {
|
||||
/// Fast metadata extraction from filename only — no archive I/O. Always succeeds.
|
||||
pub fn parse_metadata_fast(path: &Path, _format: BookFormat, library_root: &Path) -> ParsedMetadata {
|
||||
let filename = path
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| "Untitled".to_string());
|
||||
|
||||
let volume = extract_volume(&filename);
|
||||
let title = filename;
|
||||
let series = extract_series(path, library_root);
|
||||
|
||||
ParsedMetadata {
|
||||
title,
|
||||
series,
|
||||
volume,
|
||||
page_count: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse_metadata(
|
||||
path: &Path,
|
||||
format: BookFormat,
|
||||
library_root: &Path,
|
||||
) -> Result<ParsedMetadata> {
|
||||
let mut meta = parse_metadata_fast(path, format, library_root);
|
||||
|
||||
meta.page_count = match format {
|
||||
BookFormat::Cbz => parse_cbz_page_count(path).ok(),
|
||||
BookFormat::Cbr => parse_cbr_page_count(path).ok(),
|
||||
BookFormat::Pdf => parse_pdf_page_count(path).ok(),
|
||||
};
|
||||
|
||||
Ok(ParsedMetadata {
|
||||
title,
|
||||
series,
|
||||
volume,
|
||||
page_count,
|
||||
})
|
||||
Ok(meta)
|
||||
}
|
||||
|
||||
fn extract_volume(filename: &str) -> Option<i32> {
|
||||
// Common volume patterns: T01, T02, T1, T2, Vol 1, Vol. 1, Volume 1, #1, #01, - 1, - 01
|
||||
let patterns = [
|
||||
// T01, T02 pattern (most common for manga/comics)
|
||||
(r"(?i)T(\d+)", 1),
|
||||
// Vol 1, Vol. 1, Volume 1
|
||||
(r"(?i)Vol\.?\s*(\d+)", 1),
|
||||
(r"(?i)Volume\s*(\d+)", 1),
|
||||
// #1, #01
|
||||
(r"#(\d+)", 1),
|
||||
// - 1, - 01 at the end
|
||||
(r"-\s*(\d+)\s*$", 1),
|
||||
];
|
||||
|
||||
for (pattern, group) in &patterns {
|
||||
if let Ok(re) = regex::Regex::new(pattern) {
|
||||
if let Some(caps) = re.captures(filename) {
|
||||
if let Some(mat) = caps.get(*group) {
|
||||
// Parse as integer to remove leading zeros
|
||||
return mat.as_str().parse::<i32>().ok();
|
||||
}
|
||||
}
|
||||
/// Open an archive once and return (page_count, first_page_bytes).
|
||||
/// This is more efficient than calling parse_metadata + extract_first_page separately.
|
||||
pub fn analyze_book(path: &Path, format: BookFormat) -> Result<(i32, Vec<u8>)> {
|
||||
match format {
|
||||
BookFormat::Cbz => analyze_cbz(path),
|
||||
BookFormat::Cbr => analyze_cbr(path),
|
||||
BookFormat::Pdf => analyze_pdf(path),
|
||||
}
|
||||
}
|
||||
|
||||
None
|
||||
fn analyze_cbz(path: &Path) -> Result<(i32, Vec<u8>)> {
|
||||
let file = std::fs::File::open(path)
|
||||
.with_context(|| format!("cannot open cbz: {}", path.display()))?;
|
||||
let mut archive = zip::ZipArchive::new(file).context("invalid cbz archive")?;
|
||||
|
||||
let mut image_names: Vec<String> = Vec::new();
|
||||
for i in 0..archive.len() {
|
||||
let entry = archive.by_index(i).context("cannot read cbz entry")?;
|
||||
let name = entry.name().to_ascii_lowercase();
|
||||
if is_image_name(&name) {
|
||||
image_names.push(entry.name().to_string());
|
||||
}
|
||||
}
|
||||
image_names.sort();
|
||||
|
||||
let count = image_names.len() as i32;
|
||||
let first_image = image_names.first().context("no images found in cbz")?;
|
||||
|
||||
let mut entry = archive
|
||||
.by_name(first_image)
|
||||
.context("cannot read first image")?;
|
||||
let mut buf = Vec::new();
|
||||
entry.read_to_end(&mut buf)?;
|
||||
|
||||
Ok((count, buf))
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
fn clean_title(filename: &str) -> String {
|
||||
// Remove volume patterns from title to clean it up
|
||||
let cleaned = regex::Regex::new(r"(?i)\s*T\d+\s*")
|
||||
.ok()
|
||||
.and_then(|re| Some(re.replace_all(filename, " ").to_string()))
|
||||
.unwrap_or_else(|| filename.to_string());
|
||||
fn list_cbr_images(path: &Path) -> Result<Vec<String>> {
|
||||
// Try unrar lb first (fast)
|
||||
let output = std::process::Command::new("unrar")
|
||||
.arg("lb")
|
||||
.arg(path)
|
||||
.output()
|
||||
.with_context(|| format!("failed to execute unrar lb for {}", path.display()))?;
|
||||
|
||||
let cleaned = regex::Regex::new(r"(?i)\s*Vol\.?\s*\d+\s*")
|
||||
.ok()
|
||||
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
|
||||
.unwrap_or_else(|| cleaned);
|
||||
if output.status.success() {
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let images: Vec<String> = stdout
|
||||
.lines()
|
||||
.filter(|line| is_image_name(&line.to_ascii_lowercase()))
|
||||
.map(|l| l.to_string())
|
||||
.collect();
|
||||
if !images.is_empty() {
|
||||
return Ok(images);
|
||||
}
|
||||
}
|
||||
|
||||
let cleaned = regex::Regex::new(r"(?i)\s*Volume\s*\d+\s*")
|
||||
.ok()
|
||||
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
|
||||
.unwrap_or_else(|| cleaned);
|
||||
// Fallback: lsar (from unar package) handles UTF-16BE encoded filenames
|
||||
let lsar_output = std::process::Command::new("lsar")
|
||||
.arg(path)
|
||||
.output()
|
||||
.with_context(|| format!("failed to execute lsar for {}", path.display()))?;
|
||||
|
||||
let cleaned = regex::Regex::new(r"#\d+")
|
||||
.ok()
|
||||
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
|
||||
.unwrap_or_else(|| cleaned);
|
||||
if !lsar_output.status.success() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"both unrar lb and lsar failed for {}",
|
||||
path.display()
|
||||
));
|
||||
}
|
||||
|
||||
let cleaned = regex::Regex::new(r"-\s*\d+\s*$")
|
||||
.ok()
|
||||
.and_then(|re| Some(re.replace_all(&cleaned, " ").to_string()))
|
||||
.unwrap_or_else(|| cleaned);
|
||||
let stdout = String::from_utf8_lossy(&lsar_output.stdout);
|
||||
// lsar output: first line is archive info, then one file per line (indented)
|
||||
let images: Vec<String> = stdout
|
||||
.lines()
|
||||
.skip(1) // skip the archive header line
|
||||
.map(|l| l.trim().to_string())
|
||||
.filter(|line| is_image_name(&line.to_ascii_lowercase()))
|
||||
.collect();
|
||||
|
||||
// Clean up extra spaces
|
||||
cleaned.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||
Ok(images)
|
||||
}
|
||||
|
||||
fn analyze_cbr(path: &Path) -> Result<(i32, Vec<u8>)> {
|
||||
let mut image_names = list_cbr_images(path)?;
|
||||
image_names.sort();
|
||||
|
||||
let count = image_names.len() as i32;
|
||||
if count == 0 {
|
||||
return Err(anyhow::anyhow!("no images found in cbr: {}", path.display()));
|
||||
}
|
||||
|
||||
let first_name = &image_names[0];
|
||||
|
||||
// Try unrar p to extract first image to stdout (faster — no temp dir)
|
||||
let p_output = std::process::Command::new("unrar")
|
||||
.args(["p", "-inul"])
|
||||
.arg(path)
|
||||
.arg(first_name)
|
||||
.output();
|
||||
|
||||
match p_output {
|
||||
Ok(out) if out.status.success() && looks_like_image(&out.stdout) => Ok((count, out.stdout)),
|
||||
_ => {
|
||||
// Fallback: full extraction with unar (handles special chars, encoding issues)
|
||||
let image_bytes = extract_cbr_first_page(path)?;
|
||||
Ok((count, image_bytes))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Check image magic bytes to validate that bytes are a real image before decoding.
|
||||
fn looks_like_image(bytes: &[u8]) -> bool {
|
||||
if bytes.len() < 12 {
|
||||
return false;
|
||||
}
|
||||
// JPEG: FF D8 FF
|
||||
if bytes.starts_with(&[0xFF, 0xD8, 0xFF]) {
|
||||
return true;
|
||||
}
|
||||
// PNG: 89 50 4E 47 0D 0A 1A 0A
|
||||
if bytes.starts_with(&[0x89, 0x50, 0x4E, 0x47]) {
|
||||
return true;
|
||||
}
|
||||
// WebP: RIFF....WEBP
|
||||
if &bytes[0..4] == b"RIFF" && &bytes[8..12] == b"WEBP" {
|
||||
return true;
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
fn analyze_pdf(path: &Path) -> Result<(i32, Vec<u8>)> {
|
||||
let count = parse_pdf_page_count(path)?;
|
||||
let image_bytes = extract_pdf_first_page(path)?;
|
||||
Ok((count, image_bytes))
|
||||
}
|
||||
|
||||
fn parse_cbz_page_count(path: &Path) -> Result<i32> {
|
||||
@@ -192,26 +303,11 @@ fn parse_cbz_page_count(path: &Path) -> Result<i32> {
|
||||
}
|
||||
|
||||
fn parse_cbr_page_count(path: &Path) -> Result<i32> {
|
||||
let output = std::process::Command::new("unrar")
|
||||
.arg("lb")
|
||||
.arg(path)
|
||||
.output()
|
||||
.with_context(|| format!("failed to execute unrar for {}", path.display()))?;
|
||||
|
||||
if !output.status.success() {
|
||||
return Err(anyhow::anyhow!("unrar failed for {}", path.display()));
|
||||
}
|
||||
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let count = stdout
|
||||
.lines()
|
||||
.filter(|line| is_image_name(&line.to_ascii_lowercase()))
|
||||
.count() as i32;
|
||||
Ok(count)
|
||||
let images = list_cbr_images(path)?;
|
||||
Ok(images.len() as i32)
|
||||
}
|
||||
|
||||
fn parse_pdf_page_count(path: &Path) -> Result<i32> {
|
||||
// Use pdfinfo command line tool instead of lopdf for better performance
|
||||
let output = std::process::Command::new("pdfinfo")
|
||||
.arg(path)
|
||||
.output()
|
||||
@@ -238,6 +334,10 @@ fn parse_pdf_page_count(path: &Path) -> Result<i32> {
|
||||
}
|
||||
|
||||
fn is_image_name(name: &str) -> bool {
|
||||
// Skip macOS metadata entries (__MACOSX/ prefix or AppleDouble ._* files)
|
||||
if name.starts_with("__macosx/") || name.contains("/._") || name.starts_with("._") {
|
||||
return false;
|
||||
}
|
||||
name.ends_with(".jpg")
|
||||
|| name.ends_with(".jpeg")
|
||||
|| name.ends_with(".png")
|
||||
@@ -282,7 +382,6 @@ fn extract_cbr_first_page(path: &Path) -> Result<Vec<u8>> {
|
||||
let tmp_dir = std::env::temp_dir().join(format!("stripstream-cbr-thumb-{}", Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&tmp_dir).context("cannot create temp dir")?;
|
||||
|
||||
// Use env command like the API does
|
||||
let output = std::process::Command::new("env")
|
||||
.args(["LC_ALL=en_US.UTF-8", "LANG=en_US.UTF-8", "unar", "-o"])
|
||||
.arg(&tmp_dir)
|
||||
@@ -298,7 +397,6 @@ fn extract_cbr_first_page(path: &Path) -> Result<Vec<u8>> {
|
||||
));
|
||||
}
|
||||
|
||||
// Use WalkDir for recursive search (CBR can have subdirectories)
|
||||
let mut image_files: Vec<_> = WalkDir::new(&tmp_dir)
|
||||
.into_iter()
|
||||
.filter_map(|e| e.ok())
|
||||
@@ -346,3 +444,33 @@ fn extract_pdf_first_page(path: &Path) -> Result<Vec<u8>> {
|
||||
let _ = std::fs::remove_dir_all(&tmp_dir);
|
||||
Ok(data)
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
fn clean_title(filename: &str) -> String {
|
||||
let cleaned = regex::Regex::new(r"(?i)\s*T\d+\s*")
|
||||
.ok()
|
||||
.map(|re| re.replace_all(filename, " ").to_string())
|
||||
.unwrap_or_else(|| filename.to_string());
|
||||
|
||||
let cleaned = regex::Regex::new(r"(?i)\s*Vol\.?\s*\d+\s*")
|
||||
.ok()
|
||||
.map(|re| re.replace_all(&cleaned, " ").to_string())
|
||||
.unwrap_or(cleaned);
|
||||
|
||||
let cleaned = regex::Regex::new(r"(?i)\s*Volume\s*\d+\s*")
|
||||
.ok()
|
||||
.map(|re| re.replace_all(&cleaned, " ").to_string())
|
||||
.unwrap_or(cleaned);
|
||||
|
||||
let cleaned = regex::Regex::new(r"#\d+")
|
||||
.ok()
|
||||
.map(|re| re.replace_all(&cleaned, " ").to_string())
|
||||
.unwrap_or(cleaned);
|
||||
|
||||
let cleaned = regex::Regex::new(r"-\s*\d+\s*$")
|
||||
.ok()
|
||||
.map(|re| re.replace_all(&cleaned, " ").to_string())
|
||||
.unwrap_or(cleaned);
|
||||
|
||||
cleaned.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||
}
|
||||
|
||||
@@ -18,7 +18,7 @@ services:
|
||||
meilisearch:
|
||||
image: getmeili/meilisearch:v1.12
|
||||
env_file:
|
||||
- ../.env
|
||||
- .env
|
||||
ports:
|
||||
- "7700:7700"
|
||||
volumes:
|
||||
@@ -39,7 +39,7 @@ services:
|
||||
POSTGRES_PASSWORD: stripstream
|
||||
POSTGRES_DB: stripstream
|
||||
volumes:
|
||||
- ./migrations:/migrations:ro
|
||||
- ./infra/migrations:/migrations:ro
|
||||
command:
|
||||
[
|
||||
"sh",
|
||||
@@ -49,15 +49,15 @@ services:
|
||||
|
||||
api:
|
||||
build:
|
||||
context: ..
|
||||
context: .
|
||||
dockerfile: apps/api/Dockerfile
|
||||
env_file:
|
||||
- ../.env
|
||||
- .env
|
||||
ports:
|
||||
- "7080:8080"
|
||||
- "7080:7080"
|
||||
volumes:
|
||||
- ${LIBRARIES_HOST_PATH:-../libraries}:/libraries
|
||||
- ${THUMBNAILS_HOST_PATH:-../data/thumbnails}:/data/thumbnails
|
||||
- ${LIBRARIES_HOST_PATH:-./libraries}:/libraries
|
||||
- ${THUMBNAILS_HOST_PATH:-./data/thumbnails}:/data/thumbnails
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
@@ -66,22 +66,22 @@ services:
|
||||
meilisearch:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8080/health"]
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:7080/health"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
indexer:
|
||||
build:
|
||||
context: ..
|
||||
context: .
|
||||
dockerfile: apps/indexer/Dockerfile
|
||||
env_file:
|
||||
- ../.env
|
||||
- .env
|
||||
ports:
|
||||
- "7081:8081"
|
||||
- "7081:7081"
|
||||
volumes:
|
||||
- ${LIBRARIES_HOST_PATH:-../libraries}:/libraries
|
||||
- ${THUMBNAILS_HOST_PATH:-../data/thumbnails}:/data/thumbnails
|
||||
- ${LIBRARIES_HOST_PATH:-./libraries}:/libraries
|
||||
- ${THUMBNAILS_HOST_PATH:-./data/thumbnails}:/data/thumbnails
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
@@ -90,27 +90,27 @@ services:
|
||||
meilisearch:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8081/health"]
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:7081/health"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
backoffice:
|
||||
build:
|
||||
context: ..
|
||||
context: .
|
||||
dockerfile: apps/backoffice/Dockerfile
|
||||
env_file:
|
||||
- ../.env
|
||||
- .env
|
||||
environment:
|
||||
- PORT=8082
|
||||
- PORT=7082
|
||||
- HOST=0.0.0.0
|
||||
ports:
|
||||
- "7082:8082"
|
||||
- "7082:7082"
|
||||
depends_on:
|
||||
api:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8082/health"]
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:7082/health"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BASE_API="${BASE_API:-http://127.0.0.1:8080}"
|
||||
BASE_API="${BASE_API:-http://127.0.0.1:7080}"
|
||||
TOKEN="${API_TOKEN:-stripstream-dev-bootstrap-token}"
|
||||
|
||||
measure() {
|
||||
|
||||
8
infra/migrations/0012_directory_mtimes.sql
Normal file
8
infra/migrations/0012_directory_mtimes.sql
Normal file
@@ -0,0 +1,8 @@
|
||||
CREATE TABLE directory_mtimes (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
library_id UUID NOT NULL REFERENCES libraries(id) ON DELETE CASCADE,
|
||||
dir_path TEXT NOT NULL,
|
||||
mtime TIMESTAMPTZ NOT NULL,
|
||||
UNIQUE(library_id, dir_path)
|
||||
);
|
||||
CREATE INDEX idx_directory_mtimes_library ON directory_mtimes(library_id);
|
||||
114
infra/smoke.sh
114
infra/smoke.sh
@@ -1,10 +1,38 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
BASE_API="${BASE_API:-http://127.0.0.1:8080}"
|
||||
BASE_INDEXER="${BASE_INDEXER:-http://127.0.0.1:8081}"
|
||||
BASE_BACKOFFICE="${BASE_BACKOFFICE:-${BASE_ADMIN:-http://127.0.0.1:8082}}"
|
||||
BASE_API="${BASE_API:-http://127.0.0.1:7080}"
|
||||
BASE_INDEXER="${BASE_INDEXER:-http://127.0.0.1:7081}"
|
||||
BASE_BACKOFFICE="${BASE_BACKOFFICE:-${BASE_ADMIN:-http://127.0.0.1:7082}}"
|
||||
TOKEN="${API_TOKEN:-stripstream-dev-bootstrap-token}"
|
||||
# Max seconds to wait for a job to finish
|
||||
JOB_TIMEOUT="${JOB_TIMEOUT:-120}"
|
||||
|
||||
# ─── helpers ────────────────────────────────────────────────────────────────
|
||||
|
||||
auth() { curl -fsS -H "Authorization: Bearer $TOKEN" "$@"; }
|
||||
|
||||
# Wait for a job (by id) to reach status success or failed.
|
||||
wait_job() {
|
||||
local job_id="$1"
|
||||
local label="${2:-job}"
|
||||
local waited=0
|
||||
while true; do
|
||||
local status
|
||||
status="$(auth "$BASE_API/index/jobs/$job_id" | python3 -c "import sys,json; print(json.load(sys.stdin).get('status',''))")"
|
||||
case "$status" in
|
||||
success) echo "[smoke] $label finished: success"; return 0 ;;
|
||||
failed) echo "[smoke] $label finished: FAILED"; return 1 ;;
|
||||
cancelled) echo "[smoke] $label finished: cancelled"; return 1 ;;
|
||||
esac
|
||||
if [ "$waited" -ge "$JOB_TIMEOUT" ]; then
|
||||
echo "[smoke] $label timed out after ${JOB_TIMEOUT}s (last status: $status)"; return 1
|
||||
fi
|
||||
sleep 2; waited=$((waited + 2))
|
||||
done
|
||||
}
|
||||
|
||||
# ─── health ──────────────────────────────────────────────────────────────────
|
||||
|
||||
echo "[smoke] health checks"
|
||||
curl -fsS "$BASE_API/health" >/dev/null
|
||||
@@ -13,29 +41,89 @@ curl -fsS "$BASE_INDEXER/health" >/dev/null
|
||||
curl -fsS "$BASE_INDEXER/ready" >/dev/null
|
||||
curl -fsS "$BASE_BACKOFFICE/health" >/dev/null
|
||||
|
||||
# ─── libraries ───────────────────────────────────────────────────────────────
|
||||
|
||||
echo "[smoke] list libraries"
|
||||
curl -fsS -H "Authorization: Bearer $TOKEN" "$BASE_API/libraries" >/dev/null
|
||||
auth "$BASE_API/libraries" >/dev/null
|
||||
|
||||
echo "[smoke] queue rebuild"
|
||||
curl -fsS -X POST -H "Authorization: Bearer $TOKEN" "$BASE_API/index/rebuild" >/dev/null
|
||||
sleep 2
|
||||
# ─── full rebuild (2-phase: discovery + analysis) ────────────────────────────
|
||||
|
||||
echo "[smoke] list books and optional page fetch"
|
||||
BOOKS_JSON="$(curl -fsS -H "Authorization: Bearer $TOKEN" "$BASE_API/books")"
|
||||
BOOK_ID="$(BOOKS_JSON="$BOOKS_JSON" python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
echo "[smoke] queue full rebuild"
|
||||
REBUILD_JOB_ID="$(auth -X POST "$BASE_API/index/rebuild" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")"
|
||||
echo "[smoke] rebuild job id: $REBUILD_JOB_ID"
|
||||
wait_job "$REBUILD_JOB_ID" "rebuild"
|
||||
|
||||
# ─── verify books have page_count + thumbnail after analysis phase ────────────
|
||||
|
||||
echo "[smoke] verify books metadata (page_count + thumbnail)"
|
||||
BOOKS_JSON="$(auth "$BASE_API/books")"
|
||||
export BOOKS_JSON
|
||||
python3 - <<'PY'
|
||||
import json, os, sys
|
||||
|
||||
payload = json.loads(os.environ.get("BOOKS_JSON", "{}"))
|
||||
items = payload.get("items") or []
|
||||
if not items:
|
||||
print("[smoke] no books found — skipping metadata check")
|
||||
sys.exit(0)
|
||||
|
||||
missing_page_count = [b["id"] for b in items if not b.get("page_count")]
|
||||
missing_thumbnail = [b["id"] for b in items if not b.get("thumbnail_url")]
|
||||
|
||||
if missing_page_count:
|
||||
print(f"[smoke] WARN: {len(missing_page_count)} book(s) still missing page_count")
|
||||
if missing_thumbnail:
|
||||
print(f"[smoke] WARN: {len(missing_thumbnail)} book(s) still missing thumbnail")
|
||||
|
||||
print(f"[smoke] {len(items)} books, {len(items)-len(missing_page_count)} with page_count, {len(items)-len(missing_thumbnail)} with thumbnail")
|
||||
PY
|
||||
|
||||
# ─── page fetch ──────────────────────────────────────────────────────────────
|
||||
|
||||
BOOK_ID="$(python3 - <<'PY'
|
||||
import json, os
|
||||
items = json.loads(os.environ.get("BOOKS_JSON", "{}")).get("items") or []
|
||||
print(items[0]["id"] if items else "")
|
||||
PY
|
||||
)"
|
||||
|
||||
if [ -n "$BOOK_ID" ]; then
|
||||
curl -fsS -H "Authorization: Bearer $TOKEN" "$BASE_API/books/$BOOK_ID/pages/1?format=webp&quality=80&width=1080" >/dev/null
|
||||
echo "[smoke] fetch page 1 for book $BOOK_ID"
|
||||
auth "$BASE_API/books/$BOOK_ID/pages/1?format=webp&quality=80&width=1080" >/dev/null
|
||||
|
||||
echo "[smoke] fetch thumbnail for book $BOOK_ID"
|
||||
auth "$BASE_API/books/$BOOK_ID/thumbnail" >/dev/null
|
||||
fi
|
||||
|
||||
# ─── thumbnail rebuild (handled by indexer, not API) ─────────────────────────
|
||||
|
||||
echo "[smoke] thumbnail rebuild job"
|
||||
THUMB_REBUILD_ID="$(auth -X POST -H "Content-Type: application/json" -d '{}' "$BASE_API/index/thumbnails/rebuild" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")"
|
||||
echo "[smoke] thumbnail rebuild job id: $THUMB_REBUILD_ID"
|
||||
wait_job "$THUMB_REBUILD_ID" "thumbnail_rebuild"
|
||||
|
||||
# ─── thumbnail regenerate ────────────────────────────────────────────────────
|
||||
|
||||
echo "[smoke] thumbnail regenerate job"
|
||||
THUMB_REGEN_ID="$(auth -X POST -H "Content-Type: application/json" -d '{}' "$BASE_API/index/thumbnails/regenerate" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])")"
|
||||
echo "[smoke] thumbnail regenerate job id: $THUMB_REGEN_ID"
|
||||
wait_job "$THUMB_REGEN_ID" "thumbnail_regenerate"
|
||||
|
||||
# ─── route checkup supprimée (doit retourner 404) ────────────────────────────
|
||||
|
||||
echo "[smoke] /index/jobs/:id/thumbnails/checkup must be gone (404)"
|
||||
HTTP_CODE="$(curl -s -o /dev/null -w "%{http_code}" -X POST \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
"$BASE_API/index/jobs/$REBUILD_JOB_ID/thumbnails/checkup")"
|
||||
if [ "$HTTP_CODE" = "404" ]; then
|
||||
echo "[smoke] checkup route correctly returns 404"
|
||||
else
|
||||
echo "[smoke] FAIL: checkup route returned $HTTP_CODE (expected 404)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ─── metrics ─────────────────────────────────────────────────────────────────
|
||||
|
||||
echo "[smoke] metrics"
|
||||
curl -fsS "$BASE_API/metrics" >/dev/null
|
||||
|
||||
|
||||
20
openspec/config.yaml
Normal file
20
openspec/config.yaml
Normal file
@@ -0,0 +1,20 @@
|
||||
schema: spec-driven
|
||||
|
||||
# Project context (optional)
|
||||
# This is shown to AI when creating artifacts.
|
||||
# Add your tech stack, conventions, style guides, domain knowledge, etc.
|
||||
# Example:
|
||||
# context: |
|
||||
# Tech stack: TypeScript, React, Node.js
|
||||
# We use conventional commits
|
||||
# Domain: e-commerce platform
|
||||
|
||||
# Per-artifact rules (optional)
|
||||
# Add custom rules for specific artifacts.
|
||||
# Example:
|
||||
# rules:
|
||||
# proposal:
|
||||
# - Keep proposals under 500 words
|
||||
# - Always include a "Non-goals" section
|
||||
# tasks:
|
||||
# - Break tasks into chunks of max 2 hours
|
||||
Reference in New Issue
Block a user