diff --git a/.gitignore b/.gitignore index c7b5dc7..ba2b5eb 100644 --- a/.gitignore +++ b/.gitignore @@ -42,3 +42,4 @@ *.swp *.swo .DS_Store +/.worktrees/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..d27edba --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,46 @@ +# AGENTS.md + +## Project Purpose + +This repository is a plugin-based C++ video analytics system for RK3588. Development should follow the existing DAG + plugin architecture instead of adding feature-specific shortcuts into the main process. + +## Build and Verification Constraints + +- This repository does not use local cross-compilation in the current workflow. +- Final build, link, runtime verification, and hardware-dependent testing must be executed on the RK3588 target device. +- Do not assume the current workstation can build or run RKNN, RGA, MPP, ZLMediaKit, or full media pipelines. +- Do not run full project builds by default on the current workstation unless the task explicitly requires it and the needed dependencies are confirmed to exist locally. +- Prefer static inspection and targeted platform-independent tests over speculative local builds. +- If a task depends on RK3588-only capabilities, state clearly what was reviewed locally and what still must be verified on the device. +- Do not claim that a hardware-related change is verified unless it was actually built and exercised on RK3588. + +## Local Development Expectations + +- Local work should focus on architecture, plugin boundaries, data structures, configuration, documentation, code review, and platform-independent tests. +- Local build attempts are allowed only when they are clearly useful and do not depend on missing RK3588 runtime libraries or device-only features. +- If local compilation is attempted, keep it scoped to the smallest useful target. +- Do not distort production code to satisfy the local machine environment. +- Do not remove or weaken RK3588-specific paths just because they are not runnable on the current workstation. + +## Plugin and Architecture Rules + +- New analytics capabilities should be implemented as plugins when they represent pipeline nodes. +- Reuse existing shared frame metadata and plugin interfaces where possible. +- If a new feature introduces a new semantic output type, define an explicit data model instead of encoding it into unrelated existing fields. +- Keep node responsibilities narrow. Detection, tracking, event reasoning, visualization, and alarming should remain separated unless there is a strong architectural reason not to. + +## Testing and Validation Rules + +- Prefer targeted unit tests for platform-independent logic. +- For RK3588-only features, document a device-side validation checklist instead of pretending local validation is sufficient. +- When a test or build cannot run locally because of platform constraints, say so explicitly in the final report. +- Separate validation into two buckets: + - local code-level validation + - RK3588 device-side validation +- Do not merge those two buckets into a single vague statement such as "verified" or "tested" without saying where verification happened. + +## Communication Rules for Agents + +- Say what is known from code inspection versus what still requires RK3588 device verification. +- Favor root-cause fixes over environment-specific patches. +- Keep recommendations aligned with the shortest path that preserves the architecture. diff --git a/docs/superpowers/plans/2026-04-01-region-behavior-recognition.md b/docs/superpowers/plans/2026-04-01-region-behavior-recognition.md new file mode 100644 index 0000000..c73f160 --- /dev/null +++ b/docs/superpowers/plans/2026-04-01-region-behavior-recognition.md @@ -0,0 +1,696 @@ +# Region Behavior Recognition Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add first-release region behavior recognition with unified behavior event data, rule-based region events, temporal-rule action events, event fusion, and downstream OSD/alarm support. + +**Architecture:** Extend `Frame` with a dedicated behavior-event payload, then add three focused plugins: `region_event`, `action_recog`, and `event_fusion`. Keep the first release linear in the graph so each node appends or normalizes behavior events on the same `Frame`, while `osd` and `alarm` consume the normalized result without overloading detection classes. + +**Tech Stack:** C++17, existing plugin `INode` ABI, RK3588 DAG graph manager, GoogleTest, CMake + +--- + +## File Map + +- Modify: `D:\App\C++\Rk3588Sys\include\frame\frame.h` + - Add behavior event enums and result structures to `Frame` +- Create: `D:\App\C++\Rk3588Sys\include\behavior\behavior_event.h` + - Shared behavior-event definitions and helpers if `frame.h` becomes noisy +- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt` + - Register new plugins in the build +- Create: `D:\App\C++\Rk3588Sys\plugins\region_event\region_event_node.cpp` + - Rule-based intrusion and climb events +- Create: `D:\App\C++\Rk3588Sys\plugins\action_recog\action_recog_node.cpp` + - Temporal-rule fall and fight events +- Create: `D:\App\C++\Rk3588Sys\plugins\event_fusion\event_fusion_node.cpp` + - Stable event normalization, ID assignment, and lifecycle progression +- Modify: `D:\App\C++\Rk3588Sys\plugins\osd\osd_node.cpp` + - Draw behavior labels, track IDs, and durations +- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.h` + - Add behavior-event-aware alarm rule shape +- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.cpp` + - Evaluate behavior events in addition to detections +- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\alarm_node.cpp` + - Parse behavior-event alarm rules and preserve old detection rules +- Create: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp` + - Validate frame-level behavior event model +- Create: `D:\App\C++\Rk3588Sys\tests\test_region_event.cpp` + - Validate intrusion and climb rule logic +- Create: `D:\App\C++\Rk3588Sys\tests\test_action_recog.cpp` + - Validate fall and fight temporal state-machine logic +- Create: `D:\App\C++\Rk3588Sys\tests\test_event_fusion.cpp` + - Validate event merge and lifecycle behavior +- Create: `D:\App\C++\Rk3588Sys\tests\test_alarm_behavior_events.cpp` + - Validate alarm matching on behavior events +- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt` + - Add new test files to `rk3588_gtests` +- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_intrusion.json` + - Reference graph for intrusion pipeline +- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_full.json` + - Reference graph for intrusion, climb, fall, and fight +- Modify: `D:\App\C++\Rk3588Sys\Readme.md` + - Document behavior event model and sample pipeline + +### Task 1: Add Behavior Event Data Model + +**Files:** +- Create: `D:\App\C++\Rk3588Sys\include\behavior\behavior_event.h` +- Modify: `D:\App\C++\Rk3588Sys\include\frame\frame.h` +- Test: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp` +- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt` + +- [ ] **Step 1: Write the failing test** + +```cpp +#include + +#include "frame/frame.h" + +namespace rk3588 { + +TEST(BehaviorEventModelTest, FrameStoresBehaviorEventsSeparatelyFromDetections) { + auto frame = std::make_shared(); + frame->det = std::make_shared(); + frame->behavior_events = std::make_shared(); + + BehaviorEventItem item; + item.event_id = 7; + item.type = BehaviorEventType::Fall; + item.status = BehaviorEventStatus::Active; + item.track_ids = {42}; + item.duration_ms = 1600; + + frame->behavior_events->items.push_back(item); + + ASSERT_NE(frame->det, nullptr); + ASSERT_NE(frame->behavior_events, nullptr); + ASSERT_EQ(frame->behavior_events->items.size(), 1u); + EXPECT_EQ(frame->behavior_events->items[0].track_ids[0], 42); + EXPECT_EQ(frame->behavior_events->items[0].duration_ms, 1600u); +} + +} // namespace rk3588 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure` + +Expected: compile failure because `BehaviorEventResult`, `BehaviorEventItem`, and related enums do not exist. + +- [ ] **Step 3: Write minimal implementation** + +```cpp +// include/behavior/behavior_event.h +#pragma once + +#include +#include +#include + +#include "frame/frame.h" + +namespace rk3588 { + +enum class BehaviorEventType { Intrusion, Climb, Fall, Fight }; +enum class BehaviorEventStatus { Pending, Active, Ended }; + +struct BehaviorEventItem { + int event_id = -1; + BehaviorEventType type = BehaviorEventType::Intrusion; + BehaviorEventStatus status = BehaviorEventStatus::Pending; + float score = 0.0f; + Rect bbox{}; + std::vector track_ids; + uint64_t start_pts = 0; + uint64_t last_pts = 0; + uint64_t duration_ms = 0; + std::string source; + std::string region_id; +}; + +struct BehaviorEventResult { + std::vector items; +}; + +} // namespace rk3588 +``` + +```cpp +// include/frame/frame.h +#include "behavior/behavior_event.h" + +struct Frame { + // existing fields... + std::shared_ptr behavior_events; +}; +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure` + +Expected: `BehaviorEventModelTest` passes. + +- [ ] **Step 5: Commit** + +```bash +git add include/behavior/behavior_event.h include/frame/frame.h tests/test_behavior_event_model.cpp tests/CMakeLists.txt +git commit -m "feat: add behavior event frame model" +``` + +### Task 2: Implement `region_event` for Intrusion and Rule-Based Climb + +**Files:** +- Create: `D:\App\C++\Rk3588Sys\plugins\region_event\region_event_node.cpp` +- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt` +- Test: `D:\App\C++\Rk3588Sys\tests\test_region_event.cpp` +- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt` + +- [ ] **Step 1: Write the failing tests** + +```cpp +#include + +#include "frame/frame.h" + +namespace rk3588 { + +class RegionEventNode; + +TEST(RegionEventTest, EmitsIntrusionWhenTrackedPersonStaysInsideRegion) { + RegionEventNode node; + // Build config with one intrusion ROI and min_duration_ms = 0 for deterministic test. + // Feed a frame with one tracked person inside the ROI. + // Expect one active intrusion event. +} + +TEST(RegionEventTest, EmitsClimbWhenTrackCrossesBoundaryOverTime) { + RegionEventNode node; + // Build config with one climb boundary. + // Feed a short sequence where one track moves across boundary with upward motion. + // Expect one climb event with the same track_id. +} + +} // namespace rk3588 +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R RegionEventTest --output-on-failure` + +Expected: compile failure because `RegionEventNode` and plugin logic do not exist. + +- [ ] **Step 3: Write minimal implementation** + +```cpp +// plugins/region_event/region_event_node.cpp +class RegionEventNode final : public INode { +public: + std::string Id() const override { return id_; } + std::string Type() const override { return "region_event"; } + + bool Init(const SimpleJson& config, const NodeContext& ctx) override; + bool Start() override { return true; } + void Stop() override {} + NodeStatus Process(FramePtr frame) override { + if (!frame || !frame->det) return NodeStatus::DROP; + EnsureBehaviorEvents(*frame); + EvaluateIntrusion(*frame); + EvaluateClimb(*frame); + Push(frame); + return NodeStatus::OK; + } + +private: + void EnsureBehaviorEvents(Frame& frame); + void EvaluateIntrusion(Frame& frame); + void EvaluateClimb(Frame& frame); + void Push(const FramePtr& frame); + + std::string id_; + NodeContext ctx_{}; +}; + +REGISTER_NODE(RegionEventNode, "region_event"); +``` + +Implementation constraints: + +- Only inspect detections with valid `track_id` +- Append to `frame->behavior_events` without mutating `frame->det` +- Store per-track state inside the node for duration and boundary-crossing checks + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R RegionEventTest --output-on-failure` + +Expected: intrusion and climb tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/region_event/region_event_node.cpp plugins/CMakeLists.txt tests/test_region_event.cpp tests/CMakeLists.txt +git commit -m "feat: add region event plugin" +``` + +### Task 3: Implement `action_recog` for Temporal Fall and Fight + +**Files:** +- Create: `D:\App\C++\Rk3588Sys\plugins\action_recog\action_recog_node.cpp` +- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt` +- Test: `D:\App\C++\Rk3588Sys\tests\test_action_recog.cpp` +- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt` + +- [ ] **Step 1: Write the failing tests** + +```cpp +#include + +namespace rk3588 { + +class ActionRecogNode; + +TEST(ActionRecogTest, EmitsFallAfterRapidDropAndLowPosePersistence) { + ActionRecogNode node; + // Feed same track across frames with centroid drop and aspect-ratio change. + // Expect one active fall event. +} + +TEST(ActionRecogTest, EmitsFightWhenTwoTracksStayCloseWithRepeatedMotion) { + ActionRecogNode node; + // Feed two tracked persons across frames in close proximity with alternating motion. + // Expect one active fight event referencing both track ids. +} + +} // namespace rk3588 +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R ActionRecogTest --output-on-failure` + +Expected: compile failure because `ActionRecogNode` does not exist. + +- [ ] **Step 3: Write minimal implementation** + +```cpp +// plugins/action_recog/action_recog_node.cpp +struct TrackSample { + uint64_t pts = 0; + Rect bbox{}; +}; + +class ActionRecogNode final : public INode { +public: + std::string Id() const override { return id_; } + std::string Type() const override { return "action_recog"; } + bool Init(const SimpleJson& config, const NodeContext& ctx) override; + bool Start() override { return true; } + void Stop() override {} + NodeStatus Process(FramePtr frame) override; + +private: + void UpdateTrackHistory(const Frame& frame); + void EvaluateFall(Frame& frame); + void EvaluateFight(Frame& frame); + + std::string id_; + NodeContext ctx_{}; + std::unordered_map> history_; +}; + +REGISTER_NODE(ActionRecogNode, "action_recog"); +``` + +Implementation constraints: + +- Keep first release rule/state-machine only +- Do not add RKNN model coupling in this task +- Emit `track_ids` with one track for `fall` and two-or-more tracks for `fight` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R ActionRecogTest --output-on-failure` + +Expected: fall and fight tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/action_recog/action_recog_node.cpp plugins/CMakeLists.txt tests/test_action_recog.cpp tests/CMakeLists.txt +git commit -m "feat: add temporal action recognition plugin" +``` + +### Task 4: Implement `event_fusion` for Event IDs and Lifecycle Normalization + +**Files:** +- Create: `D:\App\C++\Rk3588Sys\plugins\event_fusion\event_fusion_node.cpp` +- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt` +- Test: `D:\App\C++\Rk3588Sys\tests\test_event_fusion.cpp` +- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt` + +- [ ] **Step 1: Write the failing tests** + +```cpp +#include + +namespace rk3588 { + +class EventFusionNode; + +TEST(EventFusionTest, AssignsStableEventIdAcrossFramesForSameTrackAndType) { + EventFusionNode node; + // Feed two frames with the same fall event on track 42. + // Expect the same event_id in both outputs. +} + +TEST(EventFusionTest, MarksEventEndedWhenSourceStopsEmittingIt) { + EventFusionNode node; + // Feed one frame with an active intrusion event, then one frame without it. + // Expect an ended event emission before cleanup. +} + +} // namespace rk3588 +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R EventFusionTest --output-on-failure` + +Expected: compile failure because `EventFusionNode` does not exist. + +- [ ] **Step 3: Write minimal implementation** + +```cpp +// plugins/event_fusion/event_fusion_node.cpp +class EventFusionNode final : public INode { +public: + std::string Id() const override { return id_; } + std::string Type() const override { return "event_fusion"; } + bool Init(const SimpleJson& config, const NodeContext& ctx) override; + bool Start() override { return true; } + void Stop() override {} + NodeStatus Process(FramePtr frame) override; + +private: + std::string MakeKey(const BehaviorEventItem& item) const; + + std::string id_; + NodeContext ctx_{}; + int next_event_id_ = 1; + std::unordered_map active_ids_; +}; + +REGISTER_NODE(EventFusionNode, "event_fusion"); +``` + +Normalization rules: + +- key identity by `type + region_id + normalized track_ids` +- reuse prior `event_id` for matching active events +- emit one final `Ended` item before forgetting an event + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R EventFusionTest --output-on-failure` + +Expected: fusion tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/event_fusion/event_fusion_node.cpp plugins/CMakeLists.txt tests/test_event_fusion.cpp tests/CMakeLists.txt +git commit -m "feat: add behavior event fusion plugin" +``` + +### Task 5: Update OSD to Render Behavior Events + +**Files:** +- Modify: `D:\App\C++\Rk3588Sys\plugins\osd\osd_node.cpp` +- Test: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp` + +- [ ] **Step 1: Write the failing test** + +```cpp +TEST(BehaviorEventModelTest, BehaviorEventsExposeTypeTrackAndDurationForOsdFormatting) { + BehaviorEventItem item; + item.type = BehaviorEventType::Intrusion; + item.track_ids = {9}; + item.duration_ms = 2100; + + EXPECT_EQ(FormatBehaviorEventLabel(item), "intrusion #9 2.1s"); +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure` + +Expected: compile failure because behavior-event label formatting helper does not exist. + +- [ ] **Step 3: Write minimal implementation** + +```cpp +// plugins/osd/osd_node.cpp +static std::string BehaviorEventTypeName(BehaviorEventType type) { + switch (type) { + case BehaviorEventType::Intrusion: return "intrusion"; + case BehaviorEventType::Climb: return "climb"; + case BehaviorEventType::Fall: return "fall"; + case BehaviorEventType::Fight: return "fight"; + } + return "unknown"; +} + +static std::string FormatBehaviorEventLabel(const BehaviorEventItem& item) { + std::ostringstream oss; + oss << BehaviorEventTypeName(item.type); + if (!item.track_ids.empty()) oss << " #" << item.track_ids.front(); + oss << " " << std::fixed << std::setprecision(1) + << (static_cast(item.duration_ms) / 1000.0) << "s"; + return oss.str(); +} +``` + +Render rules: + +- draw behavior boxes after detection boxes +- use distinct color per event type +- skip `Pending` by default and render `Active` and `Ended` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure` + +Expected: formatting test passes. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/osd/osd_node.cpp tests/test_behavior_event_model.cpp +git commit -m "feat: render behavior events in osd" +``` + +### Task 6: Extend Alarm Engine to Consume Behavior Events + +**Files:** +- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.h` +- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.cpp` +- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\alarm_node.cpp` +- Test: `D:\App\C++\Rk3588Sys\tests\test_alarm_behavior_events.cpp` +- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt` + +- [ ] **Step 1: Write the failing tests** + +```cpp +#include + +namespace rk3588 { + +TEST(AlarmBehaviorEventsTest, MatchesBehaviorEventByTypeAndDuration) { + RuleEngine engine; + // Init with one behavior-event rule for fall, min_duration_ms = 1000. + // Feed frame with one active fall event of 1500 ms. + // Expect one rule match. +} + +TEST(AlarmBehaviorEventsTest, PreservesLegacyDetectionRules) { + RuleEngine engine; + // Init with a classic detection rule on class_id 0. + // Feed frame->det only. + // Expect legacy path still matches. +} + +} // namespace rk3588 +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R AlarmBehaviorEventsTest --output-on-failure` + +Expected: compile failure because rule engine does not understand behavior-event rules. + +- [ ] **Step 3: Write minimal implementation** + +```cpp +// plugins/alarm/rule_engine.h +struct AlarmRule { + std::string name; + std::set class_ids; + std::set event_types; + std::set region_ids; + RoiRect roi; + float min_score = 0.0f; + bool use_behavior_events = false; + int min_duration_ms = 0; + int cooldown_ms = 5000; +}; +``` + +```cpp +// plugins/alarm/rule_engine.cpp +RuleMatchResult RuleEngine::Evaluate(const std::shared_ptr& frame) { + if (HasBehaviorRules() && frame && frame->behavior_events) { + return EvaluateBehaviorEvents(frame); + } + return EvaluateDetections(frame); +} +``` + +Compatibility rules: + +- behavior-event rules must not break existing `class_ids` rules +- keep old config shape valid +- require explicit `use_behavior_events` or `event_types` to enter new path + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R AlarmBehaviorEventsTest --output-on-failure` + +Expected: both behavior-event and legacy detection tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/alarm/rule_engine.h plugins/alarm/rule_engine.cpp plugins/alarm/alarm_node.cpp tests/test_alarm_behavior_events.cpp tests/CMakeLists.txt +git commit -m "feat: support behavior events in alarm engine" +``` + +### Task 7: Add Sample Configs and Documentation + +**Files:** +- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_intrusion.json` +- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_full.json` +- Modify: `D:\App\C++\Rk3588Sys\Readme.md` + +- [ ] **Step 1: Write the config files** + +```json +{ + "queue": {"size": 8, "strategy": "drop_oldest"}, + "graphs": [ + { + "name": "sample_region_behavior_intrusion", + "nodes": [ + {"id": "in", "type": "input_rtsp", "url": "rtsp://127.0.0.1/live"}, + {"id": "pre", "type": "preprocess", "dst_w": 1920, "dst_h": 1080, "dst_format": "rgb"}, + {"id": "person_det", "type": "ai_yolo", "model_path": "./models/yolov8n-640.rknn", "model_version": "v8", "model_w": 640, "model_h": 640, "class_filter": [0]}, + {"id": "trk", "type": "tracker", "track_classes": [0]}, + {"id": "region_evt", "type": "region_event", "events": [{"type": "intrusion", "region_id": "zone_a", "roi": {"x": 0.1, "y": 0.1, "w": 0.4, "h": 0.4}, "min_duration_ms": 1000}]}, + {"id": "fusion", "type": "event_fusion"}, + {"id": "osd", "type": "osd"}, + {"id": "pub", "type": "publish", "outputs": [{"proto": "rtsp_server", "port": 8555, "path": "/live/behavior"}]}, + {"id": "alarm", "type": "alarm", "rules": [{"name": "intrusion_rule", "use_behavior_events": true, "event_types": ["intrusion"], "min_duration_ms": 1000, "cooldown_ms": 10000}]} + ], + "edges": [["in", "pre"], ["pre", "person_det"], ["person_det", "trk"], ["trk", "region_evt"], ["region_evt", "fusion"], ["fusion", "osd"], ["osd", "pub"], ["pub", "alarm"]] + } + ] +} +``` + +- [ ] **Step 2: Document behavior-event pipeline in README** + +```md +## Behavior Events + +Behavior events are emitted through `frame->behavior_events`, not encoded as detection classes. + +First-release nodes: + +- `region_event`: intrusion, rule-based climb +- `action_recog`: temporal-rule fall, temporal-rule fight +- `event_fusion`: stable event IDs and lifecycle normalization +``` + +- [ ] **Step 3: Validate config syntax** + +Run: `cmake --build build --target rk3588_gtests` + +Expected: project still builds after adding documentation and sample configs. + +- [ ] **Step 4: Commit** + +```bash +git add configs/sample_region_behavior_intrusion.json configs/sample_region_behavior_full.json Readme.md +git commit -m "docs: add behavior event sample pipelines" +``` + +### Task 8: Full Verification Pass + +**Files:** +- Modify as needed based on failures from previous tasks + +- [ ] **Step 1: Build the full test target** + +Run: `cmake --build build --target rk3588_gtests` + +Expected: build succeeds. + +- [ ] **Step 2: Run targeted behavior-event tests** + +Run: `ctest --test-dir build -R "BehaviorEventModelTest|RegionEventTest|ActionRecogTest|EventFusionTest|AlarmBehaviorEventsTest" --output-on-failure` + +Expected: all new behavior-related tests pass. + +- [ ] **Step 3: Run existing regression tests** + +Run: `ctest --test-dir build --output-on-failure` + +Expected: no regressions in existing gtests. + +- [ ] **Step 4: Smoke-check plugin build outputs** + +Run: `cmake --build build --target region_event action_recog event_fusion osd alarm` + +Expected: all plugin targets build successfully. + +- [ ] **Step 5: Commit any final fixes** + +```bash +git add -A +git commit -m "test: verify region behavior recognition pipeline" +``` + +## Self-Review + +Spec coverage check: + +- Unified behavior event data model: covered by Task 1 +- `region_event`, `action_recog`, `event_fusion`: covered by Tasks 2-4 +- OSD behavior-event rendering: covered by Task 5 +- Alarm behavior-event support: covered by Task 6 +- Sample graph and docs: covered by Task 7 +- Verification and regression safety: covered by Task 8 + +Placeholder scan: + +- No `TBD`, `TODO`, or “implement later” markers remain +- Each task lists concrete files and exact commands +- Each code task includes a code sketch to anchor implementation + +Type consistency check: + +- Shared naming uses `BehaviorEventType`, `BehaviorEventStatus`, `BehaviorEventItem`, `BehaviorEventResult` +- Plugin names align with the approved architecture: `region_event`, `action_recog`, `event_fusion` +- Alarm config extension consistently refers to `use_behavior_events`, `event_types`, and `region_ids`