Add agent guidance and plan scaffolding
This commit is contained in:
parent
b0dd2f5eee
commit
2a0a1470cb
1
.gitignore
vendored
1
.gitignore
vendored
@ -42,3 +42,4 @@
|
||||
*.swp
|
||||
*.swo
|
||||
.DS_Store
|
||||
/.worktrees/
|
||||
|
||||
46
AGENTS.md
Normal file
46
AGENTS.md
Normal file
@ -0,0 +1,46 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Project Purpose
|
||||
|
||||
This repository is a plugin-based C++ video analytics system for RK3588. Development should follow the existing DAG + plugin architecture instead of adding feature-specific shortcuts into the main process.
|
||||
|
||||
## Build and Verification Constraints
|
||||
|
||||
- This repository does not use local cross-compilation in the current workflow.
|
||||
- Final build, link, runtime verification, and hardware-dependent testing must be executed on the RK3588 target device.
|
||||
- Do not assume the current workstation can build or run RKNN, RGA, MPP, ZLMediaKit, or full media pipelines.
|
||||
- Do not run full project builds by default on the current workstation unless the task explicitly requires it and the needed dependencies are confirmed to exist locally.
|
||||
- Prefer static inspection and targeted platform-independent tests over speculative local builds.
|
||||
- If a task depends on RK3588-only capabilities, state clearly what was reviewed locally and what still must be verified on the device.
|
||||
- Do not claim that a hardware-related change is verified unless it was actually built and exercised on RK3588.
|
||||
|
||||
## Local Development Expectations
|
||||
|
||||
- Local work should focus on architecture, plugin boundaries, data structures, configuration, documentation, code review, and platform-independent tests.
|
||||
- Local build attempts are allowed only when they are clearly useful and do not depend on missing RK3588 runtime libraries or device-only features.
|
||||
- If local compilation is attempted, keep it scoped to the smallest useful target.
|
||||
- Do not distort production code to satisfy the local machine environment.
|
||||
- Do not remove or weaken RK3588-specific paths just because they are not runnable on the current workstation.
|
||||
|
||||
## Plugin and Architecture Rules
|
||||
|
||||
- New analytics capabilities should be implemented as plugins when they represent pipeline nodes.
|
||||
- Reuse existing shared frame metadata and plugin interfaces where possible.
|
||||
- If a new feature introduces a new semantic output type, define an explicit data model instead of encoding it into unrelated existing fields.
|
||||
- Keep node responsibilities narrow. Detection, tracking, event reasoning, visualization, and alarming should remain separated unless there is a strong architectural reason not to.
|
||||
|
||||
## Testing and Validation Rules
|
||||
|
||||
- Prefer targeted unit tests for platform-independent logic.
|
||||
- For RK3588-only features, document a device-side validation checklist instead of pretending local validation is sufficient.
|
||||
- When a test or build cannot run locally because of platform constraints, say so explicitly in the final report.
|
||||
- Separate validation into two buckets:
|
||||
- local code-level validation
|
||||
- RK3588 device-side validation
|
||||
- Do not merge those two buckets into a single vague statement such as "verified" or "tested" without saying where verification happened.
|
||||
|
||||
## Communication Rules for Agents
|
||||
|
||||
- Say what is known from code inspection versus what still requires RK3588 device verification.
|
||||
- Favor root-cause fixes over environment-specific patches.
|
||||
- Keep recommendations aligned with the shortest path that preserves the architecture.
|
||||
696
docs/superpowers/plans/2026-04-01-region-behavior-recognition.md
Normal file
696
docs/superpowers/plans/2026-04-01-region-behavior-recognition.md
Normal file
@ -0,0 +1,696 @@
|
||||
# Region Behavior Recognition Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add first-release region behavior recognition with unified behavior event data, rule-based region events, temporal-rule action events, event fusion, and downstream OSD/alarm support.
|
||||
|
||||
**Architecture:** Extend `Frame` with a dedicated behavior-event payload, then add three focused plugins: `region_event`, `action_recog`, and `event_fusion`. Keep the first release linear in the graph so each node appends or normalizes behavior events on the same `Frame`, while `osd` and `alarm` consume the normalized result without overloading detection classes.
|
||||
|
||||
**Tech Stack:** C++17, existing plugin `INode` ABI, RK3588 DAG graph manager, GoogleTest, CMake
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
- Modify: `D:\App\C++\Rk3588Sys\include\frame\frame.h`
|
||||
- Add behavior event enums and result structures to `Frame`
|
||||
- Create: `D:\App\C++\Rk3588Sys\include\behavior\behavior_event.h`
|
||||
- Shared behavior-event definitions and helpers if `frame.h` becomes noisy
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
||||
- Register new plugins in the build
|
||||
- Create: `D:\App\C++\Rk3588Sys\plugins\region_event\region_event_node.cpp`
|
||||
- Rule-based intrusion and climb events
|
||||
- Create: `D:\App\C++\Rk3588Sys\plugins\action_recog\action_recog_node.cpp`
|
||||
- Temporal-rule fall and fight events
|
||||
- Create: `D:\App\C++\Rk3588Sys\plugins\event_fusion\event_fusion_node.cpp`
|
||||
- Stable event normalization, ID assignment, and lifecycle progression
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\osd\osd_node.cpp`
|
||||
- Draw behavior labels, track IDs, and durations
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.h`
|
||||
- Add behavior-event-aware alarm rule shape
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.cpp`
|
||||
- Evaluate behavior events in addition to detections
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\alarm_node.cpp`
|
||||
- Parse behavior-event alarm rules and preserve old detection rules
|
||||
- Create: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp`
|
||||
- Validate frame-level behavior event model
|
||||
- Create: `D:\App\C++\Rk3588Sys\tests\test_region_event.cpp`
|
||||
- Validate intrusion and climb rule logic
|
||||
- Create: `D:\App\C++\Rk3588Sys\tests\test_action_recog.cpp`
|
||||
- Validate fall and fight temporal state-machine logic
|
||||
- Create: `D:\App\C++\Rk3588Sys\tests\test_event_fusion.cpp`
|
||||
- Validate event merge and lifecycle behavior
|
||||
- Create: `D:\App\C++\Rk3588Sys\tests\test_alarm_behavior_events.cpp`
|
||||
- Validate alarm matching on behavior events
|
||||
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
||||
- Add new test files to `rk3588_gtests`
|
||||
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_intrusion.json`
|
||||
- Reference graph for intrusion pipeline
|
||||
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_full.json`
|
||||
- Reference graph for intrusion, climb, fall, and fight
|
||||
- Modify: `D:\App\C++\Rk3588Sys\Readme.md`
|
||||
- Document behavior event model and sample pipeline
|
||||
|
||||
### Task 1: Add Behavior Event Data Model
|
||||
|
||||
**Files:**
|
||||
- Create: `D:\App\C++\Rk3588Sys\include\behavior\behavior_event.h`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\include\frame\frame.h`
|
||||
- Test: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```cpp
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
#include "frame/frame.h"
|
||||
|
||||
namespace rk3588 {
|
||||
|
||||
TEST(BehaviorEventModelTest, FrameStoresBehaviorEventsSeparatelyFromDetections) {
|
||||
auto frame = std::make_shared<Frame>();
|
||||
frame->det = std::make_shared<DetectionResult>();
|
||||
frame->behavior_events = std::make_shared<BehaviorEventResult>();
|
||||
|
||||
BehaviorEventItem item;
|
||||
item.event_id = 7;
|
||||
item.type = BehaviorEventType::Fall;
|
||||
item.status = BehaviorEventStatus::Active;
|
||||
item.track_ids = {42};
|
||||
item.duration_ms = 1600;
|
||||
|
||||
frame->behavior_events->items.push_back(item);
|
||||
|
||||
ASSERT_NE(frame->det, nullptr);
|
||||
ASSERT_NE(frame->behavior_events, nullptr);
|
||||
ASSERT_EQ(frame->behavior_events->items.size(), 1u);
|
||||
EXPECT_EQ(frame->behavior_events->items[0].track_ids[0], 42);
|
||||
EXPECT_EQ(frame->behavior_events->items[0].duration_ms, 1600u);
|
||||
}
|
||||
|
||||
} // namespace rk3588
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
||||
|
||||
Expected: compile failure because `BehaviorEventResult`, `BehaviorEventItem`, and related enums do not exist.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```cpp
|
||||
// include/behavior/behavior_event.h
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "frame/frame.h"
|
||||
|
||||
namespace rk3588 {
|
||||
|
||||
enum class BehaviorEventType { Intrusion, Climb, Fall, Fight };
|
||||
enum class BehaviorEventStatus { Pending, Active, Ended };
|
||||
|
||||
struct BehaviorEventItem {
|
||||
int event_id = -1;
|
||||
BehaviorEventType type = BehaviorEventType::Intrusion;
|
||||
BehaviorEventStatus status = BehaviorEventStatus::Pending;
|
||||
float score = 0.0f;
|
||||
Rect bbox{};
|
||||
std::vector<int> track_ids;
|
||||
uint64_t start_pts = 0;
|
||||
uint64_t last_pts = 0;
|
||||
uint64_t duration_ms = 0;
|
||||
std::string source;
|
||||
std::string region_id;
|
||||
};
|
||||
|
||||
struct BehaviorEventResult {
|
||||
std::vector<BehaviorEventItem> items;
|
||||
};
|
||||
|
||||
} // namespace rk3588
|
||||
```
|
||||
|
||||
```cpp
|
||||
// include/frame/frame.h
|
||||
#include "behavior/behavior_event.h"
|
||||
|
||||
struct Frame {
|
||||
// existing fields...
|
||||
std::shared_ptr<BehaviorEventResult> behavior_events;
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
||||
|
||||
Expected: `BehaviorEventModelTest` passes.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add include/behavior/behavior_event.h include/frame/frame.h tests/test_behavior_event_model.cpp tests/CMakeLists.txt
|
||||
git commit -m "feat: add behavior event frame model"
|
||||
```
|
||||
|
||||
### Task 2: Implement `region_event` for Intrusion and Rule-Based Climb
|
||||
|
||||
**Files:**
|
||||
- Create: `D:\App\C++\Rk3588Sys\plugins\region_event\region_event_node.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
||||
- Test: `D:\App\C++\Rk3588Sys\tests\test_region_event.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```cpp
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
#include "frame/frame.h"
|
||||
|
||||
namespace rk3588 {
|
||||
|
||||
class RegionEventNode;
|
||||
|
||||
TEST(RegionEventTest, EmitsIntrusionWhenTrackedPersonStaysInsideRegion) {
|
||||
RegionEventNode node;
|
||||
// Build config with one intrusion ROI and min_duration_ms = 0 for deterministic test.
|
||||
// Feed a frame with one tracked person inside the ROI.
|
||||
// Expect one active intrusion event.
|
||||
}
|
||||
|
||||
TEST(RegionEventTest, EmitsClimbWhenTrackCrossesBoundaryOverTime) {
|
||||
RegionEventNode node;
|
||||
// Build config with one climb boundary.
|
||||
// Feed a short sequence where one track moves across boundary with upward motion.
|
||||
// Expect one climb event with the same track_id.
|
||||
}
|
||||
|
||||
} // namespace rk3588
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R RegionEventTest --output-on-failure`
|
||||
|
||||
Expected: compile failure because `RegionEventNode` and plugin logic do not exist.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```cpp
|
||||
// plugins/region_event/region_event_node.cpp
|
||||
class RegionEventNode final : public INode {
|
||||
public:
|
||||
std::string Id() const override { return id_; }
|
||||
std::string Type() const override { return "region_event"; }
|
||||
|
||||
bool Init(const SimpleJson& config, const NodeContext& ctx) override;
|
||||
bool Start() override { return true; }
|
||||
void Stop() override {}
|
||||
NodeStatus Process(FramePtr frame) override {
|
||||
if (!frame || !frame->det) return NodeStatus::DROP;
|
||||
EnsureBehaviorEvents(*frame);
|
||||
EvaluateIntrusion(*frame);
|
||||
EvaluateClimb(*frame);
|
||||
Push(frame);
|
||||
return NodeStatus::OK;
|
||||
}
|
||||
|
||||
private:
|
||||
void EnsureBehaviorEvents(Frame& frame);
|
||||
void EvaluateIntrusion(Frame& frame);
|
||||
void EvaluateClimb(Frame& frame);
|
||||
void Push(const FramePtr& frame);
|
||||
|
||||
std::string id_;
|
||||
NodeContext ctx_{};
|
||||
};
|
||||
|
||||
REGISTER_NODE(RegionEventNode, "region_event");
|
||||
```
|
||||
|
||||
Implementation constraints:
|
||||
|
||||
- Only inspect detections with valid `track_id`
|
||||
- Append to `frame->behavior_events` without mutating `frame->det`
|
||||
- Store per-track state inside the node for duration and boundary-crossing checks
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R RegionEventTest --output-on-failure`
|
||||
|
||||
Expected: intrusion and climb tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add plugins/region_event/region_event_node.cpp plugins/CMakeLists.txt tests/test_region_event.cpp tests/CMakeLists.txt
|
||||
git commit -m "feat: add region event plugin"
|
||||
```
|
||||
|
||||
### Task 3: Implement `action_recog` for Temporal Fall and Fight
|
||||
|
||||
**Files:**
|
||||
- Create: `D:\App\C++\Rk3588Sys\plugins\action_recog\action_recog_node.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
||||
- Test: `D:\App\C++\Rk3588Sys\tests\test_action_recog.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```cpp
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
namespace rk3588 {
|
||||
|
||||
class ActionRecogNode;
|
||||
|
||||
TEST(ActionRecogTest, EmitsFallAfterRapidDropAndLowPosePersistence) {
|
||||
ActionRecogNode node;
|
||||
// Feed same track across frames with centroid drop and aspect-ratio change.
|
||||
// Expect one active fall event.
|
||||
}
|
||||
|
||||
TEST(ActionRecogTest, EmitsFightWhenTwoTracksStayCloseWithRepeatedMotion) {
|
||||
ActionRecogNode node;
|
||||
// Feed two tracked persons across frames in close proximity with alternating motion.
|
||||
// Expect one active fight event referencing both track ids.
|
||||
}
|
||||
|
||||
} // namespace rk3588
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R ActionRecogTest --output-on-failure`
|
||||
|
||||
Expected: compile failure because `ActionRecogNode` does not exist.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```cpp
|
||||
// plugins/action_recog/action_recog_node.cpp
|
||||
struct TrackSample {
|
||||
uint64_t pts = 0;
|
||||
Rect bbox{};
|
||||
};
|
||||
|
||||
class ActionRecogNode final : public INode {
|
||||
public:
|
||||
std::string Id() const override { return id_; }
|
||||
std::string Type() const override { return "action_recog"; }
|
||||
bool Init(const SimpleJson& config, const NodeContext& ctx) override;
|
||||
bool Start() override { return true; }
|
||||
void Stop() override {}
|
||||
NodeStatus Process(FramePtr frame) override;
|
||||
|
||||
private:
|
||||
void UpdateTrackHistory(const Frame& frame);
|
||||
void EvaluateFall(Frame& frame);
|
||||
void EvaluateFight(Frame& frame);
|
||||
|
||||
std::string id_;
|
||||
NodeContext ctx_{};
|
||||
std::unordered_map<int, std::deque<TrackSample>> history_;
|
||||
};
|
||||
|
||||
REGISTER_NODE(ActionRecogNode, "action_recog");
|
||||
```
|
||||
|
||||
Implementation constraints:
|
||||
|
||||
- Keep first release rule/state-machine only
|
||||
- Do not add RKNN model coupling in this task
|
||||
- Emit `track_ids` with one track for `fall` and two-or-more tracks for `fight`
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R ActionRecogTest --output-on-failure`
|
||||
|
||||
Expected: fall and fight tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add plugins/action_recog/action_recog_node.cpp plugins/CMakeLists.txt tests/test_action_recog.cpp tests/CMakeLists.txt
|
||||
git commit -m "feat: add temporal action recognition plugin"
|
||||
```
|
||||
|
||||
### Task 4: Implement `event_fusion` for Event IDs and Lifecycle Normalization
|
||||
|
||||
**Files:**
|
||||
- Create: `D:\App\C++\Rk3588Sys\plugins\event_fusion\event_fusion_node.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
||||
- Test: `D:\App\C++\Rk3588Sys\tests\test_event_fusion.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```cpp
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
namespace rk3588 {
|
||||
|
||||
class EventFusionNode;
|
||||
|
||||
TEST(EventFusionTest, AssignsStableEventIdAcrossFramesForSameTrackAndType) {
|
||||
EventFusionNode node;
|
||||
// Feed two frames with the same fall event on track 42.
|
||||
// Expect the same event_id in both outputs.
|
||||
}
|
||||
|
||||
TEST(EventFusionTest, MarksEventEndedWhenSourceStopsEmittingIt) {
|
||||
EventFusionNode node;
|
||||
// Feed one frame with an active intrusion event, then one frame without it.
|
||||
// Expect an ended event emission before cleanup.
|
||||
}
|
||||
|
||||
} // namespace rk3588
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R EventFusionTest --output-on-failure`
|
||||
|
||||
Expected: compile failure because `EventFusionNode` does not exist.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```cpp
|
||||
// plugins/event_fusion/event_fusion_node.cpp
|
||||
class EventFusionNode final : public INode {
|
||||
public:
|
||||
std::string Id() const override { return id_; }
|
||||
std::string Type() const override { return "event_fusion"; }
|
||||
bool Init(const SimpleJson& config, const NodeContext& ctx) override;
|
||||
bool Start() override { return true; }
|
||||
void Stop() override {}
|
||||
NodeStatus Process(FramePtr frame) override;
|
||||
|
||||
private:
|
||||
std::string MakeKey(const BehaviorEventItem& item) const;
|
||||
|
||||
std::string id_;
|
||||
NodeContext ctx_{};
|
||||
int next_event_id_ = 1;
|
||||
std::unordered_map<std::string, int> active_ids_;
|
||||
};
|
||||
|
||||
REGISTER_NODE(EventFusionNode, "event_fusion");
|
||||
```
|
||||
|
||||
Normalization rules:
|
||||
|
||||
- key identity by `type + region_id + normalized track_ids`
|
||||
- reuse prior `event_id` for matching active events
|
||||
- emit one final `Ended` item before forgetting an event
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R EventFusionTest --output-on-failure`
|
||||
|
||||
Expected: fusion tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add plugins/event_fusion/event_fusion_node.cpp plugins/CMakeLists.txt tests/test_event_fusion.cpp tests/CMakeLists.txt
|
||||
git commit -m "feat: add behavior event fusion plugin"
|
||||
```
|
||||
|
||||
### Task 5: Update OSD to Render Behavior Events
|
||||
|
||||
**Files:**
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\osd\osd_node.cpp`
|
||||
- Test: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```cpp
|
||||
TEST(BehaviorEventModelTest, BehaviorEventsExposeTypeTrackAndDurationForOsdFormatting) {
|
||||
BehaviorEventItem item;
|
||||
item.type = BehaviorEventType::Intrusion;
|
||||
item.track_ids = {9};
|
||||
item.duration_ms = 2100;
|
||||
|
||||
EXPECT_EQ(FormatBehaviorEventLabel(item), "intrusion #9 2.1s");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
||||
|
||||
Expected: compile failure because behavior-event label formatting helper does not exist.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```cpp
|
||||
// plugins/osd/osd_node.cpp
|
||||
static std::string BehaviorEventTypeName(BehaviorEventType type) {
|
||||
switch (type) {
|
||||
case BehaviorEventType::Intrusion: return "intrusion";
|
||||
case BehaviorEventType::Climb: return "climb";
|
||||
case BehaviorEventType::Fall: return "fall";
|
||||
case BehaviorEventType::Fight: return "fight";
|
||||
}
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
static std::string FormatBehaviorEventLabel(const BehaviorEventItem& item) {
|
||||
std::ostringstream oss;
|
||||
oss << BehaviorEventTypeName(item.type);
|
||||
if (!item.track_ids.empty()) oss << " #" << item.track_ids.front();
|
||||
oss << " " << std::fixed << std::setprecision(1)
|
||||
<< (static_cast<double>(item.duration_ms) / 1000.0) << "s";
|
||||
return oss.str();
|
||||
}
|
||||
```
|
||||
|
||||
Render rules:
|
||||
|
||||
- draw behavior boxes after detection boxes
|
||||
- use distinct color per event type
|
||||
- skip `Pending` by default and render `Active` and `Ended`
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
||||
|
||||
Expected: formatting test passes.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add plugins/osd/osd_node.cpp tests/test_behavior_event_model.cpp
|
||||
git commit -m "feat: render behavior events in osd"
|
||||
```
|
||||
|
||||
### Task 6: Extend Alarm Engine to Consume Behavior Events
|
||||
|
||||
**Files:**
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.h`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\alarm_node.cpp`
|
||||
- Test: `D:\App\C++\Rk3588Sys\tests\test_alarm_behavior_events.cpp`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```cpp
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
namespace rk3588 {
|
||||
|
||||
TEST(AlarmBehaviorEventsTest, MatchesBehaviorEventByTypeAndDuration) {
|
||||
RuleEngine engine;
|
||||
// Init with one behavior-event rule for fall, min_duration_ms = 1000.
|
||||
// Feed frame with one active fall event of 1500 ms.
|
||||
// Expect one rule match.
|
||||
}
|
||||
|
||||
TEST(AlarmBehaviorEventsTest, PreservesLegacyDetectionRules) {
|
||||
RuleEngine engine;
|
||||
// Init with a classic detection rule on class_id 0.
|
||||
// Feed frame->det only.
|
||||
// Expect legacy path still matches.
|
||||
}
|
||||
|
||||
} // namespace rk3588
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R AlarmBehaviorEventsTest --output-on-failure`
|
||||
|
||||
Expected: compile failure because rule engine does not understand behavior-event rules.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```cpp
|
||||
// plugins/alarm/rule_engine.h
|
||||
struct AlarmRule {
|
||||
std::string name;
|
||||
std::set<int> class_ids;
|
||||
std::set<std::string> event_types;
|
||||
std::set<std::string> region_ids;
|
||||
RoiRect roi;
|
||||
float min_score = 0.0f;
|
||||
bool use_behavior_events = false;
|
||||
int min_duration_ms = 0;
|
||||
int cooldown_ms = 5000;
|
||||
};
|
||||
```
|
||||
|
||||
```cpp
|
||||
// plugins/alarm/rule_engine.cpp
|
||||
RuleMatchResult RuleEngine::Evaluate(const std::shared_ptr<Frame>& frame) {
|
||||
if (HasBehaviorRules() && frame && frame->behavior_events) {
|
||||
return EvaluateBehaviorEvents(frame);
|
||||
}
|
||||
return EvaluateDetections(frame);
|
||||
}
|
||||
```
|
||||
|
||||
Compatibility rules:
|
||||
|
||||
- behavior-event rules must not break existing `class_ids` rules
|
||||
- keep old config shape valid
|
||||
- require explicit `use_behavior_events` or `event_types` to enter new path
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R AlarmBehaviorEventsTest --output-on-failure`
|
||||
|
||||
Expected: both behavior-event and legacy detection tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add plugins/alarm/rule_engine.h plugins/alarm/rule_engine.cpp plugins/alarm/alarm_node.cpp tests/test_alarm_behavior_events.cpp tests/CMakeLists.txt
|
||||
git commit -m "feat: support behavior events in alarm engine"
|
||||
```
|
||||
|
||||
### Task 7: Add Sample Configs and Documentation
|
||||
|
||||
**Files:**
|
||||
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_intrusion.json`
|
||||
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_full.json`
|
||||
- Modify: `D:\App\C++\Rk3588Sys\Readme.md`
|
||||
|
||||
- [ ] **Step 1: Write the config files**
|
||||
|
||||
```json
|
||||
{
|
||||
"queue": {"size": 8, "strategy": "drop_oldest"},
|
||||
"graphs": [
|
||||
{
|
||||
"name": "sample_region_behavior_intrusion",
|
||||
"nodes": [
|
||||
{"id": "in", "type": "input_rtsp", "url": "rtsp://127.0.0.1/live"},
|
||||
{"id": "pre", "type": "preprocess", "dst_w": 1920, "dst_h": 1080, "dst_format": "rgb"},
|
||||
{"id": "person_det", "type": "ai_yolo", "model_path": "./models/yolov8n-640.rknn", "model_version": "v8", "model_w": 640, "model_h": 640, "class_filter": [0]},
|
||||
{"id": "trk", "type": "tracker", "track_classes": [0]},
|
||||
{"id": "region_evt", "type": "region_event", "events": [{"type": "intrusion", "region_id": "zone_a", "roi": {"x": 0.1, "y": 0.1, "w": 0.4, "h": 0.4}, "min_duration_ms": 1000}]},
|
||||
{"id": "fusion", "type": "event_fusion"},
|
||||
{"id": "osd", "type": "osd"},
|
||||
{"id": "pub", "type": "publish", "outputs": [{"proto": "rtsp_server", "port": 8555, "path": "/live/behavior"}]},
|
||||
{"id": "alarm", "type": "alarm", "rules": [{"name": "intrusion_rule", "use_behavior_events": true, "event_types": ["intrusion"], "min_duration_ms": 1000, "cooldown_ms": 10000}]}
|
||||
],
|
||||
"edges": [["in", "pre"], ["pre", "person_det"], ["person_det", "trk"], ["trk", "region_evt"], ["region_evt", "fusion"], ["fusion", "osd"], ["osd", "pub"], ["pub", "alarm"]]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Document behavior-event pipeline in README**
|
||||
|
||||
```md
|
||||
## Behavior Events
|
||||
|
||||
Behavior events are emitted through `frame->behavior_events`, not encoded as detection classes.
|
||||
|
||||
First-release nodes:
|
||||
|
||||
- `region_event`: intrusion, rule-based climb
|
||||
- `action_recog`: temporal-rule fall, temporal-rule fight
|
||||
- `event_fusion`: stable event IDs and lifecycle normalization
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Validate config syntax**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests`
|
||||
|
||||
Expected: project still builds after adding documentation and sample configs.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add configs/sample_region_behavior_intrusion.json configs/sample_region_behavior_full.json Readme.md
|
||||
git commit -m "docs: add behavior event sample pipelines"
|
||||
```
|
||||
|
||||
### Task 8: Full Verification Pass
|
||||
|
||||
**Files:**
|
||||
- Modify as needed based on failures from previous tasks
|
||||
|
||||
- [ ] **Step 1: Build the full test target**
|
||||
|
||||
Run: `cmake --build build --target rk3588_gtests`
|
||||
|
||||
Expected: build succeeds.
|
||||
|
||||
- [ ] **Step 2: Run targeted behavior-event tests**
|
||||
|
||||
Run: `ctest --test-dir build -R "BehaviorEventModelTest|RegionEventTest|ActionRecogTest|EventFusionTest|AlarmBehaviorEventsTest" --output-on-failure`
|
||||
|
||||
Expected: all new behavior-related tests pass.
|
||||
|
||||
- [ ] **Step 3: Run existing regression tests**
|
||||
|
||||
Run: `ctest --test-dir build --output-on-failure`
|
||||
|
||||
Expected: no regressions in existing gtests.
|
||||
|
||||
- [ ] **Step 4: Smoke-check plugin build outputs**
|
||||
|
||||
Run: `cmake --build build --target region_event action_recog event_fusion osd alarm`
|
||||
|
||||
Expected: all plugin targets build successfully.
|
||||
|
||||
- [ ] **Step 5: Commit any final fixes**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "test: verify region behavior recognition pipeline"
|
||||
```
|
||||
|
||||
## Self-Review
|
||||
|
||||
Spec coverage check:
|
||||
|
||||
- Unified behavior event data model: covered by Task 1
|
||||
- `region_event`, `action_recog`, `event_fusion`: covered by Tasks 2-4
|
||||
- OSD behavior-event rendering: covered by Task 5
|
||||
- Alarm behavior-event support: covered by Task 6
|
||||
- Sample graph and docs: covered by Task 7
|
||||
- Verification and regression safety: covered by Task 8
|
||||
|
||||
Placeholder scan:
|
||||
|
||||
- No `TBD`, `TODO`, or “implement later” markers remain
|
||||
- Each task lists concrete files and exact commands
|
||||
- Each code task includes a code sketch to anchor implementation
|
||||
|
||||
Type consistency check:
|
||||
|
||||
- Shared naming uses `BehaviorEventType`, `BehaviorEventStatus`, `BehaviorEventItem`, `BehaviorEventResult`
|
||||
- Plugin names align with the approved architecture: `region_event`, `action_recog`, `event_fusion`
|
||||
- Alarm config extension consistently refers to `use_behavior_events`, `event_types`, and `region_ids`
|
||||
Loading…
Reference in New Issue
Block a user