697 lines
23 KiB
Markdown
697 lines
23 KiB
Markdown
# Region Behavior Recognition Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Add first-release region behavior recognition with unified behavior event data, rule-based region events, temporal-rule action events, event fusion, and downstream OSD/alarm support.
|
|
|
|
**Architecture:** Extend `Frame` with a dedicated behavior-event payload, then add three focused plugins: `region_event`, `action_recog`, and `event_fusion`. Keep the first release linear in the graph so each node appends or normalizes behavior events on the same `Frame`, while `osd` and `alarm` consume the normalized result without overloading detection classes.
|
|
|
|
**Tech Stack:** C++17, existing plugin `INode` ABI, RK3588 DAG graph manager, GoogleTest, CMake
|
|
|
|
---
|
|
|
|
## File Map
|
|
|
|
- Modify: `D:\App\C++\Rk3588Sys\include\frame\frame.h`
|
|
- Add behavior event enums and result structures to `Frame`
|
|
- Create: `D:\App\C++\Rk3588Sys\include\behavior\behavior_event.h`
|
|
- Shared behavior-event definitions and helpers if `frame.h` becomes noisy
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
|
- Register new plugins in the build
|
|
- Create: `D:\App\C++\Rk3588Sys\plugins\region_event\region_event_node.cpp`
|
|
- Rule-based intrusion and climb events
|
|
- Create: `D:\App\C++\Rk3588Sys\plugins\action_recog\action_recog_node.cpp`
|
|
- Temporal-rule fall and fight events
|
|
- Create: `D:\App\C++\Rk3588Sys\plugins\event_fusion\event_fusion_node.cpp`
|
|
- Stable event normalization, ID assignment, and lifecycle progression
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\osd\osd_node.cpp`
|
|
- Draw behavior labels, track IDs, and durations
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.h`
|
|
- Add behavior-event-aware alarm rule shape
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.cpp`
|
|
- Evaluate behavior events in addition to detections
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\alarm_node.cpp`
|
|
- Parse behavior-event alarm rules and preserve old detection rules
|
|
- Create: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp`
|
|
- Validate frame-level behavior event model
|
|
- Create: `D:\App\C++\Rk3588Sys\tests\test_region_event.cpp`
|
|
- Validate intrusion and climb rule logic
|
|
- Create: `D:\App\C++\Rk3588Sys\tests\test_action_recog.cpp`
|
|
- Validate fall and fight temporal state-machine logic
|
|
- Create: `D:\App\C++\Rk3588Sys\tests\test_event_fusion.cpp`
|
|
- Validate event merge and lifecycle behavior
|
|
- Create: `D:\App\C++\Rk3588Sys\tests\test_alarm_behavior_events.cpp`
|
|
- Validate alarm matching on behavior events
|
|
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
|
- Add new test files to `rk3588_gtests`
|
|
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_intrusion.json`
|
|
- Reference graph for intrusion pipeline
|
|
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_full.json`
|
|
- Reference graph for intrusion, climb, fall, and fight
|
|
- Modify: `D:\App\C++\Rk3588Sys\Readme.md`
|
|
- Document behavior event model and sample pipeline
|
|
|
|
### Task 1: Add Behavior Event Data Model
|
|
|
|
**Files:**
|
|
- Create: `D:\App\C++\Rk3588Sys\include\behavior\behavior_event.h`
|
|
- Modify: `D:\App\C++\Rk3588Sys\include\frame\frame.h`
|
|
- Test: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
|
|
|
- [ ] **Step 1: Write the failing test**
|
|
|
|
```cpp
|
|
#include <gtest/gtest.h>
|
|
|
|
#include "frame/frame.h"
|
|
|
|
namespace rk3588 {
|
|
|
|
TEST(BehaviorEventModelTest, FrameStoresBehaviorEventsSeparatelyFromDetections) {
|
|
auto frame = std::make_shared<Frame>();
|
|
frame->det = std::make_shared<DetectionResult>();
|
|
frame->behavior_events = std::make_shared<BehaviorEventResult>();
|
|
|
|
BehaviorEventItem item;
|
|
item.event_id = 7;
|
|
item.type = BehaviorEventType::Fall;
|
|
item.status = BehaviorEventStatus::Active;
|
|
item.track_ids = {42};
|
|
item.duration_ms = 1600;
|
|
|
|
frame->behavior_events->items.push_back(item);
|
|
|
|
ASSERT_NE(frame->det, nullptr);
|
|
ASSERT_NE(frame->behavior_events, nullptr);
|
|
ASSERT_EQ(frame->behavior_events->items.size(), 1u);
|
|
EXPECT_EQ(frame->behavior_events->items[0].track_ids[0], 42);
|
|
EXPECT_EQ(frame->behavior_events->items[0].duration_ms, 1600u);
|
|
}
|
|
|
|
} // namespace rk3588
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
|
|
|
Expected: compile failure because `BehaviorEventResult`, `BehaviorEventItem`, and related enums do not exist.
|
|
|
|
- [ ] **Step 3: Write minimal implementation**
|
|
|
|
```cpp
|
|
// include/behavior/behavior_event.h
|
|
#pragma once
|
|
|
|
#include <cstdint>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "frame/frame.h"
|
|
|
|
namespace rk3588 {
|
|
|
|
enum class BehaviorEventType { Intrusion, Climb, Fall, Fight };
|
|
enum class BehaviorEventStatus { Pending, Active, Ended };
|
|
|
|
struct BehaviorEventItem {
|
|
int event_id = -1;
|
|
BehaviorEventType type = BehaviorEventType::Intrusion;
|
|
BehaviorEventStatus status = BehaviorEventStatus::Pending;
|
|
float score = 0.0f;
|
|
Rect bbox{};
|
|
std::vector<int> track_ids;
|
|
uint64_t start_pts = 0;
|
|
uint64_t last_pts = 0;
|
|
uint64_t duration_ms = 0;
|
|
std::string source;
|
|
std::string region_id;
|
|
};
|
|
|
|
struct BehaviorEventResult {
|
|
std::vector<BehaviorEventItem> items;
|
|
};
|
|
|
|
} // namespace rk3588
|
|
```
|
|
|
|
```cpp
|
|
// include/frame/frame.h
|
|
#include "behavior/behavior_event.h"
|
|
|
|
struct Frame {
|
|
// existing fields...
|
|
std::shared_ptr<BehaviorEventResult> behavior_events;
|
|
};
|
|
```
|
|
|
|
- [ ] **Step 4: Run test to verify it passes**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
|
|
|
Expected: `BehaviorEventModelTest` passes.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add include/behavior/behavior_event.h include/frame/frame.h tests/test_behavior_event_model.cpp tests/CMakeLists.txt
|
|
git commit -m "feat: add behavior event frame model"
|
|
```
|
|
|
|
### Task 2: Implement `region_event` for Intrusion and Rule-Based Climb
|
|
|
|
**Files:**
|
|
- Create: `D:\App\C++\Rk3588Sys\plugins\region_event\region_event_node.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
|
- Test: `D:\App\C++\Rk3588Sys\tests\test_region_event.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
```cpp
|
|
#include <gtest/gtest.h>
|
|
|
|
#include "frame/frame.h"
|
|
|
|
namespace rk3588 {
|
|
|
|
class RegionEventNode;
|
|
|
|
TEST(RegionEventTest, EmitsIntrusionWhenTrackedPersonStaysInsideRegion) {
|
|
RegionEventNode node;
|
|
// Build config with one intrusion ROI and min_duration_ms = 0 for deterministic test.
|
|
// Feed a frame with one tracked person inside the ROI.
|
|
// Expect one active intrusion event.
|
|
}
|
|
|
|
TEST(RegionEventTest, EmitsClimbWhenTrackCrossesBoundaryOverTime) {
|
|
RegionEventNode node;
|
|
// Build config with one climb boundary.
|
|
// Feed a short sequence where one track moves across boundary with upward motion.
|
|
// Expect one climb event with the same track_id.
|
|
}
|
|
|
|
} // namespace rk3588
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R RegionEventTest --output-on-failure`
|
|
|
|
Expected: compile failure because `RegionEventNode` and plugin logic do not exist.
|
|
|
|
- [ ] **Step 3: Write minimal implementation**
|
|
|
|
```cpp
|
|
// plugins/region_event/region_event_node.cpp
|
|
class RegionEventNode final : public INode {
|
|
public:
|
|
std::string Id() const override { return id_; }
|
|
std::string Type() const override { return "region_event"; }
|
|
|
|
bool Init(const SimpleJson& config, const NodeContext& ctx) override;
|
|
bool Start() override { return true; }
|
|
void Stop() override {}
|
|
NodeStatus Process(FramePtr frame) override {
|
|
if (!frame || !frame->det) return NodeStatus::DROP;
|
|
EnsureBehaviorEvents(*frame);
|
|
EvaluateIntrusion(*frame);
|
|
EvaluateClimb(*frame);
|
|
Push(frame);
|
|
return NodeStatus::OK;
|
|
}
|
|
|
|
private:
|
|
void EnsureBehaviorEvents(Frame& frame);
|
|
void EvaluateIntrusion(Frame& frame);
|
|
void EvaluateClimb(Frame& frame);
|
|
void Push(const FramePtr& frame);
|
|
|
|
std::string id_;
|
|
NodeContext ctx_{};
|
|
};
|
|
|
|
REGISTER_NODE(RegionEventNode, "region_event");
|
|
```
|
|
|
|
Implementation constraints:
|
|
|
|
- Only inspect detections with valid `track_id`
|
|
- Append to `frame->behavior_events` without mutating `frame->det`
|
|
- Store per-track state inside the node for duration and boundary-crossing checks
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R RegionEventTest --output-on-failure`
|
|
|
|
Expected: intrusion and climb tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add plugins/region_event/region_event_node.cpp plugins/CMakeLists.txt tests/test_region_event.cpp tests/CMakeLists.txt
|
|
git commit -m "feat: add region event plugin"
|
|
```
|
|
|
|
### Task 3: Implement `action_recog` for Temporal Fall and Fight
|
|
|
|
**Files:**
|
|
- Create: `D:\App\C++\Rk3588Sys\plugins\action_recog\action_recog_node.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
|
- Test: `D:\App\C++\Rk3588Sys\tests\test_action_recog.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
```cpp
|
|
#include <gtest/gtest.h>
|
|
|
|
namespace rk3588 {
|
|
|
|
class ActionRecogNode;
|
|
|
|
TEST(ActionRecogTest, EmitsFallAfterRapidDropAndLowPosePersistence) {
|
|
ActionRecogNode node;
|
|
// Feed same track across frames with centroid drop and aspect-ratio change.
|
|
// Expect one active fall event.
|
|
}
|
|
|
|
TEST(ActionRecogTest, EmitsFightWhenTwoTracksStayCloseWithRepeatedMotion) {
|
|
ActionRecogNode node;
|
|
// Feed two tracked persons across frames in close proximity with alternating motion.
|
|
// Expect one active fight event referencing both track ids.
|
|
}
|
|
|
|
} // namespace rk3588
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R ActionRecogTest --output-on-failure`
|
|
|
|
Expected: compile failure because `ActionRecogNode` does not exist.
|
|
|
|
- [ ] **Step 3: Write minimal implementation**
|
|
|
|
```cpp
|
|
// plugins/action_recog/action_recog_node.cpp
|
|
struct TrackSample {
|
|
uint64_t pts = 0;
|
|
Rect bbox{};
|
|
};
|
|
|
|
class ActionRecogNode final : public INode {
|
|
public:
|
|
std::string Id() const override { return id_; }
|
|
std::string Type() const override { return "action_recog"; }
|
|
bool Init(const SimpleJson& config, const NodeContext& ctx) override;
|
|
bool Start() override { return true; }
|
|
void Stop() override {}
|
|
NodeStatus Process(FramePtr frame) override;
|
|
|
|
private:
|
|
void UpdateTrackHistory(const Frame& frame);
|
|
void EvaluateFall(Frame& frame);
|
|
void EvaluateFight(Frame& frame);
|
|
|
|
std::string id_;
|
|
NodeContext ctx_{};
|
|
std::unordered_map<int, std::deque<TrackSample>> history_;
|
|
};
|
|
|
|
REGISTER_NODE(ActionRecogNode, "action_recog");
|
|
```
|
|
|
|
Implementation constraints:
|
|
|
|
- Keep first release rule/state-machine only
|
|
- Do not add RKNN model coupling in this task
|
|
- Emit `track_ids` with one track for `fall` and two-or-more tracks for `fight`
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R ActionRecogTest --output-on-failure`
|
|
|
|
Expected: fall and fight tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add plugins/action_recog/action_recog_node.cpp plugins/CMakeLists.txt tests/test_action_recog.cpp tests/CMakeLists.txt
|
|
git commit -m "feat: add temporal action recognition plugin"
|
|
```
|
|
|
|
### Task 4: Implement `event_fusion` for Event IDs and Lifecycle Normalization
|
|
|
|
**Files:**
|
|
- Create: `D:\App\C++\Rk3588Sys\plugins\event_fusion\event_fusion_node.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\CMakeLists.txt`
|
|
- Test: `D:\App\C++\Rk3588Sys\tests\test_event_fusion.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
```cpp
|
|
#include <gtest/gtest.h>
|
|
|
|
namespace rk3588 {
|
|
|
|
class EventFusionNode;
|
|
|
|
TEST(EventFusionTest, AssignsStableEventIdAcrossFramesForSameTrackAndType) {
|
|
EventFusionNode node;
|
|
// Feed two frames with the same fall event on track 42.
|
|
// Expect the same event_id in both outputs.
|
|
}
|
|
|
|
TEST(EventFusionTest, MarksEventEndedWhenSourceStopsEmittingIt) {
|
|
EventFusionNode node;
|
|
// Feed one frame with an active intrusion event, then one frame without it.
|
|
// Expect an ended event emission before cleanup.
|
|
}
|
|
|
|
} // namespace rk3588
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R EventFusionTest --output-on-failure`
|
|
|
|
Expected: compile failure because `EventFusionNode` does not exist.
|
|
|
|
- [ ] **Step 3: Write minimal implementation**
|
|
|
|
```cpp
|
|
// plugins/event_fusion/event_fusion_node.cpp
|
|
class EventFusionNode final : public INode {
|
|
public:
|
|
std::string Id() const override { return id_; }
|
|
std::string Type() const override { return "event_fusion"; }
|
|
bool Init(const SimpleJson& config, const NodeContext& ctx) override;
|
|
bool Start() override { return true; }
|
|
void Stop() override {}
|
|
NodeStatus Process(FramePtr frame) override;
|
|
|
|
private:
|
|
std::string MakeKey(const BehaviorEventItem& item) const;
|
|
|
|
std::string id_;
|
|
NodeContext ctx_{};
|
|
int next_event_id_ = 1;
|
|
std::unordered_map<std::string, int> active_ids_;
|
|
};
|
|
|
|
REGISTER_NODE(EventFusionNode, "event_fusion");
|
|
```
|
|
|
|
Normalization rules:
|
|
|
|
- key identity by `type + region_id + normalized track_ids`
|
|
- reuse prior `event_id` for matching active events
|
|
- emit one final `Ended` item before forgetting an event
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R EventFusionTest --output-on-failure`
|
|
|
|
Expected: fusion tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add plugins/event_fusion/event_fusion_node.cpp plugins/CMakeLists.txt tests/test_event_fusion.cpp tests/CMakeLists.txt
|
|
git commit -m "feat: add behavior event fusion plugin"
|
|
```
|
|
|
|
### Task 5: Update OSD to Render Behavior Events
|
|
|
|
**Files:**
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\osd\osd_node.cpp`
|
|
- Test: `D:\App\C++\Rk3588Sys\tests\test_behavior_event_model.cpp`
|
|
|
|
- [ ] **Step 1: Write the failing test**
|
|
|
|
```cpp
|
|
TEST(BehaviorEventModelTest, BehaviorEventsExposeTypeTrackAndDurationForOsdFormatting) {
|
|
BehaviorEventItem item;
|
|
item.type = BehaviorEventType::Intrusion;
|
|
item.track_ids = {9};
|
|
item.duration_ms = 2100;
|
|
|
|
EXPECT_EQ(FormatBehaviorEventLabel(item), "intrusion #9 2.1s");
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
|
|
|
Expected: compile failure because behavior-event label formatting helper does not exist.
|
|
|
|
- [ ] **Step 3: Write minimal implementation**
|
|
|
|
```cpp
|
|
// plugins/osd/osd_node.cpp
|
|
static std::string BehaviorEventTypeName(BehaviorEventType type) {
|
|
switch (type) {
|
|
case BehaviorEventType::Intrusion: return "intrusion";
|
|
case BehaviorEventType::Climb: return "climb";
|
|
case BehaviorEventType::Fall: return "fall";
|
|
case BehaviorEventType::Fight: return "fight";
|
|
}
|
|
return "unknown";
|
|
}
|
|
|
|
static std::string FormatBehaviorEventLabel(const BehaviorEventItem& item) {
|
|
std::ostringstream oss;
|
|
oss << BehaviorEventTypeName(item.type);
|
|
if (!item.track_ids.empty()) oss << " #" << item.track_ids.front();
|
|
oss << " " << std::fixed << std::setprecision(1)
|
|
<< (static_cast<double>(item.duration_ms) / 1000.0) << "s";
|
|
return oss.str();
|
|
}
|
|
```
|
|
|
|
Render rules:
|
|
|
|
- draw behavior boxes after detection boxes
|
|
- use distinct color per event type
|
|
- skip `Pending` by default and render `Active` and `Ended`
|
|
|
|
- [ ] **Step 4: Run test to verify it passes**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R BehaviorEventModelTest --output-on-failure`
|
|
|
|
Expected: formatting test passes.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add plugins/osd/osd_node.cpp tests/test_behavior_event_model.cpp
|
|
git commit -m "feat: render behavior events in osd"
|
|
```
|
|
|
|
### Task 6: Extend Alarm Engine to Consume Behavior Events
|
|
|
|
**Files:**
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.h`
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\rule_engine.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\plugins\alarm\alarm_node.cpp`
|
|
- Test: `D:\App\C++\Rk3588Sys\tests\test_alarm_behavior_events.cpp`
|
|
- Modify: `D:\App\C++\Rk3588Sys\tests\CMakeLists.txt`
|
|
|
|
- [ ] **Step 1: Write the failing tests**
|
|
|
|
```cpp
|
|
#include <gtest/gtest.h>
|
|
|
|
namespace rk3588 {
|
|
|
|
TEST(AlarmBehaviorEventsTest, MatchesBehaviorEventByTypeAndDuration) {
|
|
RuleEngine engine;
|
|
// Init with one behavior-event rule for fall, min_duration_ms = 1000.
|
|
// Feed frame with one active fall event of 1500 ms.
|
|
// Expect one rule match.
|
|
}
|
|
|
|
TEST(AlarmBehaviorEventsTest, PreservesLegacyDetectionRules) {
|
|
RuleEngine engine;
|
|
// Init with a classic detection rule on class_id 0.
|
|
// Feed frame->det only.
|
|
// Expect legacy path still matches.
|
|
}
|
|
|
|
} // namespace rk3588
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R AlarmBehaviorEventsTest --output-on-failure`
|
|
|
|
Expected: compile failure because rule engine does not understand behavior-event rules.
|
|
|
|
- [ ] **Step 3: Write minimal implementation**
|
|
|
|
```cpp
|
|
// plugins/alarm/rule_engine.h
|
|
struct AlarmRule {
|
|
std::string name;
|
|
std::set<int> class_ids;
|
|
std::set<std::string> event_types;
|
|
std::set<std::string> region_ids;
|
|
RoiRect roi;
|
|
float min_score = 0.0f;
|
|
bool use_behavior_events = false;
|
|
int min_duration_ms = 0;
|
|
int cooldown_ms = 5000;
|
|
};
|
|
```
|
|
|
|
```cpp
|
|
// plugins/alarm/rule_engine.cpp
|
|
RuleMatchResult RuleEngine::Evaluate(const std::shared_ptr<Frame>& frame) {
|
|
if (HasBehaviorRules() && frame && frame->behavior_events) {
|
|
return EvaluateBehaviorEvents(frame);
|
|
}
|
|
return EvaluateDetections(frame);
|
|
}
|
|
```
|
|
|
|
Compatibility rules:
|
|
|
|
- behavior-event rules must not break existing `class_ids` rules
|
|
- keep old config shape valid
|
|
- require explicit `use_behavior_events` or `event_types` to enter new path
|
|
|
|
- [ ] **Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests && ctest --test-dir build -R AlarmBehaviorEventsTest --output-on-failure`
|
|
|
|
Expected: both behavior-event and legacy detection tests pass.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add plugins/alarm/rule_engine.h plugins/alarm/rule_engine.cpp plugins/alarm/alarm_node.cpp tests/test_alarm_behavior_events.cpp tests/CMakeLists.txt
|
|
git commit -m "feat: support behavior events in alarm engine"
|
|
```
|
|
|
|
### Task 7: Add Sample Configs and Documentation
|
|
|
|
**Files:**
|
|
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_intrusion.json`
|
|
- Create: `D:\App\C++\Rk3588Sys\configs\sample_region_behavior_full.json`
|
|
- Modify: `D:\App\C++\Rk3588Sys\Readme.md`
|
|
|
|
- [ ] **Step 1: Write the config files**
|
|
|
|
```json
|
|
{
|
|
"queue": {"size": 8, "strategy": "drop_oldest"},
|
|
"graphs": [
|
|
{
|
|
"name": "sample_region_behavior_intrusion",
|
|
"nodes": [
|
|
{"id": "in", "type": "input_rtsp", "url": "rtsp://127.0.0.1/live"},
|
|
{"id": "pre", "type": "preprocess", "dst_w": 1920, "dst_h": 1080, "dst_format": "rgb"},
|
|
{"id": "person_det", "type": "ai_yolo", "model_path": "./models/yolov8n-640.rknn", "model_version": "v8", "model_w": 640, "model_h": 640, "class_filter": [0]},
|
|
{"id": "trk", "type": "tracker", "track_classes": [0]},
|
|
{"id": "region_evt", "type": "region_event", "events": [{"type": "intrusion", "region_id": "zone_a", "roi": {"x": 0.1, "y": 0.1, "w": 0.4, "h": 0.4}, "min_duration_ms": 1000}]},
|
|
{"id": "fusion", "type": "event_fusion"},
|
|
{"id": "osd", "type": "osd"},
|
|
{"id": "pub", "type": "publish", "outputs": [{"proto": "rtsp_server", "port": 8555, "path": "/live/behavior"}]},
|
|
{"id": "alarm", "type": "alarm", "rules": [{"name": "intrusion_rule", "use_behavior_events": true, "event_types": ["intrusion"], "min_duration_ms": 1000, "cooldown_ms": 10000}]}
|
|
],
|
|
"edges": [["in", "pre"], ["pre", "person_det"], ["person_det", "trk"], ["trk", "region_evt"], ["region_evt", "fusion"], ["fusion", "osd"], ["osd", "pub"], ["pub", "alarm"]]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Document behavior-event pipeline in README**
|
|
|
|
```md
|
|
## Behavior Events
|
|
|
|
Behavior events are emitted through `frame->behavior_events`, not encoded as detection classes.
|
|
|
|
First-release nodes:
|
|
|
|
- `region_event`: intrusion, rule-based climb
|
|
- `action_recog`: temporal-rule fall, temporal-rule fight
|
|
- `event_fusion`: stable event IDs and lifecycle normalization
|
|
```
|
|
|
|
- [ ] **Step 3: Validate config syntax**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests`
|
|
|
|
Expected: project still builds after adding documentation and sample configs.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add configs/sample_region_behavior_intrusion.json configs/sample_region_behavior_full.json Readme.md
|
|
git commit -m "docs: add behavior event sample pipelines"
|
|
```
|
|
|
|
### Task 8: Full Verification Pass
|
|
|
|
**Files:**
|
|
- Modify as needed based on failures from previous tasks
|
|
|
|
- [ ] **Step 1: Build the full test target**
|
|
|
|
Run: `cmake --build build --target rk3588_gtests`
|
|
|
|
Expected: build succeeds.
|
|
|
|
- [ ] **Step 2: Run targeted behavior-event tests**
|
|
|
|
Run: `ctest --test-dir build -R "BehaviorEventModelTest|RegionEventTest|ActionRecogTest|EventFusionTest|AlarmBehaviorEventsTest" --output-on-failure`
|
|
|
|
Expected: all new behavior-related tests pass.
|
|
|
|
- [ ] **Step 3: Run existing regression tests**
|
|
|
|
Run: `ctest --test-dir build --output-on-failure`
|
|
|
|
Expected: no regressions in existing gtests.
|
|
|
|
- [ ] **Step 4: Smoke-check plugin build outputs**
|
|
|
|
Run: `cmake --build build --target region_event action_recog event_fusion osd alarm`
|
|
|
|
Expected: all plugin targets build successfully.
|
|
|
|
- [ ] **Step 5: Commit any final fixes**
|
|
|
|
```bash
|
|
git add -A
|
|
git commit -m "test: verify region behavior recognition pipeline"
|
|
```
|
|
|
|
## Self-Review
|
|
|
|
Spec coverage check:
|
|
|
|
- Unified behavior event data model: covered by Task 1
|
|
- `region_event`, `action_recog`, `event_fusion`: covered by Tasks 2-4
|
|
- OSD behavior-event rendering: covered by Task 5
|
|
- Alarm behavior-event support: covered by Task 6
|
|
- Sample graph and docs: covered by Task 7
|
|
- Verification and regression safety: covered by Task 8
|
|
|
|
Placeholder scan:
|
|
|
|
- No `TBD`, `TODO`, or “implement later” markers remain
|
|
- Each task lists concrete files and exact commands
|
|
- Each code task includes a code sketch to anchor implementation
|
|
|
|
Type consistency check:
|
|
|
|
- Shared naming uses `BehaviorEventType`, `BehaviorEventStatus`, `BehaviorEventItem`, `BehaviorEventResult`
|
|
- Plugin names align with the approved architecture: `region_event`, `action_recog`, `event_fusion`
|
|
- Alarm config extension consistently refers to `use_behavior_events`, `event_types`, and `region_ids`
|