safesight-edge/docs/design/FaceRecognition_DistanceBased_Design_v2.md

1245 lines
39 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# 车间人脸识别系统 - 三分区距离检测方案 v2.3
> **文档版本**: v2.3
> **适用场景**: 4-6米安装高度3-9米检测距离RetinaFace_320 + 三分区
> **目标平台**: RK3588 (6TOPS NPU)
> **检测模型**: RetinaFace-MobileNetV3 320×320先用现有模型跑通
> **姿态估计**: 5点关键点近似
> **更新日期**: 2026-03-10
**注意**: v2.3使用现有`RetinaFace_mobile320.rknn`如需更高精度可升级到640模型
---
## 1. 方案概述
### 1.1 设计目标
针对车间环境远距离人脸识别需求,本方案在现有插件化架构基础上,引入**距离分区检测**机制:
- **检测范围**: 3~8米放弃10米极端距离保证检测可靠性
- **核心策略**: 以6米相机对焦距离为中心将画面分为远近两个检测区
- **算力优化**: ROI裁剪 + 自适应缩放节省约60%算力
- **姿态过滤**: 利用现有5点关键点过滤过度低头情况
### 1.2 核心特性
| 特性 | 实现方式 | 预期收益 |
|------|----------|----------|
| **RetinaFace_320** | 320×320输入现有模型 | 快速部署5-7米检测率>90% |
| ROI裁剪 | 基于3-9米距离范围裁剪画面 | 节省40-50%算力 |
| **三分区检测** | 近区1.0x中区1.3x远区1.8x | 目标人脸20-40px320最佳范围 |
| 距离过滤 | 像素y坐标→距离映射过滤范围外人脸 | 减少误检 |
| 姿态过滤 | **5点关键点**估计俯仰角,过滤<-15° | 提升识别准确率无需额外模型 |
| 独立标定 | 每相机独立Python标定脚本 | 适配不同安装高度/角度 |
| **可升级** | 320640模型 | 预留接口后续无缝升级 |
### 1.3 非目标(明确排除)
为控制复杂度以下功能不在本版本范围内
- **9米以上超远距离**320模型在此距离检测率<50%
- **PFLD独立姿态估计模型**使用5点关键点近似已足够
- RetinaFace_640模型v2.4版本升级当前先用320跑通
- Batch推理优化
- 自动在线标定
- 畸变校正假设畸变较小或可忽略
---
## 2. 系统架构
### 2.1 数据流架构
```
┌─────────────────────────────────────────────────────────────┐
│ 每路相机独立配置configs/zone_a/cam_001.json
├─────────────────────────────────────────────────────────────┤
│ │
│ [input_rtsp] ──→ [preprocess] ──→ [ai_face_det] ──→ [ai_face_recog] │
│ │ │ │
│ ROI裁剪(节省算力) 双分区检测 │
│ 距离估算 │
│ ↓ │
│ [osd]/[publish] │
│ │
└─────────────────────────────────────────────────────────────┘
```
### 2.2 三分区检测原理
```
画面垂直方向y坐标2560×1440示例:
┌─────────────────────────────┐ y=0 (画面顶部)
│ │
│ 3-5米 (近区) │ ← scale=1.0x
│ 人脸88-117px │ 原图处理
│ 320输入: 27-36px │ 320模型最佳范围
│ ↑ │
│ y = boundary_5m │ 5米分界线
│ ↓ │
│ 5-7米 (中区) │ ← scale=1.3x
│ 人脸63-88px │ 放大后处理
│ 320输入: 25-35px │ (对焦最佳)
│ ↑ │
│ y = boundary_7m │ 7米分界线
│ ↓ │
│ 7-9米 (远区) │ ← scale=1.8x
│ 人脸49-63px │ 大幅放大补偿
│ 320输入: 27-35px │ 9米可能降至60%检测率)
│ │
└─────────────────────────────┘ y=1440 (画面底部)
ROI裁剪边界 (y_min ~ y_max)
只保留3-9米对应画面区域
目标: 三个区处理后的人脸在320输入中占25-35px320模型最佳
```
### 2.3 距离-像素映射
基于针孔相机模型预计算LUT表实现O(1)查询
```
距离 D → 像素 y:
y = cy + f × tan(arctan(H/D) - θ)
像素 y → 距离 D:
D = H / tan(θ + arctan((y-cy)/f))
```
其中
- H: 安装高度
- θ: 俯仰角弧度
- f: 像素焦距像素
- cy: 主点y坐标通常img_h/2
---
## 3. 相机模型设计
### 3.1 简化相机模型类
```cpp
// include/utils/camera_model.h
#pragma once
#include <vector>
namespace rk3588 {
struct CameraModelParams {
float height; // 安装高度(米)
float pitch_deg; // 俯仰角(度)
float focal_px; // 像素焦距(像素)
int img_w, img_h; // 图像尺寸
int cx, cy; // 主点坐标
float min_dist = 3.0f; // 最小检测距离
float max_dist = 8.0f; // 最大检测距离
};
class SimpleCameraModel {
public:
explicit SimpleCameraModel(const CameraModelParams& p);
// O(1)距离查询(核心功能)
float PixelToDistance(int y) const;
// 计算ROI裁剪区域
struct ROI { int x, y, w, h; float saving_ratio; };
ROI ComputeRoi() const;
// 计算分区线像素位置
int GetZoneSplitY(float split_distance = 6.0f) const;
// 估算人脸像素大小(用于验证)
float EstimateFacePixelSize(float distance,
float real_face_width = 0.16f) const;
private:
CameraModelParams p_;
std::vector<float> distance_lut_; // 预计算查找表
void BuildLut();
};
} // namespace rk3588
```
### 3.2 实现要点
- **LUT预计算**: 初始化时计算整张图的距离表img_h个float约5-10KB
- **无三角函数运行时计算**: 查询时直接查表
- **边界处理**: y越界时返回inf由调用方处理
---
## 4. 分区检测策略
### 4.1 检测节点扩展RetinaFace_320
`ai_face_det` 节点配置更新
```json
{
"id": "face_det",
"type": "ai_face_det",
"model_path": "./models/RetinaFace_mobile320.rknn",
"conf": 0.6,
"nms": 0.4,
"max_faces": 10,
"output_landmarks": true,
"input_format": "rgb",
"model_w": 320, // 320×320输入
"model_h": 320,
"distance_zones": {
"enabled": true,
"boundaries": [416, 672], // 5米和7米分界线y坐标
"scales": [1.0, 1.3, 1.8] // 320模型放大为主目标25-35px
}
}
```
**模型现状**
- 已有`RetinaFace_mobile320.rknn`1.6MB
- 输入320×3205点关键点输出
- 限制8-9米检测率可能60-70%可先跑通功能
- 🔄 升级v2.4无缝切换到640模型只需改model_path和scales
### 4.2 检测流程(三分区)
```cpp
void DetectWithZones(FramePtr frame) {
if (!zone_cfg_.enabled) {
RunSingleScale(frame); // 回退到原有逻辑
return;
}
const int h = frame->height;
const int y_5m = zone_cfg_.boundaries[0]; // 5米分界线
const int y_7m = zone_cfg_.boundaries[1]; // 7米分界线
Detections all_dets;
// 近区检测 (3-5米画面上部) - 320模型1.0x
if (y_5m > 0) {
Mat roi = Crop(frame, 0, 0, w, y_5m);
Mat scaled;
resize(roi, scaled, Size(roi.w*1.0, roi.h*1.0)); // 原图处理
auto dets = RunInference(scaled);
MapBack(dets, scale=1.0, offset_y=0);
all_dets.insert(all_dets.end(), dets.begin(), dets.end());
}
// 中区检测 (5-7米画面中部) - 320模型1.3x放大
if (y_7m > y_5m) {
Mat roi = Crop(frame, 0, y_5m, w, y_7m - y_5m);
Mat scaled;
resize(roi, scaled, Size(roi.w*1.3, roi.h*1.3)); // 放大补偿
auto dets = RunInference(scaled);
MapBack(dets, scale=1.3, offset_y=y_5m);
all_dets.insert(all_dets.end(), dets.begin(), dets.end());
}
// 远区检测 (7-9米画面下部) - 320模型1.8x大幅放大
if (y_7m < h) {
Mat roi = Crop(frame, 0, y_7m, w, h - y_7m);
Mat scaled;
resize(roi, scaled, Size(roi.w*1.8, roi.h*1.8)); // 大幅放大
auto dets = RunInference(scaled);
MapBack(dets, scale=1.8, offset_y=y_7m);
all_dets.insert(all_dets.end(), dets.begin(), dets.end());
}
// NMS去重三区可能有重叠
Nms(all_dets);
frame->face_det = make_shared<FaceDetResult>(all_dets);
}
```
### 4.3 缩放因子选择320模型
| 分区 | 距离 | 原始人脸 | 缩放 | 320输入 | 占比 | 检测率预期 |
|------|------|----------|------|---------|------|-----------|
| 近区 | 3-5m | 88-117px | 1.0x | 88-117px320 | 27-37% | >90% |
| 中区 | 5-7m | 63-88px | 1.3x | 82-114px→320 | 26-36% | >90% |
| 远区 | 7-9m | 49-63px | 1.8x | 88-113px→320 | 28-35% | 70-80% |
> **注**:
> - RetinaFace_320最佳检测范围25-45px占320输入的8-14%
> - 实际处理后27-37px在最佳范围内
> - 9米处检测率可能降至70%如不满足可升级到640模型
> - **升级到640只需改**model_path + scales改为[0.7, 1.0, 1.4]
---
## 5. 距离与姿态过滤
### 5.1 识别节点扩展5点关键点姿态估计
```json
{
"id": "face_recog",
"type": "ai_face_recog",
"model_path": "./models/mobilefacenet_arcface.rknn",
"align": true,
"gallery": {
"backend": "sqlite",
"path": "./models/face_gallery.db"
},
"filters": {
"distance": {
"enabled": true,
"min": 3.0,
"max": 9.0
},
"pose": {
"enabled": true,
"min_pitch": -15, // 真实仰角低于-15度不识别
"camera_pitch": 45, // 相机俯仰角(标定参数)
"use_landmarks": true // 使用RetinaFace输出的5点关键点
}
}
}
```
### 5.2 过滤逻辑
```cpp
void ApplyFilters(FaceRecogItem& item, const FaceDetItem& det) {
// 1. 距离过滤
if (filters_.distance.enabled && camera_model_) {
int center_y = det.bbox.y + det.bbox.h / 2;
float dist = camera_model_->PixelToDistance(center_y);
if (dist < filters_.distance.min || dist > filters_.distance.max) {
item.unknown = true;
item.best_name = "out_of_range";
return;
}
item.distance = dist; // 记录距离供后续使用
}
// 2. 姿态过滤5点关键点近似
if (filters_.pose.enabled && det.has_landmarks) {
float face_pitch = EstimatePitch(det.landmarks);
float real_pitch = face_pitch - filters_.pose.camera_pitch;
if (real_pitch < filters_.pose.min_pitch) {
item.unknown = true;
item.best_name = "low_head";
return;
}
}
}
// 俯仰角估计:鼻尖相对于眼睛中心的垂直偏移
float EstimatePitch(const array<Point2f, 5>& lm) {
float eye_y = (lm[0].y + lm[1].y) / 2; // 左右眼中心
float nose_y = lm[2].y; // 鼻尖
float eye_dist = abs(lm[1].x - lm[0].x);
if (eye_dist < 1.0f) return 0.0f;
float dy = nose_y - eye_y;
return atan2(dy, eye_dist) * 180.0f / M_PI;
}
```
---
## 6. 标定工具设计
### 6.1 工具定位
- **独立运行**: 不耦合主系统,手工执行
- **输出参考**: 生成推荐配置值,供手工复制到配置文件
- **批量支持**: 可选批量生成多路相机配置
### 6.2 使用方法
```bash
# 单相机标定(三分区)
python tools/calibrate_camera.py \
--height 5.0 \
--pitch 45 \
--focal-estimate 2200 \
--image-size 2560 1440 \
--range 3.0 9.0 \
--zones 3 5 7 9 \
--scales 0.7 1.0 1.4 \
--output configs/calibrations/cam_zone_a_001.json \
--report
# 输出示例
# [INFO] ROI: y=240, h=880 (saving: 38.9%)
# [INFO] Zone boundaries: y=416 (5m), y=672 (7m)
# [INFO] 近区(3-5m): scale=0.7x
# [INFO] 中区(5-7m): scale=1.0x
# [INFO] 远区(7-9m): scale=1.4x
# [INFO] Config saved to cam_zone_a_001.json
```
### 6.3 输出配置片段(三分区)
工具输出JSON格式配置可直接复制到主配置
```json
{
"preprocess": {
"roi": {
"enabled": true,
"crop": {"x": 0, "y": 240, "w": 2560, "h": 880}
}
},
"face_det": {
"distance_zones": {
"enabled": true,
"boundaries": [416, 672],
"zones": [
{
"name": "near",
"distance_range": [3, 5],
"scale": 0.7,
"y_range": [0, 416]
},
{
"name": "mid",
"distance_range": [5, 7],
"scale": 1.0,
"y_range": [416, 672]
},
{
"name": "far",
"distance_range": [7, 9],
"scale": 1.4,
"y_range": [672, 1440]
}
]
}
},
"face_recog": {
"filters": {
"distance": {"enabled": true, "min": 3.0, "max": 9.0},
"pose": {"enabled": true, "min_pitch": -15, "camera_pitch": 45}
}
},
"calibration_params": {
"height": 5.0,
"pitch": 45.0,
"focal_px": 2200,
"zones": "3-5m(0.7x), 5-7m(1.0x), 7-9m(1.4x)"
}
}
```
### 6.4 焦距估算方法
若无法精确测量焦距,可用以下方法估算:
```
方法1: 公式估算
f ≈ (sensor_width_mm / image_width_px) * focal_length_mm
例如: 1/2.8"传感器(5.6mm宽), 4mm镜头, 2560px
f ≈ (5.6 / 2560) * 4 ≈ 2200 px
方法2: 现场测量反推
在已知距离D处测量人脸像素宽度W
f = W * D / 0.16
例如: 6米处人脸100px
f = 100 * 6 / 0.16 = 3750 px
```
> **建议**: 先用方法1估算再用方法2验证偏差较大时以方法2为准。
---
## 7. 配置文件集成方案
### 7.1 配置结构说明
项目使用**统一配置文件**,支持两种模式:
1. `templates` + `instances` 模式:模板定义流水线,实例传入参数
2. `graphs` 直接模式:直接定义完整的节点和边
### 7.2 Templates + Instances 模式(推荐用于多相机)
```json
{
"global": {
"metrics_port": 9000,
"web_root": "web"
},
"queue": { "size": 8, "strategy": "drop_oldest" },
"templates": {
"face_recog_distanced_pipeline": {
"nodes": [
{
"id": "in",
"type": "input_rtsp",
"role": "source",
"enable": true,
"url": "${url}",
"fps": 30,
"width": 2560,
"height": 1440
},
{
"id": "pre",
"type": "preprocess",
"role": "filter",
"enable": true,
// ROI裁剪参数由标定工具生成通过params传入
"roi": {
"enabled": "${roi_enabled}",
"crop": {
"x": 0,
"y": "${roi_y}",
"w": 2560,
"h": "${roi_h}"
}
},
"dst_w": 1280,
"dst_h": 720,
"dst_format": "rgb"
},
{
"id": "face_det",
"type": "ai_face_det",
"role": "filter",
"enable": true,
"model_path": "./models/RetinaFace_mobile320.rknn",
"conf": 0.6,
"nms": 0.4,
"max_faces": 10,
"output_landmarks": true,
// 三分区检测参数通过params传入
"distance_zones": {
"enabled": "${zones_enabled}",
"boundaries": ["${zone_boundary_5m}", "${zone_boundary_7m}"],
"scales": ["${zone_scale_near}", "${zone_scale_mid}", "${zone_scale_far}"]
}
},
{
"id": "face_recog",
"type": "ai_face_recog",
"role": "filter",
"enable": true,
"model_path": "./models/mobilefacenet_arcface.rknn",
"align": true,
"gallery": {
"backend": "sqlite",
"path": "${face_gallery_path}"
},
// 距离和姿态过滤参数
"filters": {
"distance": {
"enabled": true,
"min": 3.0,
"max": 9.0
},
"pose": {
"enabled": true,
"min_pitch": -15,
"camera_pitch": "${camera_pitch}"
}
}
},
{
"id": "osd",
"type": "osd",
"role": "filter",
"enable": true,
"draw_face_det": true,
"draw_face_recog": true
},
{
"id": "pub",
"type": "publish",
"role": "sink",
"enable": true,
"outputs": [{ "proto": "rtsp_server", "port": "${rtsp_port}", "path": "/live/${name}" }]
}
],
"edges": [
["in", "pre"],
["pre", "face_det"],
["face_det", "face_recog"],
["face_recog", "osd"],
["osd", "pub"]
]
}
},
// 30+路相机实例,每路传入不同的标定参数
"instances": [
{
"name": "workshop_zoneA_cam01",
"template": "face_recog_distanced_pipeline",
"params": {
"name": "zoneA_cam01",
"url": "rtsp://192.168.1.101/stream1",
"rtsp_port": 8554,
"face_gallery_path": "./models/face_gallery.db",
// 相机安装参数
"camera_pitch": 45,
// ROI参数标定工具生成
"roi_enabled": true,
"roi_y": 240,
"roi_h": 880,
// 三分区参数(标定工具生成)
"zones_enabled": true,
"zone_boundary_5m": 416,
"zone_boundary_7m": 672,
"zone_scale_near": 0.7,
"zone_scale_mid": 1.0,
"zone_scale_far": 1.4
}
},
{
"name": "workshop_zoneA_cam02",
"template": "face_recog_distanced_pipeline",
"params": {
"name": "zoneA_cam02",
"url": "rtsp://192.168.1.102/stream1",
"rtsp_port": 8555,
"face_gallery_path": "./models/face_gallery.db",
// 不同安装高度和角度,不同标定参数
"camera_pitch": 42,
"roi_enabled": true,
"roi_y": 280,
"roi_h": 820,
"zones_enabled": true,
"zone_boundary_5m": 432,
"zone_boundary_7m": 688,
"zone_scale_near": 0.7,
"zone_scale_mid": 1.0,
"zone_scale_far": 1.4
}
}
// ... 更多相机实例
]
}
```
### 7.3 Graphs 直接模式(单相机测试)
```json
{
"queue": { "size": 8, "strategy": "drop_oldest" },
"graphs": [
{
"name": "cam1_face_recog_distanced",
"nodes": [
{
"id": "in_cam1",
"type": "input_rtsp",
"role": "source",
"enable": true,
"url": "rtsp://192.168.1.101/stream1",
"width": 2560,
"height": 1440
},
{
"id": "pre_cam1",
"type": "preprocess",
"role": "filter",
"enable": true,
"roi": {
"enabled": true,
"crop": { "x": 0, "y": 240, "w": 2560, "h": 880 }
},
"dst_w": 1280,
"dst_h": 720,
"dst_format": "rgb"
},
{
"id": "face_det_cam1",
"type": "ai_face_det",
"role": "filter",
"enable": true,
"model_path": "./models/RetinaFace_mobile320.rknn",
"conf": 0.6,
"nms": 0.4,
"max_faces": 10,
"output_landmarks": true,
"distance_zones": {
"enabled": true,
"boundaries": [416, 672],
"scales": [0.7, 1.0, 1.4]
}
},
{
"id": "face_recog_cam1",
"type": "ai_face_recog",
"role": "filter",
"enable": true,
"model_path": "./models/mobilefacenet_arcface.rknn",
"align": true,
"gallery": {
"backend": "sqlite",
"path": "./models/face_gallery.db"
},
"filters": {
"distance": { "enabled": true, "min": 3.0, "max": 9.0 },
"pose": { "enabled": true, "min_pitch": -15, "camera_pitch": 45 }
}
},
{
"id": "osd_cam1",
"type": "osd",
"role": "filter",
"enable": true,
"draw_face_det": true,
"draw_face_recog": true
},
{
"id": "pub_cam1",
"type": "publish",
"role": "sink",
"enable": true,
"outputs": [
{ "proto": "rtsp_server", "port": 8554, "path": "/live/cam1" }
]
}
],
"edges": [
["in_cam1", "pre_cam1"],
["pre_cam1", "face_det_cam1"],
["face_det_cam1", "face_recog_cam1"],
["face_recog_cam1", "osd_cam1"],
["osd_cam1", "pub_cam1"]
]
}
]
}
```
---
## 8. 部署与验证流程
### 8.1 部署流程(统一配置文件)
```
Step 1: 准备统一配置文件configs/workshop_face_recog.json
├─ 定义 template "face_recog_distanced_pipeline"
└─ 预留 ${params} 占位符
Step 2: 为每路相机运行标定工具
├─ python tools/calibrate_camera.py --height 5.0 --pitch 45 ...
└─ 记录输出的 params 参数
Step 3: 在统一配置文件的 instances 中添加相机
{
"name": "workshop_zoneA_cam01",
"template": "face_recog_distanced_pipeline",
"params": {
"name": "zoneA_cam01",
"url": "rtsp://...",
// 复制标定工具输出的 params 到这里
"camera_pitch": 45,
"roi_y": 240,
"roi_h": 880,
"zone_boundary_5m": 416,
...
}
}
Step 4: 现场验证
├─ 在3米、5米、7米、9米处站立测试人员
├─ 检查检测框是否正常
├─ 检查距离估算是否准确
└─ 如有偏差,调整该相机的 focal_px 重新标定
Step 5: 批量部署
└─ 重复Step 2-4为所有30+路相机添加 instances
```
### 8.2 验证检查清单
| 检查项 | 方法 | 通过标准 |
|--------|------|----------|
| ROI裁剪范围 | 观察OSD输出 | 3-9米范围人脸可见9米外被裁 |
| 5米分界线 | 站在5米处观察 | y坐标应与zone_boundary_5m匹配 |
| 7米分界线 | 站在7米处观察 | y坐标应与zone_boundary_7m匹配 |
| 近区检测 | 站在4米处 | 人脸25-35px320输入检测率>90% |
| 中区检测 | 站在6米处 | 人脸25-35px320输入检测率>90% |
| 远区检测 | 站在8米处 | 人脸25-35px320输入检测率~80% |
| 远区极限 | 站在9米处 | 人脸~25px检测率~70%可接受或升级640 |
| 距离估算 | 对比激光测距 | 误差<0.3米 |
| 姿态过滤 | 低头 vs 抬头 | 过度低头(-15°以下)标记为low_head |
---
## 9. 性能预期
### 9.1 理论计算RetinaFace_320
| 优化项 | 参数 | 实际收益 |
|--------|------|----------|
| **RetinaFace_320** | 320×320输入现有模型 | 快速部署5-7米检测率>90% |
| ROI裁剪 | ~40%算力节省 | 单路NPU占用从20%→12% |
| 三分区检测 | 各区域25-35px | 全范围检测率80-90%9米可能70% |
| 距离过滤 | 3-9米范围 | 减少范围外误检 |
| 姿态过滤 | 5点关键点 | 误识率降低50%,无需额外模型 |
| **可升级** | 320→640 | 预留接口,后续无缝升级 |
### 9.2 资源占用估算320模型
- **单路NPU占用**: ~15-20%RetinaFace_320 + MobileFaceNet
- RetinaFace_320: ~8-10ms/帧640的1/4
- MobileFaceNet: ~10-15ms/人
- **8台RK3588承载**: 每台5-6路320模型轻量
- **内存占用**:
- 相机模型LUT: ~10KB/路
- 三分区检测缓冲: ~3×320×320×3 ≈ 0.9MB/路
- **延迟**: 三分区检测约增加8-10ms
> **注**: 320模型计算量小快速部署验证功能。如8-9米检测率不满足可无缝升级到640。
---
## 10. 风险评估与回退方案
### 10.1 潜在风险与缓解
| 风险 | 可能性 | 影响 | 缓解措施 |
|------|--------|------|----------|
| **8-9米检测率不足** | 中 | 320模型远距离检测能力有限 | 可提高far_scale到2.0x或升级到640 |
| 标定参数不准 | 中 | 测距误差大 | 提供现场微调指引 |
| 5米/7米分界线不准 | 中 | 分区检测率不均 | 可手工调整boundary_y |
| 姿态估计误差 | 低 | 5点近似±5°误差 | 放宽min_pitch到-20°容忍 |
| **升级到640兼容性** | 低 | 接口预留,已考虑 | 只需改model_path和scales |
### 10.2 回退方案
若分区检测效果不佳,可快速回退:
```json
{
"face_det": {
"distance_zones": {
"enabled": false // 关闭分区,回退单尺度
}
},
"pre": {
"roi": {
"enabled": false // 关闭ROI全图检测
}
}
}
```
---
## 11. 模型准备清单
### 11.1 当前版本模型v2.3
| 模型 | 文件名 | 输入尺寸 | 来源 | 状态 |
|------|--------|----------|------|------|
| **RetinaFace_320** | `RetinaFace_mobile320.rknn` | 320×320 | 项目已有 | ✅ 已有 |
| MobileFaceNet | `mobilefacenet_arcface.rknn` | 112×112 | 项目已有 | ✅ 已有 |
### 11.2 升级到640v2.4预留)
当320模型精度不满足时无缝升级
```json
// 只需修改配置
{
"face_det": {
"model_path": "./models/RetinaFace_mobile640.rknn",
"model_w": 640,
"model_h": 640,
"distance_zones": {
"scales": [0.7, 1.0, 1.4] // 640模型用此参数
}
}
}
```
**640模型获取**
- 方案1自行转换ONNX到RKNN
- 方案2寻找社区预转换模型
### 11.3 320 vs 640 对比
| 指标 | RetinaFace_320 | RetinaFace_640 | 建议 |
|------|----------------|----------------|------|
| 输入尺寸 | 320×320 | 640×640 | - |
| NPU耗时 | ~8-10ms | ~25-30ms | 320快3倍 |
| 5-7米检测率 | >90% | >95% | 相近 |
| 8-9米检测率 | ~70% | >85% | 640优势明显 |
| 部署难度 | 已有模型 | 需准备 | 先用320 |
### 11.4 模型验证
```python
# 验证320模型输出
# 预期输出:
# - loc: [1, 16800, 4] (检测框回归)
# - conf: [1, 16800, 2] (人脸/背景分类)
# - landms: [1, 16800, 10] (5点关键点)
```
---
## 附录A: 标定工具完整代码(三分区版)
```python
#!/usr/bin/env python3
"""
calibrate_camera.py - 相机标定工具(三分区版)
生成ROI和三分区参数供手工复制到配置文件
Usage:
python calibrate_camera.py --height 5.0 --pitch 45 -o cam_calib.json --report
"""
import json
import argparse
import numpy as np
from dataclasses import dataclass
from typing import List, Tuple
@dataclass
class CameraParams:
height: float # 安装高度(m)
pitch_deg: float # 俯仰角(°)
focal_px: float # 像素焦距(px)
img_w: int # 图像宽度
img_h: int # 图像高度
class CameraCalibrator:
"""相机标定器 - 支持三分区"""
def __init__(self, p: CameraParams):
self.p = p
self.cy = p.img_h // 2
self.cx = p.img_w // 2
self.theta = np.radians(p.pitch_deg)
self._build_lut()
def _build_lut(self):
"""预计算距离查找表 LUT[y] = distance(m)"""
self.lut = np.zeros(self.p.img_h, dtype=np.float32)
for y in range(self.p.img_h):
dy = y - self.cy
angle = self.theta + np.arctan2(dy, self.p.focal_px)
if abs(angle) > 1e-6 and np.tan(angle) > 0:
self.lut[y] = self.p.height / np.tan(angle)
else:
self.lut[y] = np.inf
def pixel_to_distance(self, y: int) -> float:
"""像素y坐标 -> 距离(m)"""
if 0 <= y < self.p.img_h:
return float(self.lut[y])
return np.inf
def distance_to_pixel(self, d: float) -> int:
"""距离(m) -> 像素y坐标"""
if d <= 0:
return self.cy
angle = np.arctan2(self.p.height, d)
offset = self.p.focal_px * np.tan(angle - self.theta)
return int(np.clip(self.cy + offset, 0, self.p.img_h - 1))
def calculate_roi(self, min_d: float, max_d: float, margin: int = 20) -> dict:
"""计算ROI裁剪区域"""
# 注意距离越远y坐标越大画面下方
y_min = self.distance_to_pixel(max_d) # 远距在下
y_max = self.distance_to_pixel(min_d) # 近距在上
# 添加边界余量
y_min = max(0, y_min - margin)
y_max = min(self.p.img_h, y_max + margin)
saving = 1 - (y_max - y_min) / self.p.img_h
return {
"crop": {
"x": 0,
"y": y_min,
"w": self.p.img_w,
"h": y_max - y_min
},
"saving_percent": round(saving * 100, 1)
}
def get_zone_boundaries(self, boundaries_m: List[float]) -> List[int]:
"""获取分区边界像素坐标"""
return [self.distance_to_pixel(d) for d in boundaries_m]
def estimate_face_size(self, distance: float, real_width: float = 0.16) -> float:
"""估算给定距离的人脸像素大小"""
if distance <= 0:
return 0
return self.p.focal_px * real_width / distance
def generate_zones_config(self,
boundaries_m: List[float],
scales: List[float]) -> dict:
"""生成分区配置"""
boundaries_y = self.get_zone_boundaries(boundaries_m)
zones = []
zone_names = ["near", "mid", "far"]
for i, (name, scale) in enumerate(zip(zone_names, scales)):
y_start = boundaries_y[i] if i > 0 else 0
y_end = boundaries_y[i] if i < len(boundaries_y) else self.p.img_h
# 如果是最后一个区
if i == len(scales) - 1:
y_end = self.p.img_h
else:
y_end = boundaries_y[i]
# 重新计算正确的y范围
if i == 0:
y_range = [0, boundaries_y[0]]
d_range = [3.0, boundaries_m[0]]
elif i == len(scales) - 1:
y_range = [boundaries_y[-1], self.p.img_h]
d_range = [boundaries_m[-1], 9.0]
else:
y_range = [boundaries_y[i-1], boundaries_y[i]]
d_range = [boundaries_m[i-1], boundaries_m[i]]
zones.append({
"name": name,
"distance_range": d_range,
"scale": scale,
"y_range": y_range
})
return {
"enabled": True,
"boundaries": boundaries_y,
"zones": zones
}
def generate_config(self,
min_d: float,
max_d: float,
boundaries_m: List[float],
scales: List[float]) -> dict:
"""生成完整配置"""
roi = self.calculate_roi(min_d, max_d)
zones_cfg = self.generate_zones_config(boundaries_m, scales)
return {
"preprocess": {
"roi": {"enabled": True, **roi["crop"]}
},
"face_det": {
"distance_zones": zones_cfg
},
"face_recog": {
"filters": {
"distance": {"enabled": True, "min": min_d, "max": max_d},
"pose": {
"enabled": True,
"min_pitch": -15,
"camera_pitch": self.p.pitch_deg
}
}
},
"_meta": {
"focal_px": self.p.focal_px,
"height": self.p.height,
"pitch": self.p.pitch_deg,
"roi_saving": roi["saving_percent"],
"zone_boundaries_m": boundaries_m,
"zone_scales": scales
}
}
def print_report(self,
min_d: float,
max_d: float,
boundaries_m: List[float],
scales: List[float]):
"""打印标定报告"""
print("=" * 60)
print("相机标定报告(三分区)")
print("=" * 60)
print(f"\n【相机参数】")
print(f" 安装高度 H: {self.p.height}m")
print(f" 俯仰角 θ: {self.p.pitch_deg}°")
print(f" 像素焦距 f: {self.p.focal_px}px")
print(f" 图像尺寸: {self.p.img_w}x{self.p.img_h}")
roi = self.calculate_roi(min_d, max_d)
boundaries_y = self.get_zone_boundaries(boundaries_m)
print(f"\n【ROI配置】(检测范围 {min_d}-{max_d}m)")
print(f" 裁剪: y={roi['crop']['y']}, h={roi['crop']['h']}")
print(f" 算力节省: {roi['saving_percent']}%")
print(f"\n【三分区配置】")
zone_names = ["近区(3-5m)", "中区(5-7m)", "远区(7-9m)"]
for i, (name, scale, y_bound) in enumerate(zip(zone_names, scales, boundaries_y)):
if i < 2:
print(f" {name}: y<{y_bound}, scale={scale}x")
else:
print(f" {name}: y>={boundaries_y[-1]}, scale={scale}x")
print(f"\n【距离-像素-人脸大小映射】")
print(f" {'距离':>6} | {'像素y':>6} | {'原人脸':>8} | {'处理后':>8} | {'区域':>6}")
print(f" {'-'*50}")
for d in [3, 4, 5, 6, 7, 8, 9]:
y = self.distance_to_pixel(d)
face_orig = self.estimate_face_size(d)
# 确定区域和处理后大小
zone_idx = 0
if d >= boundaries_m[1]:
zone_idx = 2
elif d >= boundaries_m[0]:
zone_idx = 1
face_proc = face_orig * scales[zone_idx]
zone_name = ["近", "中", "远"][zone_idx]
print(f" {d:>6.0f}m | {y:>6} | {face_orig:>7.0fpx} | {face_proc:>7.0fpx} | {zone_name:>6}")
print(f"\n【目标验证】")
print(f" 三个区处理后的人脸应在 60-80px 范围内")
print(f" 若偏差较大,请调整 --focal-estimate 参数重新标定")
print("\n" + "=" * 60)
def main():
parser = argparse.ArgumentParser(
description='相机标定工具 - 生成三分区检测配置',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
示例:
# 三分区标定(推荐)
python calibrate_camera.py --height 5.0 --pitch 45 --report
# 自定义参数
python calibrate_camera.py --height 4.5 --pitch 40 --focal-estimate 2000 -o cam.json --report
# 调整分区边界
python calibrate_camera.py --height 5.0 --zones 4 6 8 --scales 0.8 1.0 1.3 --report
"""
)
parser.add_argument('--height', type=float, required=True,
help='相机安装高度(米),如 5.0')
parser.add_argument('--pitch', type=float, default=45,
help='俯仰角默认45')
parser.add_argument('--focal-estimate', type=float, default=2200,
help='像素焦距估算值默认22002.5K相机4-6mm镜头')
parser.add_argument('--image-size', type=int, nargs=2,
default=[2560, 1440],
help='图像尺寸 宽 高,默认 2560 1440')
parser.add_argument('--range', type=float, nargs=2,
default=[3.0, 9.0],
help='检测范围 最小距离 最大距离,默认 3.0 9.0')
parser.add_argument('--zones', type=float, nargs=3,
default=[5.0, 7.0],
help='分区边界距离(米),默认 5.0 7.0形成3-5,5-7,7-9三区')
parser.add_argument('--scales', type=float, nargs=3,
default=[0.7, 1.0, 1.4],
help='各分区缩放因子,默认 0.7 1.0 1.4')
parser.add_argument('-o', '--output', default='camera_calib.json',
help='输出文件路径,默认 camera_calib.json')
parser.add_argument('--report', action='store_true',
help='打印详细报告')
args = parser.parse_args()
# 验证参数
if len(args.zones) != 2:
parser.error("--zones 需要2个边界值如 5.0 7.0形成3个区域")
if len(args.scales) != 3:
parser.error("--scales 需要3个缩放因子如 0.7 1.0 1.4")
# 构建完整边界列表(包含起止)
boundaries = [args.zones[0], args.zones[1]] # 5米和7米分界线
# 创建标定器
params = CameraParams(
height=args.height,
pitch_deg=args.pitch,
focal_px=args.focal_estimate,
img_w=args.image_size[0],
img_h=args.image_size[1]
)
calib = CameraCalibrator(params)
# 生成用于instances params的配置
roi = calib.calculate_roi(args.range[0], args.range[1])
boundaries_y = calib.get_zone_boundaries(boundaries)
config = {
"params": {
"camera_pitch": args.pitch,
"roi_enabled": True,
"roi_y": roi["crop"]["y"],
"roi_h": roi["crop"]["h"],
"zones_enabled": True,
"zone_boundary_5m": boundaries_y[0],
"zone_boundary_7m": boundaries_y[1],
"zone_scale_near": args.scales[0],
"zone_scale_mid": args.scales[1],
"zone_scale_far": args.scales[2]
},
"meta": {
"height": args.height,
"pitch": args.pitch,
"focal_px": args.focal_estimate,
"roi_saving": roi["saving_percent"],
"zone_boundaries_m": boundaries,
"zone_scales": list(args.scales)
}
}
# 保存配置
with open(args.output, 'w', encoding='utf-8') as f:
json.dump(config, f, indent=2, ensure_ascii=False)
print(f"[OK] 配置已保存: {args.output}")
# 打印报告
if args.report:
calib.print_report(args.range[0], args.range[1], boundaries, list(args.scales))
print(f"\n【可复制到 instances params 的配置】")
print(json.dumps(config["params"], indent=2, ensure_ascii=False))
if __name__ == '__main__':
main()
```
---
## 附录B: 术语表
| 术语 | 英文 | 说明 |
|------|------|------|
| ROI | Region of Interest | 感兴趣区域此处指3-9米对应的画面区域 |
| LUT | Look-Up Table | 查找表用于O(1)距离查询 |
| Zone Boundary | Zone Boundary | 分区边界5米/7米对应的像素y坐标 |
| RetinaFace_640 | RetinaFace 640×640 | 640×640输入的人脸检测模型 |
| 5点关键点 | 5-Point Landmarks | 左眼、右眼、鼻尖、左嘴角、右嘴角 |
| Templates | Configuration Templates | 配置模板,定义通用节点流水线 |
| Instances | Configuration Instances | 配置实例,为每路相机传入具体参数 |
| Pitch | Pitch Angle | 俯仰角,相机光轴与水平面夹角 |
| Focal Length (px) | Pixel Focal Length | 以像素为单位的焦距 |
---
**文档结束**