vr画面添加

This commit is contained in:
Rowland 2025-09-15 14:38:18 +08:00
parent af0a999191
commit 6c9c4339f2
5 changed files with 1327 additions and 1 deletions

194
VR_GUIDE.md Normal file
View File

@ -0,0 +1,194 @@
# VR模式使用指南
## 概述
本项目现已支持VR模式通过SteamVR将Panda3D引擎的画面传输到VR头显设备。您可以在虚拟现实环境中体验和编辑3D场景。
## 系统要求
### 硬件要求
- 支持SteamVR的VR头显设备如HTC Vive、Oculus Rift、Valve Index等
- VR头显控制器可选用于交互
- 满足VR运行要求的计算机配置
### 软件要求
- Windows 10/11 或 Linux支持OpenVR
- SteamVR运行时
- Python 3.7+
- 项目依赖库(见安装步骤)
## 安装与设置
### 1. 安装VR依赖
```bash
# 安装OpenVR Python库
pip install openvr==2.2.0
# 或者安装所有项目依赖
pip install -r requirements/requirements.txt
```
### 2. 安装SteamVR
1. 安装Steam客户端
2. 在Steam中搜索并安装"SteamVR"
3. 连接您的VR头显设备
4. 按照SteamVR设置向导完成房间设置
### 3. 验证VR设置
运行VR测试脚本来验证安装
```bash
python vr_test.py
```
## 使用方法
### 启动VR模式
1. **启动SteamVR**
- 在Steam中启动SteamVR
- 确保VR头显已连接并处于就绪状态
2. **启动应用程序**
```bash
python main.py
```
3. **进入VR模式**
- 在菜单栏中选择 `VR``进入VR模式`
- 系统会自动检测VR设备并初始化VR渲染
4. **戴上VR头显**
- 现在您可以在VR环境中查看和操作3D场景
### VR菜单选项
- **进入VR模式**: 启用VR渲染将画面输出到VR头显
- **退出VR模式**: 退出VR模式回到桌面显示
- **VR状态**: 查看VR系统状态和设备信息
- **VR设置**: 配置VR渲染参数和性能选项
### VR交互
目前支持的VR交互功能
- **头部追踪**: 自动追踪头显位置和朝向
- **房间规模移动**: 在设定的VR空间内移动
- **立体显示**: 为左右眼提供不同视角的立体图像
## 配置选项
### VR设置对话框
`VR``VR设置` 中可以配置:
- **渲染质量**: 调整VR渲染的画质等级
- **抗锯齿**: 设置抗锯齿级别
- **刷新率**: 配置VR头显刷新率
- **异步重投影**: 启用/禁用ATW技术
### 性能优化建议
1. **降低渲染分辨率**: 如果性能不足可以降低VR渲染质量
2. **减少抗锯齿**: 关闭或降低抗锯齿级别
3. **优化场景复杂度**: 减少场景中的多边形数量
4. **关闭不必要的特效**: 暂时关闭高级渲染特效
## 故障排除
### 常见问题
**Q: 无法进入VR模式提示"VR系统不可用"**
A: 请检查:
- SteamVR是否正在运行
- VR头显是否正确连接
- OpenVR库是否已安装 (`pip install openvr`)
**Q: VR画面卡顿或延迟**
A: 尝试:
- 降低VR渲染质量
- 关闭其他占用GPU的程序
- 检查VR头显连接线是否松动
**Q: VR菜单显示为灰色不可用**
A: 检查:
- VR管理器是否正确初始化
- 查看控制台错误信息
- 运行 `python vr_test.py` 进行诊断
**Q: 只能看到一只眼的画面**
A: 这可能是:
- VR渲染缓冲区创建失败
- OpenVR投影矩阵设置错误
- 检查图形驱动程序是否最新
### 诊断工具
使用内置的VR测试工具进行问题诊断
```bash
python vr_test.py
```
该工具会检测:
- 基础环境和依赖
- VR管理器状态
- VR设备可用性
- VR初始化过程
- UI菜单集成
### 日志信息
VR相关的日志信息会输出到控制台包括
- VR系统初始化状态
- 渲染缓冲区创建信息
- 追踪数据更新
- 错误和警告信息
## 开发者信息
### VR架构
```
main.py (MyWorld)
├── core/vr_manager.py (VRManager)
│ ├── OpenVR集成
│ ├── 渲染缓冲区管理
│ ├── 相机控制
│ └── 追踪数据处理
└── ui/main_window.py
├── VR菜单
├── VR设置对话框
└── VR事件处理
```
### 核心类
- **VRManager**: VR功能的核心管理类
- **MainWindow**: UI集成和事件处理
- **MyWorld**: VR管理器的宿主类
### 扩展开发
如需添加新的VR功能可以
1. 在 `VRManager` 中添加新方法
2. 在 `MainWindow` 中添加对应的UI控件
3. 更新VR设置对话框
4. 添加相应的测试代码
## 技术支持
如果遇到问题,请:
1. 运行 `python vr_test.py` 获取诊断信息
2. 检查控制台日志
3. 确认SteamVR和硬件设置正确
4. 提交问题时请包含详细的错误信息和系统配置
## 版本信息
- VR功能版本: 1.0.0
- 支持的OpenVR版本: 2.2.0+
- 兼容的SteamVR版本: 最新版本
---
**注意**: VR功能目前处于初始版本可能还有一些限制和待完善的功能。我们会持续改进和优化VR体验。

932
core/vr_manager.py Normal file
View File

@ -0,0 +1,932 @@
"""
VR管理器模块
负责VR功能的初始化渲染和交互
- OpenVR/SteamVR集成
- VR头显跟踪和渲染
- VR控制器交互
- VR模式切换
"""
import sys
import numpy as np
from panda3d.core import (
WindowProperties, GraphicsPipe, FrameBufferProperties,
GraphicsOutput, Texture, Camera, PerspectiveLens, MatrixLens,
Mat4, Vec3, TransformState, RenderState, CardMaker,
BitMask32, PandaNode, NodePath, LMatrix4, LVector3, LVector4,
CS_yup_right, CS_default, PythonCallbackObject
)
from direct.task import Task
from direct.showbase.DirectObject import DirectObject
try:
import openvr
OPENVR_AVAILABLE = True
except ImportError:
OPENVR_AVAILABLE = False
print("警告: OpenVR未安装VR功能将不可用")
class VRManager(DirectObject):
"""VR管理器类 - 处理所有VR相关功能"""
def __init__(self, world):
"""初始化VR管理器
Args:
world: 主世界对象引用
"""
super().__init__()
self.world = world
self.vr_system = None
self.vr_enabled = False
self.vr_initialized = False
# VR渲染相关
self.vr_left_eye_buffer = None
self.vr_right_eye_buffer = None
self.vr_left_camera = None
self.vr_right_camera = None
self.vr_compositor = None
# VR跟踪数据
self.hmd_pose = Mat4.identMat()
self.controller_poses = {}
self.tracked_device_poses = []
self.poses = None # OpenVR姿态数组
# VR渲染参数
self.eye_width = 1080
self.eye_height = 1200
self.near_clip = 0.1
self.far_clip = 1000.0
# VR任务
self.vr_task = None
# VR锚点层级系统
self.tracking_space = None
self.hmd_anchor = None
self.left_eye_anchor = None
self.right_eye_anchor = None
# 坐标系转换矩阵 - 使用Panda3D内置方法
self.coord_mat = LMatrix4.convert_mat(CS_yup_right, CS_default)
self.coord_mat_inv = LMatrix4.convert_mat(CS_default, CS_yup_right)
# 性能监控
self.frame_count = 0
self.last_fps_check = 0
self.last_fps_time = 0
self.vr_fps = 0
self.submit_failures = 0
self.pose_failures = 0
# VR提交策略 - 基于参考实现
self.submit_together = True # 是否在right_cb中同时提交左右眼
print("✓ VR管理器初始化完成")
def convert_mat(self, mat):
"""
将OpenVR矩阵转换为Panda3D矩阵 - 基于参考实现
"""
if len(mat.m) == 4:
result = LMatrix4(
mat.m[0][0], mat.m[1][0], mat.m[2][0], mat.m[3][0],
mat.m[0][1], mat.m[1][1], mat.m[2][1], mat.m[3][1],
mat.m[0][2], mat.m[1][2], mat.m[2][2], mat.m[3][2],
mat.m[0][3], mat.m[1][3], mat.m[2][3], mat.m[3][3])
elif len(mat.m) == 3:
result = LMatrix4(
mat.m[0][0], mat.m[1][0], mat.m[2][0], 0.0,
mat.m[0][1], mat.m[1][1], mat.m[2][1], 0.0,
mat.m[0][2], mat.m[1][2], mat.m[2][2], 0.0,
mat.m[0][3], mat.m[1][3], mat.m[2][3], 1.0)
return result
def is_vr_available(self):
"""检查VR系统是否可用"""
if not OPENVR_AVAILABLE:
return False
try:
# 检查SteamVR是否运行
return openvr.isRuntimeInstalled() and openvr.isHmdPresent()
except Exception as e:
print(f"VR检查失败: {e}")
return False
def initialize_vr(self):
"""初始化VR系统"""
if not OPENVR_AVAILABLE:
print("❌ OpenVR不可用无法初始化VR")
return False
if self.vr_initialized:
print("VR系统已经初始化")
return True
try:
print("🔄 正在初始化VR系统...")
# 初始化OpenVR - 使用Scene应用类型确保正确的焦点管理
self.vr_system = openvr.init(openvr.VRApplication_Scene)
if not self.vr_system:
print("❌ 无法初始化OpenVR系统")
return False
# 获取compositor
self.vr_compositor = openvr.VRCompositor()
if not self.vr_compositor:
print("❌ 无法获取VR Compositor")
return False
# 获取推荐的渲染目标尺寸
self.eye_width, self.eye_height = self.vr_system.getRecommendedRenderTargetSize()
print(f"✓ VR渲染目标尺寸: {self.eye_width}x{self.eye_height}")
# 创建OpenVR姿态数组
poses_t = openvr.TrackedDevicePose_t * openvr.k_unMaxTrackedDeviceCount
self.poses = poses_t()
print("✓ VR姿态数组已创建")
# 创建VR渲染缓冲区
if not self._create_vr_buffers():
print("❌ 创建VR渲染缓冲区失败")
return False
# 设置VR相机
if not self._setup_vr_cameras():
print("❌ 设置VR相机失败")
return False
# 启动VR更新任务
self._start_vr_task()
self.vr_initialized = True
print("✅ VR系统初始化成功")
return True
except Exception as e:
print(f"❌ VR初始化失败: {e}")
import traceback
traceback.print_exc()
return False
def _create_vr_buffers(self):
"""创建VR渲染缓冲区 - 基于参考实现"""
try:
# 创建左眼纹理和缓冲区
self.vr_left_texture = self._create_vr_texture("VR Left Eye Texture")
self.vr_left_eye_buffer = self._create_vr_buffer(
"VR Left Eye",
self.vr_left_texture,
self.eye_width,
self.eye_height
)
if not self.vr_left_eye_buffer:
print("❌ 创建左眼缓冲区失败")
return False
# 设置左眼缓冲区属性
self.vr_left_eye_buffer.setSort(-100)
self.vr_left_eye_buffer.setClearColor((0.1, 0.2, 0.4, 1)) # 深蓝色背景便于调试
self.vr_left_eye_buffer.setActive(True)
# 创建右眼纹理和缓冲区
self.vr_right_texture = self._create_vr_texture("VR Right Eye Texture")
self.vr_right_eye_buffer = self._create_vr_buffer(
"VR Right Eye",
self.vr_right_texture,
self.eye_width,
self.eye_height
)
if not self.vr_right_eye_buffer:
print("❌ 创建右眼缓冲区失败")
return False
# 设置右眼缓冲区属性
self.vr_right_eye_buffer.setSort(-99)
self.vr_right_eye_buffer.setClearColor((0.1, 0.2, 0.4, 1)) # 深蓝色背景便于调试
self.vr_right_eye_buffer.setActive(True)
print("✓ VR渲染缓冲区创建成功")
return True
except Exception as e:
print(f"❌ 创建VR缓冲区失败: {e}")
import traceback
traceback.print_exc()
return False
def _create_vr_texture(self, name):
"""创建VR纹理对象 - 基于参考实现"""
texture = Texture(name)
texture.setWrapU(Texture.WMClamp)
texture.setWrapV(Texture.WMClamp)
texture.setMinfilter(Texture.FTLinear)
texture.setMagfilter(Texture.FTLinear)
return texture
def _create_vr_buffer(self, name, texture, width, height):
"""创建VR渲染缓冲区 - 基于参考实现"""
# 设置帧缓冲属性
fbprops = FrameBufferProperties()
fbprops.setRgbaBits(1, 1, 1, 1)
# 可以在这里添加多重采样抗锯齿
# fbprops.setMultisamples(4)
# 创建缓冲区
buffer = self.world.win.makeTextureBuffer(name, width, height, to_ram=False, fbp=fbprops)
if buffer:
# 清除默认渲染纹理
buffer.clearRenderTextures()
# 添加我们的纹理
buffer.addRenderTexture(texture, GraphicsOutput.RTMBindOrCopy, GraphicsOutput.RTPColor)
return buffer
def _setup_vr_cameras(self):
"""设置VR相机 - 使用锚点层级系统"""
try:
# 创建VR追踪空间锚点层级
self.tracking_space = self.world.render.attachNewNode('tracking-space')
self.hmd_anchor = self.tracking_space.attachNewNode('hmd-anchor')
self.left_eye_anchor = self.hmd_anchor.attachNewNode('left-eye')
self.right_eye_anchor = self.hmd_anchor.attachNewNode('right-eye')
# 获取投影矩阵
projection_left = self.coord_mat_inv * self.convert_mat(
self.vr_system.getProjectionMatrix(openvr.Eye_Left, self.near_clip, self.far_clip))
projection_right = self.coord_mat_inv * self.convert_mat(
self.vr_system.getProjectionMatrix(openvr.Eye_Right, self.near_clip, self.far_clip))
# 创建左眼相机节点
left_cam_node = Camera('left-cam')
left_lens = MatrixLens()
left_lens.setUserMat(projection_left)
left_cam_node.setLens(left_lens)
# 创建右眼相机节点
right_cam_node = Camera('right-cam')
right_lens = MatrixLens()
right_lens.setUserMat(projection_right)
right_cam_node.setLens(right_lens)
# 附加相机到眼睛锚点
self.vr_left_camera = self.left_eye_anchor.attachNewNode(left_cam_node)
self.vr_right_camera = self.right_eye_anchor.attachNewNode(right_cam_node)
# 设置显示区域并添加渲染回调
left_dr = self.vr_left_eye_buffer.makeDisplayRegion()
left_dr.setCamera(self.vr_left_camera)
left_dr.setActive(True)
left_dr.setDrawCallback(PythonCallbackObject(self.left_cb))
right_dr = self.vr_right_eye_buffer.makeDisplayRegion()
right_dr.setCamera(self.vr_right_camera)
right_dr.setActive(True)
right_dr.setDrawCallback(PythonCallbackObject(self.right_cb))
print("✓ VR相机锚点层级系统设置完成")
return True
except Exception as e:
print(f"❌ 设置VR相机失败: {e}")
import traceback
traceback.print_exc()
return False
def _get_eye_offset(self, eye):
"""获取眼睛相对于头显的偏移"""
try:
if not self.vr_system:
# 使用标准IPD瞳距估算值
ipd = 0.064 # 64mm平均IPD
if eye == openvr.Eye_Left:
return Vec3(-ipd/2, 0, 0)
else:
return Vec3(ipd/2, 0, 0)
# 从OpenVR获取眼睛到头显的变换矩阵
eye_transform = self.vr_system.getEyeToHeadTransform(eye)
# 提取位移信息
x = eye_transform[0][3]
y = eye_transform[1][3]
z = eye_transform[2][3]
return Vec3(x, y, z)
except Exception as e:
print(f"❌ 获取眼睛偏移失败: {e}")
# 返回默认值
ipd = 0.064
if eye == openvr.Eye_Left:
return Vec3(-ipd/2, 0, 0)
else:
return Vec3(ipd/2, 0, 0)
def _start_vr_task(self):
"""启动VR更新任务"""
if self.vr_task:
self.world.taskMgr.remove(self.vr_task)
self.vr_task = self.world.taskMgr.add(self._update_vr, "update_vr")
print("✓ VR更新任务已启动")
def _update_vr(self, task):
"""VR更新任务 - 每帧调用"""
if not self.vr_enabled or not self.vr_system:
return task.cont
try:
# 性能监控
self.frame_count += 1
# 计算VR FPS
import time
current_time = time.time()
if self.last_fps_time == 0:
self.last_fps_time = current_time
elif current_time - self.last_fps_time >= 1.0: # 每秒更新一次FPS
self.vr_fps = (self.frame_count - self.last_fps_check) / (current_time - self.last_fps_time)
self.last_fps_check = self.frame_count
self.last_fps_time = current_time
# 优化的VR更新顺序
# 1. 立即调用 waitGetPoses 获取最新的姿态数据
# 这确保我们使用最新数据而不是上一帧的数据
self._wait_get_poses()
# 2. 更新相机位置(使用刚获取的最新姿态数据)
self._update_camera_poses()
# 注意:纹理提交现在通过渲染回调自动处理
# 定期输出性能报告
if self.frame_count % 1800 == 1: # 每30秒@60fps输出一次性能报告
self._print_performance_report()
except Exception as e:
print(f"VR更新错误: {e}")
import traceback
traceback.print_exc()
return task.cont
def _wait_get_poses(self):
"""调用VRCompositor的waitGetPoses来获取焦点和姿态数据"""
try:
if not self.vr_compositor or not self.poses:
return
# 调用waitGetPoses获取焦点和姿态数据
# 这个调用可能会阻塞直到下一个VR同步点
result = self.vr_compositor.waitGetPoses(self.poses, None)
# 检查姿态数据的有效性
valid_poses = 0
# 更新HMD姿态设备0通常是头显
if len(self.poses) > 0 and self.poses[0].bPoseIsValid:
valid_poses += 1
else:
# 如果HMD姿态无效不要频繁输出错误信息
if not hasattr(self, '_hmd_invalid_warning_shown'):
print("⚠️ HMD姿态数据无效")
self._hmd_invalid_warning_shown = True
# 更新控制器姿态
self.controller_poses.clear()
for device_id in range(1, min(len(self.poses), openvr.k_unMaxTrackedDeviceCount)):
if self.poses[device_id].bPoseIsValid:
device_class = self.vr_system.getTrackedDeviceClass(device_id)
if device_class == openvr.TrackedDeviceClass_Controller:
controller_matrix = self.poses[device_id].mDeviceToAbsoluteTracking
self.controller_poses[device_id] = self._convert_openvr_matrix_to_panda(controller_matrix)
valid_poses += 1
# 性能监控 - 偶尔输出姿态状态
if self.frame_count % 600 == 1: # 每10秒输出一次@60fps
print(f"📊 VR姿态状态 - 有效姿态数: {valid_poses}, 总帧数: {self.frame_count}")
except Exception as e:
# 限制错误输出频率
if not hasattr(self, '_last_error_frame'):
self._last_error_frame = 0
if self.frame_count - self._last_error_frame > 300: # 每5秒最多输出一次错误
print(f"waitGetPoses失败: {e}")
self._last_error_frame = self.frame_count
# 记录姿态失败次数
self.pose_failures += 1
def _update_tracking_data(self):
"""更新VR追踪数据"""
try:
# 获取设备姿态
poses = self.vr_system.getDeviceToAbsoluteTrackingPose(
openvr.TrackingUniverseStanding, 0.0, openvr.k_unMaxTrackedDeviceCount
)
# 更新HMD姿态设备0通常是头显
if poses[0].bPoseIsValid:
hmd_matrix = poses[0].mDeviceToAbsoluteTracking
self.hmd_pose = self._convert_openvr_matrix_to_panda(hmd_matrix)
# 更新控制器姿态
for device_id in range(1, openvr.k_unMaxTrackedDeviceCount):
if poses[device_id].bPoseIsValid:
device_class = self.vr_system.getTrackedDeviceClass(device_id)
if device_class == openvr.TrackedDeviceClass_Controller:
controller_matrix = poses[device_id].mDeviceToAbsoluteTracking
self.controller_poses[device_id] = self._convert_openvr_matrix_to_panda(controller_matrix)
except Exception as e:
print(f"更新追踪数据失败: {e}")
def _convert_openvr_matrix_to_panda(self, ovr_matrix):
"""将OpenVR矩阵转换为Panda3D矩阵
坐标系转换
OpenVR: X右, Y上, -Z前右手坐标系
Panda3D: X右, Y前, Z上右手坐标系
转换规则
OpenVR X Panda3D X
OpenVR Y Panda3D Z
OpenVR -Z Panda3D Y
"""
mat = Mat4()
# 修正的坐标转换矩阵
# OpenVR: X右, Y上, -Z前 → Panda3D: X右, Y前, Z上
# 转换规则: (ovr_x, ovr_y, ovr_z) → (panda_x, panda_y, panda_z)
# (ovr_x, ovr_y, ovr_z) → (ovr_x, -ovr_z, ovr_y)
# X轴行Panda3D的X轴对应OpenVR的X轴
mat.setCell(0, 0, ovr_matrix[0][0]) # X_x → X_x
mat.setCell(0, 1, ovr_matrix[0][1]) # X_y → X_y
mat.setCell(0, 2, ovr_matrix[0][2]) # X_z → X_z
mat.setCell(0, 3, ovr_matrix[0][3]) # 位移X分量
# Y轴行Panda3D的Y轴对应OpenVR的-Z轴
mat.setCell(1, 0, -ovr_matrix[2][0]) # -Z_x → Y_x
mat.setCell(1, 1, -ovr_matrix[2][1]) # -Z_y → Y_y
mat.setCell(1, 2, -ovr_matrix[2][2]) # -Z_z → Y_z
mat.setCell(1, 3, -ovr_matrix[2][3]) # 位移Y分量-Z位移
# Z轴行Panda3D的Z轴对应OpenVR的Y轴
mat.setCell(2, 0, ovr_matrix[1][0]) # Y_x → Z_x
mat.setCell(2, 1, ovr_matrix[1][1]) # Y_y → Z_y
mat.setCell(2, 2, ovr_matrix[1][2]) # Y_z → Z_z
mat.setCell(2, 3, ovr_matrix[1][3]) # 位移Z分量Y位移
# 齐次坐标
mat.setCell(3, 0, 0)
mat.setCell(3, 1, 0)
mat.setCell(3, 2, 0)
mat.setCell(3, 3, 1)
# 调试信息 - 验证坐标系转换
if not hasattr(self, '_coord_debug_counter'):
self._coord_debug_counter = 0
self._coord_debug_counter += 1
if self._coord_debug_counter % 600 == 1: # 每10秒输出一次@60fps
print(f"🔄 坐标系转换调试 (第{self._coord_debug_counter}帧)")
# 输出原始OpenVR矩阵信息
ovr_pos = Vec3(ovr_matrix[0][3], ovr_matrix[1][3], ovr_matrix[2][3])
print(f" OpenVR原始位置: {ovr_pos}")
# 输出转换后的Panda3D矩阵信息
# 正确的方法从矩阵中读取位移第4列前3行
panda_pos = Vec3(mat.getCell(0, 3), mat.getCell(1, 3), mat.getCell(2, 3))
print(f" Panda3D转换位置: {panda_pos}")
# 检查矩阵设置是否正确
# 手动验证位置转换OpenVR (x,y,z) → Panda3D (x,-z,y)
manual_converted_pos = Vec3(ovr_pos.x, -ovr_pos.z, ovr_pos.y)
print(f" 手动转换结果: {manual_converted_pos}")
# 检查我们的矩阵是否正确设置了位置
print(f" 矩阵位置元素: [{mat.getCell(0,3)}, {mat.getCell(1,3)}, {mat.getCell(2,3)}]")
# 验证转换是否正确
expected_panda_pos = manual_converted_pos
print(f" 预期Panda3D位置: {expected_panda_pos}")
# 检查转换是否正确
diff = panda_pos - expected_panda_pos
diff_magnitude = diff.length()
if diff_magnitude < 0.001:
print(f" ✅ 坐标转换正确 (误差: {diff_magnitude:.6f})")
else:
print(f" ⚠️ 坐标转换可能有误 (误差: {diff_magnitude:.6f})")
print(f" 差异向量: {diff}")
print(f" 实际矩阵第4行: [{mat.getCell(3,0)}, {mat.getCell(3,1)}, {mat.getCell(3,2)}, {mat.getCell(3,3)}]")
return mat
def update_hmd(self, pose):
"""
更新HMD锚点 - 基于参考实现
"""
try:
# 将OpenVR姿态转换为Panda3D矩阵
modelview = self.convert_mat(pose.mDeviceToAbsoluteTracking)
# 应用坐标系转换并设置HMD锚点
self.hmd_anchor.setMat(self.coord_mat_inv * modelview * self.coord_mat)
# 获取眼睛到头部的变换
view_left = self.convert_mat(self.vr_system.getEyeToHeadTransform(openvr.Eye_Left))
view_right = self.convert_mat(self.vr_system.getEyeToHeadTransform(openvr.Eye_Right))
# 设置眼睛锚点
self.left_eye_anchor.setMat(self.coord_mat_inv * view_left * self.coord_mat)
self.right_eye_anchor.setMat(self.coord_mat_inv * view_right * self.coord_mat)
except Exception as e:
print(f"更新HMD姿态失败: {e}")
def _update_camera_poses(self):
"""更新相机姿态 - 使用锚点系统简化处理"""
try:
# 使用锚点系统后,相机位置自动跟随锚点
# 只需要获取HMD姿态并更新锚点即可
# 从poses数组中获取HMD姿态
if hasattr(self, 'poses') and len(self.poses) > 0:
hmd_pose = self.poses[openvr.k_unTrackedDeviceIndex_Hmd]
if hmd_pose.bPoseIsValid:
self.update_hmd(hmd_pose)
else:
print("⚠️ HMD姿态数据无效")
except Exception as e:
print(f"更新相机姿态失败: {e}")
import traceback
traceback.print_exc()
def _submit_frames_to_vr(self):
"""将渲染帧提交给VR系统"""
try:
if not self.vr_compositor:
return
# 使用我们创建的纹理对象
left_texture = self.vr_left_texture
right_texture = self.vr_right_texture
if left_texture and right_texture:
# 确保纹理已准备并获取OpenGL纹理ID
gsg = self.world.win.getGsg()
prepared_objects = gsg.getPreparedObjects()
# 准备纹理 - 使用更简单的方法
try:
# 方法1: 尝试prepareNow
if hasattr(left_texture, 'prepareNow'):
left_texture.prepareNow(0, prepared_objects, gsg)
if hasattr(right_texture, 'prepareNow'):
right_texture.prepareNow(0, prepared_objects, gsg)
except Exception as prep_error:
print(f"纹理准备失败: {prep_error}")
# 继续尝试,可能纹理已经准备好了
# 获取OpenGL纹理ID
left_texture_id = self._get_texture_opengl_id(left_texture, prepared_objects, gsg)
right_texture_id = self._get_texture_opengl_id(right_texture, prepared_objects, gsg)
# 检查是否成功获取了纹理ID
if left_texture_id is not None and left_texture_id > 0 and right_texture_id is not None and right_texture_id > 0:
try:
# 创建OpenVR纹理结构
left_eye_texture = openvr.Texture_t()
left_eye_texture.handle = int(left_texture_id)
left_eye_texture.eType = openvr.TextureType_OpenGL
left_eye_texture.eColorSpace = openvr.ColorSpace_Gamma
right_eye_texture = openvr.Texture_t()
right_eye_texture.handle = int(right_texture_id)
right_eye_texture.eType = openvr.TextureType_OpenGL
right_eye_texture.eColorSpace = openvr.ColorSpace_Gamma
# 提交到VR系统
error_left = self.vr_compositor.submit(openvr.Eye_Left, left_eye_texture)
error_right = self.vr_compositor.submit(openvr.Eye_Right, right_eye_texture)
# 检查提交结果 - 在Python OpenVR中None表示成功
left_success = (error_left is None or error_left == openvr.VRCompositorError_None)
right_success = (error_right is None or error_right == openvr.VRCompositorError_None)
if not left_success:
print(f"⚠️ 左眼纹理提交错误: {error_left}")
self.submit_failures += 1
if not right_success:
print(f"⚠️ 右眼纹理提交错误: {error_right}")
self.submit_failures += 1
# 如果两个都成功了,输出成功信息(仅第一次)
if not hasattr(self, '_first_submit_success'):
if left_success and right_success:
print("✅ VR纹理提交成功缓冲区应该显示场景内容。")
print(f" 左眼纹理ID: {left_texture_id}, 右眼纹理ID: {right_texture_id}")
self._first_submit_success = True
except Exception as submit_error:
print(f"❌ VR纹理提交过程失败: {submit_error}")
if "DoNotHaveFocus" in str(submit_error):
print("🔍 这通常意味着另一个VR应用程序正在使用VR系统")
self.submit_failures += 1
else:
# 只在第一次失败时输出警告,避免太多日志
if not hasattr(self, '_texture_id_warning_shown'):
print("⚠️ 无法获取有效的纹理OpenGL ID跳过VR帧提交")
print(" 这可能是因为纹理尚未正确准备到GPU")
self._texture_id_warning_shown = True
except Exception as e:
print(f"提交VR帧失败: {e}")
import traceback
traceback.print_exc()
def _get_texture_opengl_id(self, texture, prepared_objects, gsg):
"""获取纹理的OpenGL ID"""
try:
# 方法1: 使用prepareNow获取正确的纹理上下文
texture_context = texture.prepareNow(0, prepared_objects, gsg)
if texture_context and hasattr(texture_context, 'getNativeId'):
native_id = texture_context.getNativeId()
if native_id > 0:
print(f"✓ 成功获取纹理 {texture.getName()} 的OpenGL ID: {native_id}")
return native_id
# 方法2: 备选方案 - 通过prepared_objects获取texture context
if hasattr(prepared_objects, 'getTextureContext'):
texture_context = prepared_objects.getTextureContext(texture)
if texture_context and hasattr(texture_context, 'getNativeId'):
native_id = texture_context.getNativeId()
if native_id > 0:
print(f"✓ 备选方法获取纹理 {texture.getName()} 的OpenGL ID: {native_id}")
return native_id
# 方法3: 尝试texture对象本身的方法
if hasattr(texture, 'getNativeId'):
native_id = texture.getNativeId()
if native_id > 0:
print(f"✓ 直接获取纹理 {texture.getName()} 的OpenGL ID: {native_id}")
return native_id
# 如果所有方法都失败
print(f"❌ 无法获取纹理 {texture.getName()} 的OpenGL ID")
return None
except Exception as e:
print(f"获取纹理OpenGL ID失败: {e}")
import traceback
traceback.print_exc()
return None
def enable_vr(self):
"""启用VR模式"""
if not self.is_vr_available():
print("❌ VR系统不可用")
return False
if not self.vr_initialized:
if not self.initialize_vr():
return False
self.vr_enabled = True
# 禁用主相机避免干扰VR渲染
self._disable_main_cam()
print("✅ VR模式已启用")
return True
def disable_vr(self):
"""禁用VR模式"""
self.vr_enabled = False
# 恢复主相机
self._enable_main_cam()
print("✅ VR模式已禁用")
def cleanup(self):
"""清理VR资源"""
try:
print("🔄 正在清理VR资源...")
# 停止VR任务
if self.vr_task:
self.world.taskMgr.remove(self.vr_task)
self.vr_task = None
# 清理渲染缓冲区
if self.vr_left_eye_buffer:
self.vr_left_eye_buffer.removeAllDisplayRegions()
self.world.graphicsEngine.removeWindow(self.vr_left_eye_buffer)
self.vr_left_eye_buffer = None
if self.vr_right_eye_buffer:
self.vr_right_eye_buffer.removeAllDisplayRegions()
self.world.graphicsEngine.removeWindow(self.vr_right_eye_buffer)
self.vr_right_eye_buffer = None
# 清理相机
if self.vr_left_camera:
self.vr_left_camera.removeNode()
self.vr_left_camera = None
if self.vr_right_camera:
self.vr_right_camera.removeNode()
self.vr_right_camera = None
# 关闭OpenVR
if self.vr_system and OPENVR_AVAILABLE:
try:
openvr.shutdown()
except:
pass
self.vr_system = None
self.vr_enabled = False
self.vr_initialized = False
print("✅ VR资源清理完成")
except Exception as e:
print(f"⚠️ VR清理过程中出错: {e}")
def get_vr_status(self):
"""获取VR状态信息"""
return {
'available': self.is_vr_available(),
'initialized': self.vr_initialized,
'enabled': self.vr_enabled,
'eye_resolution': (self.eye_width, self.eye_height),
'device_count': len(self.controller_poses) + (1 if self.vr_enabled else 0),
'vr_fps': self.vr_fps,
'frame_count': self.frame_count,
'submit_failures': self.submit_failures,
'pose_failures': self.pose_failures
}
def _print_performance_report(self):
"""输出VR性能报告"""
print("📊 === VR性能报告 ===")
print(f" VR帧率: {self.vr_fps:.1f} FPS")
print(f" 总帧数: {self.frame_count}")
print(f" 提交失败: {self.submit_failures}")
print(f" 姿态失败: {self.pose_failures}")
# 计算失败率
if self.frame_count > 0:
submit_fail_rate = (self.submit_failures / self.frame_count) * 100
pose_fail_rate = (self.pose_failures / self.frame_count) * 100
print(f" 提交失败率: {submit_fail_rate:.2f}%")
print(f" 姿态失败率: {pose_fail_rate:.2f}%")
print("========================")
def left_cb(self, cbdata):
"""左眼渲染回调 - 基于参考实现"""
# 执行实际的渲染工作
cbdata.upcall()
# 根据提交策略决定是否立即提交
if not self.submit_together:
# 分别提交模式:左眼渲染完成后立即提交
self.submit_texture(openvr.Eye_Left, self.vr_left_texture)
def right_cb(self, cbdata):
"""右眼渲染回调 - 基于参考实现"""
# 执行实际的渲染工作
cbdata.upcall()
# 根据提交策略决定提交方式
if self.submit_together:
# 同时提交模式:右眼渲染完成后同时提交左右眼
self.submit_texture(openvr.Eye_Left, self.vr_left_texture)
self.submit_texture(openvr.Eye_Right, self.vr_right_texture)
else:
# 分别提交模式:只提交右眼
self.submit_texture(openvr.Eye_Right, self.vr_right_texture)
def submit_texture(self, eye, texture):
"""提交纹理到VR - 基于参考实现,增强调试信息"""
try:
if not self.vr_compositor:
print("❌ VR compositor不可用")
self.submit_failures += 1
return
# 获取graphics state guardian和prepared objects
gsg = self.world.win.getGsg()
if not gsg:
print("❌ 无法获取GraphicsStateGuardian")
self.submit_failures += 1
return
prepared_objects = gsg.getPreparedObjects()
if not prepared_objects:
print("❌ 无法获取PreparedObjects")
self.submit_failures += 1
return
# 准备纹理并获取更详细的错误信息
if not texture:
print("❌ 纹理对象为空")
self.submit_failures += 1
return
print(f"🔍 准备纹理: {texture.getName()}, 大小: {texture.getXSize()}x{texture.getYSize()}")
texture_context = texture.prepareNow(0, prepared_objects, gsg)
if not texture_context:
print("❌ prepareNow返回空的texture_context")
self.submit_failures += 1
return
handle = texture_context.getNativeId()
print(f"🔍 获取OpenGL纹理句柄: {handle}")
if handle != 0:
ovr_texture = openvr.Texture_t()
ovr_texture.handle = handle
ovr_texture.eType = openvr.TextureType_OpenGL
ovr_texture.eColorSpace = openvr.ColorSpace_Gamma
eye_name = "左眼" if eye == openvr.Eye_Left else "右眼"
print(f"🔍 提交{eye_name}纹理到VR, 句柄: {handle}")
# 提交到VR系统
error = self.vr_compositor.submit(eye, ovr_texture)
print(f"🔍 VR提交结果: {error}")
# 检查错误
if error and error != openvr.VRCompositorError_None:
print(f"⚠️ VR纹理提交错误代码: {error}")
self.submit_failures += 1
else:
# 只在第一次成功时输出
if not hasattr(self, '_submit_success_logged'):
print(f"✅ VR纹理提交成功! {eye_name}")
self._submit_success_logged = True
else:
print(f"❌ 无法获取纹理OpenGL句柄: handle = {handle}")
print(f" 纹理状态: 已准备={texture.isPrepared(prepared_objects)}")
print(f" 纹理格式: {texture.getFormat()}")
self.submit_failures += 1
except Exception as e:
print(f"❌ VR纹理提交异常: {e}")
import traceback
traceback.print_exc()
self.submit_failures += 1
def _disable_main_cam(self):
"""禁用主相机 - 基于参考实现"""
try:
# 保存原始相机状态
if not hasattr(self, '_original_camera_parent'):
self._original_camera_parent = self.world.camera.getParent()
# 创建空节点并将主相机重新附加到它
self._empty_world = NodePath("empty_world")
self.world.camera.reparentTo(self._empty_world)
print("✓ 主相机已禁用")
except Exception as e:
print(f"⚠️ 禁用主相机失败: {e}")
def _enable_main_cam(self):
"""恢复主相机 - 基于参考实现"""
try:
# 恢复原始相机状态
if hasattr(self, '_original_camera_parent') and self._original_camera_parent:
self.world.camera.reparentTo(self._original_camera_parent)
else:
# 如果没有保存的父节点重新附加到render
self.world.camera.reparentTo(self.world.render)
# 清理空世界节点
if hasattr(self, '_empty_world'):
self._empty_world.removeNode()
delattr(self, '_empty_world')
print("✓ 主相机已恢复")
except Exception as e:
print(f"⚠️ 恢复主相机失败: {e}")

View File

@ -97,6 +97,15 @@ class MyWorld(CoreWorld):
from core.collision_manager import CollisionManager
self.collision_manager = CollisionManager(self)
# 初始化VR管理器
try:
from core.vr_manager import VRManager
self.vr_manager = VRManager(self)
print("✓ VR管理器初始化完成")
except Exception as e:
print(f"⚠ VR管理器初始化失败: {e}")
self.vr_manager = None
# 调试选项
self.debug_collision = False # 是否显示碰撞体

View File

@ -98,3 +98,4 @@ webencodings==0.5.1
xdg==5
xkit==0.0.0
zipp==1.0.0
openvr==2.2.0

View File

@ -438,6 +438,17 @@ class MainWindow(QMainWindow):
self.refreshAssetsAction = self.assetsMenu.addAction('刷新资源')
self.refreshAssetsAction.triggered.connect(self.refreshAssetsView)
# VR菜单
self.vrMenu = menubar.addMenu('VR')
self.enterVRAction = self.vrMenu.addAction('进入VR模式')
self.exitVRAction = self.vrMenu.addAction('退出VR模式')
self.vrMenu.addSeparator()
self.vrStatusAction = self.vrMenu.addAction('VR状态')
self.vrSettingsAction = self.vrMenu.addAction('VR设置')
# 初始状态下禁用退出VR选项
self.exitVRAction.setEnabled(False)
# 帮助菜单
self.helpMenu = menubar.addMenu('帮助')
self.aboutAction = self.helpMenu.addAction('关于')
@ -898,6 +909,12 @@ class MainWindow(QMainWindow):
# self.toggleHotReloadAction.triggered.connect(self.onToggleHotReload)
# self.openScriptsManagerAction.triggered.connect(self.onOpenScriptsManager)
# 连接VR菜单事件
self.enterVRAction.triggered.connect(self.onEnterVR)
self.exitVRAction.triggered.connect(self.onExitVR)
self.vrStatusAction.triggered.connect(self.onShowVRStatus)
self.vrSettingsAction.triggered.connect(self.onShowVRSettings)
def onCreateCesiumView(self):
if hasattr(self.world,'gui_manager') and self.world.gui_manager:
@ -1845,13 +1862,21 @@ class MainWindow(QMainWindow):
# 清理工具管理器中的进程
if hasattr(self.world, 'tool_manager') and self.world.tool_manager:
print("🧹 清理工具管理器进程...")
self.world.tool_manager.cleanup_processes()
if hasattr(self.world.tool_manager, 'cleanup_processes'):
self.world.tool_manager.cleanup_processes()
else:
print("✓ 工具管理器无需清理进程")
# 停止更新定时器
if hasattr(self, 'updateTimer') and self.updateTimer:
self.updateTimer.stop()
print("⏹️ 更新定时器已停止")
# 清理VR资源
if hasattr(self.world, 'vr_manager') and self.world.vr_manager:
print("🧹 清理VR资源...")
self.world.vr_manager.cleanup()
# 清理Panda3D资源
if hasattr(self, 'pandaWidget') and self.pandaWidget:
print("🧹 清理Panda3D资源...")
@ -2000,6 +2025,171 @@ class MainWindow(QMainWindow):
else:
QMessageBox.warning(self, "错误", "高度图地形创建失败!")
# ==================== VR事件处理 ====================
def onEnterVR(self):
"""进入VR模式"""
try:
if hasattr(self.world, 'vr_manager') and self.world.vr_manager:
success = self.world.vr_manager.enable_vr()
if success:
# 更新菜单状态
self.enterVRAction.setEnabled(False)
self.exitVRAction.setEnabled(True)
QMessageBox.information(self, "成功", "VR模式已启用\n请确保您的VR头显已正确连接。")
else:
QMessageBox.warning(self, "错误", "无法启用VR模式\n请检查:\n1. SteamVR是否正在运行\n2. VR头显是否已连接\n3. OpenVR库是否已正确安装")
else:
QMessageBox.warning(self, "错误", "VR管理器不可用")
except Exception as e:
QMessageBox.critical(self, "错误", f"启用VR模式时发生错误\n{str(e)}")
def onExitVR(self):
"""退出VR模式"""
try:
if hasattr(self.world, 'vr_manager') and self.world.vr_manager:
self.world.vr_manager.disable_vr()
# 更新菜单状态
self.enterVRAction.setEnabled(True)
self.exitVRAction.setEnabled(False)
QMessageBox.information(self, "成功", "已退出VR模式")
else:
QMessageBox.warning(self, "错误", "VR管理器不可用")
except Exception as e:
QMessageBox.critical(self, "错误", f"退出VR模式时发生错误\n{str(e)}")
def onShowVRStatus(self):
"""显示VR状态"""
try:
if hasattr(self.world, 'vr_manager') and self.world.vr_manager:
status = self.world.vr_manager.get_vr_status()
status_text = f"""VR系统状态
可用性: {'✅ 可用' if status['available'] else '❌ 不可用'}
初始化: {'✅ 已初始化' if status['initialized'] else '❌ 未初始化'}
启用状态: {'✅ 已启用' if status['enabled'] else '❌ 未启用'}
渲染分辨率: {status['eye_resolution'][0]}x{status['eye_resolution'][1]}
追踪设备数: {status['device_count']}
提示
- 如果VR不可用请确保已安装SteamVR并连接VR头显
- 如果OpenVR库未安装请运行pip install openvr
"""
QMessageBox.information(self, "VR状态", status_text)
else:
QMessageBox.warning(self, "错误", "VR管理器不可用")
except Exception as e:
QMessageBox.critical(self, "错误", f"获取VR状态时发生错误\n{str(e)}")
def onShowVRSettings(self):
"""显示VR设置对话框"""
try:
if hasattr(self.world, 'vr_manager') and self.world.vr_manager:
dialog = self.createVRSettingsDialog()
dialog.exec_()
else:
QMessageBox.warning(self, "错误", "VR管理器不可用")
except Exception as e:
QMessageBox.critical(self, "错误", f"打开VR设置时发生错误\n{str(e)}")
def createVRSettingsDialog(self):
"""创建VR设置对话框"""
dialog = QDialog(self)
dialog.setWindowTitle("VR设置")
dialog.setModal(True)
dialog.resize(400, 300)
layout = QVBoxLayout(dialog)
# VR状态显示
status_group = QGroupBox("VR状态")
status_layout = QVBoxLayout()
if hasattr(self.world, 'vr_manager') and self.world.vr_manager:
status = self.world.vr_manager.get_vr_status()
available_label = QLabel(f"VR可用性: {'' if status['available'] else ''}")
available_label.setStyleSheet(f"color: {'green' if status['available'] else 'red'};")
status_layout.addWidget(available_label)
enabled_label = QLabel(f"VR状态: {'已启用' if status['enabled'] else '未启用'}")
enabled_label.setStyleSheet(f"color: {'green' if status['enabled'] else 'gray'};")
status_layout.addWidget(enabled_label)
resolution_label = QLabel(f"渲染分辨率: {status['eye_resolution'][0]}x{status['eye_resolution'][1]}")
status_layout.addWidget(resolution_label)
status_group.setLayout(status_layout)
layout.addWidget(status_group)
# 渲染设置
render_group = QGroupBox("渲染设置")
render_layout = QFormLayout()
# 渲染质量
quality_combo = QComboBox()
quality_combo.addItems(["", "", "", "超高"])
quality_combo.setCurrentText("")
render_layout.addRow("渲染质量:", quality_combo)
# 抗锯齿
aa_combo = QComboBox()
aa_combo.addItems(["", "2x", "4x", "8x"])
aa_combo.setCurrentText("4x")
render_layout.addRow("抗锯齿:", aa_combo)
render_group.setLayout(render_layout)
layout.addWidget(render_group)
# 性能设置
perf_group = QGroupBox("性能设置")
perf_layout = QFormLayout()
# 刷新率
refresh_combo = QComboBox()
refresh_combo.addItems(["72Hz", "90Hz", "120Hz", "144Hz"])
refresh_combo.setCurrentText("90Hz")
perf_layout.addRow("刷新率:", refresh_combo)
# 异步重投影
async_check = QCheckBox("启用异步重投影")
async_check.setChecked(True)
perf_layout.addRow("", async_check)
perf_group.setLayout(perf_layout)
layout.addWidget(perf_group)
# 按钮
button_layout = QHBoxLayout()
apply_button = QPushButton("应用")
ok_button = QPushButton("确定")
cancel_button = QPushButton("取消")
button_layout.addWidget(apply_button)
button_layout.addStretch()
button_layout.addWidget(ok_button)
button_layout.addWidget(cancel_button)
layout.addLayout(button_layout)
# 连接信号
apply_button.clicked.connect(lambda: self.applyVRSettings(dialog))
ok_button.clicked.connect(dialog.accept)
cancel_button.clicked.connect(dialog.reject)
return dialog
def applyVRSettings(self, dialog):
"""应用VR设置"""
try:
# 这里可以实现设置的保存和应用逻辑
QMessageBox.information(dialog, "成功", "VR设置已应用")
except Exception as e:
QMessageBox.critical(dialog, "错误", f"应用VR设置时发生错误\n{str(e)}")
def setup_main_window(world,path = None):
"""设置主窗口的便利函数"""
app = QApplication.instance()