网络建设文章网站,wordpress模板是什么意思,网络营销是什么传销吗,wordpress百度主动提交#x1f3ae; HY-World 1.5: 具有实时延迟与几何一致性的交互式世界建模系统框架
#x1f4d6; 简介
尽管HY-World 1.0能够生成沉浸式3D世界#xff0c;但其依赖耗时的离线生成流程且缺乏实时交互能力。HY-World 1.5通过WorldPlay填补了这一空白——这是一个支持实时交互式… HY-World 1.5: 具有实时延迟与几何一致性的交互式世界建模系统框架 简介尽管HY-World 1.0能够生成沉浸式3D世界但其依赖耗时的离线生成流程且缺乏实时交互能力。HY-World 1.5通过WorldPlay填补了这一空白——这是一个支持实时交互式世界建模的流式视频扩散模型具备长期几何一致性解决了现有方法在速度与内存之间的权衡难题。我们的模型依托四大核心设计实现突破双重动作表征通过键盘鼠标输入实现鲁棒的动作控制重构上下文记忆动态重组历史帧上下文采用时序重构技术保留几何关键帧的可访问性有效缓解记忆衰减世界罗盘创新的强化学习后训练框架直接提升长时序自回归视频模型的动作跟随性与视觉质量上下文强制蒸馏专为记忆感知模型设计的新蒸馏方法通过对齐师生模型的记忆上下文保留远程信息处理能力在实现实时速度的同时避免误差漂移。综合而言HY-World 1.5能以24FPS生成具备卓越一致性的长时序流式视频性能优于现有技术。该模型在多样化场景中展现出强大泛化能力支持第一人称与第三人称视角兼容写实与风格化环境可应用于3D重建、可触发事件及无限世界扩展等多元场景。系统概览HY-World 1.5开源了实时世界模型的系统化训练框架涵盖数据、训练、推理部署全流程。技术报告详细披露了模型预训练、中期训练、强化学习后训练及记忆感知蒸馏的具体实现并介绍了一系列降低网络传输延迟与模型推理延迟的工程技术最终为用户提供实时流式推理体验。推理流程给定一张图像或描述世界的文本提示我们的模型执行下一片段16个视频帧预测任务根据用户动作生成未来视频。对于每个片段的生成我们会动态重组过去片段的上下文记忆以确保长期的时间和几何一致性。 使用示例我们开源了双向扩散模型和自回归扩散模型的推理代码。对于提示词改写推荐使用Gemini或通过vLLM部署的模型。当前代码库仅支持兼容vLLM API的模型。如需使用Gemini需自行实现接口调用。具体实现可参考HunyuanVideo-1.5项目。推荐使用generate_custom_trajectory.py脚本生成自定义相机运动轨迹。exportT2V_REWRITE_BASE_URLyour_vllm_server_base_urlexportT2V_REWRITE_MODEL_NAMEyour_model_nameexportI2V_REWRITE_BASE_URLyour_vllm_server_base_urlexportI2V_REWRITE_MODEL_NAMEyour_model_namePROMPTA paved pathway leads towards a stone arch bridge spanning a calm body of water. Lush green trees and foliage line the path and the far bank of the water. A traditional-style pavilion with a tiered, reddish-brown roof sits on the far shore. The water reflects the surrounding greenery and the sky. The scene is bathed in soft, natural light, creating a tranquil and serene atmosphere. The pathway is composed of large, rectangular stones, and the bridge is constructed of light gray stone. The overall composition emphasizes the peaceful and harmonious nature of the landscape.IMAGE_PATH./assets/img/test.png# Now we only provide the i2v model, so the path cannot be NoneSEED1ASPECT_RATIO16:9RESOLUTION480p# Now we only provide the 480p modelOUTPUT_PATH./outputs/MODEL_PATH# Path to pretrained hunyuanvideo-1.5 modelAR_ACTION_MODEL_PATH# Path to our HY-World 1.5 autoregressive checkpointsBI_ACTION_MODEL_PATH# Path to our HY-World 1.5 bidirectional checkpointsAR_DISTILL_ACTION_MODEL_PATH# Path to our HY-World 1.5 autoregressive distilled checkpointsPOSE_JSON_PATH./assets/pose/test_forward_32_latents.json# Path to the customized camera trajectoryNUM_FRAMES125# Configuration for faster inference# For AR inference, the maximum number recommended is 4. For bidirectional models, it can be set to 8.N_INFERENCE_GPU4# Parallel inference GPU count.# Configuration for better qualityREWRITEfalse# Enable prompt rewriting. Please ensure rewrite vLLM server is deployed and configured.ENABLE_SRfalse# Enable super resolution. When the NUM_FRAMES 121, you can set it to true# inference with bidirectional modeltorchrun --nproc_per_node$N_INFERENCE_GPUgenerate.py\--prompt$PROMPT\--image_path$IMAGE_PATH\--resolution$RESOLUTION\--aspect_ratio$ASPECT_RATIO\--video_length$NUM_FRAMES\--seed$SEED\--rewrite$REWRITE\--sr$ENABLE_SR--save_pre_sr_video\--pose_json_path$POSE_JSON_PATH\--output_path$OUTPUT_PATH\--model_path$MODEL_PATH\--action_ckpt$BI_ACTION_MODEL_PATH\--few_stepfalse\--model_typebi# inference with autoregressive model#torchrun --nproc_per_node$N_INFERENCE_GPU generate.py \# --prompt $PROMPT \# --image_path $IMAGE_PATH \# --resolution $RESOLUTION \# --aspect_ratio $ASPECT_RATIO \# --video_length $NUM_FRAMES \# --seed $SEED \# --rewrite $REWRITE \# --sr $ENABLE_SR --save_pre_sr_video \# --pose_json_path $POSE_JSON_PATH \# --output_path $OUTPUT_PATH \# --model_path $MODEL_PATH \# --action_ckpt $AR_ACTION_MODEL_PATH \# --few_step false \# --model_type ar# inference with autoregressive distilled model#torchrun --nproc_per_node$N_INFERENCE_GPU generate.py \# --prompt $PROMPT \# --image_path $IMAGE_PATH \# --resolution $RESOLUTION \# --aspect_ratio $ASPECT_RATIO \# --video_length $NUM_FRAMES \# --seed $SEED \# --rewrite $REWRITE \# --sr $ENABLE_SR --save_pre_sr_video \# --pose_json_path $POSE_JSON_PATH \# --output_path $OUTPUT_PATH \# --model_path $MODEL_PATH \# --action_ckpt $AR_DISTILL_ACTION_MODEL_PATH \# --few_step true \# --num_inference_steps 4 \# --model_type ar 评估结果HY-World 1.5 在多项量化指标上超越现有方法包括不同视频时长的重建指标和人工评估结果。ModelReal-timeShort-termLong-termPSNR⬆SSIM⬆LPIPS⬇R d i s t R_{dist}Rdist⬇T d i s t T_{dist}Tdist⬇PSNR⬆SSIM⬆LPIPS⬇R d i s t R_{dist}Rdist⬇T d i s t T_{dist}Tdist⬇CameraCtrl❌17.930.5690.2980.0370.34110.090.2410.5490.7331.117SEVA❌19.840.5980.3130.0470.22310.510.3010.5170.7211.893ViewCrafter❌19.910.6170.3270.0290.5439.320.2710.6611.5733.051Gen3C❌21.680.6350.2780.0240.47715.370.4310.4830.3570.979VMem❌19.970.5870.3160.0480.21912.770.3350.5420.7481.547Matrix-Game-2.0✅17.260.5050.3830.2870.8439.570.2050.6312.1252.742GameCraft❌21.050.6390.3410.1510.61710.090.2870.6142.4973.291Ours (w/o Context Forcing)❌21.270.6690.2610.0330.15716.270.4250.4950.6110.991Ours (full)✅21.920.7020.2470.0310.12118.940.5850.3710.3320.797 更多示例https://github.com/user-attachments/assets/6aac8ad7-3c64-4342-887f-53b7100452edhttps://github.com/user-attachments/assets/531bf0ad-1fca-4d76-bb65-84701368926dhttps://github.com/user-attachments/assets/f165f409-5a74-4e19-a32c-fc98d92259e1 引用article{hyworld2025, title{HY-World 1.5: A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency}, author{Team HunyuanWorld}, journal{arXiv preprint}, year{2025} } article{worldplay2025, title{WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Model}, author{Wenqiang Sun and Haiyu Zhang and Haoyuan Wang and Junta Wu and Zehan Wang and Zhenwei Wang and Yunhong Wang and Jun Zhang and Tengfei Wang and Chunchao Guo}, year{2025}, journal{arXiv preprint} } inproceedings{wang2025compass, title{WorldCompass: Reinforcement Learning for Long-Horizon World Models}, author{Zehan Wang and Tengfei Wang and Haiyu Zhang and Wenqiang Sun and Junta Wu and Haoyuan Wang and Zhenwei Wang and Hengshuang Zhao and Chunchao Guo and Zhou Zhao}, journal {arXiv preprint}, year 2025 } 致谢我们要感谢 HunyuanWorld、HunyuanWorld-Mirror、HunyuanVideo 和 FastVideo 的杰出贡献。