Ambiguity in the frame boundary of the rendered image

[Jinjing98]

Hi, thanks for the great work!

I observe that the rendering results of the method seem to show ambiguity/blurring effect on the boundary of the frame, is there some speicial reason here? I am a bit curious as the work you built upon (4D-GS https://github.com/hustvl/4DGaussians) seems not to display such effect in their results, I assume it is due to some special design in the method while struggle to figure out...

Also I am curious to the visualizer you used in the above left image, is it a 3D pointcloud, or a tilted 2D image?

Thanks in advance!

[yifliu3]

Hi, the blurry effect is perhaps caused by TV loss, and you can remove the loss to check. The left image shows the point cloud reconstructed by the function in render.py. 获取 Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ 发件人: Jinjing98 ***@***.***> 发送时间: Friday, November 29, 2024 1:00:31 AM 收件人: CUHK-AIM-Group/EndoGaussian ***@***.***> 抄送: Subscribed ***@***.***> 主题: [CUHK-AIM-Group/EndoGaussian] Ambiguity in the frame boundary of the rendered image (Issue #32) Hi, thanks for the great work! I observe that the rendering results of the method seem to show ambiguity/blurring effect on the boundary of the frame, is there some speicial reason here? I am a bit curious as the work you built upon (4D-GS https://github.com/hustvl/4DGaussians) seems not to display such effect in their results, I assume it is due to some special design in the method while struggle to figure out... image.png (view on web)<https://github.com/user-attachments/assets/01be2005-3571-4719-9a2a-9e2b0627130c> Also I am curious to the visualizer you used in the above left image, is it a 3D pointcloud, or a tilted 2D image? Thanks in advance! ― Reply to this email directly, view it on GitHub<#32>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVCSCGLFYJD4YP7ZO3KGGV32C5D27AVCNFSM6AAAAABSVRQXECVHI2DSMVQWIX3LMV43ASLTON2WKOZSG4YDENZXGI4TSMQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Jinjing98]

@yifliu3 Hi, thanks for your quick reply! :)

Just tried as you suggested, I disabled the tv_loss is the coarse stage (it is already disabled in fine stage as i followed your default setting of time_smoothness_weight to 0 ). Below are the comparision for your reference(EndoNeRF pulling frame 1):
with coarse tv_loss(by default):

without coarse tv_loss:

Regarding rendering, thanks for enabling the reconstruction visualisation! :D