Computer Graphics
TU Braunschweig

PlenopticPoints: Rasterizing Neural Feature Points for High-Quality Novel View Synthesis


PlenopticPoints: Rasterizing Neural Feature Points for High-Quality Novel View Synthesis

This paper presents a point-based, neural rendering approach for complex real-world objects from a set of photographs.

Our method is specifically geared towards representing fine detail and reflective surface characteristics at improved quality over current state-of-the-art methods. From the photographs, we create a 3D point model based on optimized neural feature points located on a regular grid. For rendering, we employ view-dependent spherical harmonics shading, differentiable rasterization, and a deep neural rendering network. By combining a point-based approach and novel regularizers, our method is able to accurately represent local detail such as fine geometry and high-frequency texture while at the same time convincingly interpolating unseen viewpoints during inference.

Our method achieves about 7 frames per second at 800×800 pixel output resolution on commodity hardware, putting it within reach for real-time rendering applications.

 

Resources

Please send an E-Mail to hahlbohm@cg.cs.tu-bs.de if you would like access to our source code, pretrained models, as well as the images used for evaluation.


Author(s): Florian Hahlbohm, Moritz Kappel, Jan-Philipp Tauscher, Martin Eisemann, Marcus Magnor
Published: September 2023
Type: Article in conference proceedings
Book: Proc. Vision, Modeling and Visualization (VMV) (The Eurographics Association)
ISBN: 978-3-03868-232-5
DOI: 10.2312/vmv.20231226
Presented at: Vision, Modeling and Visualization (VMV) 2023
Project(s): Immersive Digital Reality  Point-Based Neural Rendering 

BibTex PDF Link 



@inproceedings{hahlbohm2023plenopticpoints,
  title = {PlenopticPoints: Rasterizing Neural Feature Points for High-Quality Novel View Synthesis},
  author = {Hahlbohm, Florian and Kappel, Moritz and Tauscher, Jan-Philipp and Eisemann, Martin and Magnor, Marcus},
  booktitle = {Proc. Vision, Modeling and Visualization ({VMV})},
  organization = {Eurographics},
  isbn = {978-3-03868-232-5},
  doi = {10.2312/vmv.20231226},
  editor = {T. Grosch and M. Guthe},
  pages = {53--61},
  month = {Sep},
  year = {2023}
}

Authors