Loading...
Research Focus

Neural Holography & 3D Vision

The 3D Video Coding (3DV) team conducts research across two major frontiers of next-generation 3D media: phase-aware hologram compression (Learned-based/hand-crafted based) and 3D Gaussian Splatting (3DGS)

Holography

Holography reproduces the full light field of objects in space, eliminating the vergence-accommodation conflict of conventional 3D displays. Phase hologram data, however, has two properties that make standard image processing fundamentally unsuitable:

  • 2π-Periodicity (Circularity): Phase values are defined on a circular manifold, causing conventional difference, clipping, and loss computations to fail.
  • Randomness: Phase distributions approximate uniform distributions, frustrating the statistical assumptions of existing compression models and neural networks.

3DV formalizes both properties at every layer of the pipeline — from conventional codec extensions (HEVC, VVC) to end-to-end neural video compression (NHVC), phase-aware activation functions (FlexMU), and neural hologram generation under variable propagation conditions (PaD). The team’s work spans 17+ publications and 22+ patent applications, with industry partnerships at Samsung and ETRI.

3D Gaussian Splatting

3DGS represents 3D scenes as millions of learnable Gaussian ellipsoids, enabling real-time novel-view synthesis at >100 fps — far exceeding NeRF in speed.

Neural Holography & 3D Vision
Research Timeline

Research Highlights

HEVC/VVC Extension for Phase Holograms
2022–Present Optics Express 2023

Standard video codecs treat all signals as living on a linear number line.

Phase signals live on a circle, so conventional difference computation, clipping, and RD optimization all produce incorrect results.

We introduced two foundational operations and integrated them into HEVC’s RDO, residual computation, and sample reconstruction processes:

  • Shorter Circular Difference: Resolves the distance ambiguity between two phase values by always selecting the shorter arc on the phase circle.
  • Circular Clipping: Wraps out-of-range values back onto the valid phase interval instead of hard-clipping to the boundary.

Applied across seven holographic test sequences, the proposed method achieves –63.3% BD-Rate (phase domain) and –65.5% BD-Rate (NR domain) compared to standard HEVC. VVC extensions with SAO and deblocking filters are submitted.

NHVC: Neural Holographic Video Compression with Scalable Architecture
2022–2025 IEEE VR 2024 (Oral)

Conventional approaches treat hologram generation and compression as separate pipelines. NHVC is the first unified end-to-end neural holographic video codec, jointly optimizing generation and compression within a single scalable architecture.

Key design choices:

  • Deformable convolution for large receptive fields and implicit temporal motion handling.
  • Strong band-limiting to suppress high-order diffraction noise.
  • Scalable architecture: a single model supports image/video generation and compression, switchable at inference time.

NHVC achieves >33 dB NR-PSNR quality, outperforming cascaded baselines by –75.8% BD-Rate (image) and –75.6% BD-Rate (video).

PDA: Phase Distribution Alignment
ACM MM 2025 (Oral)

Phase holograms are shift-invariant: adding a global constant (mod 2π) leaves the 3D reconstruction unchanged. This means two physically identical holograms can show arbitrarily large phase-domain error — making direct phase comparison unreliable.

PDA (Phase Distribution Alignment) maps each hologram to a unique canonical form before computing error, resolving this ambiguity. PDA-based metrics show consistently higher correlation with NR-domain metrics across 13 noise types and 5 strength levels.

PDA has three demonstrated applications:

  • Quality evaluation: reliable phase-domain quality assessment without computationally expensive numerical reconstruction.
  • Pre-processing for conventional codecs: –14% BD-Rate with HEVC, –25.41% BD-Rate with VVC phase hologram extension.
  • Training loss for neural models: +0.37 dB generation quality and –11% BD-Rate vs. NHVC, with faster training convergence.
PaD: Propagation as Data
IEEE VR Workshop 2024

Existing neural hologram generators are optimized for a fixed propagation distance.

Changing the distance, wavelength, or pixel pitch requires full retraining — impractical for real deployments where hardware specifications vary.

PaD (Propagation as Data) treats the propagation kernel as input data rather than a fixed physical operation baked into the architecture. A dedicated Kernel Encoder ingests the propagation kernel at inference time, enabling a single trained model to generalize to arbitrary propagation conditions without retraining.

Architecture highlights: pre-trained LDM image encoder for stable feature extraction; FFC (Fast Fourier Convolution) for feature–kernel fusion; enlarged receptive field for global scene structure modeling.

Publications

Related Papers & Contributions

TitleVenueYear
Phase Distribution Matters: On the Importance of Phase Distribution Alignment (PDA) in Holographic ApplicationsProceedings of the 33rd ACM International Conference on Multimedia. 2025.2025DOI
Distribution-Shifting: Novel phase-distortion metrics for hologram processing2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)2024DOI
NHVC: Neural Holographic Video Compression with Scalable Architecture2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)2024DOI
P-Hologen: An End-to-End Generative Framework for Phase-Only HologramsPacific Conference on Computer Graphics and Applications (Pacific Graphics)2024DOI
Propagation as Data (PaD): Neural phase hologram generation with variable distance support2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)2024DOI
VVC 기반의 위상 홀로그램 압축2024년 한국방송미디어 공학회 하계학술대회 학부생 논문2024
위상 홀로그램 압축 기술 동향실감미디어 디스플레이 및 표준화 동향2024
위상 홀로그램의 픽셀피치 변화에 따른 적정 전파거리에 대한고찰방송공학회논문지2024DOI
DHM을 위한 간섭무늬 압축 방법과 위상 압축 방법의 성능 비교2023년 1월 홀로그래픽 신호처리 특집호 방송공학회논문지2023DOI
HEVC extension for phase hologram compressionOptics Express2023DOI
신경망 기반 블록 단위 위상 홀로그램 이미지 압축2023년 1월 홀로그래픽 신호처리 특집호 방송공학회논문지2023DOI
전파거리에 따른 위상 홀로그램 복원성능 분석 및 BL-ASM 개선 방안 연구2023년 1월 홀로그래픽 신호처리 특집호 방송공학회논문지2023DOI
디지털 홀로그래픽 현미경 데이터를 위한 위상 영상 압축2022년 한국방송미디어 공학회 하계학술대회 대학생 논문2022
복소 홀로그램 표현방식에 따른 압축 성능 분석2022년 한국방송미디어 공학회 하계학술대회 학부생 논문2022
심층 학습 기반 위상 홀로그램 생성2022년 한국방송미디어 공학회 하계학술대회 학부생 논문2022
순환 손실 함수를 이용한 딥러닝 기반 위상 홀로그램 초해상도2021년 한국방송미디어 공학회 추계학술대회 대학생 논문2021
위상 홀로그램 동영상 압축 성능 분석2020년 한국방송미디어 공학회 추계학술대회2020
Back to About