AllSky header

Project Deep Sky explores DNN approaches to generating photorealistic and physically accurate sky models for artistic, scientific and engineering communities.

arXiv Google Scholar Semantic Scholar dblp

Key Project Milestones:

  • AllSky: Towards Physically-Based Sky-Modeling For Image Based Lighting
  • Icarus: Full Dynamic Range Sky-Modelling For Image Based Lighting
  • Helios: Coming soon!

Research Focus

Sky models are essential for:

  • Artists: Realistic lighting in 3D graphics and rendering with Image-Based Lighting (IBL)
  • Scientists: Understanding natural light’s impact on biological processes including the human circadian rhythm, plant growth, and animal behaviour.
  • Engineering: A wide range of applications including architectural design, urban planning, and solar energy applications

Our work develops DNN approaches to capture the complexity of real-world skies—including sun positioning, atmospheric conditions, and exposure range—enabling artists, researchers, and engineers to control generation through an intuitive set of parameters and/or modalities.

Though conventional Low Dynamic Range (LDR) imagery is suitable for some applications, High Dynamic Range (HDR) imagery is integral to sky-models and the capture of outdoor scenes. HDR images capture a greater range of illumination and, in the particular case of outdoor lighting, can capture the estimated 22 f-stops of exposure necessary for the highlights and shadows of an average real-world outdoor scene. To distinguish the proportionality of scene exposure captured by an HDRI, we define the following:

  • Low Dynamic Range (LDR) Imagery: Display-referenced images with compressed dynamic range which can be clipped and displayed in 8-bit colour.

  • High Dynamic Range (HDR) Imagery: Scene-referenced measures of illumination with uncompressed dynamic range and precision greater than LDR 8-bit colour for later display as LDR. This includes imagery captured by conventional cameras in 12-bit RAW.

  • Extended Dynamic Range (EDR) Imagery: HDR images captured using techniques such as LDR-bracketing for greater exposure range than a singular image from a conventional camera.

  • Full Dynamic Range (FDR) Imagery / Physically-Captured Imagery: HDR images that fully-capture the exposure range of a reference scene without saturation of the exposure range.

We demonstrate the importance of FDR imagery for sky-modeling and the limitations of LDR and HDR paradigms through AllSky. Through Icarus, we demonstrate modelling of the Full Dynamic Range (FDR) of skies, generating photorealistic and physically accurate weathered skies and sun. Project Deep Sky targets interchangeability with physically-captured imagery, facilitating direct use in existing rendering (IBL) pipelines.

Icarus

Full Dynamic Range Sky-Modelling For Image Based Lighting

 Posted on March 5, 2026

[arXiv PrePrint]

Scenes rendered per \(512^2\) environment maps. [Centre] Our DNN sky-model (Fig.\(\textcolor{red}{b}\), per \(\textcolor{red}{R}\textcolor{green}{G}\textcolor{blue}{B}\)-Icarus with Robertson fusion) recreates the illumination, tones and light transmission of real-world Full Dynamic Range imagery (\(\textit{FDR}\) ground truth in Fig.\(\textcolor{red}{a}\)). Icarus accurately models solar illumination for unprecedented lighting directionality (shadows, Fig.\(\textcolor{red}{c}\)). [Border] Icarus enables intuitive user-control over positioning and styling of solar and atmospheric formations.

Icarus

Scenes rendered per \(512^2\) environment maps. [Centre] Our DNN sky-model (Fig.\(\textcolor{red}{b}\), per \(\textcolor{red}{R}\textcolor{green}{G}\textcolor{blue}{B}\)-Icarus with Robertson fusion) recreates the illumination, tones and light transmission of real-world Full Dynamic Range imagery (\(\textit{FDR}\) ground truth in Fig.\(\textcolor{red}{a}\)). Icarus accurately models solar illumination for unprecedented lighting directionality (shadows, Fig.\(\textcolor{red}{c}\)). [Border] Icarus enables intuitive user-control over positioning and styling of solar and atmospheric formations.

[Read More]
Tags: DeepSky Full-Dynamic-Range-Imaging Sky-Modeling Image-Based-Lighting 

AllSky - 3DV'26

Towards Physically-Based Sky-Modeling For Image Based Lighting

 Posted on December 15, 2025

AllSky header

Abstract

Accurate environment maps are a key component to modelling real-world outdoor scenes. They enable captivating visual arts, immersive virtual reality and a wide range of scientific and engineering applications. To alleviate the burden of physical-capture, physically-simulation and volumetric rendering, sky-models have been proposed as fast, flexible, and cost-saving alternatives. In recent years, sky-models have been extended through deep learning to be more comprehensive and inclusive of cloud formations, but recent work has demonstrated these models fall short in faithfully recreating accurate and photorealistic natural skies. Particularly at higher resolutions, DNN sky-models struggle to accurately model the 14EV+ class-imbalanced solar region, resulting in poor visual quality and scenes illuminated with skewed light transmission, shadows and tones. In this work, we propose AllSky, an all-weather sky-model capable of learning the exposure range of Full Dynamic Range (FDR) physically captured outdoor imagery. Our model demonstrates conditional multi-exposure image environment map generation with intuitive user-positioning of solar and cloud formations, extending the current state-of-the-art to user-controlled texturing of atmospheric formations. Through our evaluation, we demonstrate AllSky is interchangeable with FDR physically captured outdoor imagery or parametric sky-models, and illuminates scenes with unprecedented accuracy, photorealism, lighting directionality (shadows), and tones in Image Based Lighting (IBL).

[Read More]
Tags: DeepSky Full-Dynamic-Range-Imaging Sky-Modeling Image-Based-Lighting