Tone Mapping
HDR images
High Dynamic Range (HDR) images contain information about the luminosity in a scene that can be well outside the range of standard file formats like PNG or JPEG.
In non-HDR images (Low Dynamic Range (LDR) images), the value for any particular pixel usually lie in the range [0, 1] or [0, 255]. In HDR images, there is no real limit to the upper value.
Other considerations for HDR image
- There may be more than 4 channels of data. EXR images are often used to stored extra information such as rendering depth, UV texture coordinates, coordinates etc
- It will often be necessary to extract the channels of interest for plotting. In the EXR image format, channels are stored alphabetically, which means that careful selection will be needed to extract RGBA channels in order.
Tone mapping is the process of manipulating the HDR values into a lower dynamic range - usually with values in the range [0, 1].
There is no “correct” tone mapping operation - just different techniques depending on your requirements.
Example image
When an EXR image is loaded it is just a numeric array of data.
file <- system.file("image/rstats.exr", package = "picohdr")
im <- picohdr::read_exr(file)
dim(im)
#> [1] 270 360 25The names on the array indicate the channel names. EXR stored all channels in alphabetical order.
Here there are RGBA channels but also extra information from the 3D
renderer which created this image e.g. the u and
v texture coordinates at each output pixel.
dimnames(im)[[3]]
#> [1] "Albedo.B" "Albedo.G" "Albedo.R"
#> [4] "B" "G" "N.X"
#> [7] "N.Y" "N.Z" "Ns.X"
#> [10] "Ns.Y" "Ns.Z" "P.X"
#> [13] "P.Y" "P.Z" "R"
#> [16] "RelativeVariance.B" "RelativeVariance.G" "RelativeVariance.R"
#> [19] "Variance.B" "Variance.G" "Variance.R"
#> [22] "dzdx" "dzdy" "u"
#> [25] "v"A peek at the first plane in the array shows that it is just numeric data
im[1:5, 1:5, 1]
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 0.4577637 0.4567871 0.4526367 0.4565430 0.4577637
#> [2,] 0.4567871 0.4560547 0.4548340 0.4538574 0.4550781
#> [3,] 0.4560547 0.4521484 0.4553223 0.4533691 0.4553223
#> [4,] 0.4562988 0.4567871 0.4541016 0.4521484 0.4543457
#> [5,] 0.4548340 0.4567871 0.4548340 0.4528809 0.4577637We can create a standard RGB array from this image data
If this was a normal image loaded from PNG or JPEG, we could do the
following to view it in R - but this code will cause an error with an
HDR image as there is no guarantee that the pixel values lie between 0
and 1 (which is what as.raster() requires)
plot(as.raster(rgb_arr))
#> Error in rgb(t(x[, , 1L]), t(x[, , 2L]), t(x[, , 3L]), maxColorValue = max): color intensity 1.00977, not in [0,1]library(ggplot2)
df <- array_to_df(rgb_arr)
ggplot(df) +
geom_density(aes(value, group = channel, colour = channel)) +
theme_bw() +
theme(legend.position = 'bottom') +
coord_cartesian(xlim = c(0, 1.5), ylim = c(0, 5)) +
scale_color_manual(values = c('blue', 'green', 'red')) +
labs(
title = "Raw HDR pixel values",
subtitle = "Some values greater than 1.0 in this image"
)
Tone mapping is then the process by which these pixel values can be shifted, truncated, adapted to squeeze the pixel values into the range [0, 1] so that we can view it properly in our low dynamic range R session.
Tone-map by clamping
A simple technique for tone-mapping is to just clamp the values at [0, 1] with values outside this range being pulled back to these limits.
If we do this, then the image is now viewable as a raster, but some parts of it look blown out and overexposed.
rgb_clamped <- rgb_arr |>
adj_clamp(lo = 0, hi = 1) |>
adj_gamma()
oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_clamped))
Tone-map by linear rescaling
rgb_rescale <- rgb_arr |>
adj_rescale(lo = 0, hi = 1) |>
adj_gamma()
oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_rescale))
Tone-map with Reinhard’s technique
At its core, Reinhard’s technique is a non-linear rescaling of the values back into the range [0, 1].
rgb_rh1<- rgb_arr |>
tm_reinhard() |>
adj_gamma()
oldpar <- par(mai = c(0, 0, 0, 0))
plot(as.raster(rgb_rh1))
