I appreciate a good graph. This stems from my love of images, pictorial or abstract. Columns of numbers are senseless. A graph uncovers the nascent relationships within a data set through pragmatic and precise organization.
I create a lot of graphs and require my students to do the same. I advocate for learning this skill because the cinematic and photographic arts are full of numerical relationships that are opaque until delineated. Learning Adams and Archer’s Zone System in order to improve my black-and-white photography made such scientifico-mathematical analysis necessary. I owe a huge debt to Phil Davis’ excellent Beyond the Zone System (1988). This book taught me how to produce my own sensitometry graphs and connect the results to my materials and processes. (I wrote about my personal journey to learn sensitometry and the benefits it conferred on my image-making process in this article.)
Graph literacy, the ability to read and interpret the information presented in graphs, is critical to understanding image fidelity and quality. Sensitometry, the science of tone reproduction is replete with graphs, many which connect to each other to demonstrate the chain of tone reproduction throughout an imaging process (Davis, 1988). Colorimetric coordinates allow one to identify if two different radiant spectra produce the same visual impression (Koenderink, et al., 2020, pp. 3). The modulation transfer function (MTF) helps reveal the quality of sharpness in a lens or sensor (Kiening, 2008). Graphing a sensor’s signal-to-noise ratio or information capacity opens a complex world of quantum interactions and communication problems (Dainty & Shaw, 1974). (N.B. – These citations are my personal recommendations for learning more on the subject, not to a foundational piece of research on the topic.)
Currently, I am trying to find substantive information about HDR (high dynamic range) post-processes and displays. I found a set of videos on the American Society of Cinematographers (ASC) website where artists discuss their HDR experiences. Amongst the frequently invoked and vague descriptors “better” and “more” there is a rare moment of hard data. During Erik Messerschmidt’s interview I was excited to finally see the same image displayed on an SDR and HDR waveform. (See Episode 2 – HDR from Set to Post at approximately 30:00 to 35:00 minutes.) My elation receded once I leaned in closer to the data. What I observed was not so much a case for HDR, but a series of critical lessons in the display and interpretation of information from graphs.
Preliminary Notes
A few points of instruction on this article:
This is not so much an analysis of SDR or HDR, although, the savvy image-maker will certainly take away important points. I also want to set aside questions about the author’s intentions. Or rather, I think the best of intentions should be assumed. The ASC and the cinematographers interviewed are doing their best to wrap their head around a new and complicated discussion. (One not helped by marketing departments executing corporate interests.) The waveform information was most likely shown in a problematic manner due to constraints of time and resources.
Even though HDR is both an output transform and color space, I focus on the transfer of code values to luminance from the display. This discussion will be black-and-white. (Perhaps even metaphorically speaking.)
I consider the terms waveform and graph synonymously. Waveforms are a form of graph since they display the horizontal axis of the image against the output code value or monitor luminance.
As I was proofreading this piece Steve Yedlin published his Debunking “HDR” video. This was an eery moment of synchronicity. While his video focuses on the technology and equipment, I believe his conclusions support the graphic evidence of this article.
Identifying the Graph’s Purpose: A look at the HDR waveform
Graphs are created to express a relationship. Your first task, as graph interpreter, is to identify the labels of the x and y-axis (or abscissa and ordinate if you happen to be an Ancient Roman surveyor). For example, the HDR waveform plots the horizonal axis of the image to the luminance from the screen. This makes sense, we wish to understand the relationship between the objects in the image to the light output from the monitor for practical and artistic purposes.
The HDR waveform. Notice that the labeled identification in the video conflicts with the readings on the y-axis. The window is at 225 cd·m^-2 whereas the wall is at 2 cd·m^-2.
Once the relationship is recognized (image to luminance), identify the units of the axes. The x-axis is a spatial scale – the position of data within the image from left to right. The y-axis is using the SI unit of luminance, candelas per meter squared (cd·m^-2) or ‘nits’. The derivation of this unit is not important at this moment. We only to know two critical facts:
The higher the luminance the brighter the display and vice versa.
Luminance adds linearly, as per Lambert’s 1st Law of Photometrics (Lambert, 1760/2001, p. 18, §51).
Finally, we determine whether the graph provides any emergent analysis. An important metric mentioned by Mr. Messerschmidt is translating this information into stops for comparison – particularly because of the ‘huge difference’ between the wall and window (American Cinematographer, 2021, 35:00).
A ‘stop’ is a flexible metric used in the photographic arts and is a change to a quantity by a factor of two. Stops can be applied to temporal measures (shutter speed, frame rate), physical measures (shutter angle, sensor sensitivity), or various scales of light intensity (f-number, footcandles, cd·m^-2, etc.). Stops are a useful measuring stick to apply against all the different scales demarcated as they are by disparate numerical conventions. The reason the visual arts rely on changes of intensity by a factor of two is due to our perception of brightness. (I explain in more detail in this article.)
None of the graphs in this video are marked in stops, but this number can be derived from any two y-axis values. The range of luminance in stops can be calculated from this formula:
The difference in stops can be calculated by taking the log of two different luminance values divided by log(2). L_1 is the higher luminance and L_2 the lower luminance. This formula is extremely useful - so memorize it!
The HDR waveform introduces the two objects in the frame that will serve our analysis: the white window and the dark wall. While Mr. Messerschmidt claims that the luminance of the window approached 1,000 cd·m^-2, he identifies the display output of the window as approximately 650 cd·m^-2. The video’s editor kindly includes an arrow to indicate the window on the waveform along with a label of ‘~650 nits’. However, this data actually intersects the y-axis at around 225 cd·m^-2. This discrepancy by more than a factor of two provides our first substantial lesson in graph literacy:
Lesson #1: Check all reported data against the graph and vice versa.
While our purpose is not to speculate on why or how this occurred, there is a highly innocuous reason this could have occurred – the scales were misinterpreted as being linear. This detail provides our second lesson:
Lesson #2: Identify the scale of each axis. Is it linear or logarithmic?
Photometric quantities are linear, but the photographic industry relies on geometric sequences – the factor of two of the ‘stop’, for example. Look closely and you will notice that the HDR waveform uses a logarithmic scale to conveniently compress the data into a visually useful relationship. In this example, it is using a base-10 logarithmic scale. (Identifiable by the sequence 0.1, 1.0, 10, 100, 1000, etc.) Log scales are useful because of the fact that changes to our sensation of brightness diminishes as the light stimulus increases. In other words, our sensitivity to changes of light in a dim room are greater than changes to light in bright sunlight. A logarithmic scale helps the data accord with our perception – the lower region of luminance is expanded and the upper region compressed. Take a look at the small yellow interval marks on the waveform.
If we wanted the waveform to accord more appropriately with perceptual scales, then the luminance would be arrayed along a base-2 scale. (A sequence such as 2, 4, 8, etc. or in logarithmic values such as 0.3, 0.6, 0.9, etc.) By using this interval, which matches the factor of two change in photographic materials, then every ‘stop’ would be an equal distance along the y-axis. I marked the HDR waveform in ‘stops’ below (selecting 1,000 cd·m^-2 as an arbitrary starting point) to demonstrate how stops fit along a base-10 scale. Notice that each change in stop is just a factor of two.
The scale of luminance in this waveform is displayed using a logarithmic scale - specifically a base-10 log scale. On the left I marked changes in luminance by a factor of two, a ‘stop’, from 1,000 cd·m^-2 to 1 cd·m^-2. Notice that the steps are not equal, a base-2 log scale would make every ‘stop’ an equal step on the graph.
Before moving on to the SDR waveform, let us calculate the number of stops between the window and wall in this example. The difference between the window and wall is approximately 6.8 stops. (Log10(225/2)/0.3 = 6.8)
Applying Our Lessons: A look at the SDR waveform
The SDR waveform. Rather than using the obsolete unit IRE, I identified the numbers in 10-bit code values (CV_10-bit) and normalized code values (CV_N).
Applying our first two lessons to the SDR waveform reveals two important differences.
The y-axis is not in luminance, but 10-bit code values (0 to 1023).
The code values follow a linear sequence and are displayed on a linear scale.
As in the HDR waveform, two scenic elements are identified; the window and the wall. However, the labels curiously introduce a different unit – IRE. This brings us to another lesson:
Lesson #3: Refer to data in the graph using the same units as the scales in the graph. If two different units are comparable, label the intervals with two scales.
Since 100 IRE corresponds to the highest code value, and 0 IRE to code value 0, the waveform could have been labeled in both. If two scales correspond, both can be labeled on opposite sides of the graph for ease of use as well as clarity. In the case of this waveform, it would be better to abandon IRE for normalized code values.
The formula for normalized code values (CV_N). Simply divide the measured code value (CV_M) by the total bit-depth (CV_Bit-depth).
You should notice that the normalized code value is just the fraction of the total bit-depth. This scale allows for easy translation to any bit-depth. (IRE is a bit irrelevant, as it was originally the scale of a composite video signal in millivolts (mV). The use of IRE today is comparable to calling film sensitivity ASA despite the fact that the American Standards Association changed its name.) The IRE label is not an egregious error, but I would advocate that it is better to call a spade a spade - the data should be referenced in code values or normalized code values. The window is at a normalized code value of .95 (975 CV_10-bit) and the wall at .16 (160 CV_10-bit).
In an effort to relate the y-axis of the SDR to the HDR waveform, the author’s of the video introduce a troublesome short circuit. The window is labeled as ‘~ 100 IRE / 100 nits’ and the wall as ‘~ 15 IRE / 15 nits’. The problem is that IRE and luminance are not in a 1:1 relation. There is a gamma transform applied to the signal. This provides the crucial lesson that:
Lesson #4: The identification of data points, whether using units of the axes or not, must be expressed using the correct relationship.
While we do not know the display gamma in this instance, it is safe to assume from the surrounding discussion that it is 2.4. This allows us to calculate the percentage of display output as follows:
Window: 0.95^2.4 = 0.88
Wall: 0.16^2.4 = 0.012
Since their display white is set at 100 cd·m^-2, this means the window is at 88 cd·m^-2 and the wall at 1.2 cd·m^-2. It is absolutely certain that the wall is not at 15 cd·m^-2. I used my Gossen monitor meter on my own calibrated monitor to determine that a CV_N of 0.46 produced 15 cd·m^-2. I then used Photoshop to assemble four achromatic tones to demonstrate the difference between code values and luminance for an SDR monitor at a gamma of 2.4.
With a monitor calibrated so that the maximum code value produces 100 cd·m^-2, I adjusted the code values in Photoshop until my monitor meter read 15 cd·m^-2. This luminance is close to middle gray! Out of curiosity I set the normalized code value to 0.15 and read the luminance from the screen. Notice that even a high quality monitor will never be truly black, in my case it read 0.2 cd·m^-2 with a code value of 0.
Now, calculate the range of stops for the window and wall in SDR. ((Log_10(88/1.2)/0.3) This is only 6.2 stops, as compared to the 6.8 stops of the HDR display. The difference in stops between the HDR and SDR luminances is somewhere between half to one stop, allowing for errors. The difference makes sense in light of vision research and sensitometry. However, this is not the subject of this article and may be addressed in a future piece of writing. (I have already explained this in my, as of now, unpublished book.)
Comparing Graphs:
The HDR waveform is on the left, the SDR waveform is on the right. This is how the two are displayed in the ASC video.
The raison d’etre of the ASC video is to compare HDR and SDR. To facilitate this comparison the two waveforms are displayed side-by-side. However, this is taxing on the viewer because the graphs use different units and scales. Placing them side by side is confusing and, if anything, makes the SDR waveform appear to encompass a greater range. An intuitively obvious takeaway is:
Lesson #5: When comparing two different sets of data, place both on the same graph and use the same scales.
Since much of the conversation in this video is centering on the display’s luminance it is best to plot both in cd·m^-2. Despite the fact that SDR output transforms the data in relation to a maximum signal, we can still select a common and useful luminance. The discussion in the video treats monitor white as 100 cd·m^-2, and this is a common recommendation. (An in-depth analysis could adjust the SDR information to a variety of luminance levels.)
Adjusting the size of the SDR graph to fit from 100 to 0 cd·m^-2 is immensely helpful to see the relationship between the two different systems.
The SDR graph on the right has been compressed so that its maximum and minimum code values match 100 and 0 cd·m^-2 from the HDR scale on the far left. However, this is still not an appropriate comparison because the SDR y-axis scale is still linear and not logarithmic.
However, remember that the SDR waveform uses a linear scale on the y-axis. I used Photoshop to stretch and compress bands of the SDR waveform to roughly emulate a log-10 scale. This is critical because it brings luminance into proper relationship between the two graphs, and we can compare apples to apples.
Now, the SDR waveform has been (by dint of force) stretched to a base-10 log scale. Notice how similar the two outputs actually are except for the location of the window luminance, which is 1.4 stops brighter.
Finally, the information is consistent. The harmony in graphic expression allows the interrogator, whether artist or engineer, to correctly interpret the two systems. In this case, we can see that the HDR and SDR image are not too dissimilar. This makes sense because tone reproduction in the visual arts is based on relative differences in tones. The majority of the information in the frame, regardless of how bright the monitor may be, must maintain its relative proportions. For example, notice that as the window diminishes in luminance by about 1.4 stops, that the wall falls in luminance by about 1 stop. The shift in the tones is due in part to the adaptation of the visual system viewing the image, and partly by aesthetic interests harnessing the fact that HDR affords a brighter white. (However, notice that the window does not need to be much brighter to produce a feeling of brilliance in the viewer.)
The Case for Graph Literacy
These five simple lessons are useful for the artist who is performing their own technical analyses. I find that graphs of data help to buttress or question a visual assessment of image fidelity and quality (Silverstein & Farrell, 1996). If the image information supports our visual impression, than we can feel confident in connecting a technique to an aesthetic. However, this marriage of the scientific and artistic can only be correctly expressed by properly treating the information and its geometric expression.
Even if you are internally expressing the opinion that “I will never be testing and graphing camera information”, graph literacy will help you understand the work of other engineers and scientists. Consider what we have accomplished with these two waveforms - we have identified issues, changed the graphs to visually correspond, and calculated hard numbers.
The human mind is an outstanding piece of pattern recognition wetware. However, for most of us, we lack the capability to recognize patterns in large data sets. Graphs are the means to ‘visualize’ the numbers so that relationships emerge and patterns are expressed. Good graphs beget more graphs, or a table with a few pertinent values. While some mathematical knowledge is required to make and interpret graphs, the concepts can be learned in a short period of time. (Or, in this case, can be learned by investigating claims.)
At the same time, graphs can deceive - intentionally or unintentionally. Graph literacy remedies this problem by equipping your mind to enter, explore, and seize upon substantive relationships. In turn, this frees you to pursue your mental visualization to its aesthetic ends rather than getting bewildered in a maze of engineering conventions that (while useful) are often arbitrary. A good graph allows you to see further. Graph literacy closes the distance between scientific understanding and artistic expression.
References:
American Cinematographer. (2021, February 21). ASC Insights - episode 2 - HDR from set to Post. Vimeo. https://vimeo.com/507723649
Dainty, J.C., & Shaw, R. (1974). Image Science: principles, analysis, and evaluation of photographic-type imaging processes. Academic Press.
Davis, P. (1988). Beyond the Zone System: Second Edition. Focal Press.
Kiening, H. (2008, April 1), 4K+ Systems: Theory Basics for Motion Picture Imaging. Arnold & Richter Cine Technik, Systems [Technology brochure, K5.40455.0].
Koenderink, J.J., & van Doorn, A. J. & Gegenfurtner, K (2020, November). Colors and Things. i-Perception, 11(6), https://doi.org/10.1177/2041669520958431
Lambert, J. H. (2001). Photometry or On the Measure and Gradations of Light, Colors and Shade (D.L. DiLaura, Trans.). Illuminating Engineering Society of North America. (Originally published 1760)
Silverstein, D.A., & Farrell, J.E. (1996). The relationship between image fidelity and image quality. Proceedings of 3rd IEEE International Conference on Image Processing, 881–884.