Should you normalize RGB values by 255 or 256?

Let’s say you’re writing an image processing program. The program takes in an image, converts it to floating point, does some processing and finally saves the modified pixels to disk as 8-bit colors. The question today concerns how exactly the integer-to-float conversion should be done. There are two approaches which, written in Python and NumPy, look like this:

pixels = img / 255.0
result = process(pixels)
output = np.trunc(result * 255 + 0.5)

pixels = (img + 0.5) / 256.0
result = process(pixels)
output = np.trunc(result * 256)

I assume that in both cases the output values are clamped before the final typecast:

# Clamp and cast to 8 bits
output_8bit = output.clip(0, 255).astype(np.uint8)

The standard approach maps the integer 0 to 0.0 and 255 to 1.0. It works perfectly fine and is how GPUs do it. The alternative adds a 0.5 bias and divides by 256 instead, so the integer 0 gets mapped to 0.5/256=0.001953125. This is inconvenient because your image processing code can’t detect black pixels, for example, without knowing the above constant. As a consequence, you tie your logic to 8-bit inputs even if you compute in floating point. With the standard approach, you can always assume black is 0.0.

But some programmers still feel a pull towards the alternative. What is going on? What do they see in it?

The case against 255.0

The standard approach does look quite strange when plotted on the number line. Below you can see an exaggerated version with 3-bit integers in the range [0..7] being mapped to [0,1]:

On the X-axis we’ve got a number line and the locations of brown circles on it represent the decoded floating-point values. The numbers inside are the integer inputs. Each integer has arrows pointing to it; these show a range of floating-point values that round to it. I’ll call these ranges “bins” in the rest of this article.

Smaller bins at the extremes

The first issue really apparent in the diagram is how the standard formula’s extreme bins jut beyond the [0,1] range. Perhaps this visualization is unfair – both approaches clamp their output so the extreme bins could extend infinitely – but it clearly shows how “stretched” the standard range is. The stretched range is wider than the assumed operating range [0, 1] in image processing.

This means that when converting floating-point values in the [0, 1] range back to integers, the extreme bins have effectively half the width of other bins. As a consequence, it will be “harder” to output extreme values from your algorithm. For example, if you generate uniform [0,1] noise and round it using the standard formula, the values 0 and 255 will occur only half as frequently as other integers.

We can verify this claim empirically by generating a million uniform random numbers, plotting them as a histogram, and observing that both the 0 and 255 bins are indeed only half as tall as other bins:

The highlighted crop:

import numpy as np
import matplotlib.pyplot as plt

result = np.random.uniform(0, 1, 1000000)
final_values = np.trunc(result * 255 + 0.5).clip(0, 255).astype(np.uint8)
plt.hist(final_values, bins=256, range=(0, 255))
plt.show()

Still, I’m having a hard time coming up with an example situation where the bias away from the extremes would prove problematic. Sure, the standard approach’s floats are spread over a wider range, but the original image will still round-trip convert losslessly (uint8 → float → uint8).

Also, any result value just beyond 0.0 or 1.0 will still round to the right bin, evening out the output distribution. An example of what I mean. Assume your processing subtracts 0.005 from the floating-point colors. In the standard approach this pushes blacks below zero – outside the [0,1] range – but in the alternative the values stay positive. In the end both output the integer 0 anyway:

Standard:
trunc(255 * (-0.005) + 0.5) = 0

Alternative:
trunc(256 * (0.5 / 256 - 0.005)) = 0

It didn’t matter that in the standard approach the zero bin was only “half the size”.

Inexactness

The second issue is that the standard approach’s floating-point values aren’t exact. For example 128/255.0 \approx 0.501961 but 128/256.0 = 0.5. Due to this round-off error, the distances between floating-point values vary a tiny bit. But this isn’t a real problem since the error is truly tiny. A 32-bit floating-point number has a 23-bit fraction (“significand”). We are talking about round-off error in its least-significant bit; jitter with the magnitude less than 2^{-23}. Surely a relative error of 0.00001 % is immaterial even in the most sophisticated image processing task. In this case, inexactness is an aesthetic question, not a technical one.

Values not in between integers

The alternative approach always places each floating-point value exactly in the middle of two integers. See how the vertical bars align in the number line diagram above. The halfway position can be thought of as a compromise; we don’t know what the original quantized value was exactly, and thus the average point between two successive integers is a good guess.

I’m sure there are applications where this property is useful, even though I’m having a hard time coming up with examples myself. Well, at least dithering is more convenient, argues a 2015 blog post “Converting Color Depth” by Andrew Kesler (known for his business card raytracer). The reasoning goes that noise can be added without worrying about edge cases. In contrast, the standard formula’s awkward extremes require careful handling to keep the noise distribution consistent.

Two types of quantizers

So far the standard “divide by 255” formula still looks solid, or at least firm enough to still be worth it. Another way to think about the question is to zoom out a bit and see the two approaches as two different uniform scalar quantizers. If we check the Wikipedia page on quantization, we’ll quickly learn that there are two main types of quantizers:

Most uniform quantizers for signed input data can be classified as being of one of two types: mid-riser and mid-tread. The terminology is based on what happens in the region around the value 0, and uses the analogy of viewing the input-output function of the quantizer as a stairway. Mid-tread quantizers have a zero-valued reconstruction level (corresponding to a tread of a stairway), while mid-riser quantizers have a zero-valued classification threshold (corresponding to a riser of a stairway).

As a source Wikipedia cites a 1977 paper that has such an incredible combined title and abstract layout that I must reproduce it here:

Anyway. When plotted on a graph, the mid-riser and mid-tread quantizers differ where they cross zero:

Mid-tread indeed maps zero to zero and mid-riser maps zero to the middle of two integers (sound familiar?). The notation chosen by Wikipedia represents an input real number with x, its encoded (“classified”) integer value with k, and reconstructed real number with y_k. The corresponding quantizer formulas look like this:

L stands for the number of distinct output levels (for example 256).

If we apply these definitions to our two competing approaches, we can call the standard formula a “mid-riser” with L=255 and the alternative a “mid-tread” with L=256. Actually, I’ll show their code again with the new labels to make the connection to the new formulas above clear. The code snippets themselves are the same as in the beginning.

pixels = img / 255.0
result = process(pixels)
output = np.trunc(result * 255 + 0.5)

pixels = (img + 0.5) / 256.0
result = process(pixels)
output = np.trunc(result * 256)

From this perspective we can say the standard approach is a strange combination of a mid-riser quantizer for unsigned inputs (the quote said “for signed input data”) and a choice of L=255 integer codes. Clearly this is not optimal for 8-bit inputs. Again, this is all for the programming convenience of having the extremes map to 0.0 and 1.0. This leads to the final criticism of the standard formula.

Higher quantization error but not really

If we were designing a system that receives a uniformly distributed real number x \in [0,1], encodes it as an 8-bit integer k, and finally reconstructs it as another real number y_k, the standard formula would waste bandwidth. Remember how the 0 and 255 bins poked slightly beyond the [0,1] range’s edges? In the standard approach, the range of representable values is actually [-0.5/255, 255.5/255], meaning the bins are spaced further apart than strictly needed for [0, 1] inputs, leading to a higher reconstruction error. The increase in error is small, however. According to StackOverflow user Peter Mudrievskij’s calculation, the mean absolute errors are 1/1020 and 1/1024 for 255 and 256 divisors, respectively. Thus division by 256 is theoretically more precise.

The subtle part is that this kind of reconstruction is not what we’re doing. The premise was that we are loading 8-bit RGB images, doing processing on them, and saving them again. We have no control over how they were quantized when saved; all information lost is gone forever. In other words, if an image’s color were multiplied by 255 and rounded, dividing them by 256 at load time does not bring back any precision. Only when we control both saving and loading does an appeal to lower reconstruction error make sense.

In fact, using the alternative formula to load other people’s images will introduce more error. Most likely the images were quantized via the standard formula, so decoding them with the wrong scale factor is incorrect, in theory. In practice, the colors aren’t absolute measurements (even if the sRGB spec claims so), and all that happens is that we’ll do our processing in a slightly smaller range with a small offset. End of the subtle part.

Finally, one should never mix the encode and decode steps of the two quantizers. That’s just broken code. It’s an easy mistake to make, though. Update: Or perhaps it’s fine? See the end.

Conclusion

To answer the question posed in the title: if you’re processing images given to you by strangers, you should normalize RGB values by 255. Neither inexact floating-point values nor some abstract feeling of a higher reconstruction error is a good reason to go for the alternative. But if you control both image saving and loading, don’t need zero to map to zero, and feel OK about tying your processing code to the 8-bit dynamic range, then you can consider division by 256 to eke out a bit more precision. Just don’t blame me when your colleagues load your images with the standard formula anyway, ruining your master plan.

I’m writing a book on color reduction algorithms. Sign up here if you’re interested.

Other takes

Jonathan Blow’s 2002 article talks about mid-riser and mid-tread quantizers without mentioning them by name. I got the diagram idea from there.

The already mentioned 2015 blog post by Andrew Kesler advocates for the alternate formula. Unfortunately the comparison is to the standard formula but without rounding, which invalidates most of the analysis.

An update two months later: The Third Option

When skimming Angelo Pesce’s “Thursday links”, I learned about a 2023 blog post titled Accurate color conversions by Chris Lomont. It’s about converting 8-bit colors to floats and back.

Lomont arrives at a different conclusion than I did. He uses the names “BAD METHOD” and “BETTER METHOD” for what I call the standard and alternative formulas, respectively.

With the standard method, he identifies as the main issue the half-sized bins at 0 and 255:

[–] the integers 0 and 255 have half the size of interval that map to them as all the other integers. This means that image processing will lose some representation of the end colors, which is bad.

Then he moves on to the alternative formula and highlights how it doesn’t use the full [0,1] range. Lomont illustrates this limitation with an image comparison scenario:

Having the darkest, lowest byte value not map to the darkest, lowest floating point value causes some problems. Consider checking 2 images for differences - often the values are subtracted, then the difference scaled up to increase its visibility. Having a nonzero value, then multiplying, will imply errors that should not be there.

He then proposes the “BEST METHOD” formulas that should keep both full-range floats and uniformly sized bins (and also roundtrip-convert correctly in the presence of noise). Using the variable names compatible with Wikipedia’s quantization article, Lomont’s method looks like this:

y_k = \dfrac{k}{255.0}

k = \text{clamp}(\text{floor}(x\times 256.0),0,255)

where we have the input x \in [0,1], the decoded value y_k \in [0,1], and the quantized integer k \in [0..255].

As we established earlier, quantization error really isn’t a huge deal in this context. Especially since in this case, the errors cancel out on uniformly-distributed inputs. Thus, the integer outputs are still compatible with readers that use the standard approach.

Error example

To give an example where the error distribution makes a difference, suppose you’re darkening an image. You convert the 8-bit values to floats via

colors = img/255.0

and then process them via

result = colors*0.5.

If you save them via the proposed

floor(result * 256).clip(0,255),

you’ll get on average 0.4% darker image than if you did the standard

floor(result * 255 + 0.5).

That’s the tradeoff. Not bad for the benefits. Perhaps I wouldn’t use this myself but calling it “broken code” like I did earlier is definitely unfair.

Earlier discussion

I believe this approach is also the mentioned in a comment by Kornel Lesiński. From it we also learn that clamping isn’t needed if you subtract a tiny epsilon from 256.0:

k = \text{floor}(x\times 255.999)

From the same comment thread we’ll find a second, subtle downside. High dynamic range values suffer from a clear upwards bias:

[–] on the scale where 0 is black and 255 is white, 1024 is a real color. If you convert it to float by dividing by 255 (which simply must be correct, otherwise white isn’t white), then if you convert it back to an integer by multiplying by 256, you just increased its energy, even if you quantize (you get something like 1028).

–Comment by wareya on 2nd of June, 2026.

This could be also read from the reconstruction error plot above if its X-axis range was extended to [0,4].

import numpy as np

import matplotlib.pyplot as plt

if False:
    # Plot mapping
    floats = np.linspace(start=0.0, stop=1.0, num=1000, endpoint=True)
    int_third = [int(np.floor(y*256).clip(0,255)) for y in floats]
    int_standard = [int(np.floor(y*255+0.5).clip(0,255)) for y in floats]
    int_alt = [int(np.floor(y*256).clip(0,255)) for y in floats]

    fig, ax = plt.subplots()
    plt.plot(floats, int_third, label='Third')
    plt.plot(floats, int_standard, label='Standard')
    plt.legend()
    plt.show()

# Error comparison

original = np.linspace(start=0.0, stop=1.0, num=10000, endpoint=True)
rng = np.random.default_rng(seed=1336)
# original = rng.uniform(0.0, 1.0, size=10000) # uniform random values show a difference in means

# Simulate image editing followed by quantization
gain = 0.5

# The new "third" method
ks = [int(np.floor(y*256).clip(0,255)) for y in original]
ks_edited = [int(np.floor(gain*y*256).clip(0,255)) for y in original]

# "Standard" division by 255
ks_standard = [int(np.floor(y*255+0.5).clip(0,255)) for y in original]
ks_standard_edited = [int(np.floor(gain*y*255+0.5).clip(0,255)) for y in original]

# "Alternative" division by 256 and centering (integers identical to "third")
ks_alt = [int(np.floor(y*256).clip(0,255)) for y in original]
ks_alt_edited = [int(np.floor(gain*y*256).clip(0,255)) for y in original]

yk = [k/255 for k in ks]
yk_standard = [k/255 for k in ks_standard]
yk_alt = [(k+0.5)/256 for k in ks_alt]

yk_edited = [k/255 for k in ks_edited]
yk_standard_edited = [k/255 for k in ks_standard_edited]
yk_alt_edited = [(k+0.5)/256 for k in ks_alt_edited]

yk_edited_mean = np.array(yk_edited).mean()
yk_standard_edited_mean = np.array(yk_standard_edited).mean()
yk_alt_edited_mean = np.array(yk_alt_edited).mean()

print(f"third edited mean:    {yk_edited_mean}")
print(f"standard edited mean: {yk_standard_edited_mean}, relative mean diff: {(yk_edited_mean - yk_standard_edited_mean)/np.abs(yk_standard_edited_mean) * 100:.3f} %")
print(f"alt edited mean:      {yk_alt_edited_mean}")
print()

sq_diffs = np.mean((original - yk)**2)
sq_diffs_standard = np.mean((original - yk_standard)**2)
sq_diffs_alt = np.mean((original - yk_alt)**2)

signed_diffs = np.mean(original - yk)
signed_diffs_standard = np.mean(original - yk_standard)
signed_diffs_alt = np.mean(original - yk_alt)

mean_before = original.mean()
mean_after = np.array(yk).mean()
mean_after_standard = np.array(yk_standard).mean()
mean_after_alt = np.array(yk_alt).mean()

std_before = original.std()
std_after = np.array(yk).std()
std_after_standard = np.array(yk_standard).std()
std_after_alt = np.array(yk_alt).std()

print(f"squared error (third):    {sq_diffs}")
print(f"squared error (standard): {sq_diffs_standard}")
print(f"squared error (alt):      {sq_diffs_alt}")
print()

print(f"signed diff (third):    {signed_diffs}")
print(f"signed diff (standard): {signed_diffs_standard}")
print(f"signed diff (alt):      {signed_diffs_alt}")
print()

print(f"mean before:            {mean_before}")
print(f"mean after (third):     {mean_after}")
print(f"mean after (standard):  {mean_after_standard}")
print(f"mean after (alt):       {mean_after_alt}")
print()

print(f"mean diff (third):      {mean_after - mean_before}")
print(f"mean diff (standard):   {mean_after_standard - mean_before}")
print(f"mean diff (alt):        {mean_after_alt - mean_before}")
print()

print(f"std before:            {std_before}")
print(f"std after (third):     {std_after}")
print(f"std after (standard):  {std_after_standard}")
print(f"std after (alt):       {std_after_alt}")

diffs = (yk - original)
diffs_standard = (yk_standard - original)
diffs_alt = (yk_alt - original)

import matplotlib.pyplot as plt

fig, ax = plt.subplots()
lw=0.75
alpha=0.9
ax.plot(original, diffs_standard, label="Standard", alpha=alpha, lw=lw)
ax.plot(original, diffs_alt, label="Alternative", alpha=alpha, lw=lw)
ax.plot(original, diffs, label="Third", alpha=alpha, lw=lw)
ax.set_xlabel("Input value")
ax.set_ylabel("Reconstruction error")
ax.legend()

fig, ax = plt.subplots()
bins = 32
ax.set_title(f"Byte distribution after x{gain} gain")
ax.hist(ks_standard_edited, label='Standard', bins=bins)
ax.hist(ks_alt_edited, label='Alternative', bins=bins)
ax.hist(ks_edited, label='Third', bins=bins)

plt.show()

third edited mean:    0.24902
standard edited mean: 0.2499964705882353, relative mean diff: -0.391 %
alt edited mean:      0.250000390625

squared error (third):    2.5533533745531147e-06
squared error (standard): 1.2814301037946933e-06
squared error (alt):      1.2718200937932334e-06

signed diff (third):    1.709743457922741e-17
signed diff (standard): 1.7161272403143356e-17
signed diff (alt):      1.6286971771251046e-17

mean before:            0.5
mean after (third):     0.5
mean after (standard):  0.5
mean after (alt):       0.5

mean diff (third):      0.0
mean diff (standard):   0.0
mean diff (alt):        0.0

std before:            0.2887040035517924
std after (third):     0.2898338491803864
std after (standard):  0.28870827938982857
std after (alt):       0.2887016857070255

Article history

• Aug 17, 2026: Added discussion of Lomont’s article.

---

Mastodon Bluesky YouTube Feed

30fps.net