Probability Distributions, Gaussian and Uniform

stough 202-

In the matplotlib tutorial we saw how to produce a histogram of color distribution in an image. These histograms represent the probability distribution on color; that is, the probability of a particular color in an image corresponds to the height of the histogram at that color (given the histogram of that image).

In this demo we’ll look at Uniform and Gaussian probability distributions, and what corresponding “random” images look like.

Uniform
Gaussian

Imports¶

We’ll produce random numbers using numpy’s updated random-number-generation techniques. Read more about it here.

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

# For importing from alternative directory sources
import sys  
sys.path.insert(0, '../dip_utils')

from matrix_utils import arr_info
from vis_utils import (vis_rgb_cube, 
                       vis_pair)

from numpy.random import default_rng
rng = default_rng() # could add parameter "seed="

rng

Generator(PCG64) at 0x7F880ADFCD60

At this point the object rng is our random number generator, and it has been seeded (with the current time for example) to produce different random sequences each run through this notebook. You could also seed the default random number generator to ensure the same random sequence (good for debugging).

Uniform Random Distribution¶

When you think of the word “random,” you probably mean that any outcome of the “random” process is equally likely. Let’s say we’re generating integers between 0 and 255 (inclusive). We would expect over many samples that no number is a lot more likely than any other.

X = rng.integers(0, high=256, size=2**20)

plt.figure(figsize=(6,3))
unif_counts, _, _ = plt.hist(X, bins = np.arange(257));

arr_info(X)

((1048576,), dtype('int64'), np.int64(0), np.int64(255))

X.mean()

np.float64(127.59317111968994)

In the above we generate about 1M samples in the range [0,255]. In fact we generate $2^{20} = 1048576$ elements. Since these random numbers are split among $256 = 2^{8}$ possibilites, that means on average we should get $2^{12} = 4096$ hits for each output integer. What makes the randomness uniform is that most of the bins have about the same number of outcomes, hovering near 4K as expected.

Little complication in generating the plot: plt.hist expects the bins argument to include both left end of the first bin and the right end of the last bin. Since arange provides the sequence $[0, N)$ exclusive, we call it with 257 to make sure we get all the way to $[0,...,256]$ .

Now let’s generate an image of uniform random colors

I_uniform = np.stack([rng.integers(0,256,2**14).reshape(2**7,2**7) for i in range(3)], axis=2)

plt.figure(figsize=(3,3))
plt.imshow(I_uniform)

arr_info(I_uniform.ravel())

((49152,), dtype('int64'), np.int64(0), np.int64(255))

The above is what a truly random image looks like.

Is it anyhing like the images we ourselves capture, communicate to each other, or attempt to study?
What is missing from it?
What does this say about the kinds of images that we actually care about?

Viewing in the Color Cube Space

If any possible color is equally likely to occur, as in the image above, what would that look like in the color cube visualization we saw in the matplotlib_tutorial? Think about it, then execute the next cell.

vis_rgb_cube(I_uniform)

Gaussian Random Distribution¶

There is a different kind of random that is more practically interesting in the world: the Gaussian, or Normal, Distribution, also called the Bell Curve. Many phenomena we deal with every day often closely follow the normal distribution, including heights, measurement errors, blood pressure, test scores. Here we’ll take a look a couple of Guassian-distributed samples.

As an aside, most everything that we try to model with a Gaussian is better modelled with some sort of skew normal, weibull, or pareto. These distributions have a heavier tail on one side than the other, common when there is some physical limit on the measurement. This is a complication we’ll concern ourselves with only when relevant.

X = rng.standard_normal(100000)

plt.figure(figsize=(6,3))
plt.hist(X, bins=100);

X.mean(), X.var()

(np.float64(-0.0024453854048798935), np.float64(0.9937574389054398))

Above we sample from a standard_normal, which you can see has a mean of 0 and a variance (average square distance from the mean) of 1, symbolized with $\mathcal{N}(\mu = 0,\sigma^{2} = 1)$ . We can reposition this kind of distribution anywhere we want by

adding a constant, to change the mean: $C + \mathcal{N}(\mu,\sigma^{2}) \sim \mathcal{N}(C + \mu,\sigma^{2})$
multiplying by a constant, to change the variance: $K*\mathcal{N}(\mu,\sigma^{2}) \sim \mathcal{N}(\mu,K^2\sigma^{2})$

Using the above rules, we could make a Gaussian distributed image in the range $[0,255]$ . We can also use numpy’s normal to do the same thing.

# Similar idea using our rules.
# X_im = (128 + 30*X).clip(0,255) 
I_normal = np.stack([rng.normal(128,30,2**14).reshape(2**7,2**7) for i in range(3)], axis=2)
I_normal = np.uint8(I_normal).clip(0,255)

plt.figure(figsize=(6,3))
norm_counts, _, _ = plt.hist(I_normal.ravel(), np.arange(257));

vis_pair(I_uniform, I_normal, 
         first_title='Uniform Random', second_title='Gauss Random',
         show_ticks=False)

In the above cells, we plot the distribution of the Gaussian normal image (I_normal) and then show both the uniform (I_uniform) and normal images side by side.

You should immediately notice that the “Gauss Random” image is much more bland, or muted in its tones, than the “Uniform Random” image. This kind of difference is related to contrast. Contrast measures how distinguishable different image parts are from one another. In I_normal, most pixel intensities in red, green, and blue hover around 128. This results in a middling gray tone to the whole image, with neighboring pixels only mildly distinguishable from one another, especially compared to I_uniform.

Color Cube Again

So what would I_normal look like in the color cube space, with more pixels hovering in the middle of their component ranges rather than equally all over the place? Think about it, then execute the next cell.

vis_rgb_cube(I_normal)

Plotting the two distributions together¶

A little bit of matplotlib-ary to show the two distributions together without y-axis.

#Normalize the heights.
norm_counts = norm_counts/norm_counts.sum()
unif_counts = unif_counts/unif_counts.sum()

f, ax1 = plt.subplots(figsize=(7,3))
ax1.bar(np.arange(256), unif_counts, alpha=.5)

plt.bar(np.arange(256), norm_counts, alpha=.5);
ax1.get_yaxis().set_visible(False)
plt.tight_layout()

Here we looked into the difference between uniformly and normally-distributed randomness, and what that looks like in image data. Having this background will be critical in many later image processing and analysis topics, from contrast enhancement to compression and many others.