Weber's Law and Weberized TV Restoration

Weber’s Law and Weberized TV Restoration

Jianhong Shen

School of Mathematics

University of Minnesota

Minneapolis, MN 55455, USA

Abstract

Most conventional image processors consider little the inﬂuence of human vi-

sion psychology. Weber’s Law in psychology and psychophysics claims that hu-

man’s perception and response to the intensity ﬂuctuation of visual signals are

weighted by the background stimulus , instead of being plainly uniform. This

paper attempts to integrate this well known perceptual law into the classical total

variation (TV) image restoration model of Rudin, Osher, and Fatemi [Physica D,

60:259-268, 1992]. We study the issues of existence and uniqueness for the pro-

posed Weberized nonlinear TV restoration model, making use of the direct method

in the space of functions with bounded variations. We also propose an iterative

algorithm based on the linearization technique for the associated nonlinear Euler-

Lagrange equation.

PACS (2001): 87.19.Dd (information processing in vision and hearing); 87.57.Nk

(image analysis).

Keywords: Weber’s Law; Vision; Psychophysics; Image restoration.

1 Introduction: Weber’s Law

As information carriers, all images are eventually perceived and interpreted by the

human visual system. As a result, human vision psychology and psychophysics play

an important role in the successful communication of image information. From the

imaging science point of view, this fact implies that any ideal image processor should

take into account the consequences of vision psychology and psychophysics.

The current paper makes an attempt in this direction. We develop an image restora-

tion model that intends to incorporate one of the most well known and inﬂuential psy-

chological results - Weber’s Law for sound and light perception. We study its major

mathematical properties (e.g., existence and uniqueness), as well as the computational

strategy for the associated nonlinear PDE.

Email: [email protected]. Tel: (612) 625-3570. Fax: (612) 626-2017.

Weber’sLawwas ﬁrst described in 1834 by German physiologist E. H. Weber [Web34],

and was later formulated quantitatively by the great experimental psychologist Gustav

Fechner [Fec58], founder of the modern psychophysics. The law reveals the universal

inﬂuence of the background stimulus on human’s sensitivity to the intensity incre-

ment , or the so called JND (just-noticeable-difference), in the perception of both

sound and light. It claims that the so-called Weber’s fraction is a constant:

const. (1)

Many experiments have demonstrated that in a large range of stimuli , Weber’s Law

indeed provides a good approximation.

Empirical evidence is easy to gather from daily life for a qualitative understanding

of Weber’s Law. In a fully packed stadium where the background sound intensity is

high, one has to speak close to shouting in order to be effectively heard by the other

folks. The same observation is true for visual communication. The stars can be clearly

spotted in a dark night without a bright full moon, and away from the urban neon

lights. But otherwise our naked vision has much difﬁculty in ﬁnding them. In the

current paper, we apply Weber’s Law in the context of visual perception. Therefore,

stands for the background light intensity and the intensity ﬂuctuation.

Since almost all images are eventually to be observed and interpreted by humans,

an ideal digital image processor has to take into account the effects of human psychol-

ogy and psychophysics, such as that of Weber’s Law. This is an important new area that

needs to be further explored. The current paper makes the ﬁrst infantile attempt of in-

tegrating Weber’s Law into image restoration schemes. We demonstrate our main idea

through “Weberizing” the well known classical model of total variation (TV) denoising

and enhancement by Rudin, Osher, and Fatemi [ROF92, RO94].

The organization goes as follows. In Section 2, we quickly review the TV restora-

tion model in image processing, and explain the idea behind its “Weberization.” In

Section 3, we ﬁrst rigorously interpret the Weberized TV restoration energy and its ad-

missible space, and then apply the direct method to study the existence and uniqueness

of the minimizers. The computational approach to the minimization of the Weberized

TV energy is addressed in Section 4, accompanied by some typical numerical results.

The conclusion goes into Section 5.

2 Weberized TV for Image Restoration

Let denote the observed raw image data, which is assumed to be a degraded version

of the original good image . Distortions in are typically modeled by blurring and

noising:

(2)

where is a linear blurring operator, or a lowpass ﬁlter with , and denotes

white noise. The goal of image restoration is to recover the original good image

from one single observation of (since strictly speaking, is a random ﬁeld). In

this paper, we shall assume that the noise is spatially homogeneous, and can be well

approximated by Gaussian. We shall also focus on the pure denoising case when there

is no severe blurring.

The TV restoration model ﬁrst proposed by Rudin, Osher, and Fatemi [RO94,

ROF92] is to minimize the following Bayesian type energy [GG84, Mum94, CS02]:

(3)

in the space of functions with bounded variations BV . (Here, imitating the con-

ditional expectation in probability theory, the vertical bar deﬁnes two domains for the

known variables and the unknowns.) The ﬁrst regularity term is understood beyond

the conventional Sobolev space

, instead, as the TV Radon measure [Giu84].

BV has been proven a sufﬁciently good image space for most images without much

texture [CS02]. The main characteristic of the BV image model is that it legalizes 1-

dimensional singularities, or popularly referred to as “edges,” an important visual cue

in human and computer vision [MH80].

Ever since their ﬁrst introduction into digital image processing in [RO94, ROF92],

the BV image model and TV restoration model (3) have witnessed many successful new

developments during the past decade. We refer to our recent survey paper [CS02] and

the references therein for more detail. In particular, with his collaborators, the author

of the present paper has been able to extend and generlize the models onto digital graph

domains [COS01], onto the so-called nonﬂat image features that live on Riemannian

manifolds [CS00], and onto the novel area of image inpainting and geometric image

interpolations [CS01]. Figure 1 shows one application of the TV restoration model for

the error concealment of a blurry image transmitted through a wireless network with

randomly lost packets.

A blurred image with 80 lost packets Deblurring and error concealment by TV inpainting

Figure 1: TV restoration of a blurry image with simulated random packet loss.

Most conventional restoration models do not take into account that our visual sen-

sitivity to the regularity or local ﬂuctuation depends on the ambient intensity level

. That is, models such as (3) assume that a local variation, say, should be

treated equally independent of the background intensity level , no matter whether it

is or . But this exactly violates Weber’s Law, according to which, a

ﬂuctuation level of against a background intensity is much more

signiﬁcant than the same amount against . In fact it is approximately equivalent

to a level of

in the latter situation.

It is out of this consideration that we propose to “Weberize” the classical TV

restoration model (3). The key is to replace the uniform local variation

by the Weberized local variation ,

which encodes the inﬂuence of the background intensity according to Weber’s Law (1).

Therefore, instead of (3), we propose the Weberized TV restoration model

(4)

The current paper is devoted to the study of the mathematical properties of this new

model, including issues related to the existence and uniqueness of the minimizers, and

its computational approach.

3 Existence and Uniqueness

We ﬁrst work out some natural assumptions on the data model, and then investigate the

existence and uniqueness of the minimizers to the Weberized TV restoration model (4).

3.1 Assumptions and the admissible space

In Weber’s fraction , denotes the intensity value. Thus . We shall call

the “blackhole” since physically it means no photons are emitted or reﬂected. The

blackhole is the singularity of both Weber’s fraction and the Weberized local variation

. Therefore, technically we should stay away from the blackhole and

assume that .

The blackhole similarly imposes some natural restrictions on the noise model

Since also represents the intensity value, we must have , which implies that

. Now that in both the TV restoration model (3) and its Weberized version (4),

noise reduction has been controlled by the least square energy, we are implicitly as-

suming that the noise can be well approximated by some Gaussian [Str93].

In particular, (or its probability density function) should be almost symmetric, and

the condition is equivalent to , which can be roughly translated to that

the signal-to-noise ratio SNR . Therefore,

and we obtain a natural technical constraint for the Weberized TV restoration model (4):

(5)

Before investigating the existence of the Weberized TV retoration (E:Ew)

(6)

we need to explain the exact meaning of the Weberized TV, and the admissible space

for the restoration energy .

First, due to the least square energy control in , we assume that . As

a result, . Throughout the paper, we also assume that is a Lipschitz open

domain with a ﬁnite Lebesgue measure .

Second, the Weberized TV energy

is understood in the sense of the coarea formula [Giu84]. More generally, let

be a continuous function. Then, for any BV and ,

we deﬁne

Per (7)

Here for any , the perimeter of the set is deﬁned as [Giu84]:

Per Per (8)

(Note: in this paper we shall always use the conventional notation to denote

the TV Radon measure [Giu84]. ) When , (7) is precisely the classical

coarea formula.

Another equivalent way is to introduce the integral of : . Then

the deﬁnition (7) is identical to

TV (9)

For instance, for the Weberized TV, , . Therefore, the

Weberized TV restoration can be rewritten as

(10)

The combination of all the three elements discussed above leads to the following

natural admissible space for the Weberized TV restoration (10):

TV (11)

This is the space that we shall work with from now on.

3.2 Existence of Weberized TV restoration

First we prove a Maximum Principle type result for the Weberized energy form (10).

The technique is characteristic to total variation related energies, but becomes difﬁcult

for conventional Sobolev type regularity energies.

Lemma 1 Suppose that

for all , and is a minimizer of the

Weberized TV restoration energy restricted in the admissible space as deﬁned

in (11). Then (in the Lebesgue a. e. sense). In particular, .

Proof. Deﬁne . Then

The equality in the inequality line holds if and only if has Lebesgue measure 0.

Meanwhile, by the coarea formula (7),

Per Per

Per

Per Per

Per

where for the second equality, we have applied Per . Together, we have

established directly that

and the equality holds if and only if Since is a minimizer in and

, the equality must hold and thus

Notice that such direct method is almost unique to TV related energies. The ceiling

operator (or the ﬂooring operator ) is readily compatible

with the TV measure. The same technique is valid even for more general nonnegative

functions as in (7).

Theorem 1 (Existence) Assume that the observation satisﬁes

and

loc

Then the Weberized TV restoration model (6) or (10) has at least one minimizer when

restricted in the admissible space (see (11)).

Proof. Notice that the admissible space is nonempty since . Let

be a minimizing sequence of the Weberized TV restoration energy

restricted in . In the spirit of Lemma 1, we can assume that .

Then

Therefore, is a bounded sequence in the Banach space BV endowed with the

BV norm:

By the weak compactness, has a subsequence, still denoted by for conve-

nience, that converges strongly in to some : . Furthermore, after a

reﬁnement of the subsequence if necessary, we can assume that

Then by the Lebesgue Dominated Convergence Theorem,

(12)

For the control over the Weberized TV, deﬁne and

. Then

and since , we also have

(13)

For any compactly supported vectorial test function

with

we have

and

(14)

Now that the observation

loc

and that is compactly supported,

the right hand side of (14) must belong to

. Hence, again by the Lebesgue

Dominated Convergence Theorem,

Consequently, for each of such test functions ,

Therefore, by deﬁnition [Giu84], we must have

(15)

Finally, the combination of (12) and (15) gives

It is easy to see that . Since is a minimizing sequence, we therefore have

shown that is in fact a minimizer. This completes the proof.

We close this subsection with a remark on the conditions of the existence theorem.

In numerous digital applications, the conditions in Theorem 1 are naturally satisﬁed

since intensity values are always scaled to a positive interval . For instance,

and in most 8-bit display systems. When , one level of

elementary shifting, say, resolves the blackhole problem. Such practice is

equivalent to what some vision psychologists have called the modiﬁed Weber fraction

, in which the small intensity value is called the activation level.

3.3 Uniqueness of Weberized TV Restoration

Unlike the classical TV restoration model (3), the Weberized energy

is not convex. As a result, uniqueness is no longer a direct product of convexity.

We start with a computational lemma that is again unique to total variation related

energies.

Lemma 2 Let be a function, and

then the formal Euler-Lagrange differential of is

(16)

Proof. We witness an intrinsic cancellation mechanism characteristic to the TV energy

during the standard computation of Calculus of Variation :

where denotes the arc-length element of the boundary. The cancellation occurs in

the ﬁrst integral of the second line. This completes the proof.

Applying the lemma to the Weberized TV restoration energy , we obtain the

formal equilibrium Euler-Lagrange equation:

along (17)

Or, applying the time marching scheme along the gradient descent direction,

(18)

with the same Neumann adiabatic boundary condition, and an appropriate initial guess.

Notice that (18) is a nonlinear diffusion-reaction type equation. At each ﬁxed pixel

, set . Then the pure reaction mechanism is given by the ODE

(19)

which has an unstable repelling boundary , and the unique globally stable attrac-

tor . It is this property that hints that the Weberized TV restoration model may

have a unique solution.

The following uniqueness theorem is at a formal level in the sense that our proof

relies on the formal Euler-Lagrange equations (17). A mathematically more rigorous

proof, or an appropriate reformulation of the uniqueness issue, still remains an open

problem for our readers.

Like the Existence Theorem 1, the following Uniqueness Theorem is again estab-

lished in the natural admissible space in Section 3.1.

Theorem 2 (Uniqueness) Assume that is a minimizer of the Weberized

TV restoration energy restricted in . If for all , then

is unique.

Proof. Since , is away from the boundary of the admissible

space and Calculus of Variation is valid, which leads to the Euler-Lagrange equation

for :

along (20)

or equivalently,

along (21)

Deﬁne a new reference energy for the Weberized TV restoration as fol-

lows:

(22)

It is easy to derive that (21) is exactly the Euler-Lagrange equilibrium equation for

. At each ﬁxed pixel , set , and deﬁne a cubic potential of

(23)

Then

and

In particular, , for all . As a result,

is strictly convex when restricted on . Now that the

TV Radon measure is semi-convex, together we conclude that the reference energy

is strictly convex:

for any BV and . The equality holds if and

only if when . Consequently, its equilibrium Euler-Lagrange equation (21)

(even unnecessarily being a global minimum) has at most one solution that satisﬁes

, which implies that is indeed unique as claimed.

The lower bound has been initially motivated by the blackhole constraint

on Weber’s fraction, as discussed in Section 3.1. It is perhaps more than a coincidence

that it also naturally appears as the inﬂection point of the reference energy in the

proof of uniqueness.

4 The Computational Approach and Examples

As for the classical TV restoration, there are many computational tools available for

digitally implementing the energy minimization (see, for example, [CGM99, ROF92,

MO00, VO96]). In this paper, as in [COS01, VO96], we propose to apply the lineariza-

tion technique to iteratively solve the Euler-Lagrange equation (17).

Deﬁne . Then Eq. (17) can be re-written as

(24)

with the Neumann adiabatic condition along the boundary of the image domain. It is

formally identical to the classical TV denoising equation [ROF92, VO96], except that

the ﬁtting constant now depends on . Notice that since .

To numerically solve (24), we apply the linearization technique. Eq. (24) is to be

solved iteratively ( ) based on the linearization

(25)

where , and stands for the linear elliptic operator

As well practiced in the TV restoration literature [CL97, VO96], is computation-

ally better conditioned to

(26)

where the notation stands for for some positive parameter . Such

conditioning gets rid of the singularity of

on the homogeneous regions of the

current guess where is close to zero. Notice that in terms of the energy

formulation, (25) and (26) are equivalentto tracking down the unique minimizer

of the quadratic energy

Figures 2 and 3 have been generated by this algorithm, and the central-difference

based ﬁnite difference schemes for the linearized equation (25). Figure 2 displays the

noisy test image (the left panel) and its Weberized TV restoration (the right panel). Fig-

ure 3 shows a typical 1-dimensional horizontal slice taken from both the noisy image

and its Weberized TV restoration. One clearly observes that unlike the conventional

TV restoration, the Weberized version is able to distribute the minimum amount of

irregularity adaptively over the image domain according to Weber’s Law. Therefore,

in the restored image, the minimum ﬂuctuation is allowed to be larger on regions

where the background intensity is higher and human’s visual sensitivity is weaker.

5 Conclusion

Most conventional image processors consider little how human subjects “feel” about

the outputs. Weber’s Law claims that human’s perception and response to the inten-

sity ﬂuctuation of both aural and visual signals are not simply uniform, instead,

A test image with uniform white noise Weberized TV denoising

Figure 2: Weberized TV restoration of a test image with homogeneous noise.

should be weighted by the ambient stimulus . The current paper has attempted to in-

tegrate this famous psychological effect into the classical TV image restoration model

of Rudin, Osher, and Fatemi [ROF92].

We have studied the issues of existence and uniqueness for the proposed Weber-

ized TV restoration model, based on the direct method in the space of functions with

bounded variations BV . We have also proposed an iterative algorithm based on the

linearization technique for the nonlinear Euler-Lagrange equation.

We consider the present work as an infantile step in the big blueprint of integrating

important psychological and psychophysical results (either empirical or statistical) into

the contemporary imaging science and technology. Our long-term goal has been set on

the exploration of all possible important interactions.

Acknowledgments

I deeply thank my great teachers and friends Professors Dan Kersten and Paul Schrater

at the Department of Psychology, University of Minnesota, for patiently teaching me

both vision psychology and psychophysics.

References

[CGM99] T. F. Chan, G. H. Golub, and P. Mulet. A nonlinear primal-dual method for

total variation-based image restoration. SIAM J. Sci. Comput., 20(6):1964–

1977, 1999.

[CL97] A. Chambolle and P. L. Lions. Image recovery via Total Variational mini-

mization and related problems. Numer. Math., 76:167–188, 1997.

0 50 100 150 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

A horizontal slice of the noisy image u

0 50 100 150 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

The same slice after Weberized TV restoration

Figure 3: A typical horizontal slice from the noisy image (left) and its Weberized TV

restoration (right). One is able to clearly observe two main features of the Weberized

TV restoration model: (1) like the classical TV model, it is able to maintain the sharp-

ness of edges in the noisy image; (2) but unlike the direct TV restoration model, it

is able to distribute the minimum amount of irregularity adaptively over the image do-

main according to Weber’s Law: the minimum amount of ﬂuctuation (in the minimized

energy) is allowed to be larger on regions where the background intensity is higher and

human’s visual sensitivity gets weaker.

[COS01] T. F. Chan, S. Osher, and J. Shen. The digital TV ﬁlter and nonlinear de-

noising. IEEE Trans. Image Process., 10(2):231–241, 2001.

[CS00] T. F. Chan and J. Shen. Variational restoration of non-ﬂat image features:

models and algorithms. SIAM J. Appl. Math., 61(4):1338–1361, 2000.

[CS01] T. F. Chan and J. Shen. Mathematical models for local non-texture inpaint-

ings. SIAM J. Appl. Math., 62(3):1019–1043, 2001.

[CS02] T. F. Chan and J. Shen. On the role of the BV image model in image restora-

tion. UCLA’s Mathematics Department CAM Report 02-14 available at:

www.math.ucla.edu/˜imagers, 2002.

[Fec58] G. T. Fechner.

Uber ein wichtiges psychophysiches Grundgesetz und dessen

Beziehung zur Sch¨azung der Sterngr¨ossen. Abk. k. Ges. Wissensch. Math.-

Phys., K1, 4, 1858.

[GG84] S. Geman and D. Geman. Stochastic relaxation, Gibbs distributions, and the

Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intell.,

6:721–741, 1984.

[Giu84] E. Giusti. Minimal Surfaces and Functions of Bounded Variation.

Birkh¨auser, Boston, 1984.

[MH80] D. Marr and E. Hildreth. Theory of edge detection. Proc. Royal Soc. Lon-

don, B:207: 187–217, 1980.

[MO00] A. Marquina and S. Osher. Explicit algorithms for a new time dependent

model based on level set motion for nonlinear deblurring and noise removal.

Siam. J. Sci. Comput., 22:387–405, 2000.

[Mum94] D. Mumford. Geometry Driven Diffusion in Computer Vision, chapter “The

Bayesian rationale for energy functionals”, pages 141–153. Kluwer Aca-

demic, 1994.

[RO94] L. Rudin and S. Osher. Total variation based image restoration with free

local constraints. Proc. 1st IEEE ICIP, 1:31–35, 1994.

[ROF92] L. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise

removal algorithms. Physica D, 60:259–268, 1992.

[Str93] G. Strang. Introduction to Applied Mathematics. Wellesley-Cambridge

Press, MA, 1993.

[VO96] C.R. Vogel and M. E. Oman. Iterative methods for total variation denoising.

SIAM J. Sci. Comput., 17(1):227–238, 1996.

[Web34] E. H. Weber. De pulsu, resorptione, audita et tactu. Annotationes anatomi-

cae et physiologicae, Leipzig: Koehler, 1834.