Bounded Bias A-Optimal Designs for Regression Models

Shawn X. Liu
Department of Mathematics and Computing
Mount Royal University

June, 2020

Copyright and all rights reserved by the author. This is a pre-print.

Abstract
In 1992, Doulas Wiens suggested a problem that considers the optimal design minimizing the
variance of the estimator of the parameters of regression function when the fitted model is
correct, subject to a bound on the bias term which occurs when the true model is different from
the assumed one. The corresponding optimal designs can be called bounded bias optimal
designs. Some general results for D-optimality was obtained and published (see Liu and Wiens,
1997). In this paper, we study mainly A-optimal designs. For some special cases of bounded
functions, explicitly design measures are given.

Key Words and Phrases: approximately regression models, bounded bias, optimal design,
robustness

1. Introduction
The robust optimal design can be traced back to 1950’s. Box and Draper (1959) noticed that the
strict formulation of the regression function is dangerous in the situations when the “true”
regression function is only approximated by the assumed one.
The problem of optimal design has been studied extensively by many authors. The famous
theorem about the equivalence of D-optimal and minimax designs was due to Kiefer and
Wolfowitz (1960). Many topics about optimal design theory can also be found in Fedorov
(1972). There is also a major consideration in robust experimental design that dealing with the
possible model violations. Started from Box and Draper (1959), the problem of finding robust
design has been studied by many researchers in different aspects. For details, see Marcus and
Sacks (1977), Pesotchinsky (1982), Li (1984), Wiens (1992) Blanchard, Field and Ronchetti
(1997), Agostinelli (2002) and Khan, Van Aelst and Zammar (2007) to name a few.
In this paper, we assume the following regression model:
𝑦𝑖 = 𝜽𝑇 𝒇(𝑥𝑖 ) + 𝜖𝑖 𝑖 = 1, … , 𝑛,
where 𝜽𝑇 = (𝜃0 , 𝜃1 , … , 𝜃𝑝−1 ), 𝒇𝑇 (𝑥) = (1, 𝑥, … , 𝑥 𝑝−1 ), and 𝜖𝑖 ′𝑠 are independent and identically
distributed random variable with mean 0 and some constant unknown variance 𝜎 2 . However it is
very often that the ‘true’ response is the following:
𝑦𝑖 = 𝜽𝑇 𝒇(𝑥𝑖 ) + 𝑥 𝑝 𝜓(𝑥 ) + 𝜖𝑖 𝑖 = 1, … , 𝑛,

where 𝜓 ∈ Ψ = {𝜓 ∶ |𝜓(𝑥 )| ≤ ∅(𝑥 ) 𝑜𝑛 𝑆 = [−1, 1]} and ∅(𝑥) is a known function with
certain properties. Then we have
𝐸 [𝑦|𝑥] = 𝜽𝑇 𝒇(𝑥)

(1.1)

𝐸 [𝑦|𝑥] = 𝜽𝑇 𝒇(𝑥 ) + 𝑥 𝑝 𝜓(𝑥)

(1.2)

and

̂ be the least squares estimator of 𝜽. Under (1.2), we have
Let 𝜽
̂ − 𝜽)(𝜽
̂ − 𝜽)𝑇 ] =
𝑀𝑆𝐸 (𝜓, 𝜉 ) = 𝐸[(𝜽

𝜎 2 −1
𝐵 (𝜉 ) + 𝐵 −1 (𝜉 )𝒃(𝜓, 𝜉 )𝒃(𝜓, 𝜉 )𝑇 𝐵−1 (𝜉)
𝑛
1

̂ − 𝜽)] = 𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 ) where 𝐵(𝜉 ) = ∫ 𝒇(𝑥 )𝒇𝑇 (𝑥)𝑑𝜉(𝑥) and 𝒃(𝜓, 𝜉 ) =
and 𝐸[(𝜽
−1
1

∫−1 𝒇(𝑥 )𝑥 𝑝 𝜓(𝑥 )𝑑𝜉(𝑥). Here 𝜉 is a design measure belong to some set ℱ.
Let
ℒ [𝑀𝑆𝐸 (𝜓, 𝜉 )] ∶= 𝑡𝑟[𝑀𝑆𝐸(𝜓, 𝜉 )] =
=

𝜎2
𝑡𝑟𝐵−1 (𝜉 ) + 𝑡𝑟𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )𝒃(𝜓, 𝜉 )𝑇 𝐵−1 (𝜉)
𝑛

𝜎2
𝑡𝑟𝐵−1 (𝜉 ) + 𝑡𝑟 𝐵𝑖𝑎𝑠(𝜓, 𝜉 )
𝑛

where 𝑡𝑟[𝐵𝑖𝑎𝑠(𝜓, 𝜉 )] = 𝑡𝑟𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )𝒃(𝜓, 𝜉 )𝑇 𝐵−1 (𝜉).

We consider the following problem, called bounded bias A-optimal design, that
𝑚𝑖𝑛𝜉𝜖ℱ 𝑡𝑟 𝐵−1 (𝜉 ) 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑚𝑎𝑥𝜓∈Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤ 𝑐.
When 𝜓(𝑥) ≡ 0, the problem becomes 𝑚𝑖𝑛𝜉𝜖ℱ 𝑡𝑟 𝐵−1 (𝜉 ), which is the usual A-optimal design.

Ideally, we hope that 𝑚𝑎𝑥𝜓∈Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| will be achieved on the boundary function ∅.
It could be true in some cases, see Liu and wiens (1997). We have that
𝑚𝑎𝑥𝜓∈Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )||2 = 𝑚𝑎𝑥𝑖=1,2 ||𝐵−1 (𝜉 )𝒃(𝜙𝑖 , 𝜉 )||2

(1.3)

where 𝜙𝑖 (𝑥) = (𝑠𝑔𝑛𝑥 )𝑖−1 𝜙(𝑥). However it is not true in general. A contra-example will be
given in Section 2. This will raise difficulty to maximize the loss function over the class Ψ. The
cause of the difficulty is due to the non-linearity and non-convexity of the 𝐵−2 (𝜉) as a function
of 𝜉.

Here are some possible ways to overcome this difficulty:
(i)
(ii)
(iii)

Let Ψ ∗ = {𝜓 ∶ 𝜓 ∈ Ψ, ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤ 𝑚𝑎𝑥𝑖=1,2 ||𝐵−1 (𝜉 )𝒃(∅𝑖 , 𝜉 )||.
Let ℱ ∗ = {𝜉 ∶ 𝜉 ∈ ℱ, 𝑚𝑎𝑥𝜓 ∈ Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| = 𝑚𝑎𝑥𝑖=1,2 ||𝐵−1 (𝜉 )𝒃(∅𝑖 , 𝜉 )||.
Choose some 𝜙 ∗ such that, for given 𝜉 ∈ ℱ we have
𝑚𝑎𝑥𝜓 ∈ Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| = 𝑚𝑎𝑥𝑖=1,2 ||𝐵−1 (𝜉 )𝒃(𝜙𝑖∗ , 𝜉 )||.

Another approach: Instead of ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤ 𝑐, we use ||𝒃(𝜓, 𝜉 )|| ≤ 𝑐 (modified bias).
i.e., we consider the problem:
𝑚𝑖𝑛𝜉𝜖ℱ ℒ[ 𝐵−1 (𝜉 )] 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| ≤ 𝑐

(1.4)

The reasons for us to consider the A-optimality are:
(i)
(ii)
(iii)

The loss function is additive.
||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )||2 = 𝑡𝑟 𝐵𝑖𝑎𝑠(𝜓, 𝜉 ) = 𝑡𝑟𝐵 −1 (𝜉 )𝒃(𝜓, 𝜉 )𝒃(𝜓, 𝜉 )𝑇 𝐵−1 (𝜉) =
𝒃(𝜓, 𝜉 )𝐵−2 (𝜉)𝒃(𝜓, 𝜉 )𝑇 .
If we use modified bias, then we have

𝑚𝑎𝑥𝜓 ∈ Ψ ||𝒃(𝜓, 𝜉 )||2 = 𝑚𝑎𝑥𝑖=1,2 ||𝒃(𝜙𝑖 , 𝜉 )||2 ; and 𝜆𝑛 ||𝒃(𝜓, 𝜉 )|| ≤ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤
𝜆1 ||𝒃(𝜓, 𝜉 )||, therefore ||𝒃(𝜓, 𝜉 )|| → 0 𝑖𝑚𝑝𝑙𝑖𝑒𝑠 ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| → 0.

2. Bounded Bias A-Optimal Design
We start this section by showing a contra-example of (1.3). Consider the regression model (1.2)
with p = 2, i.e.,
𝐸 [𝑦|𝑥] = 𝜃0 + 𝜃1 𝑥 + 𝑥 2 𝜓(𝑥) .
Let 𝜓 ∈ Ψ = {𝜓: | 𝜓(𝑥 ) ≤ 𝜙(𝑥 ) = 𝑥 2 𝑜𝑛 𝑆 ≔ [−1, 1 ]}. We are going to show that
𝑚𝑎𝑥𝜓 ∈ Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )||2 > 𝑚𝑎𝑥𝑖=1,2 ||𝐵−1 (𝜉 )𝒃(𝜙𝑖 , 𝜉 )||2 .

In this case, we have 𝑚𝑎𝑥𝑖=1,2 ||𝐵−1 (𝜉 )𝒃(𝜙𝑖 , 𝜉 )||2 = 0.619141. Let

−𝑥 2
𝜓𝑑 (𝑥 ) = { 0
𝑥2

− 1 ≤ 𝑥 ≤ −𝑑
−𝑑<𝑥 <𝑑 ,
𝑑≤𝑥 ≤1

where 0 ≤ 𝑑 ≤ 1, and let 𝜉 be the uniform probability measure on [-1, 1]. It is clear that
𝜓𝑑 (𝑥 ) ∈ Ψ. If we choose 𝑑0 = 0.6, we find that 𝒃𝑇 (𝜓𝑑0 )𝐵−2 𝒃(𝜓𝑑0 ) = 0.629725. Hence we
have
𝑚𝑎𝑥𝜓 ∈ Ψ 𝒃𝑇 (𝜓)𝐵−2 𝒃(𝜓) ≥ 𝒃𝑇 (𝜓𝑑0 )𝐵−2 𝒃(𝜓𝑑0 ) > 𝑚𝑎𝑥𝑖=1,2 {𝒃𝑇 (𝜙𝑖 )𝐵−2 𝒃(𝜙𝑖 )}.
A contra-example for continuous 𝜓 can be easily constructed as follows:
−𝑥 2
𝑑22
(𝑥 + 𝑑1 )
𝑑2 − 𝑑1
0
𝜓𝑑1 ,𝑑2 (𝑥) =
𝑑22
(𝑥 − 𝑑1 )
𝑑2 − 𝑑1
{ 𝑥2

− 1 ≤ 𝑥 ≤ −𝑑2
− 𝑑2 < 𝑥 ≤ −𝑑1
− 𝑑1 < 𝑥 ≤ 𝑑1
𝑑1 < 𝑥 ≤ 𝑑2
𝑑2 ≤ 𝑥 ≤ 1

With 𝑑1 and 𝑑2 near 0.6 and close to each other.

For some special cases, (1.3) may be true. Let us consider again the regression model (1.2) with
p = 2. Let ℱ𝑠 = {𝜉: 𝜉 ∈ ℱ 𝑎𝑛𝑑 𝜉 (−𝑥 ) = 1 − 𝜉 (𝑥 )} and Ψ𝑠 (𝜙) = {𝜓: |𝜓(𝑥 )| ≤
𝜙(𝑥 ) 𝑤𝑖𝑡ℎ 𝑏𝑜𝑡ℎ 𝜓(−𝑥 ) = 𝜓(𝑥 ) 𝑎𝑛𝑑 𝜙(−𝑥 ) = 𝜙(𝑥 ) 𝑜𝑛 𝑆}. Then we have
||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| = |∫ 𝑥 2 𝜓(𝑥 )𝑑𝜉(𝑥 )| ≤ ∫ 𝑥 2 |𝜓(𝑥)| 𝑑𝜉(𝑥) ≤ ∫ 𝑥 2 𝜙(𝑥 )𝑑𝜉(𝑥).
We conclude that 𝑚𝑎𝑥𝜓 ∈ Ψ ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| = ∫ 𝑥 2 𝜙(𝑥 )𝑑𝜉(𝑥) as we know that 𝜓 ∈ Ψ.

For 𝜙(𝑥) we consider two scenarios A and B.
Scenario A: Let 𝑙(𝑥 ) = 𝑥𝜙(√𝑥) such that 𝑙(𝑥) is convex on [0, 1].
Scenario B: Let 𝜙(𝑥 ) = ∑𝑘𝑖=0 𝑎2𝑖+2 𝑥 2𝑖 such that 𝜙(𝑥 ) ≥ 0 on S.

For scenario A, we have

𝑚𝑖𝑛𝜉 ∈ ℱ𝑠 𝑡𝑟𝐵−1 (𝜉 ) 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑚𝑎𝑥𝜓 ∈ Ψ𝑠 ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤ 𝑐

(2.1)

which is equivalent to
𝑚𝑎𝑥𝐸[𝑍] 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝐸 [𝑙(𝑍)] ≤ 𝑐
where 𝑍 = 𝑋 2 . By Jensen’s inequality, we have 𝑙(𝐸 [𝑍]) ≤ 𝐸[𝑙(𝑍)]. We know that 𝑙 (𝐸 [𝑍]) =
𝐸[𝑙(𝑍 )]. If and only if P(Z= z) = 1. Let 𝑧0 be a real number between 0 and 1 such that 𝑙 (𝑧0 ) =
𝑐 and P(Z = 𝑧0 ) = 1. We find that the solution to (2.1) is the design measure 𝜉0 (±√𝑧0 ) = 1/2
where 𝑧0 = 𝑙 −1 (𝑐) and max 𝜇2 = 𝑧0 .

For scenario B, (2.1) is equivalent to
𝑚𝑎𝑥𝜉 ∈ ℱ𝑠 𝜇2 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜

∑𝑘𝑖=0 𝑎2𝑖+2 𝜇2𝑖+2 ≤ 𝑐

(2.2)

Let
𝑘

𝑐∗ = 𝑚𝑖𝑛𝜉 ∈ ℱ𝑠 ∑

𝑘

𝑎2𝑖+2 𝜇2𝑖+2 𝑎𝑛𝑑 𝑐 ∗ = 𝑚𝑎𝑥𝜉 ∈ ℱ𝑠 ∑

𝑖=0

𝑎2𝑖+2 𝜇2𝑖+2 .

𝑖=0

Then (2.2) has solution for any 𝑐∗ ≤ 𝑐 ≤ 𝑐 ∗ . (2.2) has no solution for 𝑐 ≤ 𝑐∗ . For 𝑐 > 𝑐 ∗ ,
(2.2) has the same solution as the regular optimal design problem.

The above problem is solvable as it only depends on the first 𝑘 even moments. Numerical search
is needed to find the optimal solution. However, when k is small we can solve the problem
explicitly.
Case 1.

𝑘=1

𝜙 ( 𝑥 ) = 𝑎2 + 𝑎4 𝑥 2

In this case, we know that 𝜙(𝑥 ) ≥ 0 if and only if (i) 𝑎4 > 0, 𝑎2 ≥ 0 or (ii) 𝑎4 < 0, 𝑎2 + 𝑎4 ≥ 0.
The following lemma is trivial:
Lemma 2.1 If 𝜙(𝑥 ) ≥ 0, we have c∗ = 0 and
𝑐 ∗ ∶= 𝑚𝑎𝑥𝜉∈ ℱ𝑠 {𝑎2 𝜇2 + 𝑎4 𝜇4 } = 𝑚𝑎𝑥𝜉∈ ℱ0 {𝑎2 𝜇2 + 𝑎4 𝜇4 }
𝑎2 + 𝑎4
= {
𝛼

𝑎22

− 4𝑎

4

𝑖𝑓 𝑎4 > 0, 𝑎2 ≥ 0 𝑜𝑟 𝑎4 < 0, 𝑎2 + 2𝑎4 > 0
𝑖𝑓 𝑎4 < 0, 𝑎2 + 𝑎4 ≥ 0, 𝑎2 + 2𝑎4 ≤ 0

where ℱ0 = {𝜉: 𝜉 = 2 Δ±√𝑥 + (1 − 𝛼 )Δ0 , 0 ≤ 𝛼 ≤ 1, 0 ≤ 𝑥 ≤ 1}.

Let 𝜇2∗ be the maximum value of 𝜇2 in (2.2) and 𝜉 ∗ be the corresponding design measure. We
then have the following:
Theorem 2.2 If 𝜙(𝑥 ) ≥ 0, then for any 0 ≤ 𝑐 ≤ 𝑐 ∗ , we have
1

(i)

𝜉 ∗ = 2 Δ±1 𝑎𝑛𝑑 𝜇2∗ = 1, 𝑖𝑓 𝑎2 + 𝑎4 ≤ 𝑐;

(ii)

𝜉 ∗ = 2 Δ±√𝑧 𝑎𝑛𝑑 𝜇2∗ = 𝑧, 𝑤ℎ𝑒𝑟𝑒 𝑧 =

(iii)

𝜉∗ =

−𝑎2 +√𝑎22+4𝑎4 𝑐

1
τ

2𝑎4

Δ±1 + (1 − 𝜏)Δ0 𝑎𝑛𝑑 𝜇2∗ = 𝜏, 𝑤ℎ𝑒𝑟𝑒 𝜏 =

2

, 𝑖𝑓 𝑎2 + 𝑎4 > 𝑐, 𝑎4 > 0 𝑎𝑛𝑑 𝑎2 ≥ 0;
𝑐

𝑎2 +𝑎4

, 𝑖𝑓 𝑎2 + 𝑎4 > 𝑐, 𝑎4 < 0 𝑎𝑛𝑑 𝑎2 +

𝑎4 ≥ 0.

Case 2.

𝑘=2

𝜙 ( 𝑥 ) = 𝑎2 + 𝑎4 𝑥 2 + 𝑎6 𝑥 4

In this case, we have the following two lemmas:
Lemma 2.3 𝜙(𝑥 ) ≥ 0 on [-1, 1] if and only if
(i)
(ii)
(iii)
(iv)

𝑎6 < 0, 𝑎2 ≥ 0 𝑎𝑛𝑑 𝑎2 + 𝑎4 + 𝑎6 ≥ 0; or
𝑎6 > 0, 𝑎4 ≥ 0 𝑎𝑛𝑑 𝑎2 ≥ 0; or
𝑎6 > 0, 𝑎4 < 0, 𝑎2 > 0, 𝑎4 + 2𝑎6 ≥ 0, 𝑎𝑛𝑑 4𝑎2 𝑎6 − 𝑎42 ≥ 0; or
𝑎6 > 0, 𝑎4 < 0, 𝑎2 > 0, 𝑎4 + 2𝑎6 < 0, 𝑎𝑛𝑑 𝑎2 + 𝑎4 + 𝑎6 > 0.

Lemma 2.4 Let (𝑎2 , 𝑎4 , 𝑎6 ) satisfies one of the four conditions (i) – (iv) in Lemma 2.3. Let
𝑥0 =

−𝑎4 − √𝑎42 − 3𝑎2 𝑎6
.
3𝑎6

Then we have,
(i)
(ii)

𝑐∗ = 0.
𝑐∗ =
1

(2𝑎43 + 2𝑎42 √𝑎42 − 3𝑎2 𝑎6 − 9𝑎2 𝑎4 𝑎6 − 6𝑎2 𝑎6 √𝑎42 − 3𝑎2 𝑎6 ) 𝑖𝑓 0 < 𝑥0 < 1
{27𝑎62
𝑎2 + 𝑎 4 + 𝑎6
𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

Before we present our next theorem, we first define the following notations.
𝑐

𝑐

N1: 𝛼1∗ = 𝑎 +𝑎 +𝑎 , 𝛽1∗ = 𝑥1∗ = 0, 𝜈1∗ = 𝑎 +𝑎 +𝑎 , 𝑎𝑛𝑑 𝜉1∗ =
2

4

6

2

4

6

𝛼1∗
2

Δ±1 + (1 − 𝛼1∗ )∆0 .

8𝑎 2 𝑐

𝑎

𝛽∗

4𝑎 𝑐

6
N2: 𝛼2∗ = 0, 𝛽2∗ = 𝑎 (𝑎2 −4𝑎
, 𝑥2∗ = − 2𝑎4 , 𝜈2∗ = 4𝑎 𝑎 6−𝑎2 , 𝑎𝑛𝑑 𝜉2∗ = 22 Δ±√𝑥 ∗ + (1 − 𝛽2∗ )∆0 .
𝑎 )
4

2 6

4

6

2 6

(𝑎 +𝑎 )(𝑎 2+4𝑎 𝑎 −𝑎 2 )+8𝑎 2 𝑐

2

4

8𝑎 2 (𝑎 +𝑎 +𝑎 −𝑐)

𝑎 +𝑎

4
6
2 6
4
4
6
4
6
6
6
6 2
∗
N3: 𝛼3∗ = (𝑎 +3𝑎
, 𝛽3∗ = (𝑎 +3𝑎 )(3𝑎
,
2 +2𝑎 𝑎 +4𝑎 𝑎 −𝑎 2 ) , 𝑥3 = − 2𝑎
)(3𝑎 2+2𝑎 𝑎 +4𝑎 𝑎 −𝑎 2 )
4

6

4 6

6

2 6

4𝑎6 𝑐−(𝑎4 +𝑎6 )2

𝜈3∗ = 3𝑎2+2𝑎 𝑎 +4𝑎 𝑎 −𝑎2 , 𝑎𝑛𝑑 𝜉3∗ =
6

4 6

2 6

4

4

4

𝛼3∗
2

6

4 6

6

2 6

4

6

𝛽3∗

Δ±1 + 2 ∆±√𝑥 ∗ .
3

Furthermore, we set the following conditions.
C1: 𝑎2 + 𝑎4 + 𝑎6 ≤ 𝑐.
8𝑎 2 𝑐

𝑎

6
C2: 𝑥 ∗ = − 2𝑎4 ∈ (0, 1) 𝑎𝑛𝑑 𝛽 ∗ = 𝑎 (𝑎2−4𝑎
≤ 1.
𝑎 )
6

4

4

2 6

8𝑎 2(𝑎 +𝑎 +𝑎 −𝑐)

𝑎 +𝑎

4
6
6 2
C3: 𝑥 ∗ = − 42𝑎 6 ∈ (0, 1) 𝑎𝑛𝑑 𝛽 ∗ = (𝑎 +3𝑎 )(3𝑎
2 +2𝑎 𝑎 +4𝑎 𝑎 −𝑎 2) ∈ (0, 1).
6

4

6

6

4 6

2 6

4

After some analysis and calculation, we have found the solutions to (2.2). We omit the detailed
proof and summarize the results into the next theorem:

Theorem 2.5 Let (𝑎2 , 𝑎4 , 𝑎6 ) satisfies one of the four conditions (i) – (iv) in Lemma 2.3 and
0 ≤ 𝑐 ≤ 𝑐 ∗, where 𝑐 ∗ is indicated in Lemma 2.4. Then the solution to (2.2) is listed below:
1

(i)

If C1 is true, then 𝜉 ∗ = 2 Δ±1 𝑎𝑛𝑑 𝜇2∗ = 1.

(ii)

If C1 is not true, but C2 and C3 are true, then 𝜇2∗ = max{𝜐1∗ , 𝜐2∗ , 𝜐3∗ } ≔ 𝜐𝑖∗
and the corresponding design measure is 𝜉𝑖∗ , 𝑖 ∈ {1, 2, 3}.

(iii)

If C1 and C2 are not true, but C3 is true, then 𝜇2∗ = max{𝜐1∗ , 𝜐3∗ } ≔ 𝜐𝑗∗
and the corresponding design measure is 𝜉𝑗∗ , , 𝑗 ∈ {1, 3}.

(iv)

If C1 and C3 are not true, but C2 is true, then 𝜇2∗ = max{𝜐1∗ , 𝜐2∗ } ≔ 𝜐𝑘∗
and the corresponding design measure is 𝜉𝑘∗ , 𝑘 ∈ {1, 2} .

(v)

If C1, C2 and C3 are all not true, then 𝜇2∗ = 𝜈1∗ , and the corresponding design
measure is 𝜉1∗ =

𝛼1∗
2

Δ±1 + (1 − 𝛼1∗ )∆0 .

The following corollary is obvious.
Corollary 2.6 Let 𝜙(𝑥 ) = 𝑎2 + 𝑎4 𝑥 2 + 𝑎6 𝑥 4 ≥ 0 on [-1, 1], and 𝑎2 + 𝑎4 + 𝑎6 = 0. Then the
1

solution to (2.2) is 𝜉 ∗ = 2 Δ±1 , and 𝜇2∗ = 1.
Remark: Note that 𝑎2 + 𝑎4 + 𝑎6 = 0 if and only if 𝜙(±1) = 0, i.e., there is no violation at ±1.
The implication of Corollary 2.6 is that the bounded bias optimal design will be the same as
usual optimal design if the regression model is not violated at ±1.
Let us define the efficiency of the bounded bias optimal design 𝜉 ∗ as 𝑒(𝜉 ∗ ) = ℒ(𝜉 𝑜 )⁄ℒ(𝜉 ∗ ),
where 𝜉 𝑜 is the usual optimal design. It is clear that 𝑒(𝜉 ∗ ) is increasing in 𝜇2∗ . Consider the
𝑐
situation that 𝜇2∗ = 𝑎 +𝑎 +𝑎 . It is obvious that 𝑒(𝜉 ∗ ) is decreasing when c is decreasing for 𝑐∗ ≤
2

4

6

𝑐 ≤ 𝑐 ∗. This implies that we lose efficiency (smaller 𝑒(𝜉 ∗ )) to gain more protection on the
possible bias (smaller c) when the amount of model violations at ±1 are fixed.

Similar results would be expected for Ψ𝑠 (𝜙) = {𝜓: |𝜓(𝑥 )| ≤ 𝜙(𝑥 ) 𝑤𝑖𝑡ℎ 𝑏𝑜𝑡ℎ 𝜓(−𝑥 ) =
𝜓(𝑥 ) 𝑎𝑛𝑑 𝜙(−𝑥 ) = 𝜙(𝑥 ) 𝑜𝑛 𝑆}.
Extension to the multiple linear regression model is straight forward:
𝐸 [𝑦|𝒙] = 𝜽𝑇 𝒇(𝒙) + 𝜓(𝒙)
where 𝜽𝑇 = (𝜃0 , 𝜃1 , … , 𝜃𝑘 ) and 𝒇𝑇 (𝒙) = (1, 𝑥1 , … , 𝑥𝑘 ). Let Ψ𝑠𝑘 = {𝜓: |𝜓(𝒙)| ≤
𝜙(𝒙), 𝜓(𝑥1 , … , −𝑥𝑖 , … , 𝑥𝑘 ) = 𝜓(𝑥1 , … , 𝑥𝑖 , … , 𝑥𝑘 )}, 𝜙 (𝒙) = ∑𝑘0 𝑎𝑖2 𝑥𝑖2 and ℱ𝑠𝑘 =
{𝜉: 𝜉 𝑖𝑠 𝑠𝑦𝑚𝑚𝑒𝑡𝑟𝑖𝑐 𝑜𝑛 ? ? ? ⊆ 𝑹𝑘 }. Then we have

𝐵−1 (𝜉 ) = (

1
0
0

0
∫ 𝑥12 𝜉(𝒙)

⋯

⋮

⋱
⋯

0

1 0
0
0 𝜇2
0
)=(
⋮
⋮
2 ( )
∫ 𝑥𝑘 𝜉 𝒙
0 0

⋯
⋱
⋯

0
0
)
⋮
𝜇2

and
||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| = |∫𝑆 𝑘 𝜓(𝑥 )𝑑𝜉(𝑥 )| ≤ ∫𝑆 𝑘 |𝜓(𝑥 )| 𝑑𝜉 (𝑥 ) ≤ ∫𝑆 𝑘 𝜙(𝑥 ) 𝑑𝜉 (𝑥 ) = (∑𝑘0 𝑎𝑖2 )𝜇2 .

For A-optimality, it is clear that

𝑚𝑖𝑛𝜉∈ℱ𝑠 𝑡𝑟𝐵−1 (𝜉 ) 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑚𝑎𝑥𝜓∈Ψ𝑘𝑠 ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤ 𝑐
is equivalent to
𝑚𝑎𝑥𝜉∈ℱ𝑠 𝜇2 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 (∑𝑘0 𝑎𝑖2 𝜇2 ) ≤ 𝑐.

We find
𝑐

1

𝑐

𝜇2∗ = max 𝜇2 = ∑𝑘 𝑎2 with 𝜉 ∗ (… , ±√𝑥𝑖 , … ) = 2𝑘 𝑎𝑛𝑑 𝑥𝑖 = ∑𝑘 𝑎2 ≤ 1.
1 𝑖

1

𝑖

1

If 𝑐 ≥ ∑𝑘𝑖 𝑎𝑖2 , then 𝜇2∗ = 1 with 𝜉 ∗ (… , ±√𝑥𝑖 , … ) = 2𝑘 .

3. Modified Bounded Bias A-Optimal Design

As we mentioned before, instead of ||𝐵−1 (𝜉 )𝒃(𝜓, 𝜉 )|| ≤ 𝑐, we may use ||𝒃(𝜓, 𝜉 )|| ≤ 𝑐
(modified bias). i.e., we consider the problem:
𝑚𝑖𝑛𝜉𝜖ℱ ℒ[ 𝐵−1 (𝜉 )] 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| ≤ 𝑐.
We choose ℒ[ 𝐵−1 (𝜉 )] = 𝑡𝑟 𝐵−1 (𝜉 ). That is
𝑚𝑖𝑛𝜉∈ℱ 𝑡𝑟𝐵−1 (𝜉 ) 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| ≤ 𝑐,

(3.1)

where ℱ = {𝜉: 𝜎(𝜉 ) ⊆ [−1,1]} and Ψ = {𝜓: |𝜓(𝑥 )| ≤ 𝜙(𝑥 ), 𝜙(−𝑥 ) = 𝜙(𝑥 ), 𝜙(𝑥 ) ≥
0 𝑜𝑛 [−1,1]}.

With the modified bias, we have the following:
Theorem 3.1
𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| = 𝑚𝑎𝑥𝑖=1,2 ||𝒃(𝜙𝑖 , 𝜉 )|| 𝑤ℎ𝑒𝑟𝑒 𝜙𝑖 (𝑥 ) = (𝑠𝑔𝑛𝑥 )𝑖−1 𝜙(𝑥 ), 𝑖 = 1,2.
Proof:

Given 𝜉 ∈ ℱ, we have
2

||𝑏(𝜓, 𝜉 )|| 𝒃𝑇 (𝜓, 𝜉 )𝒃(𝜓, 𝜉 )

∫ 𝑦 𝑝 𝜓(𝑦)𝑑𝜉(𝑦)
= (∫ 𝑥 𝑝 𝜓(𝑥 )𝑑𝜉(𝑥 ), … , ∫ 𝑥 2𝑝−1 𝜓(𝑥 )𝑑𝜉(𝑥 ))

⋮
(

∫ 𝑦 2𝑝−1 𝜓(𝑦)𝑑𝜉(𝑦)

)

= ∬ 𝑥 𝑝 𝑦 𝑝 𝜓(𝑥 )𝜓(𝑦)𝑑𝜉(𝑥 )𝑑𝜉(𝑦) + ⋯ + ∬ 𝑥 2𝑝−1 𝑦 2𝑝−1 𝜓(𝑥 )𝜓(𝑦)𝑑𝜉 (𝑥)𝑑𝜉(𝑦)
𝑆×𝑆

𝑆×𝑆

= ∬ 𝑥 𝑝 𝑦 𝑝 (1 + 𝑥𝑦 + ⋯ + 𝑥 𝑝−1 𝑦 𝑝−1 )𝜓(𝑥 )𝜓(𝑦)𝑑𝜉(𝑥 )𝑑𝜉(𝑦)
𝑆×𝑆

1 − (𝑥𝑦)𝑝
(𝑥𝑦)𝑝 𝜓(𝑥 )𝜓(𝑦)𝑑𝜉(𝑥 )𝑑𝜉(𝑦)
1
−
𝑥𝑦
𝑆×𝑆

=∬

1 − (𝑥𝑦)𝑝
(𝑥𝑦)𝑝 𝜓(𝑥 )𝜓(𝑦)|𝑑𝜉(𝑥 )𝑑𝜉(𝑦)
≤∬ |
1
−
𝑥𝑦
𝑆×𝑆
1 − (𝑥𝑦)𝑝
|(𝑥𝑦)𝑝 |𝜓(𝑥 )𝜓(𝑦)𝑑𝜉(𝑥 )𝑑𝜉(𝑦)
1
−
𝑥𝑦
𝑆×𝑆

≤∬

1 − (𝑥𝑦)𝑝
=∬
(𝑠𝑔𝑛𝑥)𝑝 𝑥 𝑝 (𝑠𝑔𝑛𝑦)𝑝 𝑦 𝑝 𝜙(𝑥 )𝜙(𝑦)𝑑𝜉 (𝑥 )𝑑𝜉(𝑦)
𝑆×𝑆 1 − 𝑥𝑦

1 − (𝑥𝑦)𝑝 𝑝 𝑝
𝑥 𝑦 𝜙(𝑥 )𝜙(𝑦)𝑑𝜉 (𝑥)𝑑𝜉(𝑦)
𝑆×𝑆 1 − 𝑥𝑦
=
1 − (𝑥𝑦)𝑝 𝑝 𝑝
∬
𝑥 𝑦 (𝑠ℎ𝑛𝑥)𝜙(𝑥 )(𝑠𝑔𝑛𝑦)𝜙(𝑦)𝑑𝜉(𝑥 )𝑑𝜉(𝑦)
{ 𝑆×𝑆 1 − 𝑥𝑦
∬

={

||𝒃(𝜙1 , 𝜉 )||2
||𝒃(𝜙2 , 𝜉 )||2

𝑖𝑓 𝑝 𝑒𝑣𝑒𝑛
𝑖𝑓 𝑝 𝑜𝑑𝑑.

Hence, we have proved 𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| = 𝑚𝑎𝑥𝑖=1,2 ||𝒃(𝜙𝑖 , 𝜉 )||.

𝑖𝑓 𝑝 𝑖𝑠 𝑒𝑣𝑒𝑛
𝑖𝑓 𝑝 𝑖𝑠 𝑜𝑑𝑑

Corollary 3.2
(i)
(ii)

𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| = ||𝒃(𝜙1 , 𝜉 )|| if p is even.
𝑚𝑎𝑥𝜓∈Ψ ||𝒃(𝜓, 𝜉 )|| = ||𝒃(𝜙2 , 𝜉 )|| if p is odd.

Let 𝑇: ℱ → 𝑹1 by 𝑇(𝜉 ) = 𝑚𝑎𝑥𝑖=1,2 ||𝒃(𝜙𝑖 , 𝜉 )|| and ℱ𝑐 = {𝜉: 𝜉 ∈ ℱ, 𝑇(𝜉) ≤ 𝑐}. Then we have
the following three Lemmas.
Lemma 3.3
(i)

ℱ𝑐 is convex. (ii) 𝜉 (𝑥 ) ∈ ℱ𝑐 𝑖𝑓𝑓 1 − 𝜉 (−𝑥 − ) ≔ 𝜉 − (𝑥 ) ∈ ℱ𝑐 .

Proof: (i) ∀ 𝜉1 , 𝜉2 ∈ ℱ𝑐 , we have ||𝑏(𝜙𝑖 , 𝜉𝑗 )|| ≤ 𝑐, 𝑖 = 1,2, 𝑎𝑛𝑑 𝑗 = 1,2.Let 𝜉 ∗ = 𝜆𝜉1 +
(1 − 𝜆)𝜉2 (0 ≤ 𝜆 ≤ 1). Then we have ||𝒃(𝜙𝑖 , 𝜉 ∗ )|| = ||𝜆𝒃(𝜙𝑖 , 𝜉1 ) + (1 − 𝜆)𝒃(𝜙𝑖, 𝜉2 )|| ≤
λ||𝒃(𝜙𝑖 , 𝜉1 )|| + (1 − λ)||𝒃(𝜙𝑖 , 𝜉2 )|| ≤ λc + (1 − λ)c = c for i = 1,2 . Hence 𝜉 ∗ ∈ ℱ𝑐 .
(ii)

∀𝜉 ∈ ℱ𝑐 , let 𝜉 − = 1 − 𝜉(−𝑥 − ) we then have
1
𝑇(

𝑏 𝜙𝑖 , 𝜉

−)

1
𝑝

= (∫ 𝑥 𝜙𝑖 (𝑥 )𝑑𝜉

−(

𝑥 ) , … , ∫ 𝑥 2𝑝−1 𝜙𝑖 (𝑥 )𝑑𝜉 − (𝑥 ))

−1

−1

1

1

= (∫ 𝑥 𝑝 𝜙𝑖 (𝑥 )𝑑(1 − 𝜉 (−𝑥 − ) , … , ∫ 𝑥 2𝑝−1 𝜙𝑖 (𝑥 )𝑑(1 − 𝜉 (−𝑥 − ))
−1
−1
1
1
𝑝 (
= (∫−1(−𝑥) 𝜙𝑖 −𝑥 )𝑑𝜉(𝑥 ) , … , ∫−1(−𝑥)2𝑝−1 𝜙𝑖 (−𝑥 )𝑑𝜉(𝑥 ))
1
1
= ((−1)𝑝+𝑖−1 ∫−1 𝑥 𝑝 𝜙𝑖 (𝑥 )𝑑𝜉(𝑥 ) , … , (−1)2𝑝+𝑖−2 ∫−1 𝑥 2𝑝−1 𝜙𝑖 (𝑥)𝑑𝜉(𝑥 )).

We have shown ||𝒃(𝜙𝑖 , 𝜉 )|| = ||𝒃(𝜙𝑖 , 𝜉 − )||. Hence 𝜉 − (𝑥) ∈ ℱ𝑐 ,

Lemma 3.4

Let 𝐴 = (𝑎𝑖𝑗 )𝑛×𝑛 𝑎𝑛𝑑 𝐵 = (𝑏𝑖𝑗 )𝑛×𝑛 be two non-singular matrices. Denote 𝐴−1 =

∗
∗
∗
(𝑎𝑖𝑗
)𝑛×𝑛 𝑎𝑛𝑑 𝐵 −1 = (𝑏𝑖𝑗
)𝑛×𝑛 . If 𝑏𝑖𝑗 = (−1)𝑖+𝑗 𝑎𝑖𝑗 then 𝑏𝑖𝑗∗ = (−1)𝑖+𝑗 𝑎𝑖𝑗
.

Proof:

Let 𝑃 = (𝑝𝑖𝑗 )𝑛×𝑛 where
𝑝𝑖𝑗 = {

(−1)𝑖
0

𝑖𝑓 𝑖 = 𝑗
𝑖𝑓 𝑖 ≠ 𝑗

We then have 𝑃 −1 = 𝑃, and 𝐵 = 𝑃𝐴𝑃. Hence, we get 𝐵−1 = 𝑃 −1 𝐴−1 𝑃 −1 = 𝑃𝐴−1 𝑃. This
∗
implies that 𝑏𝑖𝑗∗ = (−1)𝑖+𝑗 𝑎𝑖𝑗
.

Let ℱ𝑐 (𝐴) = {𝜉𝑜 : 𝜉𝑜 ∈ ℱ𝑐 , 𝑡𝑟𝐵−1 (𝜉𝑜 ) = 𝑚𝑖𝑛𝜉∈ℱ𝑐 𝑡𝑟𝐵−1 (𝜉 )}. We have
Lemma 3.5
(i)
(ii)

ℱ𝑐 (𝐴) is a convex subset of ℱ𝑐 .
𝜉(𝑥) ∈ ℱ𝑐 (𝐴) if and only if 𝜉 − (𝑥) ∈ ℱ𝑐 (𝐴).

Proof:
(i)

For any 𝜉1 , 𝜉2 ∈ ℱ𝑐 (𝐴), we have 𝜉1 , 𝜉2 ∈ ℱ𝑐 and hence 𝜉 ∗ = 𝜆𝜉1 + (1 − 𝜆) 𝜉2 ∈ ℱ𝑐 .

Therefore, 𝑡𝑟𝐵−1 (𝜉1 ) = 𝑡𝑟𝐵−1 (𝜉2 ) = 𝑚𝑖𝑛𝜉∈ℱ𝑐 𝑡𝑟𝐵−1 (𝜉 ) . Because 𝐵(𝜉 ∗ ) = 𝜆𝐵(𝜉1 ) +
(1 − 𝜆)𝐵(𝜉2 ), hence we get
𝐵−1 (𝜉 ∗ ) = [𝜆𝐵(𝜉1 ) + (1 − 𝜆)𝐵(𝜉2 )]−1 ≤ 𝜆𝐵−1 (𝜉1 ) + (1 − 𝜆)𝐵−1 (𝜉2 )
and
𝑡𝑟𝐵−1 (𝜉 ∗ ) ≤ 𝜆𝑡𝑟𝐵−1 (𝜉1 ) + (1 − 𝜆)𝑡𝑟𝐵−1 (𝜉2 ).
This implies that 𝑡𝑟𝐵−1 (𝜉 ∗ ) = 𝑡𝑟𝐵−1 (𝜉1 ) = 𝑚𝑖𝑛𝜉∈ℱ𝑐 𝑡𝑟𝐵−1 (𝜉 ). We have proved 𝜉 ∗ ∈ ℱ𝑐 (𝐴).
(ii)

Let 𝜉1 ∈ ℱ𝑐 (𝐴). Then 𝜉1 ∈ ℱ𝑐 and 𝑡𝑟𝐵−1 (𝜉1 ) = 𝑚𝑖𝑛𝜉∈ℱ𝑐 𝑡𝑟𝐵−1 (𝜉 ). By Lemma 3.3

we know that 𝜉1− (𝑥) ∈ ℱ𝑐 . We need only to show that 𝑡𝑟𝐵−1 (𝜉1−1 ) = 𝑡𝑟𝐵−1 (𝜉1 ). It follows by
Lemma 3.4.

The implication of Lemma 3.5 is that there exists a symmetric optimal design as the solution to
(3.1) namely,

𝜉(𝑥)+ 𝜉 − (𝑥)
2

𝑖𝑓 𝜉 (𝑥) ∈ ℱ𝑐 . Therefore, we only need to search for the optimal

solutions within the class of symmetric design measures.

References

Agostinelli, C. Robust model selection in regression via weighted likelihood methodology.
Statistics and Probability Letters, 2002, 56, 289-300.
Blanchard, W. Field, C.A. and Ronchetti, E.M. Robust linear model selection by crossvalidation. J. Amer. Stat. Assoc., 1997, 92, 1017-1023.
Box, G.E.P. and Draper, N.R. A basis for selection of a response surface design. J. Amer.
Statist. Assoc. 1959, 54, 622-654.
Fedorov, V.V. Theory of optimal experiments. Academic Press, 1970, New York.
Khan, J.A. Van Aelst, S. and Zamar, R.H. Robust Linear Model Selection Based on Least
Angle Regression, J. Amer. Stat. Assoc., 2007, 102, 1289-1299.

Kiefer, J. and Wolfowitz The equivalence of two extreme problems. Canad. J. Math 1960, 12, 3,
363-366.
Li, K.C. Robust regression design with the design space consists of finitely many points. Ann.
Statist, 1984 12, 2, 269-282.
Marcus, M.B. and Sacks, J. Robust designs for regression problems. Statistical Theory and
Related Topics II. 245-268, Academic Press, 1977, New York.
Pesotchinsky, L. Optimal robust designs: Linear regression in Rk, Ann. Statist, 1982, 10, 511525.
Wiens, D.P. Minimax designs for approximately linear regression. J. Statist. Planning and
Inference, 1992, 31, 353-371.