1. science
2. mathematics
3. arithmetic

RATES OF GROWTH AND SAMPLE MODULI FOR WEIGHTED
EMPIRICAL PROCESSES INDEXED BY SETS
by
Kenneth S. Alexander
TECHNICAL REPORT No. 86
Revised June 1986
Department of Statistics, GN-22
University ()fWashiI1gtrin
Seattle, W;lshington 98195 USA
Rates of Growth and Sample :Moduli for Weighted
Empirical Processes Indexed by Sets
Kenneth S. Alexander"
University of Washington**
Revised June 1986
ABSTRACT
Probability inequalities are. obtained for the supremum of a
weighted empirical process indexed. by a Vapnik-Cervonenkis
class C of sets. These inequalities are particularly useful under
the assumption P(ufC E: C : P(C) < t D -? 0 as t -? o. They are used
to obtain almost sure bounds on the rate of growth of the process as the sample size approaches infinity, to find an asymptotic
sample modulus for the unweighted empirical process, and to
study the ratio Pn/P of the empirical measure to the actual
measure.
Key words and phrases:
Cervonenkis class,
process, law of the
modulus.
it
""~,,,t..rl
Icgarfthm, Vaonik-
60FI0
AM'S 1980 subject classifications:
Abbreviated title: Growth rates for
~mnlrl(>al processes
1 Research 5l1l!)nlllrte~d under an NSF Postdoctoral Fel]o~rship
11686, and in
NSF
No. DMS~301807.
DelJartm.ent
No. MeS8S-
of Maithemsltics, UrJ.versitv of Southern
L Introduction
ure
n s:
Pn'. -- n- 1",Lli=lUX..
,
lin;
=n
-P).
may be
as a stochastic process
C.
C
1)d: = H-"".x] : x E
$. 1171. becomes a normalized
distribution
tion; we call
the d.f. case. Properties of 1171. in
case,
in
related
case
C
class ld of
of ]Rd.
been extensively
studied. Recently, attention has been given to more general classes of sets or
functions, both in the theory (Dudley (1978), Gine and Zirin (1984),
Cam
1, 198
1111.
To
detailed information about the behavior of 1171. on small sets. it is
often helpful to weight 1171. at each set C by a function of
c;2(C): = P(C)(l - P(C)), the variance of IIn (C ). Since c;2(C) '" P(C) as P(C) -7 0,
this is often equivalent to weighting by the same function of P(C); when convenient we do the latter. In particular, given a nonnegative nondecreasing
function q E C[O,l] and sequences 771. .... and (an). we may ask for a finite R
and a sequence (on) such that
°
$ =R
P[lvn(C)1 > onq(c;2(C))
(1.2)
for someC
E
C with c;2(C) ~ 711.] .... 0,
a particular rate.
ask for a functioIl gsuch that (1.1) or (1.2)
some
< 0 < 00,
i.e.
:C
-
co
E
<
I
00
< {J < 1,
<00
(1
Lx
a.s.
{
a.s,
-2is
:C
E
on
I
a.s.
1
This is f!.enelralize:d
exact rates
that (1.5)
,-,"'-l"C,
1982b). Note
1
(1.6)
li17"LnSUpHlIn(C)I/(2P(C)LP(C)-1)2: C E I 1,{57n :?, P(C):?' 0711J = 1 a.s.
1
which essentially says that the function t (Lt -2) 2' is a local asymptotic
modulus of continuity for lin on 11 , A precise definition of "local asymptotic
modulus" will be given in section 4.
Van Zuijlen
showed
d-dimensional
for each
P(C)·~ 1
)Ln
and
(1.8)
p[Pn(C) ~ (K Ln)-l P(C) for some C
E
D d with Pn(C) r6 0]
< n-{1+o)
for all n ;;;;; no. These results were applied to the asymptotic theory of rank
statistics. Breiman et al. (1984) showed that if C is a Vapnik-Cervonenkis, or
"VC", class of sets (defined below), then for o,e > 0 there exist K andno such
that
(1.9)
P[supHPn(C)/P(C) -
11: P(C)
;;;;;Kn- 1Ln, C
Eel> e]
=0(n-{1+0).
In Atexanaer
more general classes of sets and functions. mcrucnng
constants and cutoff
(the (b n ), (7n).
for VC classes as
in the speera;
"'.J.'-'......."', however, do
full
1.9)
only 11,",'"'''''' bounds, except
For T> 0 define fJT to be the solution B.> 1 of fJ(logfJ - 1)
fJ",,: 1.
,-1)-1
as
T -l> 0,
fJT . . .
=
(1.
fJT -
1 ......
- 1~
T-l>
as
for all T> 0,
=
(1.1
=
+
00,
+
=(1 -
T)/T, and
-3in the one-cnmenstonai d.f. case with P
(1.13)
): C
E:
D 1,a2(C) ~
I,.J
=R
a.s.
where
t
t
R = {2(a + 1»2 and bn = (LLn)2 if n- 1LLn = o{,n) and LL,:;:;l/LLn
t
t
R = max(2,T2 {P-r - 1» and bn = (LLnf2i f ,71. =
Tn-
-+
a
1LLn for all n
t
R = 1 and bn = LLn/«n,n)2L(LLn/n,n» if
(1.14)
In
=o (n-
1LLn)
Wellner (1978) showed that,
(1.15)
and LLn/L(LLn/n , n)
t eo
P uniform on [0,1), if n- 1
~
E: D
-+
= 0(,71.) then
0 in probability,
while if
(1.16)
SU7JJll-'_(C)/P{C) - 11: C
E: D l,P{C)
In~ -eDa.s.
Note that the growth rates (bn ) in (1.13) differ from those in (1.3)-(1.6),
and the cutoff levels (,n) in (1.15) and (1.16) differ from those in (1.9); we
would like to understand such differences from a general point of view. In this
paper we present an approach which unifies all the results (1.3)-(1.9), (1.13),
and (1.15)-(1.16). We will extend them to all VC classes of sets,including
extension to higher dimensions of the d.f. and interval cases, and show how to
choose optimal q ,(bn ), and (,n) in general.
aSjrm.1Pto,tic mqdulus for 1171. ,call
Gaus:sian process Gpwhich is
it'ljll
1
the weak limit of
(1.17)
lin
on C. That is, ql(t):= 1/Jl(t 2 ) should satisfy
0 < lirnsuPt ->0 sup Hcp(C)I/q 1{U 2(C» : C
E:
C, u 2 (C ) ~ t ~ < co
a. s.
The problem of findingsuchaqtwas considered in Alexander (1986). The
main result can be summarized as follows.
t ~ 0:
c,
:=
lC
E:
) ~ t,P(C) ~ 2 U lce : C E: C,u 2 {C) ~ t,P{C) > ~ ~
C:
Ct,s:= lC\D: C
aCt) :=
) :=
t
E:
Ct , u 2(C\D) ~ s ]
-4-
give a lower bound on the number needed. This is quantified
section 3 using
the concept of a
C. Note that since IIn (C ) = - IIn(CC) , one can
often simplify matters, especially the definition of Ct , by
the concntion that P(C) ;;;; 2 for all C E C,
considering separately
C EC
with P(C) ;;;; ~
and the complement of those with p(C) > 2' This means
a 2(C) is of the order of P(C) for those sets of principal interest, i.e. those
with small a 2 (C ) . For a reasonably regular VC class C.
1
ql(t):= (2t(Lg(t)+LLt- I
(1.18)
»"2
satisfies (1. 17).
Suppose q ;;;; qi is given, and we wish to find growth constants (bn )
such that (1.1) or (1.2) holds. Since lin is less well-behaved on smaller sets,
we might expect that roughly
sup lilln (C)I/q (a 2(C» : C
2(C»
llIn (C)I
q l(a
q(a 2C»
~ sup qI(a 2(C»
!
for some 0 < R <
00.
E
C, r«
.
. C
; ; a 2(C) s an}
2
E
_
C. a (C) - Y«
I
This. along with the standard LIL. leads us to expect (b n
)
1
in (1.1) to be on the order of qI(7n)/q(7n) or (LLn)2, whichever is greater.
1
For example. when q(t)
= t"2
we expect bn to be on the order of
(1.19)
which is true provided 7n is not too small, as Theorem 3.1 shows.
Upper bounds for (bn ) can be obtained by replacing get) with its upper
bound t- I ; this is equivalent to disregarding the possibility that E t may be
only a small part of the whole space X. This is reasonable in the non-I.i.d.
case. where g (t) might vary with n in a complicated way, and is the reason
the present results improve on those in Alexander (1985). In the onedimensional d.f, case, for example, g (t) == t , so if L7:;; 1 rv Ln then the rate
1
1
(1.19) is (LLnf2 (d. (1 13» while the upper bound rate is (Ln)"2.
approach remedies this deficiency and gives exact results in a large number
of cases.
-5H. An Inequality
~~!S:S:~~~~or VC
sup~card
n C : C E: Cs : card{F)
=nF c Xs < Zn
for some n ~ 1. The
..... '"'u.~ ..... '" the
of C and denoted V(C).
eHJLPSOIC[S, all
Examples in R d ~H'_J.U"''orthants {-oo,x], or all
with at most lc
(Ie
Any
of
a VC class is a VC class, and ~C\D : C ,D E: C 5 or !C ill) : C ,D E: C 5 a VC class if C
is one. See Dudley (1978, 1984)
VC classes.
and
s ,t
separable for all
i, s for the topology of pointwise convergence (i.e. the topology in which Ci ~ C
if and only if le.\ ~ lc pointwise). Further, we assume that the r.v.'s Xi are
canonically formed, i.e. that they are defined on the probability space
(X"', a,"',P) with Xi the ith coordinate function, where P = P"'. This terminology
comes from Gaenssler (1983).
Given the law P, define a
functions
on
!::!~~~.9:
andC(t)
c
s
P(C) ~ ~,0'2(C) ~ (t) for all C
E:
C(t).
Given a P-stratified VC class,
Et
:= Uc € C{t)C
get) := aU )/«t).
Q :=
a convention,
E:
:q
~
0, q t, q(t
monotone runctrons f
.-
t
t
if
if
f
-6¢ to be O.
Theorem 2.1.
(C
I), t E [-l,aJ,
a
g E Q, n ~ 1, and p,b,u ~ O.
z(t):= g(t)/~(t)
z(t ),g(t), and
)/t are nonincreasrng. Define
VC
and
1
(2.1)
r:= [(a- 1(p/n)Az- 1(2n 2/b»vg-1(u/b)vJ'Jl\a
1
s := in:f<t ~ 0: z(t);:i 2n2/b~.
K
There exists a
=K(V(C(a») such that if
q2(t)/~(t)Lg(t) ~
(2.2)
Kb-2 for all r v s
-1
g(r)
(2.4)
(2.5)
-1
;:i
t
;:i
rvs;:i t
f
a,
;:i a,
<s and r
if
~
-~
q(t)L(bg(t)/n2~(t»/L(na(t»~Kn 26- 1
for all t
E
[r,s)n[r,aJ,
then
(2.6) P[Il/n(G)! > bg(t) + u: for some
~p
J';:i
t
;:i a
and G E C(t)J
a
+ 36 f t-1exp(-b2q2(t)/512~(t»dt
1'/2
1
+ 68exp(-bg(r)n 2/256).
1
q(t)PLg(t)t and g(t)PL(bg(t)/n2~(t»ton [/"aJ
(2.7)
for some 0
;:i {J
< 1, and if
1
1
bg(r)n 2L(g(r)A (bg(r)/n2~(r»)~ (1_{J)-1,
(2.8)
(2.9) P[!l/n(G)! > bg(t) + u for some
;:i P
+
J';:i
t
;:i
a and G E C(t)J
+
(bg(r)/n
and
- 7-
Remark 2.2.
the
function 9 0.-"; I.Uctn
Hence for a
nonincreasing.
lion.
I
,or", 111""".' we
assume h",'''r>C'""f"\lr1"
possibility of modifying ?f makes (2.7) a
,-,,.,,,,"'''' to
fol9 (or ?f) is
mild condi-
-8HI. The Square Root Weight Function
section may
compared to
lies
to obtamtng
eenavior of
Nz(u, C. P):
lYnn"'",
bounds
=minlk ;;;; 1: there
13).
lim sup
existCt..... Ck
miTl.;.iO:k P(C 6.C;.)
E
C such that
< u Z for all C
E C~.
The function log N z( •• C.P) is called the
entropy of C in LZ(P). (Note
P(C aD)
/lle - 1n lIi2(p)') It measures the size of C, telling us "how totally
Cis.
< t &1/4)byPt(A
peA () ef}/p(Et ) .
Hence usingLernrna2. 7 of1\1exancier
of Dudley (1978), we have for any
=
1.
(3.1)
1._1.
N Z(u t 2.Dt .P ) = N Z(u t 2P (E t ) 2,D toP t )
& 2(16g(t)u- 2L(8g(t)u-2» V(C)- t
& K(g (t )/u Z ) V(C)-
for some constant K
sets C 1.' " ,C"
E
=K(o,V(C»
1+6
for each 0 > O.
the following concepiwillhe use{::Jv,'r~T sufficiently small A>· o there
sumcierrtty
t > a there are k ;;;; E:>.11 (t )1->..
C with
(T2(C,;)
if
=t
and p(ein (Uj",,';Cj
»& A peed.
of nearly-disjoint sets of any
Ci's
be cnosen
Theorem 3.1.
Cbe a
Won:=
Yn :=
c 1 := lim
SUPnWn
Cg:= lim SUPnWn
for some
-9-
A =A(6) <
oo
for all t,6 > o and
U
E
(0,1).
(A) Then
limsuPnsuPfllln(C)VbnO"(C): C E C,0"2(C) ~ I'n~
(3.4)
=R
a.s,
where
(I)
if
n -l wn
1
=0(/71.)
.1
n- 1w;
and
decreases,
then
bn
=w;J.
and
R ~ (2(PC1+C2+Cs)f2;
(li) if 771. '" Tn- 1w,; for some
1
T>
1
R ;;;; max«2(pc 1+ C2+ ca)) 2", T-Z(fia-r-
(iii) if 771. =
o(n-lwn ) , n-1Yn
1», where
J.
w; decreases,
B = (pc 1+ ca)-l;
0 and n- 1
J.
then bn = W; and
decreases, and
(3.5)
=
then bn Yn and R ~ pc 1 + Cll'
(B) If p ~ 1, C is full, the lim sups in (3.2) are actually limits, and (for (i)
and (li) only) Lg(t )/Lr 1 is nondecreasing, then the upper bounds for R in (i)(iii) above are also lower bounds, so (i)-(m) give the true values of R in (3.4).
(C) If the assumptions in (B) hold, C is spatially full, and Cll 0, the "lim
sup" may be replaced by "lim" in (3.4) for each of (i)-(ui).
I
By (3.1), (3.3) is always valid with 7]/2 = p = V(C)- L Later
however, willshow< thatthis need notb~ optimal. In fact, we will
with V(C) arbitrarilylarge.
=
Remark 3.2.
The condition (3.3) is related to the "relative metric entropy" condition
(1.25) in Alexander (1985). In fact, is easy to verify that
1
N 2(ut 2 ' C t \C(1-u 2/4)t 'P ) ;;;; NU(u/2,C,P)
for all 0 < U < 2 and t > 0,
Wh,F'T"p
NU is defined
Alexander (1985).
aO-l
in (3.4)
of R
(3.4) are as follows.
C. c 1 comes
.,..,.1'1,,,,1'1 must be considered; this
numner of
values
cr 2( C)
must be
C3 comes
requirement that sums of certain
probabilities, for geometrically increasina subsequences of values of n, must
be finite, as
some proofs of the
If C is full, it is clear that pmust be at
1 if g is unbounded, hence in
T'lRrtlI""1l1Fll'" if C 1 >
there
Theorem 3.1 (B) arising
When
(3.6)
LL-,:;;.1 = o (LLn) and Lg(-'r,J = o (LLn),
1
we see from Theorem 3.1 (i) that the value of R in (3.4) is 2'2. This is true, for
ing to the tails in the d.f. case, is the sole determiner of the rate of growth of
the weighted empirical process when (3.6) holds .
.Kxa.mple 3.3
p=
J.
Theorem
Y1I.
=LLn/((n-'1I.)2·L(LLn/n-'rzJ),
we have (3.5)
3.1.
I
Th'''rll'''PTn
.c!
=
2
= aca.
and
- 11 -
In (ii}, if 71'1. . . . An- ILLn, we get
T
= "A(d -1)-1, pc 1 + c2 + Cs = (d + 1)/(d -1). and 0 = (d -1)/d.
In (iii). if (3.5) holds. i.e. if n- 1(Ln)-t = 0(71'1.) for all e > 0, the values are
W n ......
(d -1)LLn.cl = l.cs = (d -1)-1. and (d -1)(PCl +cs) = d.
To see that Ud is full. :fix
(3.7)
F:
a < "A < 1 and a < t < I, and let J1.:
="Ad- 1 and
= ~[a.x]: x =(f (t)J1.-it.J1.i 2•... ,J1.id ) E [a.l]d
for some integers ji G a with
it = El;'2ji~
where f (t) ~ ~ is given by ! (t )(1-! (t» = t. Then
(3.8)
IFI
=lHh.... ,jd) E ints~-1
: Et=2it ~
e« {(log r 1)/ (log J1.- 1»
d- l
(lo,g! (t)-I)/(log J1.- 1HI
1
G l:d),.g (t )
for some constants l:d and eo: where ints ... denotes the nonnegative integers.
Since P{C n (VD€F.D"cCD» ;;i dp.?{C) for C E F. this shows thatD d is full.
This establishes (up to the proof of (3.3» the
for d
> 1.
•
Coronary 3.5.
P be
LL7;:;1/LLn
-+
on [0,
(d G 1) and let 7n.j,
a for some aGO.
~ 7n~
R=
R
+
=
R=dand
and
=
=0
)
+
=
=R
11'1. . . .
71'1.
=
a.s,
a
so that
- 12-
I
Example 3.6.
Let P be a nondegenerat
on ]Rd,
Cbv :
E
: X<i1.J ~ b 5 for
v in the
Sd-l and b E
and let C: fCbv : b E lR, v E Sd-I S be the class
of all closed half spaces
]Rd.
an
map A : lRd -'?
1. (A) the normal law N(O,!).
A preserves all the relevant structure of C,
we may assume P N(O,!).
Let Ij? be the d.f. on ]R of N(O,l), and let x£ be a chi-squared r.v. with d
degrees of freedom. Now for t ~ ~, (j2(Cbv ) ~ t(l-t) if and only if
=
=
=
Ib I ~ 1j?-1(1- t),
Tt ...... tzu
P(Et {1- t »
so
EW-t)
= fx E Rd :
~ 1j?-1(1- t)j.
Let
Tt:
= 1j?-1(1- t);
0, it
=F[1X11 2
as t
-'?
since
0,
=
. . . K tTF- 2exp(-Tt2/ 2) . . . K 2
where K I and K 2 are constants depending on d., so get) . . . K 2 (Lt -
I ) {d -
0 /2 .
We will show in the proof of the next corollary that C is full and (3.3) is
valid with p 1, though V(C) ~ d + 2. Suppose LL"Y;;,I/LLn -'? a for some a ~ O.
Then in Theorem 3.1 (i) we have
=
cs> 0, W n '"
ca1LLn, c 1
=(d -
1)acs,/2,
c2
=acs, and pc 1 + c2 + Cs = cs(l + a(d+l)/2).
In (ii), if "Yn
=(d-
C2=CS
T
= ACs,
(J
= 2/ca(d + 1), and pc 1 + (;2 + C3 = (d + 3)cs/2.
In (iii), we find
=
The
+ 1)cs,/2.
summarizes this.
I
Corollary 3.7.
P
a> O.
a{C) : C
E
C, a(C) ~
v« f =R a.s.
- 131
R
-= (2(1+a(d + 1)/2)>"2 and
1
1
bn
1
R = max«d + 3) 2 ,A2 (,8ZA/{d + 1)
=(LLnf2
if n- 1LLn
=o ()'n)
1
-
1» and bn = (LLn)2 if )'n "" An- 1LLn
1
R
=(d + 1)/2 and bn =LLn/«n)'n)2L{LLn/n)'n»
Example 3.8.
Let P be the uniform law on [O,l]d (d ~ 1), recall that I d is the class of all
subreetangles of [O,l]d, and let C: = fC E: Iii : P(C) ~ ~ $. The bound on P(C) is
assumed for convenience, to avoid the technicalities of dealing with complements of large rectangles. Then P(Et } ; 1, so get) = t- 1•
is clear that Cis
spatially full. (3.3) with p 1 will be checked in the proof of the. next corollary, using the techniques of Theorem 2.1 of Orey and Pruitt (1973). Suppose
LLn/Ly;;1 ~ a for some 0 ~ a ~ co. If a ~ 1, the constants in Theorem 3.1 are
=
C1
= 1, =0, =a,
C2
n "'"
L)':;;,1
E: C,er(C)
~ )'nS
Cs
W
while if a > 1 they are
By Theorem 3.1 (i), if L< a <
co
we have
1
limsuPnsupfjvn(C)I/(er2(C)LLn)2: C
= (2(1 + a-
1
I»2
a.s,
which is equivalent to
l
l
limsuPnsupHvn(C)I/(er2(C)L)':;;,1)Ja: C
In
er 2(C) ~ )'n5 = (2(1 + a»)2 a.s.
3.1 (ii), if)'n "'" An -1Ln the constants are
a
and we can
e > 0, we
E: C,
= 0,
use
T
= A, () = 1, W n
"" Ln , C 1
if (3.5)
(1.
a = 0,
C1
=
1, ca
=
1,
C2
=
Cs
if
UU.lUL::::l,
= o.
I
=0
=
0
)
- 14Corollary 3.9.
Let P be
uniform law on [O,1]d (d ~ 1) and
decreases and LLn/L7;;:1 -7 a for some 0 ~ a ~ 00,
1
711. J. 0 so that n-1(L7;;:1f~·
where
1
R
1
=2 2 and bn =(LLn)2 if a =
co
[i.e, if (Ln)-& = 0(711.) for all s
1
(3.10)
R
> 0)
1
= (2(1+ a»2 and bn = (L7;;:1)2 if n- 1Ln = 0(711.)
andO<a<oo
1
(3.11)
R
1
= 22 and bn =(L7;;:1) 2 if n- 1Ln = (711.) and a =0
= "A.-iurA - 1) and bn = (Ln)2 if 711. '" An- 1Ln
= 1 and bn =Ln/((n7n)2L(Ln/n7n» if 711. =0(n- 1Ln)
and n -{1+&} = 0 (711.) for all s > 0,
0
1
(3.12)
R
(3.13)
R
1
1
provided in each case that n"' l bn decreases. For (3.11)-(3.13), "lim sup" may
be replaced by "lim' in (3.9).
I
Taking 711. n- 1(Ln )P(- 00 < (3 < 1) in Corollary 3.9, we see that the lim sup
in (1.4) is (1 - p)-1 a.s.
=
For d = 1, (3.11) is Stute's result (1.5), and (3.10) and (3.12) are due to
Mason. Shorack, and Wellner (1983). For d > 1 (3.11) is related to other
results of Stute (1984).
IV. The .4symptotic Modulus of the Empirical Process
nonaeereasmg runctron 'ljI on [0,1/4] an ~:!!!.E~!:9 ~~~~
of~~~
(4.1)
'ljI(O)
(4.2)
'ljI(s
+ t)
~ 'ljI(s)
=0
+ 'ljI(t) for all
s,t E [0,1/4]
there exist sequences 7'11.' an J. 0 satisfying nan t, 7'11. ~ an, and
(4.3)
n- 1Ln =
(4.4)
0
(an)
such that
(4.5)
.
lim: S'UJ>nSUPI
Ivn(C)-vn(D)!
'ljI(a(C tJ)
» : C ,D
E
C, P(C sn ~ 1/2,
co
a.s.
'ljI is
a local aSY!Ilptotic modulus at!k(the empty set) for (vn) on C if (4.1)-(4.3)
hold with (4.5) replaced by
.
'vn (C)l
2(C)
lim. SUPnSUPI 'ljI(a(C» : C E C,7n ~ a
~ an~
(4.6)
< co
a.s,
The requirement (4.4) ensures that, ill (4.5) and (4.6), we are not considering
only sets so small that Pn need not be much like P.
Define glCt) by
Lg 1(t):=Lg(t} + LLr 1,
and set
=i(Lg 1 (t 2 » 1/2
7floCt): =t(Lt- 2) 1/2 ,
Theorem 4.1.
liC is a VC class then
asymptotic modulus of continuity and
I
is a local asymptotic modulus
soace case
modulus of both
a variant
some cases, as the
Theorem 4.2.
C be s
vc
and lim ....
171
Cl:=
lim
(t
~.'"'
> o.
- 16/
c'a:= lim SUPt ...oLU- 1/Lg 1(t)
:= lim suPrtLL
c
)/Lg 1(cx"J ,
Ca:= lim suPrtLLn/Lg 1(cx71.) .
(i) Suppose
(4.9)
entropy condition (3.3) holds for some p ;;;; O. Then
limsuPnsupfllln(C)I/lf1(a{C»:C
E:
C.)'n ~ a 2(C) ~ cxnS
~(2(PCl+C2+C3»1../2
as.
1, Lg (t )/Lt- 1 is nondeoreasing, and the lim sups in (4.8)
are actually limits, then equality holds in (4.9).
(ii) If C is full, p
~
=
(iii) If C is
(4.10)
CXnS =
: C E: C.l'n
In
(iii) tells
a.s.
<H1l1JleIlslIOllial. uniform interval case (Example 3.8). Theorem4.2
holds provided l'n.CXn J., ncxn t, n- 1Ln o (l'n), l'n ~ CXn.
=
1
=
and Llm. 0 (L cx.;1). Since here 1/I1(t) . . . t(Lt- 2 )2, this generalizes Stute's (1982a)
result (1.9).
Dudley (1978) proved that lin, indexed by a VC class C. converges weakly
(under some measurability conditions) to a Gaussian process Gp on C with the
same covariance as lin' Lemma 7.13 of Dudley (1978) and Theorem 2.1 of Dudley (1973) show that 1/10 is a sample modulus for Gp on C. For C Dd or
=
: C E: Id,P(C) ~
(4.12)
(4.13)limg ... oSUpHGp(C)!/1/Il(a(C»: C
(4.
E: DdP(C)
l.,a 2(C ) < d
1
=22"
~ ~ ,a2(C ) < eS
=0
;;;;
p
1
= 22"
(4.13) are also obtained as special cases of
Ale:l{ander (1986).
1) and
Corollary 4.3.
C
Dd
v« J. 0 and CXn J. 0 so
a.s.
on
a:s.
'I'h"",",1"''''',",",
some a, b > O.
and LL
7/I!Ct) . . .
lim sUPnsupi!vn(C)l/7/ll(a(C» : C
E
Dd,.P(C) ~ ~ ,"In ~ a 2(C) ~
=(2(d-l+(lAb)+a- 1)/d)2.l
Our final theorem in
anS
I
section is an in-probability version of Theorem
4.1.
Theorem 4.4.
Let C be a VC class and let "In
I
-+
0 with n- 1Lg ('111,)
=
Q
(rn)' Then
- 18V. The RatioPn/P.
q(t)
behavior
=t
and bn
=n 1/2
in (1.1) and (1.2), we can
I
IPn(C)
the
(
supf I P (C) . - 1 : C E: C, PC) ~ 1n ~
As with q(t)=t 1/2 , analogs of Wellner's (1978) results (1.15) and
(1.16) will depend on the behavior of P (E t ) as t -l' O.
as
n-l'OO.
Theorem 5.1.
Let Cbe a VC class and let 1n
-l'
O. If
o (1n)
then as n
-l' 00,
Pn(C)
supfl P(C) - 11: C E: ep(c) ~ 1n~
(5.2)
-l'
0
in
probability.
If also
then the convergence in (5.2) is a.s.
Collversely if C is full and either
(5.4)
then
(5.5)
.
tirn.
T
Theorem 5.2.
C be s
>0
SUPnSUP\!(,Pn(C)
P(C) - 1I : C
h,,,,y,..,,rn mctudes (1
P(»
C
E: C,
as a
~
1n) > 0 a.s.
I
case.
vc
=
for some C E: e with
for
19 -
and
(5.7)
p*lsup~;(~!: C EC,P(C) G e7n*~ >R] <g(7n*)-·4I\e-
A
.
If g is bounded, then supc Pn(C)/P(C) is bounded in probability (Le. we can
take e 0 above).
I
771. * is well-defined in Theorem 5.2 since g(t) is assumed nonincreasing. In
the d-dimensional uniform d.f. case (Example 3.4),
g(7n *)
- 1)! if d > 1,
In the d-
=
bound on the
1.7).
- 20VI. Proof of the Inequality
1
be an mequanty
> O.
hI(>'):= (1 +>.-I)log(l +>.) - 1. >.
It is readily shown that
hI(>.)t. >.-tht(>.)J..
(6.1)
j
>./2 as
h t (>.) '" L~
(6.2)
A
x-;. 0
as A -;. co.
>.
h t (>.) ~ 2(1-11.) for all A> O.
(6.3)
and
ht(X)
·.·I·v4
if X ~ 4
>. ~ 4.
~(LX)/2.if
Bennett's ill equality (Bennett, 1962) tells us that
1
1
FElvn (C)! >M] ~ 2exp(-Mn 2ht(M/n 2u2(C»)
(6.5)
for all M ~ 0 and all C. Hence (6.4) and Theorem 2.8 of Alexander (1984) give
us the following.
Proposition 6.l.
Let C be a VC class of sets. n ~ 1. M > O. and a ~ supc u 2(C). There exists a
constant Ko
(V(C» s uc h that if ....... ".,.. ""' . .
1
1
M 2 ~Koet.L(n/a) and M ~ KoL (n / a )/ n 2L (M/ n 2 a}
(6.5)
or (li)
(6.7)
then
1
1
(5.8) F[supclvn(C)!>M] ~·16exp(-M2/6a}+ 16exp(-lMn2L (M/n 2 a}}.
(6.8) corresponds to a
1.
If M/n 2a
Poisson
(2.8) hold.
q{a»" r for all j ~ 1.
~ 0:
.
>
tj+t
~
)+u
approximation. the
the Gaussian approximation is domdominates.
Proof of Theorem 2.1.
.N ,,-
I
d. and r:' :=
f or some ., ~
t
~
a and C €
- 21 -
> bq(t)
for some 7 ~ t < r' and C E
>
+
for
E
+
:= pCC}
pCO).
We
~U'pplJ8e 7 ~
.!
t < t:', C
.!
C(t),
E
.!
Ivn(C)1 = n 2 p (C ) ~ 2n 2 0'a(C ) ~ 2n 2 q (t )/ z (t) ~ bq(t).
It follows that
(6.10) p(o) ~ P[Pn (C) > 0 for some C
nP (Er ·)
;?;
E:
C(t) and
"I;?;
t < r']
;?; p.
Turning next to the
pP}, we have
(6.11) pp} ~
+ P[lvn(.E't;)1 > ~ bq
._ p{2}
.-
j
+ p{3)
j'
Fixj and let k h k a be nonnegative integers such that
IVn (.E'dl
J
(6.12)
1
~ a1 bq (t j )fj(tj) a if and only if k 1 ~ nPn (JJ: t . ) ~ k a.
J
Then
(6.13)
De:fin.e a new prooamnta measure
pti(.)
=P(-IEt)
and let P tj := (pti)OO be the corresponding product measure on(X OO , (tOO). (C(ti )
is easily shown to be deviation-measurable for ptj , since it is deviation(6.14)
ptj(C)
=P(C)/p(Et.) and
J
O'~(C) := pti(C)(l- ptj(C» ~ O'a(C)/p(E\)
Fix k , kl;?;, k
~ ka,
and define
14: := k
C E
1
ap
k
1
_ kn -aptj (C
= --=-~...,-'-IJ-'
k
n
;?;
fj(t j )-1 for C E: C(tj
).
Now
P« to e(tj}'
k]
P n.lC(t . )
= Ft,[kPk:.
E.],
1
(6.15) F[suPc(tj)lvn (C}l > ~ o.q (t i }lnPn (E\)
the restriction of
l
=k]
1
=Fti[suPc(tj) n £2( kPk:.(C)
n
-P(C)}I > Z-bq(ti } ]
J
=Ft.[suPc(t.)!(k/n)
1
s
> ; (n/k)Zoq(ti ) ]·
1
1
2
__1,. t
,uk:. (C) + len 2p i(C) -n 2P(C)!
1
> -2,bq (t i
)]
1
1
We now wish to apply Proposition6.! with M:= ;(n/kyabq(tj), and g(tj)-t in
the role of the
€X
there (see (6.14».
Case 1:
1
z(t j
)
~ 2n z/ b , i.e.
(6.16)
=
Hence by (2.2),
M2 ·.~b2q2(tj }/32~(tj)g(tj )3~.Kg(tj )-tL'fj(tj)
(6.18)
by
.In
4
(na(tj
)}
~ KLk .
Proposition 6.1 (Ii), and use (6.17) and
If K
(6.18) to get for
~
(6.19)
so
(6.
of (6.11),
)/za~(tj»
(6.20)
+
1
- 23-
.
:i 2exp (- b 2 g2(t j )/28t(tj)}
since, by (6.
the
(6.11) and (6.20), we see
FP>:i
(6.22)
of hi above is at most
for the Fi O of (6.9),
Combining
1
1
18exp (-b 2g 2(t j)/zSt(tj» + 18exp(-2-8n 2bg (t j )L(bg(tj )/n2t(t j»)
.1
+ 34exp(-2-8n 2bg(tj )L'i/(tj ».
(The last term in (6.22) is superfluous now but will be used Iater.)
Case 2;
1
z(t j ) > 2n 2/b. Then r ~ t j < 5, so (2.4) holds, and
.1
( 6 . 2 3 ) b g ( t j) >211. 2t(t j).
We wish to use Propositioni~.l(i). To prove (6.6) it issuificient to show that,
for the constant Ko of that proposition,
.
1
1
M2'i/(tj) ~ ~Mk-iL(M?/(tj)/k2)~KoL(k'i/(tj»'
(6.24)
We need two subcases, according to which of the two terms added in (6.17) is
the larger.
Case 2a;
(6.25)
bg(t j)
~
.1
.1
8n2t(tj)'i/(tj) 2 .
again valid. By (6.17), (6.23),.an.d (2.4),
1
M'fj(tj)/k2
1
=n 2bg(t")'i/(tj)/4k
.1
(6.26)
~ bg(tj)/Bn2t(tj)
(6.27)
~ (1/4)
v(K/Bnt(tj».
1
BY(6.2'7),.M9(ti)7lc-a~~L(M?/
and the .flrst inequality in (6.24) follows.
The second follows, if K is large enough, from the following inequalities, which
are consequences of (6.17), of (2.4) and (6.27), and of (6.26) and (2.5), respectively.
L(k'i/(tj»~ L(2na(t j)'i/(tj
»
» ~ L (2/nt(t j + 2L (na(tj »
~ 4
1
(6.28)
1
)/k 2) ~ ~ n 2bg(tj)L(bg
»
- 24argument of hl in (6.21) is at most 1, so by (6.5)
Thereforeso does (6.22).
(6.4), (6.21) again holds.
Case 2b:
1
1
bq{t j) > Bn2t(tj)g{tjf2.
(6.29)
Here, by (6.12) and (6.29),
(6.30)
lc
1
n ~ a{t j) + an
.1
bq{t j)g{tj)2 ~
1
4n
-.1
.1
2bq(t j)g(tj)2.
It follows that
1
Mg{tj)/lc 2
(6.31)
1
1
=n 2bq(tj)'g{tj)/4lc ~ g(tj)2 ~ 1,
so the :first inequality in (6.24) holds.
in (6.31), if
large>enoughthen
(6.30), (2.4), and the :first inequality
1
KoL(lcg(tj »
KoL(~n-ibq(tj»+ ~KoLg(tj)
1
~ 3~n2bq(tj)Lg(tj)
1
1
~ ~ Mk 2L(Mg(tj )/lc 2).
Thus we have (6.24). From Proposition 6.1 (I), (6.24), and (6.31) we conclude
that
1
1
P1~4) ~ 16exp(-~M2g(tj» + 16exp(-~Mlc2L(Mg{tj)/lc2»
1 .••
1
;i
3Zexp(- 3~Mlc2L(Mg(tj)/lc2»
;i
3Zexp (-z-8n 2bq (t j )Lf/(tj »
.1
so by (6.13),
1
pF') ti3Zexp(-Z-,an 2bg{tj)Lg(tj
(6.32)
».
To bound pP) we use (6.5) to conclude that
(6.33) P}3);i Zexp (- ~ bq (tj )f/(tj f~n~hl(bq (tj )/Bn ~t(tj )g(tj yli»
;i
nct\I1.IUI
Zexp
--:--00
(6.22) holds.
it
to sum
estabnsned (6.22)
(t
(t
- 25monctoructty of g{t)/r;{t) and q{t)fJ- l9'i{t) (which follows
t i + 1 ~ t,/2 and 9'i(t j + 1) ~ 2-(1-P>!Pi(t j
(6.34)
)
(2.7», we
(j <
Hence, using also the monotonicity of r;(t )/t,
(6.35) '£f=oexp{-b 2q2(t i )/z8r;{tj
=
'£1=02 t i
»
-tt:/2 exp(-b 2q2(t j )/28r;(t i
»
J
a
~ 2
J
r/2
t- 1exp (-b 2q2(t )/29r;(t
».
Summing the second term in (6.22) over i: (6.34) and (2.8) may be applied to
show,
= 1,2,
where ():= 2 1- P - 1 ~ (1- (3)log 2. The theorem now follows from (6.9), (6.10),
(6.22), (6.35), and (6.36).
When we do not have (2.7) or (2.8), we replace L(.) by its lower bound 1 in
(6.22), and observe that the second term in (6.22) is then not needed, since
the third term in (6.22) becomes an upper bound for the second term in
(6.20). Otherwise the proof remains essentially the same.
I
- 26-
VII.. Proofs of the General Results
a temma demcnstratm that a well-known
about stopping
, remains true even if , is not measurable It is included solely to
avoid unwieldy measurability
to our arguments.
Lemma 7.1.
=
Let , be given on (X""', "'" ,P) by , : min~m : (X 1,'" 'xm) E Am ~ for some sets
Am c X'", and let n ~ 1, 0 < f3 <
and F c X",
Suppose for each
(Zh ... ,zm)
,m < n
is a
=
(7.1)
(7.2)
F.
(X'm+h'" ,zn)E8(Zh ...
ThenP*[,
~
n]
~
(:r1p·[(X h ... 'xn)
].
Proof.
We may write P*(D) forp·«X 1... 'xm) ED) for anym andD cxm. Let C and
D 1,••• ,Dn be measurable sets with G::) F,Dm ::) Ir ~ m], peG) = P·(F), and
P(Dm) = P*[, ~ m]. Define
on X"'" by ,·(Col) := minim : Col EDm~' Then ,. ~,so
,*
p.[, ~ n] ~ pC,· ~ n]
(7.3)
=
pC,·
=m] =
P(Dm \Dm- 1)
where Do
a copy of xm on
O2 is a copy of X"'" on which
Xm+ 1'xm+2' . . .
are defined, P 1 is P'", and P2 is POO.
For Coli EOI let
SC{Coll) := (Col2 : (Coll,Col2) E GJbe the section of Gover Col1' Then
(7.4)
.P[G nDm Dm-tJ
=[P2(SC(Col1»dP1(Coll)
1
f3].
Now if
Coli
E
~
m] n (Dm\Dm - l ) (viewed as a subset of X m),
(7.2),
It
LUl..1Un';:,
a measuraote
=
by (7.1) and
I
asymptotic upper bounds
the weighted
process will be
obtained from Theorem 2.1 with the help of
following lemma.
Lemma 7.2.
Let C
a class of sets. let q
E
Q. and let (bll.). (Un), (Yn), (an) be nonnega-
tive sequences with
(7.6)
UCJ.J.J.J.'"
events
u 2(C ) ~ an]
with
+ Un) for some C
> (1-
E
C with")'n
~u2(C) ~ (1 +e)O:n]
Suppose
(7.7)
and suppose that for some
e.e > O.
(7.8)
e: (A ',Je»
Then
= O«Ln)-(1+6).
=
1.0.
Let r be the infimum in (7.7) and choose 6 > 0 small enough so
262 +.2r- 16 < e. Fix rn , (1-6 2 )n < m ~ n. If (Xit ... ,x.m) EAm • then there exists
C = C(Xl' ... ,x.m)EC with
11lm(C}! >bmq(u2(C» + Urn ~ (1- ( 2 )(bn q {u2(C»
(7.9)
Cand
here we have
(7.10)
by the dejjnition
~
(1-
~
(1-
1, ••.
1
» +Un)
,x.m)
=
Un)
- 28For n(k):= [(1+0 2/2)k] (the integer part), we have by (7.8) that
2:k ;<:1P*(Ain(k)(e» < 00,
the lemma then follows from (7.11) and BorelCancelli.
I
For asymptotic lower bounds on full classes, our method
modeled
1
somewhat after that of Stute (1982a). Let b(j,n,p):= (j)p1(1_p)n- denote
the binomial probability,
n
:E
B+(k,n,p):=
b(j,n,p)
1=k
the binomial upper tail, and
h 2(A) := A/(1 + A) - log(1 + A) = -(1 + A- 1)- 1h 1(A).
The following generalizes Lemma 1 of Kiefer (1972).
Lem.m.a 7.3.
Let (Pn), (kn.), (In) be nonnegative sequences (With n ~ 1) satisfying
Pn ~ 0, k n ~ 00, and k n ;:;i In o(n). Let Ao > O. Then there exists TJn ~ 0 such
that
=
whenever
(7.12)
k
= {1+A)np, kn;:;i k:
;:;i
In, 0 <p ;:;iPn. and A ~
Aa.
I
Proof.
Fix n,k,p, and A satisfyin.g (7.12). Comparison to a geometric series shows
b(k,n,p);:;i B+(k,n,p);:;i (1+A- 1)b(k,n,p),
so
(7.13)
Define
bo (k,n,p) := (n!)k (~l~f) )n-k, b 1(n,k) := (27rk(:-k)
Stirling's formula, if n is large,
(7.14)
Now
(7.
for some TJ'n
b
~
+
,k)l=
0, since kn.
~ 00
,n,p)l
+ log(1- (1 + A)p)1 ;:;i TJ'nk
(1+A)p
=
A
~
n -+
;:;i
In/n ~ O. Also
0 as n
-+ 00,
)i.
- 29-
s : if n
- x}+ x 2 +
::: 0{x 2 )
16), since h 2(A) is bounded
;:i
~
Lemma 7.3 can be translated
use. This we
as
l.n./n
~
and
(7.13)-
0,
I
'-0.
a statement about
IIn
(C ) for easier later
Lemma 7.4.
Pn
-?
Let '-0 > 0 and let (Pn), (77tn), and (.lvfn ) be nonnegative sequences satisfying
0, n- 1/2 = o (77tn), and.lvfn = o (n 1/2 ) . Then there exists 7111. -? 0 such that
1
1
l-logP[lIn(C) >.lvf] - .lvfn 2h 1(.lvf/n 2a2(C»1
1
;:i 7Jn .lvfn 2h 1(M /
1
n 2a2(C»
1
whenever P(C);:i Pn' M/n 2a2(C) ~ '-0, and 77tn
The next •lemma
ers the
I
of Stout (1974). It cov-
excluded in Lemma
near O.
Lemma 7.5.
For each
f)
> 0 there exist K,'-o > 0 such that
P[lIn(C) >.lvf] ~ exp(-(1+e).lvf2/2a2(C»
whenever n
~
1-
1,.lvf2 ~ Ka2(C), and M/n 2a2(C)
;:i
I
'-0.
Proof of Theorem 3.1 (A).
It is easily
that
(i)..(iii). It follows that
_1
1'nl~(nb~n) ,2
=
0
(1).
From the Kolmogorov 0-1 law we then conclude that the lim sup in (3.4) is
some constant a.s, By Lemma 7.2, to prove (A) it suffices to show that for
each 0 >0,
P-[llIn(c)1 > (1+8o)R obnl7(C) for some C
(7.17)
€
Cwith a2(C) ~ 1'11.]
where R 0 J. R o aso ~ 0 and R(J is
Ro
(i), (ii), or (iii) we are
0<0 < 1/8. Let (a",) and (Un.) be
with an ~ 1'11. and 0 < Un. < o.
'i
j
:=
E(j) := Ct.'1
~
given in whichever of
sequences, to be
0,
and
;:i
1'n~'
c E (j) for
j ;:i
,
+
and A
+
) €
=A
< 00
for all j
s
(7.19)
Since
we have
(7.20) P*[lvn(C)! > (l+Bo)R obntT(C)Js with
v« ;;;; a2(C) ;;;; an]
;;;; P*[lvn(C)! > (l+Bo)Robntl.(f for some j ;;;; N n and C
E
E(j)J
N",
; ; l: P[lvn(C)1 > (1+4o)R obnt//2 for
some C
FU))
E
i=O
Ne",
:=
l:.Pj+Pn *'
i=O
We now consider separately the cases (i)-(iii) of (3.4).
Proof of (A) (I).
Here we take an == 1/4. Un. == u for some 0 < u < 0 to be specified later, and
R o := (2«P + o)(c 1 + 0) +
J.
c2
+
Ca
+ 20»2.
1
n
-+
Now
a2(C) ;;;; t i for C E F(j),
and
maxiW", wJ/2/n 2ti
00 since n- 1wn = 0 (-Yn). so by (6.5). (6.2), and (7.18).
-+
0
as
2IF(j)le~(-{1+40)RSwn/2)
;;;; exp (-[(1+4o)RS/2 - (p+o)(c 1+0)]Wn)
;;;; e~(-(1+o)(c2+ca+2o)wn)'
SinceNn ;;;; (log (1-u 2/4}-1)-1 £-y:;;1, it follows that
N".
(7.21)
l: Pi s Nne~(-(1+o)(C2+C3+2o)wn)
1==0
;;;;
e~{-(l+o)LL7l.).
'rneerem 2.1,
(7.
g(t)
w n / 4u 2n
"'nT~"'T",nT
from
-y := -Yn'
of Th~:>nl"prn 2.1, aCt)
u- 2g (t) (see Remark 2.2),
0 (-Yn). so r > s
(2.5)
1; if we
u ;;;; 6RoK- 1/2
b := "'... (J.....",
=
for
=0
771.
t
1
(')''11)
~
for all t
=o «nWn Y2)
and
Yn, so
_J.
Kt 2L{na{t»
~Kt
_.1
_.1
2L{nt) +Kt 2Lg(t)
for all t ~ On, and (2.3) follows.
u 2 < 0 2/512,
~
.1
6R 6(nwn)2
Theorem 2.1 now tells us that, if we take
1/4
JPn •
s
36
J
t-1exp(-02R~n/512u2)dt
"''''/''2
.1
+ 6Bexp(-oR 6{nYn wn)2/256)
~
140exp (-{l+o)(ca+o)Wn)
~
140exp{-{1+6)LLn).
With (7.20) and (7.21) this proves (7.17), and (A) (i) follows.
Proof of (A) (li).
Here we take Un
o
for
.1
with
R 6 := max(R 61,Roz)
to bevspeeified later, and
1
R 62 := T2(Pe(o)T-1), where B(6) := {(P+ 6)(Cl + 6) + Ca+ 0)-1. We take
(7.22)
CXn
J. 0
= o(an) and U{y;l cx·nJ = o(w,J.
7'11
.-
. Since
=
Lg(an} ~Lg(7n) ~ L(cx.;;:ta(cxn » + L(y;ICXn) Lg(an) + o(wn ),
we have W n "" wn and n- 1wn 0 (cxn ) . It is
then VPl"ift,,'rl
stants C1. Cz. Ca are unchanced if Yn is replaced by an in (3.2).
=
above proof of
(7.23)
(A) (i)
(C)! >
) for some C
E C with
=
sumces to barilla
>
and
R o1 := (2«P+O)(Cl+0)+Cz+ca+26»2
for some C
E
a2(C) ~
CXn
1
with
- 32-
~ 2IF(j)!exp( -(1+40)2R12wnAi';hl(A.,J).
Set
Tn := n7nw~1
and
t := fJa{o)'T' -
Tn
1.
-l>
T,
(1+40)R o2 T n ~ (1+30)t. From (1.11) we know that thl(t)
we
have
=
=(8{0)T)-1.
It follow
that
A.,,,h 1(A.,J ~ >..rn2{1+30 )t h l(t )
= (1 +30)Tn/( 1+40)2R l2 B(O)T
~ (1+Zo)/(1+40)2Rl2 B(o)
so by (7.24) and (7.18),
N",
2J ZIF(j)Jexp(- (1+ZO)8{0)-1U/n)
1=0
~
exp(-(1+6)LLn)
»
since Llln = O{LL(7,;l exn =<o(wn ) b y{ 7.22).
To bound J?~ we use. Theorem 2.1,.· a.sin the. proof of (A) (i),witht(t ):=u 2t ,
b := 6R o2w J /2 , 7 :=7n' IX:=!ln, g(t):= t 1/2, and eft) from (7.19). Again aft) is at
=
_.1
a{t),
get) is u-2g(t),
z(t) u- 2t 2, r
is
"In' and s
is
Sn := 62RI2wn/4"l.dn ..... 62R127n/4U4T > 27n provided we take u < (62R12/8T)1/4.
most
Thus r v S
=
Sn'
lfwetakeu < 6R o2K
_.1
2
then
b 2/ u 2K for
and (2.2) follows .. Also
(7.26)
(7.27)
(t)
-l>
)+
(t
~
00.
- 33on the
consider t
(7.28) increases
=7'11.
of (7.28)
we take
i.
U
i:>J..LlGll
it
of
1
./ 2,
enough so L{o~82/211,4,)~ aK/OR o
1
1
ORo2(n'U..ln7n)2L(02Rt2'U..ln/n11,47n) ~ 2 OR62,,2w nL(02Rj.z/211,4,) ~ 4KLg(-yn)
and (7.28), and then (2.5), follow. Since g(t) decreases and art) and t- P/2Lt
increase, we have t P/2Lg(t) = 11,-{1a(t ){1/2(11,-2g (t »-{1/2 L(11,-2g(t}) increasing, and
(2.7) follows. Theorem 2.1 and (7.22) can now be applied, and the result is
that
r; - ~
a.".
36
f
r
exp (- 02R 12w n / 5 12u 2)dt
1
1.. /2
1
1
+ 36exp(-2-9,-ioRo2WnL(6Ro~2,211,2»
~
140exp(-(1+6)LLn)
provided 11, ~ 11,0 for some Uo(,.6) > O. In combination with (7.20) and (7.25)
this proves (7.17), and (A) (ii) is proved.
Proof of (A) (iii).
This time we
later, and take Cln
Un
=n- 1wn
=(n7n/wn)stfor
some (large) p.. > 0 to be specified
and
R o := (p + 6)(c 1 + 6) + C3 + O.
Set wn :=
)
w'"n1/2 -- 0 (Yn,
for some C
E C
(ii) it SUIIlCI~S to bound
of
=
since
Nn
=
(6.2).
Now
LU;;1 = Q (tlJn)
by (3.5), so
(7.25), (7.29)
(7.18)
by
h I(An) '"
O(LU;;2+LL
~
and
e::cp(-(l+o)LLn).
Once again Theorem 2.1 will provide the needed bound on P n ". As before
we take (t):= u1t, b := oR 6Yn, 1':= 1'11" a:= an, g(t}:= t 1/2, and Crt) as
(7.19),
),g(t)
so
=
(nt )-l/2L(nt) ~ (nl'nJ
z(t)=U;; 2t
_l.
_l.
".»
=Ynw;;lL(wn/n'YrJL (ni'n'>
2 L(nl'n)
~ Ynw;;I(4/-L}-lL(U;;4}Lwn ~ (4/-L}-l ynL(wn/nu,Jt).
For the second term,
(ii).
LL (27;;,lcx n )
now
apply
=LL (Z'Y;;,ln -lwn) =
esraonsned as in the
and
use
the fact that
1
to obtain, if /-L is large enough,
l.
l.
+ 36e::cp(-Z-8oR 6Yn(n'Yn )2L (oR 6Yn/(n '111, )~u1»
~
140exp
(A)
by
(3.5),
- 35The result now follows as
the proof of (A) (ii).
I
We introduce now some notation and
preliminary calculations for use
in the proofs of the next three propositions. Let R > 0 and 6, A,J,L E (0,1) be constants to be specified later.
C be a full VC class. For each t E (0,1/4] let
D t c C with eA!l(t}l-A ~
cr(C)
=
lD t
~ eA!l(t}l-A
t, P(C} ~ 1/2, and P(Cfi(UJ)€Dt,D"oCD»
+
1, and
s AP(C)
for all C EDt,
where ex ~ 1/8 is the constant in the definition of "full". Let (i'71.)' (an), and (on)
be nonnegative sequences with 7'11. ~ an ~ 1/4 and
n V 2bnQ(7n) .....
(7.31)
00,
Let (n(k ),k ;;;: 0) be a strictly increasing sequence of integers with nCO)
set
= 0,
and
m(k} :=n(k} - n(k -1}
n~)
Y.I:{C) :=
lc(Xi )
i=n{.I:-l)+l
1
S.I:(C)
:=
Y.I:(C) - m(k)P(C)
1
= n(kfivn(.I:)(C)
- n(k
-lf2 Vn (.I: _ l )( C )
t.l:j := an (.I: )p/,
N.I:
= minU ;;;: 0
: tk.j+l
N'1e := minU ;;;: 0 : t.l: j ~
< 7n(k)L
7J,f(16j,
1.1: := Hi,i): N'.I: ~ i ~ Nk> 1
r(k,i)
si
~
IDtkilL
:=IIltkil,
and observe that
D tki
= ~Ckji : 1 s i
;;;; r(k,i)~
for some sets C.l:ji. Note the k indexes the number of sample points, j
indexes the sizes of the sets, and i indexes the collection of sets correspondto each
and i. The Ckji are nearly di:.;joint for fixed k and i : we wish
to replace them with fully disjoint sets Dk ji • Define
G'.l:ji :=
C.l:ji
D'kji :=
C.l:ji \G'leji'
G
fi (Um"oiCkjm),
:= D 'kji fi (Ul:>j Um:lir(.I:.l)Cktm),
while
P(UiC"kfd ~
2tkLr(k,l)
~ EL>f(2e>..a(tkL)1-.A.t,Q
+ 2tkd
~ EL>f (2e>..a(tkj)1- Att,p!'(L-f)
+ 2t kjp!'(L-j»
~ 2(e.A. + 1)p!'(1- f.LA)-l a(tkj )l-Atk~'
If,
as
we
henceforth
assume, f.L is
chosen small enough so
2(e.A.+1)f.L.A.(l-~tl;;i>..e>..(l->")/2, it follows that P(UiC"kji);;i>"P(UiD'kjd/2.
Hence for fixed k,j, for at least half the values of i we have P(C"kfi) ;;i AP(D'kfi)'
By reducing E:.A. and ID t,l;jl by half if necessary, we may assume this is valid for
all i; it then follows from C
P(Dkji)·~ (1-
(1- 2>,,)P(Ckjd, so
(7.32)CT~(Dkji).~(1 ...
2>..P(Ckji).
Observe also that
(7.33)
Ckji
=Dkji U Gkji
as a disjoint union, and
Gkji () (UL~j Um~r{k.j,.nkLm.)
= ¢.
".'
We now define events
1
A kji := [Sk(Ckji) ~ (l-26)Rn (k)2b n{k)q (a2(C kji»]
A'kji :=
1
Ekji := [V n(k-l)(Ckji) ~ -oR (n(k )/n(k -1»2b n{k)q(a2(Ckji »]
r,
:= u{j,i)€h"Ei.ji
B k ..-
B' k
U{j.i)e:Ik A 'kji'
Note that A
and A "kfi
imply Akj i,
that Akji and E kji together
imply that Vn{k)(Ckji) ~ (1-3o)Rb n{k)q(cr(Ckji». Thus
(7.34)
lim SUPnSUPlVn(C}/bn{k)q(cr(C» : C
C, 7n
E
a.s.
i.o. }
Sk are mdependent,
EP
so we
)
= 1,
(7.35)
P(Fk i.o.)
.LVl.'LU"
)=00, EP
= O.
if
< 00,
;;i cr(C} ~
Cln
$
Ukj := [Yk (Hkj) ~ Zkj]
Zkj
by
=6Rn(k)l/2on{k)Q(tkj)/2.
16Atkj (zJcj - m ( k )P (H kj »
(7.37)
k and
""
B 'k by
E
T I (",, ) := minU : "" EA'kji for some
is.
T e(",,):= minfi: "" EA'kTt{t..l)iS,
and let T I = T e = CX) off B'k. If 5k is large on Dkji, it is probably also large on
because D kji is most of Ckji by (7.32). To make this precise, we will show
that
Ckji ,
P(BkIB'k,Ukj,(TitTe) = (j,i»
(7.38)
~ P(A"kjiIB
T z)
Once (7.38) is establishe(f, we have
2P(BJ<,T I=j) ~ 2P(Bk,Ukj,Tl:::j)~P(B'J<,UJcj,T I=j)
~ P(B'k,T I=j) - P(Ufj)
so that
2P(B k) ~ P(B'k) -
(7.39)
N"
2:;
P(Ufj)'
j=N'k
,
The :first inequality in (7.38) follows directly from the definitions. To prove
thesecpnd., . oiJserve thatiJ), (7.33) ,8k (Gkji) . . i~conditi()nallYitndependentof
(Sk(J)J.:jm),m .•~r(k,j» . givenSk(Hki) (or equivalent!y,givenYk(Hkj»' n·.follows
that
(7.40) P(A "kjilB '",Ukj,(T hT2 )
=2:;
= (j,i»
peA "kjilTl ~ j,Yk(Hkj)
=l)P{Yk(Hkj) =II B 'k,Ukj,(T I,T z) =(j ,i»
l':i'Zki
, ..u .v ...........
=
Yk{Hk;j)
l-, Yk{Gkji} has a binomial distribution
m{k) - l and p
=
bmomtat dis:tribution
=
one
) = l]
once we eatannsn
~
1/2.
- 38(7.37), an~ (7.31), on the event [Yk(Hkj)
m(k)P(Gkji) - Np
=l],
=P(H£j)-lp(Gkji)Sk
~ 16>" dG(Ckjd(Zkj - m{k )P{Hkj »
~ oRn(k )l/2bn{k)q(tkj )/2
~ -1
+ oRn{k}l/2bn{k)q(dG(Ckji ».
(7.41), then (7.38) and (7.39), now follow.
It is clear that for any m and M and any collection J o .... ,Jm of disjoint sets,
P[Yk (J(j) ~ MIYk (Ji ) = li for all 1 ~ i ~ m] is monotone increasing in each 4.
From this "negative dependence" of Yk on disjoint sets, it follows that
(7.42)
n
P«B'k)C) ~
P«A'kjd C) ~
(j,i)£I/i;
n
N/i;
(1- mi7ti1if:r{k,j)P(A'kji»r{k.j).
j=N'/i;
the remainder of the proof it is necessery to split into cases. We state
each of these asa separate proposition. In each proof we will specify
(n(k», R, 0, and >.., then continue the present calculation.
Proposition 7.9.
Let C be a full VC class and q
E
Q, and suppose
1
q(t)/t 2
(7.43)
,l.,
q(t}/(tLt-l}l/2 t, Lg(t}/Lr 1 t.
Let (7n), (an), (b n) be nonnegative sequences satisfying
and
1
(7.44)
bnq(7n}/n27n
-+
O.
Suppose the following limits exist and are finite:
(7.45)
c 1 := lilTl.n 7nLg(7n}/b.fq2(7n)
c2
:
=lilTl.n YnLL (y;;: 1ex'll. }/b.fq 2( Yn)
Ca:= lilTl.n YnLLn/b.fq2(7n}.
Then
(7.46)
Proof.
o < 0 < 1 and
V:=
R=
assume R > O.
, we ccntinue our
, >..=
-.;aJLI,.; U.LaLJLUU
takmg
].
Set
u(k):= e>..g(7n{k»1-0/8,N(k) :=Nk -N'k
Since tg(t) and Lg(t)/Lr 1 increase, for j
+ I.
we have
;E N'k
r(k,j);E eA!J(tkj)l->";E eA!J()'~(f{16)1-0/16 G u(k).
Hence by (7.42) and (7.47),
IfW'k
> o then
so
while similarly
;E (-1 + (o/16log J.l-l) log (7;;(k)(Xn{k») v L
Since 7;;(k)CXn{k) ~
00
0, it follows that
c2(1-o/4)b,r{k)q2(7n{k»/7n{k)
i::51Illi1arJ.y since
logpk-2 •
For i ;EN'k
Hoetfdulg (1963)
"rlttf>/2.
J
rtence by (7.50),
we see
Using this and Theorem 1 of
- 408P(Bk ) ~ N(k)u(k}Pk A 1.
(7.52)
(7.49) we obtain
(7.53)
N(k luCk )Pk = £).N(k )g(7n(k)P-6/B exp( -(1-0/4)R2b~(k)q2(1n(k»/27n(k»
~ £).exp( -(1-o/4)csb~(A:)q2(7n(k»/7n(k»
~
e).exp(-(1-o/8)LLn(k})
~
£).k-(1-6/16).
=
With (7.52) this shows EP(B k ) 00.
To establish (7.36) it remains
7n(k) ~ t ~ £k := 7~(f(16,
to
bound P(Fd.
By
(7.43),
for
and
~:2: q2(7n(k»
t
,n(k)
Lei 1 :2: q2(7n(k»
L';;(k) - 2,n(k) .
Also
N(k) ~ N k + 1 ~ (log j.l-1)-11 0 !1. (' ;;(k )(Xn(k » + L
Combining these facts with (6.5), (6.4), (7.44), and (7.43) we obtain
P(Fk.) . .~.E(j.i)~h.2er[J(--o2VIi2~~(k)q2(a2{C~ji»/tk;2·(Ck#}}
~ Ef:N'1; 4g (t kj )exp (-4R2b~(k)q2(tkj)/tkj)
~ E:'~N'I;4exp (- (B 2+2c 2+2c s)b~(k )q2(t~j )/tkj )
~ 4N{k)exp(-(R 2/2+C2+ Cs)b;(k)q2(,n{k»/,n(k»
~
4exp(-(5/4)LLn(k})
and (7.36), and
(7.34), follow. Since 0 may be arbitrarily small, this
TI1"'I"1V"""Z the proposition.
•
Remark 7.10.
C4:=
lim
- 41 one, for us to obtain some lower bound on
ccrresponding lim
I
Proposition 7.11.
Let C be a full VC class. Let 7T'" w n , Cit and Ca be as in Theorem 3.1 and
0:= (Cl+Ca)-l, and suppose 771. "" Tn- 1W n for some T> O. Suppose the lim sups
in (3.2) are actually limits.
.1
lim sUPnsupHvn(C)I/w~(T(C): C E C, a2(C)
(7.54)
= 771.5
1
;;;; T
2 ({JeT - 1)
a.s,
Proof.
q (t)
A :::
-w
1/'2",
_ .... 11.
take
<0
=
-_·,1tt
'V
f\.A.ll
where V ;;;;80- L 1s large enough so
oR T1/'2h 1(oVR /2T 1/'2) ;;;; 6(c l+Ca).
(7.55)
We may assume R > 0, so C1 + Ca::: 0- 1 > O. Note that N k ::: N'k ::: 0, so j is always
O. By Lemma 7.4 and (1.11), similarly to (7.47),
P(A'kjd;;;; P[vm(k)(D kji);;;; (1-0/2)R(Wn(k)7n(k)1/'2]
;;;; exp(-(1-0/4)Wn(k,RT 1/'2h 1(R T- 1/'2»
=earp (-(1-0/4)(c 1+Ca)Wn(k»
.As in Proposition
(d. (7.48» it rouows
P(B'k);;;; (U(k)pk/2)A(1/2)
for u(k)
then
;=
so E P(B k )
eA9(7n(k»1-A. Analogously to (7.51)-(7.53), we obtain P(Ufj) ~ pt and
00.
By (6.5) and
1
1
P (Fk) ~ 2r (k ,0 )exp (-oR (h (k }Wn(k)7n(k)y2h 1(oVR /2T 2 »
~ 4g(7n(k»exp(-5(c l+ C a)Wn (k»
-GLd.Hl.\K;
as
Proposiiticm 7.9,
»
I
- 42Proposition 7.12.
C
a
7'11.' w n• Yn'
=
(7.56)
7'11.
(7.57)
L(wn/n7n)
0
Ct,
and Cs be as in Th~:>£'lrpTn 3.1,
(n-twn) and
=o (w n).
I
Proof.
Let 0 < 0 < 1 and this time continue the calculation preceding
bn
an
7'11.'.8.
A
and q(t)
then
=
L(n(k )/n(k-l»
'Wn(kV4
takIf
30-2 L(Wn(k)/n(k hn(k»
and Wn(k) !!;
zu:
Again we may assume R > O. and j is always O. By Lemma 7.4. (6.2), and (7.56).
P(A ·k;,.) ~ P[lIm(k)(Dk;,.)
1
!!;
(1-0/2)RYn(k)(7n(k)n(k )/m(k» 2]
1
1
~ exp (-(1-6/2)RYn(k)(n(k hn(k»2L (RYn(k)/(n(k hn(k» 2»
:=Pk·
As in Proposition 7.11 it follows from this that
8P(Bk)~ u (k)Pk A 1
wh,prp
u(k)
u(k
&A9(7n(k»t->... If Cs
=0
~
exp(-(1-6/4)cs'Wn(k»
!!;
exp(-(1-6/16)Lk)
then U(k)Pk
~ 1
so EP(B..;)
exp(-(1-0/8)LLn(k»
and
(7.59)
lJI'!;i:lfn fk 1fn
(k
)/n(kIf
Cs
>0
L
(7.57),
=
oe ,
If cs> 0
) ~ 4g (7n(k)}exp
P
~
< 00.
so
P(F;J ~ 4g
(-3Rwn (k»
~ 4exp(-Wn(k» ~ 4k- 2
once more
result now follows.
SO
) <:
00.
As in
previous two propositions, the desired
I
Proof of Theorem 3.1 (D).
For (i) and (iii) the result is immediate from Propositions 7.9 and 7.12
by
; to get
Proof of Theorem 3.1 (C).
We return now to the notation of the calculation preceding Propositions
1
7.9, 7.11, and 7.12, but with the following changes:
Dk j i := Ck j i •
We specify
now 8 k := k: 2 Vk and
the theorem it suffices to show that, whatever <5 may be, P(BC i.o.] =
(7.42), since the
(7.60)
7.
while as in
Since
Cg
=0,
r(k,O)Pk ;G; """'-Yl.
exp(-(1-<5/4)Lg (7k»
»~
so
-;. 0,
,0
and
I
o.
As in
- 44-
4.2.
of
AtLg (At)
~
AtL (A-lg (t
t, which is
f'lp::::Irlv
to prove (i)
p·[lvn(C}1 > (1+8o)R oql(q2(C»
(7.61)
with "'In
~
u2(C)
~
cxn]
some C
e
€
=O«Ln)-{1+o»,
1
where R 0 := (Z«P+o)(c 1+0)+C2+CS+ZO»"2.
Fix n and 0 < 0 < 1/8, then 0 < u < 0 small enough so
o2R3,/512u 2 ;E Z(cs+Z).
(7.62)
tettt:= (l.... u}i cxn&ndlet E(j),F(j), N n ,
3.1 (A). As in (7.20),
p·Ilvn(C)1
as in the proof of Theorem
(1+8o)R oQl(q2(C» for some C
€
e with "'In
u2(C)
~ 2:f:oP[!v n(C)1 > (1+4o)R oQl(tj) for some C
+ p·[llIn(C)1 > OR oQl(t) for some "'In
~
t
~
€
€
cxn]
F(j)]
CXn and C
€
e(t)]
..... "Nn~ lD
lD •
.....
'-'j=O" j + u:
n .
By (3.3), we can take
1
~«Lgl(7n))/n7n)"2~
for.soIIleK=K(q,u)<oo.
using (6.5) and (6.2)·that
(7.63)
Pj
s Z!F(j)lexp(.... (1+4o)R 3(LU1(tj»/2)
~2Kexp("'(1+4o)(C2+Cs+2o)Lg
l (tj»
~
2Kexp(....{1+4o)(cg+o)Lg t {tD .... (1+4o)LLn)
»
~
~
exp(-(1+40)LLt j
2:;=o(jiog (1-u
=0
asn~oo
so
=O«Ln
(7.64)
If
C2
=c
=
=0
we obtain
- 45so again (7.64) holds.
As'
proof of 'rneorern 3.1
and (4.7) that
p,/
an
;:;i
36
f
(I), we
V'-''',GUJ.
from
Thf'/Tr,PTn
2.1, (7.62),
t-lexp(-o2RKLg l(t )/512u2)dt
7,,/2
1
+ 68exp( -oR6(n1nLg 1(1n»2/256)
an
;:;i
36
f r lexp (- 2LLr
l
- 2(c s+l )Lg l (a n » dt
7,,/2
=O(exp(-2LLn»,
and (7.61), and then (i), follow.
For (ii), fix 0 > 0 and set Y« * := 1n v aJ+6. We wish to apply Proposition 7.9.
Consider
the
sequences
in
(7.45):
since
Lg(t)/Lt- 1
increases,
Lg l(1n *) ~ (l+o)Lg l(an) so
li'mn.1~Lg(1~)/qr(1:)=
Cl
lim infn1:LL«1~)-lan)/qf(1:) ~ (1+0)-lc2
Since 0 is arbitrary, Proposition
and Remark 7.10 prove (u).
For (iii), by increasing 1n we may assume 1n an' The proof is then just
like that of Theorem 3.1 (C), since C2 = 0 and C 1 = 1 whenever Cs O.
I
=
=
Proof of Theorem 4.1.
localasym.ptotic modulus
at ¢for (v n ) .
To show 1/10 is an asymptotic modulus of
let 1n,an .!. 0 with
nan t, 1n ~ an, n-1Ln 0 (1n),
LLn O(La~l). It suffices in (4.5) to consider
C,D
~P(CAD
Let
D:=
: C,D E C,
1n/2;:;i cf4(C\D) ~ an!
: C.IJ EC, 1n ;:;i
) ;:;i
We
g(t) := t- l
) ,we
=
=
EC
» :C
E
E.1n ~
: C ED
- 46-
I
so the result follows from Theorem 4.2.
Proof of Theorem 4.4.
We use
2.1, with C{t) := Ct. ((t):= t , q{t):= '¢'1{t 1/2 ) , 1:= 711.' and
0::=
we use p:= 0.)
r = 711. > S
(2.1),
n- 1Lg{7n) e (7n). If b is large enough then (2.2) is clear, (2.3) follows easily
from the observation that
=
L{na{t)) ~ L{nt) + Lg{t),
(7.65)
and (2.4) and (2.5) are vacuous. Hence (2.6) holds. If b is large then
exp{-bZqz(t )/512t) ~ b- 1{Lt - 1)Z, so the second term on the right side of (2.6)
1
can be made small. Since q{1n)n 2 ~ (n1n)VZ
small for large n,and the theorem fellows.
-'> 00
as n
-'> 00,
the third term is
I
Proof()fTheorems 5.1 arid 5.2.
Observe that for M > 0,
Pn{C) .
P[supfl P{C) -
(7.66)
11 : C
E
C,p{C) ~ 111. ~ > M]
1
~ P[lvn{C)1 > Mn 2a2{C) for some C
E
Cwith a2{C) ~1n/2].
Thus to prove the desired results we use Theorem 2.1 with C{t):= Ct>
1
and b := Mn 2". If (2.2)-(2.5) hold then
({t) := t , 1:= 111./2, 0::= 1/4,
(2.6) bounds the
of
68exp ( -u« 1n/256)
(7.67)
= 0(exp(-M2n1n/1024» + 0(exp{-.Mn1n/256».
In (2.1) the values
(7.68)
if
s=
r=
M
~
2
To prove (5.2), we take M fixed but arbitrarily small in (7.66). Then (5.1),
that n1n -'> 00
and (2.3). (2.4) and (2.5) are
(7.65), and the
vacuous by (7.68), so (5.2)
(7.67). If (5.3) .u."'.1."'-'",
same
O({Ln)-Z), and a.s. C011verlH:::I1c:e
~ n- 1
~ 2,
(7.69)
(C) for some C E C with P n
S1J.'lI)l-~":-
-
1/ : C
E
C,P{C)
~ R1~j
Pn{C) F 0,
) F
>
are vacuous.
(7.66),
vacuous.
If
9
is
t,
then
all e > 0 by
bounded,
supfPn(C)/P(C) : C € C, P(C)
;S e/~S
(5.7), while for R > 0,
P[sup~Pn.(C)/P{C) : C € C,P(C)
~ P[supfPn (C)
:C
€
< e/~S > R]
CP(C) < e/~S > 0]
~ naky~) ~ nAe/~ ~ AeLA
-l'
0 as e
-l'
0,
and the last statement in Theorem 5.2 follows.
It remains to prove (5.5) when C is full and (5A) holds. By (5.4) we have
Un) for all n
some
so if wedefinet'n to be the solu7=
then -''11.
for some 0 <
supfl
m-1(£g( -,)
Set w'n := Lg(Yn)vLLn. By Proposition 7.11 and Remark 7.10,
(J
< 1 we have infinitely often
Pn(C)
P{C) - 11: C € C,P(C) ~ 1/2,a2(C) ;S InS
;S
sup~lvn(C)I/2(n-,'n)1/2(T{C) : C € C,a2(C) = Y'nS
~
(Par - 1)/4.
•
})=t,
f
»=
f
if P{C);l;; Z
Proof of Coronary 3.5.
From
3.4 we see that if suffices to prove (3.3) for C := Dd , P = 1,
and some Tj < 00.
u € (0,1)
t € (0,1/4], and set
C' := fC € D d
:
(1-u 2/4)t < if{C)
C" := fCc: C € D d
so C t\C(1_u 2/4,)t
:
;l;;
t , P(C) ;a; l/zi
(1-u 2/4)t < if(C) ;a; t , P(C)
> l/Zi
= C' u C"; we will consider C' and C" separately.
For C", let /L:= 1 - u 2/ 8d and let F be
(3.7). We have (d. (3.8))
, Ci€ C",and
Clearly
then exists
/LX;. ;l;; Yi /L-1 Xi
for
all
i ;a; d;
so
2t/Z.
u
Hence P(CAC z) ;l;; uZt and C' is taken care
C:«: C l ,
=
C2 [O,x] € F
with
P(C 1AC z) ;a; Zd(/L-l_1)P(C l)
;l;;
of.
For C" the proof is somewhat similar. Let N be an integer between ZOdu- 2
and Zldu- z and set
G
:= f[o,x] : X;. = 1 - ru f (t )/N for some
'rti ;a; N
Then we can
€ G with X;. - f (t )/N ;a; Yi ;a; X;. for
each i ;a; d - 1. Now I\d=11Xi;l;; (TIf=11yD(1+4df(t)/N) .50 Xd;a; Yd;l;; xd(1+4dj(t)/N).
Thus
Since
IG
C" is now
uu..n.... .w.
of,
Proof of Coronary 3.7.
to
assume P
r .·-
8>
;l;;
(22d)d-1 U-2(d-l)g(t),
I
=
6
- 49-
(8.1) Ct \C(I-'Ul3/4)t
=iCatl : r
~
b < r\
v E sa-IS u
iCCu : -r- < b ~
-r, v E sa-IS
Let
two vectors
=
(8.2)
~
MO-(d-l)
for some o,i! depending only on d.
We now prove (3.3).
specify 0 := u Z/ 16r . Let C E C t \C(I-u 2/4)t ' It is
clear that there is awE Sd-l for which C C Crw and
(8.3) P(C ACrw }
~ P(Crw ) - P(Cr ' w )
=f (t) -
f
«1-u z/ 4)t ) ~ u Zt / 2.
Since V is maximal, there is a v E V making an angle ex
can show that
~
0 with w. Suppose we
(8.4)
With (8.3) and (8.2)
I
M un-(d-l) s;
16d-IMu-Z(d-l)rd-1
N Z(ut 2 ,Ct\C (1-u2/4)t' P) s;
.
Since tfJ-l(l-f(t» "" (2Lf(t)-l)VZ as t
-)0
0, there exists K= K(d) such that
K-1g (t) ~ r
(8.5)
d- 1
and (3.3) follows.
The equality in (8.4) is clear. Since
H
(j
~
Kg (t)
etc; \Crw )
depends only on r and the
T
closed half .plane bounded by lz and disjoint from Crw , and W the wedge
Crv with vertex at TV. Then
(8.6)
(3.3) follows
=tfJ(r) -
tfJ(r cos ex)
~
r(1-cosex)exp{-rZ(cos2ex)/2)
~ r 02 exp (r 2 0 z/2) exp (-r z/2)/2
~
~ u Z( 1- tfJ(r» / 8 ~ u Zt / 4.
TV
we obtam
(8.8)
peW)
=
Combining (8.6). (8.7). and (8.8) proves (8.4). and (3.3) follows.
To show C is full we use similar ideas. but change 8 to (16dLr )V2/r . Fix
A E (0.1) and take b = b(d) large enough so
v.w
since the angle between
If xECrv nCrw for
v and w is at least 8. we have IIxl1 2 ~ r 2 + r 2tan2( 8/ 2). so IIx!l ~ r(1+8 2/16). It
follows using (8.9) and (8.10) that
etc;
n
(UW€v.w,",vCrw)
~
POx:
~
b (2r )d-2 exp (-r 282/16)exp (-r 2/2)
Ilxll ~ r(l+02/16)D
Since by (8.2) and (8.5).
IVI ~ 0 8-{d-i) ~ or tt - 1/(H3dLr ){tt-i)/2 .~ Eg(t )1-],.
for some constant
thatC is f u l l . ;
E
P 1.
u E (0.1)
(0,1/4] and set
r :=
and r :=
. Let Zl+ denote
nonnegative
For
each j ,k E Zl~
~ r 131;/" for alIi ~ d and E}i, ~ rr, define
,bik E
1Jd by
,
numoer of rectangles
b 'tjk
...--
J --.............. 13
·
1.
+13
]
)
(t
}'i.
:=
.- max)!,
r
for
i ~ d.
Now
2r- 1(Vi -Wi) ~ a?7t: ~ Vi and
[aik.b ik] and
Vi -
so [v.w]
C
Wi
~ bi7c ~
P([aik .b i k ] ) ~ (1+5r- 1)dp ([v ,w])
(3.3) now follows.
I
Wi
+ 3r- 1(Wi -Vi).
P([v ,w]) + u 2t.
KS.:
ineeualtties for empizieal processes and a law of
the iterated l,..,r'::I7'iH'rrl Ann. Probability. 12
Alexander, KS.: Rates of growth
weighted
Proceedings of the Berkeley Conference in honor of Jerzy Neyman and Jack
Kiefer, Vol. 2 (L. LeCam and R. Olshen, eds.) Belmont, CA: Wadsworth 1985.
Alexander, KS.: Sample moduli for set-indexed Gaussian processes. Ann.
Probability 14, 598-611 (1986).
Breiman,L., Friedman, J.E.,
and Stone, C.J.:
Regression Trees. Belmont, CA: Wadsworth 1984.
Classification and
Csaki, E.: The law of the iterated logarithrn for normalized empirical distribution function. Z. Wahrsch. verui. Cebieie 38, 147-167 (1977).
Diaconis, P. and Freedman, D.: Asymptotics of graphical projection pursuit.
Ann. Statistics 12, 793-815 (1984).
Probability 1,
66-103 (1973).
Dudley, RM.: Central limit theorems for empirical measures. Ann. Probability
6,899....929 (1978).
P.: Empirical Processes. IMS Lecture Notes - Monogra:ph Series 3
(1983).
Hoertdmg, W.:
J.
sums
- 53Jogdeo,
~an:lue>Ls, S.M.: MOJ:lotlone convergence of bin,orrual prclbalolli1:1es
and a
Ann. Math.
39,
1
-1195 (1968).
J.:
Pn. ,J. O.
Proc. 6th Berkeley Sym/pos. Math. Statist. Probab. Vol. 1, 227-244. Berkeley:
Univ. of
(1972).
LeCam,
A remark on
measures. A Festschrift for Erich L. Lehmann
Honor of
65th Birthday
Bickel, K. Doksum, and J. Hodges,
editors). Belmont, CA: Wadsworth (1983).
Mason, D.M., Shorack, G.R., and Wellner, J.A.: Strong limit theorems for oscillation moduli of the uniform empirical process. Z. Wahrsch. ueru: Gebiete 65,
83-97 (1983).
Orey, S. and Pruitt, W.E.: Sample functions of the N-parameter Wiener process. Ann. Probability 1, 138-163 (1973).
Pollard, D.: A central limit theorem for empirical processes. J. Austral. Math.
Soc. (Ser. A) 33, 235-248 (1982).
Pollard, D.: Convergence of Stochastic Processes, New York: Springer Verlag
(1984).
Shorack, G.R. and Wellner, J.A.: Limit theorems and inequalities for the uniform empirical process indexed by intervals. Ann. Probability 10, 639-652
(1982).
Stout, W.: Almost Sure Convergence. New York: Academic (1974).
Stute, W.:
oscillation behavior of empirical processes. Ann. Probability
10, 86-107 (1982a).
Stute, W.:
bility 10,
IrHT"",.,l>n'lTl
for 1r""1"',.,,,,, 1 deIlsit.y estsmatcrs Ann. Proba-
Wellner, J.A.: Limit theorems for the
the empirical distrfbution
tion to the
distribution function. Z. Wahrsch. verui. Gebiete 45,
(1978).
Yukich, J.:
Laws of large numbers for classes of functions.
Analysis 17, 245-260 (1985).
Department of Statistics
University of Washington
Seattle, WA 98195
J. Multivar.