2007IEEE
International
Symposium
on
Signal
Processing
and
Information
Technology
Drowsy
Driver
Assistant
System
M.
ElSabrouty,
A.
Hamdy,
A.
Fawky
and
S.
Khalil'
1
Faculty
of
InformationEngineering
and
Technology
CommunicationDepartment
German
University
in
Cairo
New
cairo,
Cairo,
Egypt.
Email.
maha.elsabrout
4.
uc.
edu.eg
ahmed
elsayedna~student.guc.
edu._!eg
abdeiattah.solimana
agdn.uc
d.e
sherifkhail/&student.guc.
edu.!eg
Many
applications
were
developed
for
face
detection
Abstract
This
paper
proposes
a
new
approach
to
help
the
which
is
Neural
Networks
(NN),
Principle
Component
drowsy
drivers
on
the
roads
by
developing
a
real
time
Analysis
(PCA),
wavelets,
edge
detection,
Hough
transform,
computerized
system
which
has
the
ability
to
detectthe
face
and
'
.
'
.
'
.
a
to
know
the
status
of
theeyes,
if
the
eyes
are
closed
the
system
and
template
matching
and
geometrical
matching
are
some
strikes
out
analarm
toalert
the
driver.
We
used
PCA
and
skin
techniques
taken
for
major
approaches.
Neuralnetworks
and
tone
for
face
detection.
For
eye
detection
we
used
edge
detection,
matching
algorithms
needs
a
huge
training
set
of
face
and
wavelets
transform
and
YCbCr
transformation.
The
algorithm
nonface
imagesand
take
a
longtime
in
execution
as
performs
well
regardless
the
face
position,
lighting
varieties
and
immune
to
any
distracting
noise.
The
designedalgorithm
is
able
discussed
in
[2],[7]
&
[18].
Principle
component
analysis
to
attain
90%
accuracy
when
it
was
tested
on
different
takes
only
gray
scale
images
as
well
as
it
is
used
for
databases.
recognition
as
discussed
in
[5]
[6],
[10],[13],[15],
[20]
&
[21].
Edge
detection
is
severely
affected
by
light
condition
as
KeywordsDrowsy
driver,
edge
detection,
eye
detection,
face
in
[3],
[4]
&
[8].
As
for
wavelets
it
uses
a
big
size
window
so
detection,
Principal
Component
Analysis,
skin
tone
,
YCbCr,
wavelets
transform
it
cannot
detect
small
face
and
fail
in
extreme
lighting
condition
as
discussed
in
[10]
&
[22].
In
theother
hand,
eye
detection
techniques
are
not
as
popular
as
face
detection
I.
INTRODUCTION
techniques
or
it
only
works
with
certain
conditions.
So
we
had
to
introduce
a
new
algorithm
that
could
detectthe
eyes
Sleeping
while
driving
happens
more
often
more
than
within
the
faceregion.
mostof
the
people
think.
According
to
the
United
State
As
our
application
is
online
and
thus
is
governed
by
Department
of
Transportation
(USDT)
[11I]
&
[
14],
execution
deadlines,
we
have
proposed
a
system
to
improve
drowsiness
is
responsible
for
10%
to
lO0%
percent
of
the
20
millsioness
araiesontswhic
foccu
eachto
yearcent
the
U,
the
driver's
safety
by
detecting
the
status
of
the
eye
using
as
million
car
accidents
which
occur
each
year
inthe
US,
PA&sioefrac
dtcintomiizth
therefore,
developing
an
alarm
system
is
a
crucial
task.
It
is
PCA
&
skintone
for
face
detection
to
minimize
the
thus
important
to
develop
a
fast
alarmsystem
thatwill
searchingregion
for
the
eyes.
For
eye
detection,
we
have
notify
drivers
upon
drowsiness
and
yet
produce
as
few
false
proposed
using
wavelets
transformation
and
edge
detection
alarms
as
possible.
with
the
addition
of
YCbCr
transformation.
At
last,
we
use
The
human
face
can
be
utilized
as
a
preprocessing
step
to
Kalman
filters
for
trackingthe
eyes
to
continuously
know
the
limit
thesearch
for
eyes
within
a
limited
space
and
thus
eyesposition
and
status.
The
whole
alarmsystem
is
reduce
false
detection
result.
Face
detection
is
considered
as
implemented
on
the
DSP
processor,
ADSPBF561
Blackfin
the
backbone
of
many
systems
as
in
security
systems,face
Symmetric
MultiProcessor
for
Consumer
Multimedia.
The
recognition
and
human
interface
applications.
In
Drowsy
microprocessor
contains
512
MB
of
RAMs
and
8
MB
of
Driver
Application,
facedetection
has
to
be
combined
with
flash
memory.
an
eye
detection
algorithm
to
get
the
desired
performance.
9781
42441
8350/07/$25.00
©C2007
IEEE
176
Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.
The
rest
of
the
paper
is
organized
as
follows.
Section
11,
chrominanceof
the
skin
toextract
some
facial
features
such
presents
the
backgroundon
the
used
algorithms.
We
present
as
the
mouth
or
the
eyes.
our
modifications
and
implementation
on
the
used
B.
Eye
Detection
Algorithms
algorithms
in
Section
III.
Section
IV
presents
the
simulation
and
implementation
results.
Finally,
we
conclude
the
paper
in
Section
V.
1)
Edge
detection
II.
THEORETICAL
BACKGROUND
It
may
beoneof
the
easiest
ways
for
face
or
eye
detection[16]that
comes
into
our
minds
at
first
as
it
has
been
A.
Face
Detection
Algorithms
discussedbefore
by
many
researchers.
Themain
aim
of
the
edge
detection
techniques
is
to
find
points
in
an
input
digital
1)
Principle
Components
Analysis
image
at
which
the
light
intensity
changessuddenly
(sharply).
These
sharp
changes
in
the
image
can
reveal
some
As
discussed
in
[13],
PCA
starts
with
a
random
vector
x
important
things
from
an
image
like:
with
n
elements,
which
contain
a
sample
as
second
order
*Nonuniform
surface.
statistics
(SOS).
No
probabilitydensity
of
the
vectors
are
*Changes
in
the
property
of
a
material.
made,
it
only
matters
that
the
first
and
second
order
statistics
'~~~~~~~~~~~~~~
Luminance
change.
are
known
or
can
be
estimated
for
vector
x,
so
the
elements
*~~~~~~~~~~~
Discontinuities
in
surface
orientation.
of
x
are
measurements
like
pixel
gray
levels
or
values
of
a
signal
at
different
time
instants.
It
is
important
that
the
The
edge
detection
for
an
image
consequently
reduces
the
elements
are
mutually
correlated
and
there
is
some
size
of
the
image
and
the
amount
of
data
in
it;
it
filters
out
redundancy
in
x
which
makes
the
compression
possible.
some
unnecessary
information.
It
is
easy
to
be
implemented,
PCA
transformation
starts
with
centering
the
vector
x
by
because
less
processing
bandwidth
is
needed,
but,
it
is
very
subtracting
its
mean:
affected
by
the
luminance
conditions,
works
only
on
x~x,(E
()
grayscale
images.
This
mean
is
estimated
from
the
available
sample
x(1),
...
,x(T).
After
centering
the
vector,
the
mean
E{x}
will
2)Discrete
wavelets
transform
be
zero.
Then,
x
is
linearly
transformed
to
another
vector
y
In
this
method
[1]
we
are
suggesting
a
new
method
for
with
m
elements
such
that
m
<
n,
that
leads
to
induce
the
redundancyas
the
corelationsis
remove
.
Thsi.oeb
eye
recognition
using
discrete
wavelets
transform
with
fidunding
y
asrathe
orthogtional
is
cordinatesemove.
sh
tt
te
b
'Haar'
as
the
mother
wavelet.In
spite
of
many
researchers
elementsaoftatxdinrthegonew
coordinateystbeome
suncorreatede
opinions
although
it
is
being
computational
intensive
but
it
elements
of
x
in
the
new
coordinates
become
uncorrelated.
is
powerful
as
it
is
based
on
strong
mathematical
concepts
However,
the
variances
of
the
projections
of
x
on
the
new
and
relations
that
ensures
and
proves
its
reliability
and
its
coordinate
axes
are
maximized
so
that
the
first
axis
corresponds
to
the
maximal
variance;
the
second
axis
expected
efficiency
as
it
is
robust
against
different
lighting
corresponds
to
the
maximal
variance
in
the
direction
condition,
noise
and
low
computational
coast.
Wavelets
are
used
not
only
in
eye
recognition
but
also
in
many
other
fields
orthogonal
to
the
first
axis
and
so
on.
as
it
facilitates
the
analysis
of
a
specific
signal.
We
experienced
some
difficulties
in
dealing
with
wavelets
and
in
2)
SkinTone
Extraction
applying
some
algorithms
to
recognize
the
eyes
in
an
image.
Due
to
the
variation
in
luminance,
different
environmental
Many
algorithms
dealt
before
with
the
skincolor
and
how
conditions,
visual
angle
and
facial
expressions;
facial
organs
to
detectfaces
by
finding
the
skin
such
in
[9],
[12],[17]
&
and
features
are
recognized
by
this
order:
eyes,
nose
and
[19].
Mostof
these
techniques
were
based
on
finding
a
face
mouth.
Factors
that
must
be
taken
in
consideration:
pattern
by
comparing
it
with
a
certain
mask.
However,
they
eyae
e
'
yelahes,
eylidLs
movements,
head
tiltig,_
makeun,
glasses.
all
start
by
finding
the
suitable
color
space
such
as
YCbCr,
HWIU
or
others.
YCbCr
is
the
most
used
color
space
as
most
Of
the
previously
mentioned
algorithms
operate
on
the
177
Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.
a)
Discretised
wavelet
transforms
face
edge
detection
is
preformed
to
detect
the
edges
in
the
The
main
reason
for
using
this
type
of
wavelets
face.
4)
The
output
of
the
edge
is
taken
as
an
input
to
the
transformations
is
to
analyze
a
specified
signal
to
pick
just
a
wavelet
transformation.
discrete
subset
of
the
upper
half
plane
to
be
able
to
A.
Face
Detection
Algorithm
reconstruct
a
signal
from
the
corresponding
wavelet
..
coefficients.
The
corresponding
discrete
subset
of
the
half
1)
itnitalization
m
r
The
first
step
iS
initialization,
by
estimating
the
eigenface
plane
consists
of
all
thepoints
,.
with
integers
from
the
database
to
get
the
mean
position
and
size
of
the
while
m:
nI
E
Z.
The
corresponding
baby
wavelets
are
faces.
First,
we
load
the
name
of
the
training
set
from
a
text
known
as:
file.
Then,
we
get
the
principle
component
analysis
of
the
TM
=
fT

(2)
eigenfaces
resulting
from
thebasic
eigenface
algorithm.
At
a>1,
b>O
last
bymanual
processing
we
identify
the
mean
position
and
For
a
reconstruction
ofany
signal
x
of
finite
energy
by
the
size
of
the
face.
formula:
2)
Training
:(t)
=
E
)
X,
''6
3,
4}
We
load
the
training
set
from
the
listfile.
Then,
the
nI.EZnEZL
(3)
training
set
in
cropped
to
meet
the
criteria
from
phase
1.
Then,
we
apply
the
algorithm
discussed
in
[6].
At
the
end,
b)
Mother
wavelet
we
reduce
the
dimension
of
the
output
by
choosing
the
In
reallife
practical
applications,
the
designers
and
the
principle
components.
researchers
prefer
the
most
efficient
ways,
so
they
use
1.
The
first
step
was
getting
the
mean
of
the
training
continuously
differentiable
functions
withcompact
support
set.
as
mother
(prototype)
wavelet
(functions).
Therefore,
to
Al
satisfy
some
analytical
requirements
we
should
choose
the
(8)
wavelet
functions
from
a
subspace
of
the
space.
(8)
L(R)
n2
,
inthis
space
thefunctions
are
both
2.
Then,
we
centers
the
training
set
around
zero
by
L2R
subtractingthe
mean
from
it.
absolutely
and
square
integrable:
n
t
m
frt.
/~
~
~~~~~~
~~~~~D
=
Fi
T
(t)
I'
dt
<
oc
~~~~~~~~~~(9)
4tH2dt
K
(4)
3.
Afterwards,
we
calculate
the
eigenvectors
and
the
J
K(t)dt
<
(5)
eigenvalues
of
the
covariance
the
matrix.
It
can
be
ensured
that
we
can
formulate
the
conditions
of
C
Ln=
'?3+
zero
mean
and
square
norm
one:
=
AT
.C
(10)
_00
0(0t
dt
=O
0
(6)
Where
A[1
42
.4M1
f/
K(t)Kdt
=
(7)
4.
Then
finally,
we
get
theprinciple
components
of
In
the
case
of
continuouswavelet
transform,
the
T
must
the
eigenfaces
depending
on
its
eigenvalues.
be
a
wavelet.
The
mother
wavelet
must
satisfy
an
3)
Image
Preprocessing
admissibility
criterion
to
get
a
stably
invertible
transform.
We
insert
any
color
image
as
an
input
image.
This
image
is
In
the
case
of
discrete
wavelet
transform,
the
multi
resized
to
a
256x256
pixel
image.
Then,
the
image
is
resolution
analysis
is
used
in
many
constructions,
whichnormalized
to
adjust
the
lightening
condition.
Afterwards,
defines
the
wavelet
by
a
scaling
function.
we
apply
a
skindetection
algorithm
to
eliminatethe
background
from
the
resized
image.
At
the
end,
the
image
is
III.
SYSTEM
ALGORITHM
transformed
into
a
grayscale
image
of
PGM
format.
Our
algorithm
containsfour
main
techniques:
1)
we
uses
a
4)
Face
C
i
ProjectIon
skintone
extraction
as
a
preprocessing
step.
2)
Then,
we
use
The
gray
scaled
image
is
scanned
for
collecting
samples.
principle
component
analysis
to
successfully
detecttheface
These
samples
are
subtracted
by
thetraining
set
mean.
And
using
YaleFace
Image
Database
B.
3)
After
detectingthe
178
Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.
then,
the
output
samples
are
projected
on
the
reduced
color
in
the
cropped
region
and
we
took
thehighest
10%
of
dimension
matrix
from
the
training
phase.
them
to
lower
the
number
of
eye
locations
5)
Estimation
8)
Locating
the
eyes
For
the
final
phase,
theprojection
matrix
is
sorted
in
After
getting
10%
of
the
croppedimages
we
then
make
ascending
order
and
we
get
the
smallestprojection
as
our
sure
that
there
is
no
eye
position
close
to
theother
(we
final
output
of
theface
and
draw
arectangle
of
fixed
size
choose
5
pixels)
then
we
cropped
these
position
20X20
,of
150x180
pixel.
theeyeposition
in
512X512
colored
image
,to
look
for
white
region
in
this
area
which
will
be
identified
as
the
eye
if
not
B.
Eye
Detection
Algorithm
foundan
alarm
will
sound.
1)
Normalization
Due
to
the
undesirable
effects
of
the
luminance
the
YCbCr
code
gives
us
undesirable
output,
so
we
decided
to
IV.
EXPERIMENTAL
RESULTS
use
a
normalization
code
to
normalize
the
luminanceof
the
As
we
can
see
in
the
below
output
image,
acceptable
eye
input
picture.
detection
is
performed
despite
some
facial
conditions
like
2)
Edge
Detection
status
of
the
eyes,
the
existence
of
glasses,
different
skin
Edge
detection
as
mentionedwas
our
first
approach
but
as
colors,
and
head
positions.
This
algorithm
hasproven
we
witnessed
it
was
not
thebest
approach
when
it
comes
to
flexibility
and
reliability.
If
we
use
perfect
background
and
eye
detection
due
to
luminance
and
other
factors
but
after
all
perfectfacedetectionthe
efficiency
will
increase
up
to
9500
it
gave
us
the
edges
of
the
eye
so
we
decided
that
it
will
be
our
input
to
the
wavelet
transform,
so
that
it
will
make
the
eye
socket
detection
easier.
3)
YCbCr
transformation
We
developed
an
algorithm
that
has
the
ability
to
identify
the
human
skincolor
and
it
turns
it
to
blue
basedon
certainthresholds.
Its
main
idea
that
it
converts
the
input
RGB
image
to
YCbCr
space,
using
for
loops
and
conditional
if
R
statements
it
could
set
the
skin
color
into
blue
(value:
255),
and
it
cancels
any
objects
at
the
background
(value:
0).
4)
DWT
Themain
aim
of
this
function
is
to
compute
the
second
order
DWT
of
an
input
matrix
(image).
We
did
it
on
two
steps,
first
we
calculated
the
DWT
coefficients
using
'haar'
method,
second,
we
recomputed
the
second
order
DWT
of
the
computed
coefficients,
then
we
took
the
absolute
value
Fig2
Successful
eye
detection
the
second
coefficients
ina
matrix
(Z).
V.
CONCLUSION
AND
FUTURE
WORK
5)
Performing
IDWT
We
have
presented
an
algorithm
for
Drowsy
Driver
Computing
the
inverse
DWT
of
the
image.
We
calculated
the
Assistant
System
that
detect
skins
as
apreprocessing
step
addition
of
the
selected
node
from
the
wavelet
tree
and
its
and
then
apply
PCA
rules
to
detectfacessuccessfully.
The
maxima.
algorithm
begins
with
initialization
and
then
training
which
6)
Combination
Between
YCbCr
&
Wavelets.
are
applied
only
once.
Then,
we
taketheinput
image
are
Given
the
output
of
the
wavelet
we
combined
the
wavelet
preprocess
it
to
getaskin
only
image
in
grayscale.
The
to
YCbCr
by
adding
the
two
images
together.
image
is
then
sampledand
project
on
the
trainingspace.
7)
Cropping
the
eyeregion
Finally,
an
estimation
phase
is
applied
to
detect
the
face.
After
combining
the
YCbCr
with
wavelet
transform
all
After
getting
the
face,
the
algorithm
performs
edge
detectionpossibleeye
position
is
found
but
that
was
it
not
enough
we
and
wavelet
transformation
to
detectthe
eye
position.
At
last,
still
had
to
chose
from
the
many
eye
position
selected
.To
do
we
zoom
into
the
eye
position
to
know
the
status
of
the
eye
that
we
cropped
1
Oxi
0
image
of
the
YCbCr
of
every
selected
by
finding
the
white
components
in
it.
eye
position,
then
we
calculatedthe
percentage
of
white
179
Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.