1

Project Milestone 2

A Multimodal Human-Computer Interaction Framework into

Crisis Management

Submitted By:
Mahwish Ghani (06-0149)
Mohsan Naqi (06-0114)

Submitted to:
Mr. Jamal Ahmad

Dated: 17 March, 2009

National University of Computer and Emerging Sciences

Peshawar Campus

A Multimodal Human-Computer Interaction Framework into

Crisis Management
Mohsan Naqi

Mahwish Ghani

FAST university of Peshawar

Peshawar

FAST university of

P060114@nu.edu.pk

P060149@nu.edu.pk

Milestone 1
We selected this topic Multimodal HumanComputer Interaction Framework into Crisis
Management. The idea behind selecting this topic
was that today the world is facing main problems
like unpredictable disasters such as 9/11, earth
quakes etc. These disasters occur so suddenly that
the outcome of this results into great human loss.
However, information and communication
technology (ICT) make some help in the crisis
management and get details and share information
among different civil defense departments. But in
some major disasters these communication link
breaks down. So we must have an ad-hoc
framework and their modules connected with each
other and provide an environment to communicate
as an alternative source of communication. The
victim in the crisis area and control room operator
can communicate and help the victim. The
operator at control room can help the victim by
sending either rescuer teams, police or first aid
teams over there at crisis area and get updated
information about the situation and reduce the
dead rate. The system framework must be stable
and be based on multimodal with the help of which
it can get information from the victim to operator
at control room.

Milestone 2: Introduction
In our research there are three main key words
multimodal
human
computer
interaction,
framework and crisis management. We have
worked on these things and get information how
can these things relate to each other and make
efficient human actor and machine interaction.

Multimodal human computer interaction

The multimodal interaction means provide users
many modes to interact with the system. The
multimodal HCI is the modalities which belong to
human nature. These modalities, such as text,
speech, visual language, gesture, pen writing and
drawing, and facial expression. There are already
many search made on these modalities. But in our
research we use available modules and connect
with our framework. These modules can be added
or remove without from the framework without
affect the all system.

Framework
It is underlying conceptual structure that provides
solution for the problem. In our paper we mainly
focus on the framework architecture used for this.
In this framework input take from available
modules of multimodal HCI. The input takes from

3
cell phone, PDA and other resources which is
today use to take input from the user in the form of
multimodal modules form.

Crisis management
The crisis is unpredictable events that creates
disasters and harm the organization and peoples
The practice of crisis management involves
attempts to eliminate technological failure as well
as the development of formal communication
systems to avoid or to manage crisis situations. In
our research we related our framework with crisis
management. It means we will handle crisis
management through some framework. The main
actors involves in this process are victim, control
room operator and artificial intelligent agent and
which decide and help the operator to solve the
problem providing them best solution.

Targets

In this paper we mainly focus on the

framework architecture of the system,
which are the mainly parts of the frame
work and how the interface make better
interact between different roles in this
scenarios? We simulated the whole
scheme. The main structure of the
framework and many different parts of
the system, there are framework of
different type but we mainly focus on
the hybrid agent-based architecture,
they are used for Collaborative Decision
Making. It means that Decision making
between man and machine, between
actors and agents. There is involve
some artificial intelligent involve but we
try our best to explain there concept
behind work on it. Which multimodal HCI
modules or some other modules more
helpful in the particular situation and
which one more needs to proceed?
Which technology is the most suitable in
the major crisis? How these system
made affect?

Problems
The topic is very critical and because there are
involving the latest technologies that why there
are a lot of problem regarding the topic.

There are a lot of available multimodal

modules and other modules to select them
appropriate and interrupt them according
to the condition are not an easy task. Every
module has own limitations.

There are many fields relate behind on this

research like Data mining, Artificial
intelligent and Networks, so the use of all

4
cognitive base , both have their

these concept at one time create difficulty

for us to analyze this project

If the framework architecture is found then

what kind of system designed, what kind
of interface that user and control operator
can operate is also a main problem?

We found some framework architecture

but there term are so difficult and not
relate to our subjects, and each research
paper refer to other papers.

There is research of this kind may be easy

but implementation of this really a difficult
task.

What techniques should used for

architecture because basically there are
mostly using hybrid agent based and other

benefit and limitations.

The nature of crisis is dynamic

so the planning for the crisis
management is also difficult
and no body predicted in
advance about that.

If we made or get some

simulation environment then to
execute a simulation on one pc
is not possible, because there
are involves a lot of tools and
roles, which can not present
interact on the same pc.