\chapter{Evaluation}\label{chapter-evaluation}

... Introduction ...

\section{Methods \& Tools}

Algorithms for mobile ad-hoc networks can be evaluated through various
methods.  Basically, these evaluation methods can be diveded into three
categories\cite{Jiang06}: Measurements or field testbeds, emulation, and
simulation. With measurements the algorithms run on mobile devices in the
specific real-world scenario. Measurements need a lot of hardware and
manpower in order to arrange a test run but they also provide the most
realistic results for a scenario. The main disadvantage of measurements is
that they are hard to reproduce. A changing environment, like people or
cars moving differently than in the previous test, can have a huge impact
on the performance of the network and the results of the test. Hence with
measurements, it is hard to tell whether a better performance comes from a
better version of the algorithm or a changed environment 

Secondly, network emulation tries to bypass the problem of reproducibility
in a changing environment by imitating the network. Here, mobile devices
are connected directly to a network emulator which is usually a single or
more computer. This computer recreates predetermined network
characteristics in order to create a stable environment for the algorithm.
Because the network is only emulated, the same scenario can be tested over
and over again. The main disadvantage of emulation is that accurate timing
of data signals can be in the order of several microseconds which can be
difficult to achieve. In order to achieve reproducible results, emulation
of MANETs often use wired networks instead of transmitting data wirelessly.
This can either be done by using standard Ethernet or by replacing antennas
with cables. 

Finally, when using network simulation not only the network but mobile
devices are recreated inside a computer as well. Because all parts of a
test run is embedded inside the simulation, it is able to produce an
optimal time resolution because of it's ability to run the simulation
slower than real-time. On the other hand, simulations try to mimic
everything from top to bottom which may even be more unrealistic than
network emulation.

We conclude that the extra care should be taken when evaluating algorithms
for mobile ad-hoc networks. Each presented method has its distinct set of
advantages and shortcomings. Table~\ref{table-evaluation-methods} shows a
more detailed comparison of all three methods.

\TODO{...}

Why we choose network simulation, why we choose NS2\cite{kurkowski05}

\begin{table}\begin{center}
\setlength\doublerulesep{.5pt}\doublerulesepcolor{white} 
\setlength\arrayrulewidth{2pt}\arrayrulecolor{white} 
\begin{tabular}{>{\columncolor[gray]{.70}[.80\tabcolsep]}l
>{\columncolor[gray]{.85}[.80\tabcolsep]}c
%@{\extracolsep{\fill}}
>{\columncolor[gray]{.70}[.80\tabcolsep]}c
%@{\extracolsep{\fill}}
>{\columncolor[gray]{.85}[.80\tabcolsep]}c
%@{\extracolsep{\fill}}
>{\columncolor[gray]{.70}[.80\tabcolsep]}c
%@{\extracolsep{\fill}}
>{\columncolor[gray]{.85}[.80\tabcolsep]}c
%@{\extracolsep{\fill}}
>{\columncolor[gray]{.70}[.80\tabcolsep]}c
%@{\extracolsep{\fill}}
}
%\hline
& \multicolumn{2}{>{\columncolor[gray]{.85}[.80\tabcolsep]}c}{Practicability} & \multicolumn{2}{>{\columncolor[gray]{.70}[.80\tabcolsep]}c}{Realism} \\ \hline
Method		& Effort	&	Reproducible	& Timing 	& Real hardware\\ \hline\hline
Measurement	& +		&	--		&	+	&		 	+				\\ \hline
Emulation		& o		&	+		&	--	&			o			\\ \hline
Simulation	& --		&	+		&	+	&			--			\\ \hline
\end{tabular}\end{center}
\label{table-evaluation-methods}
\caption{Comparison of different evaluation methods} 
\end{table}


        \begin{tabular}{|l|l||c|c|c|}
        \cline{3-5}
        \multicolumn{2}{l|}{} & Real world & NS-2 Std. & NS-2 INRIA \\
        \cline{3-5}\hline
          & 802.11 &a/b/g  & b & a \\ \cline{2-5}
         MAC & Rate Adapt. & Various & -- & ARF, AARF \\ \cline{2-5}
         & Retries  & Multi-Rate & Same Rate& Same Rate \\ \hline
                  & & Realistic & FreeSpace & FreeSpace\\       
                  & Propagation & & TwoRay & {\em Shadowing}\\  
                  & & & Shadowing  &\\ \cline{2-5}
                  & Multipath & X & -- & --\\ \cline{2-5}
         \raisebox{.5em}[-.5em]{Radio} & Fading & X & -- & --\\ \cline{2-5}
               %   &                      &               & TwoRayGround & Shadowing 
        \raisebox{.5em}[-.5em]{model} & &  & & Energy, SNRT\\ 
              & \raisebox{.5em}[-.5em]{Reception}& \raisebox{.5em}[-.5em]{Realistic} & \raisebox{.5em}[-.5em]{SNRT}  &  BER\\ \cline{2-5}
                  
                  & && Other & Thermal noise \\ 
                  & \raisebox{.5em}[-.5em]{Noise} &\raisebox{.5em}[-.5em]{Realistic} & signals & + oth. signals \\ \hline \hline
        Env. & Obstacles & LOS/NLOS & LOS only & LOS only \\ \hline
        \end{tabular}
        
        
\subsection{Used Network Parameters in Simulation}
