Skip to content

Add Theos introduction #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions paper/main.tex
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
\maketitle

\begin{abstract}
This research paper treats different optimization algorithms applied on the classical image classification problem. The algorithms are implemented in C++, using the Eigen library for linear algebra, and applied on two different datasets: the MNIST dataset of handwritten digits (grayscale images), and the ORL dataset of faces (grayscale images). The results of the different algorithms are visualized by applying the Principal Component Analysis and plotting the 2D data. Throughout this paper, the algorithms in question will be described, then compared based on their execution times and their success rates.

\end{abstract}

\begin{IEEEkeywords}
Expand Down
29 changes: 11 additions & 18 deletions paper/src/introduction.tex
Original file line number Diff line number Diff line change
@@ -1,24 +1,17 @@
\section{Introduction}
The importance of data analysis and machine learning in our modern day world is increasing day by day.
And so is the number of different algorithms used.
In this paper, five of the most basic classification algorithms are used to solve an images classification problem on two very popular datasets: The MNIST dataset of handwritten numbers and the ORL dataset of faces.
Using the corresponding labels, each algorithm is trained in a supervised training process and then applied on a test set.
The resulting success rate in combination with the execution time will be a measure on how efficient each algorithm is.
The image classification problem is a very common case of study when it comes to machine learning and data analytics. There is a substential amount of different algorithms that can be used to treat this problem, however, the lack of computational power has slowed down the study of those. Nowadays, this is no longer a problem, as computers have become significantly faster than when these algorithms were thought and designed, and it is now a lot easier to compare them in depth. Machine learning is becoming a trend, and as it can be utilized on affordable hardware, it can be applied to a large variety of problems.

This study focuses on classification of two sets of images: the MNIST dataset, which is a collection of 70000 images representing handwritten digits from 0 to 9, in grayscale, and the ORL dataset, containing 400 grayscale facial images, representing a total of 40 different individuals.

In addition to applying the algorithms on the raw data, they are also applied on a reduced version of the dataset.
This reduced version is obtained by applying the common \textit{Principal Component Analysis}.

The following algorithms have been choosen and implemented in \texttt{Matlab}:
The following optimization algorithms will be applied on the said data samples:
\begin{itemize}
\item Nearest Centroid Classifier
\item Nearest Subclass Centroid Classifier
\item Nearest Neighbor Classifier
\item Perceptron, trained with Backpropagation
\item Perceptron, trained using the Minimum Square Error
\item Nearest Centroid Classifier
\item Nearest Sub-class Centroid Classifier
\item Nearest Neighbour Classifier
\item Perceptron trained using Backpropagation
\item Perceptron trained using Mean Square Error
\end{itemize}

The MNIST dataset consists of 60000 images while the ORL dataset only consists of 400.
This opposes a problem since some of the algorithms rely on a large set of parameters that have to be optimized during the supervised learning.
To overcome this problem and still get sufficient results, each algorithm is applied multiple times and the success rates and execution times are averaged.
This helps to make the overall results more predictable.
In order to establish a benchmark for these classifiers, their computation time and their accuracy will be compared, and they will be applied on the two datasets reduced to two dimensions after applying the Principal Component Analysis (PCA), which will also allow for the plotting of the results.


Loading