Machine Learning an Artificial
algorithm tends to be pretty sophisticated. It gives the computers the ability
to learn from the surrounding data and make decisions. Instead of building
heavy machines we have built such algorithms that eventually helps to decrease
the number of complex algorithms and helps the computer become independent. In
such cases pattern recognition becomes one the most important challenge faced.
It is used by most of the algorithms to make optimized decisions. It is mainly
a study of how to observe the environment, distinguish between what should be
considered amongst the whole environment and to take particular decision based
on the observations. This report talks about different machine learning
techniques. Also the pattern recognition process, design cycle, applications
Different types of machines have
different machine learning algorithms, building these algorithms is a challenge
for the scientists. Different algorithms give different learning experience to
the machines. It certainly doesn’t depend on the nature completely but also the
data structures used as well as the theories of cognitive and genetic
structures. Many of them are borrowed from the current neural networks and
cognitive sciences. Overall, learning is to improve performance based on some
measure defined to know if the machine has learned something. We have two main
types of algorithms that is supervised and unsupervised algorithm. Humans have developed high abilities to sense
the environment like recognizing handwriting, taste, colour, faces etc. We need
to make the machines analyze the same. Pattern recognition was developed in the
1960s. But in spite of all these years of research the goal of designing a
general purpose pattern recognizer is still not accomplished.
Supervised algorithm perceives both
the input as well as the output and generalizes in a way that it can be used by
all possible inputs. After analyzing the training data it produces an immediate
example which can be used to map new examples. It follows the following steps:
1) Determining the training examples types – The
user should know what kind of data should be used for the training set.
Gathering a training set – An input –
output set is gathered.
Determining the features of the input of
learned function – The learned function accuracy depends on the representation
of the input. The input object is transformed into a feature vector containing
features describing the input object.
4) Determining the learning function structure
and the corresponding learning algorithm.
the design of the data and running the learning algorithm on the same.
6) Evaluating the learned function accuracy.
learning are of 2 categories:
algorithm applies to just nominal responses with few values.
for responses that are a real number.
supervised algorithms are as follows:
SVMs are models used to analyze data for classification and regression
analysis. It has good speed and memory usage when the vectors are few. Even
when the default linear scheme is easy to interpret while using a kernel it is
difficult to know how o=the data is being classified.
Bayes – It has good speed and memory usage for simple distributions but is poor
for large datasets and kernel distributions.
Neighbor – Nearest neighbor can have either of categorical or continuous
predictors at a time. It has poor predictions with high dimensions and also
does not perform fitting with linear search.
Analysis – It is accurate when the modelling assumptions are satisfied else the
The machine receives input but does
not obtain the target output nor the rewards from the environment. But it can
develop a framework based on the knowledge it develops from the environment.
unsupervised algorithms are as follows:
Hierarchical clustering – Vectors are given as input and a dendogram is
returned as output. It creates a multilevel cluster tree.
2. K-means structuring – It is more efficient than hierarchical
clustering. In this algorithm each observation is classified into different
clusters depending on it’s nearest mean.
SUPERVISED VS UNSUPERVISED LEARNING
supervised algorithm the classes are predetermined whereas in unsupervised the
basic task is to develop classification labels.
supervised algorithm the data can be divided into segments and then the machine
searches patterns and mathematical models based on the data. In unsupervised
algorithm the data is divided into clusters based on their similarities.
supervised algorithm the models are evaluated on the basis of their predictive
capacity in relation to measures of variance in the data itself. Whereas in
unsupervised the machine is told in advance how many clusters should be formed.
PROBLEMS ENCOUNTERED DURING LEARNING
depends on the machine and the algorithm completely. Since the machines relies
on the information it perceives from the environment, the machines should be
ready to face the challenges it comes across. Such problems affects the
learning process of the machine. As different input gives different output it
becomes important to take into consideration appropriate and optimize output by
the machine. The problems faced during learning are:
The machine tends to prefer one hypothesis over another. Say for example we
have two agents N ad P. Since both the agents have their own hypothesis
predicted by taking all the data into consideration, it becomes difficult for
the learning agent to distinguish between which one is the best. Till the
learning agent cannot choose between the two hypothesis, the agent cannot
resolve the disagreement. In order to come to a conclusion, the agent needs a
bias. A good bias is the one which works best in the practical environment
asking which hypothesis suits the best to the data.
2) NOISE- In
the real world, data can never be perfect(without noise) . Noise is created
when some of the attributes have missing values, have been assigned
inappropriate values. Handling these noises becomes important for the learning
3) PATTERN RECOGNITION – Pattern
recognition is used in the classification of objects (2D or 3D) and abstract
multidimensional patterns into categories. There are many pattern recognition
systems for character and handwriting, speech and speaker recognition, document,
fingerprint, white blood cell classification, military target recognition. The
machines train the pattern recognition techniques to identify objects for
sorting, inspection, and assembly. The design of a pattern recognition system
requires the following modules: sensing, feature extraction and selection,
decision making, and system performance evaluation. The availability of low
cost and high resolution sensors (e.g., CCD cameras, microphones and scanners) and
data sharing over the Internet have resulted in huge repositories of digitized
documents (text, speech, image and video). Need for efficient archiving and
retrieval of this data has fostered the development of pattern recognition
algorithms in new application domains.
PATTERN RECOGNITION GOALS
the models that describe
the two populations.
the data to get rid of the noise in it.
the model that best represents the pattern.
AREAS IN PATTERN RECOGNITION
Template matching:- The
pattern to be recognized is matched against a stored template while taking
account all the translation, rotation and scale changes.
Statistical pattern recognition:-
It focuses on the statistical properties of the patterns
Artificial Neural Networks:-
It focuses on biological neural models.
Syntactic Pattern Recognition:-
It’s decisions are based on logical rules and grammars.
STEPS INVOLVED IN PATTERN RECOGNITION
acquisition and sensing: Measurements of physical variables,
Important issues: bandwidth, resolution, sensitivity, distortion, SNR, latency,
2) Pre-processing: Removing
noise from the data, separate the patterns of interest from the background.
Feature extraction: Finding a new
representation in terms of features.
Model learning and estimation:
Learning to map between features, pattern and categories.
Using features and learned models for assigning pattern to a category
Post-processing: Evaluating confidence
in decisions, Exploitation of context to improve performance,
Combination of experts.
FOR DESIGNING THE SYSTEM OF PATTERN RECOGNITION
Definition of pattern classes.
Feature extraction and selection.
Selection of training and test examples.
PATTERN RECOGNITION SYSTEM:
the pattern has the following steps:
1) Data collection: First step is to collect our training and test data and the
the data collected has adequate set of values or not.
2) Feature selection: In this step we study the data in terms of it’s domain dependence and prior information, it’s
computational cost and feasibility, values having patterns,
values having different patterns, invariant features with respect to
translation, rotation and scale, robust features with respect to occlusion,
distortion, deformation, and variations in environment..
3) Model Selection:- In this step we select the model based on the following
criteria:- It’s domain dependence and prior information, Design criteria,
parametric and non-parametric models, handling features with missing values and
also it’s computational complexity.
various models are:- Templates, theoretic or statistical
decision, syntactic or structural, neural, and hybrid.
Using these models we can identify hoe close we are to the final model having
the underlying patterns.
4) In this phase we decide how to learn the rules from the provided data.
being of 2 types:-
learning – Here a categorical label is provided for each and every pattern in
the training set.
learning – The machine itself forms clusters and groups based on the input
learning – Here the agent provides a feedback of the decision is right or wrong
even when the category is not initially designed.
5) Evaluation – This is the final step in which we need to evaluate how we can
estimate the performance of the training dataset in the present and also in the
near future. And also evaluate the problems faced due to over fitting.
PATTERN RECOGNITION MODELS
Techniques for analyzing multidimensional data
of various types and scales along with
for projection, dimensionality reduction, clustering and classification of data
Pattern recognition models can be designed using the following steps:
matching – For template matching the patterns are represented in the form of pixels,
curves etc. and the recognition function used is correlation between the
patterns and the distance measure. The typical criterion for this approach is
the classification error.
pattern recognition – For statistical pattern recognition the patterns are represented
in the form of features of the patterns and the recognition function used is the
discriminant function. The typical criterion for this approach is the classification
or Structural – For statistical pattern recognition the patterns are represented
in the form of primitives of the patterns and the recognition function used are
the rules and the grammar. The typical criterion for this approach is the acceptance
4) Neural network – For statistical pattern
recognition the patterns are represented in the form of pixels, features of the
patterns and the recognition function used is the network function. The typical
criterion for this approach is the mean square error.
recognition has it’s application in the following areas:
Some examples of pattern recognition
applications are as follows:
Bioinformatics – It is used in sequence
analysis with DNA/Protein sequence as the input. Here the pattern classes are
known types of genes.
Data mining – It is used in searching
meaningful patterns with points in the multidimensional space as the input.
Here the pattern classes are Compact and well as separated
Document Image Analysis – It is used in optical
character recognition with document image as the input. Here the pattern classes
Document classification – It is used in
the internet search with text document as the input. The patterns are classified
in semantic categories.
Industrial automation – It is used in
printed circuit board inspection with intensity image as input. The pattern
classes are either defective or non-defective depending on the nature of the
Multimedia database retrieval – Internet searching
is one of the major application having video clips as input and patterns classified
on the basis of video genres.
Biometric recognition – Personal identification
uses biometric recognition and has fingerprints, iris, face as input and the
patterns are classified based on authorized users with access control to those
Remote sensing – Remote sensing applies in forecasting the
crop yields with a multi spectral image as an input and classes in the form of
land usage and growth patterns of the crop.
Speech recognition – The telephone
directory uses speech recognition after receiving the speech wave form and
forms classes based on the spoken words.
Medical – Computer aided diagnosis use pattern
recognition with microscopic images.
Military – Automatic target recognition
has classes in the form of target type and optical / infrared image as input.
Natural language processing – It is used
in information extraction with sentences as input and pattern classes as parts
pattern recognition is used to cover all stages of an investigation from
problem formulation and data collection through to discrimination and
classification, assessment of results and interpretation.
basic terminologies are described below:
Steps involved in the statistical pattern
the problem – Understanding completely the aim of investigating and also
planning the remain stages in the entire process.
collection – Recording details of the data collection procedure and measuring all
the appropriate variables.
examination of the data – Verify the data, calculate the summary statistics and
produce the plots in order to get the structure.
selection / feature extraction – Select variables from the sets that are appropriate
for the given task which are gained from the either linear of non linear
transformation of the original set. This feature extraction is artificial to
pattern classification / clustering – We analyze the data and provide a successful
conclusion to our study and also it acts as a pre procesing for the supervised learning.
discrimination or regression procedures as appropriate – Here the classifier is
designed using the training set.
of results – The trained classifier is
applied to the independent test set of patterns that are labeled.
– To analyze the results we need further hypothesis that need further data
collection. This cycle can be terminated at different stages: The hypothesis
posed can be answered at the initial study of the data or maybe it is later
discovered that the data cannot answer the stated hypothesis and hence it has
to be reformulated.
pattern recognition approach
this approach all the patterns are represented in the form of d features that
are viewed as a point in the d-dimensional space. The main aim is to select the
features in different categories having pattern vectors so that they can capture
compact and d-dimensional feature space. The separation of different patterns from
the classes determine how effective the representation space is. After
obtaining training data from different classes the main aim is to generate
decision boundaries that separate the patterns that belong to different
classes. In statistical decision theory approach we generate the decision
boundary depending on the probability distribution of the patterns belonging to
different classes and these boundaries should be either specified or learnt.
The discriminate analysis approach can also be used for classification where we
first form a decision boundary in the parametric form and then based on the training patterns we choose the
best decision boundary. These boundaries are created using the mean square error
criterion. According to Vapnik’s philosophy “If we give a restricted amount of data
to solve some problem and try to solve such problem but never try to solve a
more generic problem then we can never conclude based on the information
provided as it is insufficient.”
recognition is a field of study developing significantly from 1960s. It was
very much an
subject, covering developments in the areas of statistics, engineering,
computer science, psychology and physiology, among others. It has huge number
of applications in the field of Bioinformatics, Data Mining, Document
Classification, Document Image Analysis, and Industrial Automation, Multimedia,
Database retrieval, Biometric recognition, Remote sensing, Speech recognition,
Medical, Military, Natural language processing.
the performance of learning algorithms and some classifiers have been seen and
analyzing the evaluation methods with metrics they use to measure the
performance by defining formal framework. We have concluded that the
performance of the classifiers are measured on the basis of the classification
accuracy. Some methods can be used to evaluate classifier or algorithm in
general while some others are applicable only to few algorithms. We have also
seen how pattern recognition is important in the field of artificial
intelligence. It is emerging as human beings have their own limits in
recognizing patterns. The report also shows how the statistical approach covers
various stages of investigation from formulating the data to interpreting the