ORL face classification using Adaboost classifier in sklearn

Preface:The blogger surfed the Internet to browse the use of Adaboost to achieve face classification, found that there is no classification, most of them are all about face recognition detection, and does not achieve accurate classification of a person (for example, what is the name of this person), about the ORL recognition, most of the PCA dimensionality reduction or SVM implementation, but also did not use Adaboost, so therefore wrote an accompanying article.

algorithmic principle

Algorithmic Process:

Suppose we have a binary classification training dataset T = {(x₁,y₁),...,(x_N,y_N)},y_i∈{−1,+1}.

1) At the beginning, we let each variable x_iall have the same weight, i.e., at first each sample has the same role in the base classifier.

D_j = （w_j1，w_j2，…. w_ji，…. w_jN） w_jN = 1/N

D_jdenotes a vector of weight distributions, where j denotes that the current is the first round because there are many more rounds of learning to follow, changing this distribution each time. w_jidenotes the xth round corresponding to the jth round_ivariant
2) For m = 1 , 2 , . . . , M , for a total of M base classifiers Perform the following steps:

①: learning to obtain the basic classifier G_m(x)

Using a weighted distribution D_mof the training dataset is learned to obtain the basic classifier G_m(x)，G_m(x) can give an output of +1 or -1 depending on the inputs

②: calculate the classification error rate of a single base classifier e_m

e_m = \(\sum_{i=1}^N\) w_mi I(G_m(x_i) \(\neq\)y_i)

What does this formula mean: for the mth round the error e_m, there are corresponding x_iCorresponding weights w_miI is the indicator function that takes 1 when the equation in the parentheses after it holds and 0 when it does not, i = 1, 2, ...N for all x_i So e_m is to add up the weights of all the misclassified samples, if the initial weights are equal, then 2 out of 5 samples are classified incorrectly, the error rate is 0.4, and if the weights are [0.5, 0.1, 0.1, 0.1, 0.1, 0.1] and the sample with weight 0.5 is classified incorrectly, the final error rate is 0.5.Thus this design makes it possible to classify the samples with high weight correctly can significantly reduce the classification error rate.

③: Calculation of G_m(x) of the classifier voting weight α_m(i.e., importance in the final outcome)

Assuming that e_m = 0.3, i.e., smaller classification error and smaller error rate, then, α is larger.

For binary classification problems with an error rate of more than 0.5, a simple full inversion can bring it down to less than 0.5. For example, if the classification result is [1,1,1,-1] with an error rate of 0.75, then just change it to [-1,-1,-1,-1,1] with an error rate of 0.25. (Another way to think about this is that random classification has an error rate of 0.5, and weak learning has an error rate of 0.5.) (Another way to think of it is that random classification has an error rate of 0.5, and the weak learner, while weak, has a slightly higher correct rate than random classification.

④: update the weight distribution of the training set

for the m+1st round:

w_m+1,i = \(w_{mi}e^{-α_my_iG_m(x_i)}/Z_m\)

Among them.\(Z_m = \sum_{i=1}^Ne^{-α_my_iG_m(x_i)}\)

Get the final classifier

code implementation

Platform:kaggle online notebookkaggle has no need to match the environment, there are a large number of online public datasets, to save a lot of trouble

ORL data sets

We use a public database for algorithm implementation.

The ORL face database was selected as the experimental sample, with a total of 40 people, 10 images per person, and an image size of 112*92 pixels. The images themselves have already been processed and do not need to be normalized or calibrated. The experimental samples are divided into training samples and test samples. First, we set up the training sample set and select the first 6 images of 40 people as the training samples. Then we set up the test sample set and select the last 4 pictures of 40 people as test samples for recognition.

Source of this dataset:/datasets/jagadeeshkasaraneni/orlfaces

Specific code:

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (. pd.read_csv)

import os
for dirname, _, filenames in ('/kaggle/input'):
    for filename in filenames:
        print((dirname, filename))

Output: /kaggle/input/orlfaces/ORL_faces.npz

orlfaces = ("/kaggle/input/orlfaces/ORL_faces.npz")

Output: ['testY', 'testX', 'trainX', 'trainY']

The training and test sets in this dataset have been divided for us, 60% training set, 40% test set.

print(orlfaces['trainX'].shape)
print(orlfaces['testX'].shape)

Output: (240, 10304)
(160, 10304)

240 is a total of 24 people with 10 pictures each, so 240
The test set has 16 individuals, so the value of the first dimension is 160
112 * 92 = 10304, i.e. the picture we want is 112 * 92, he puts it in the same dimension, so it's 10304.

X_train = (orlfaces['trainX'], (240, 112, 92))
Y_train = orlfaces['trainY']
X_test = (orlfaces['testX'], (160, 112, 92))
Y_test = orlfaces['testY']

print(Y_train.shape)
Y_train

Output:
(240,)

array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4,
4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5,
5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7,
7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11,
11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12,
12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15,
15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18,
18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19,
19, 19], dtype=uint8)

We will only show 3 of these images: the

## import as plt
%matplotlib inline

def show_images(images) -> None:
    n: int = len(images)
    for i in range(10):  # Too many pictures.，showcase10Forget about the picture. Otherwise, fill in here range(n)
        ()
        (images[i])

show_images(X_train)

Output:

Extra images omitted...

from import AdaBoostClassifier
from import accuracy_score


# Assume X_train and y_train are training samples and labels, and X_test and y_test are test samples and labels.
# Here you need to load and preprocess according to the actual image data

# Initialize the Adaboost classifier
ada_clf = AdaBoostClassifier(n_estimators=200,learning_rate = 0.1, random_state=42)



# Train the model
ada_clf.fit(X_train.reshape(-1,112*92),Y_train) # Adaboost classifier requires X_train dimension <= 2

# Predict the test sample
Y_pred = ada_clf.predict(X_test.reshape(-1,112*92))

# Calculate the accuracy
accuracy = accuracy_score(Y_test, Y_pred)
print(f "Recognition accuracy: {accuracy:.2f}")

Output:
Recognition accuracy: 0.76

print(Y_test,Y_pred)

We can print out the predicted Y and the true Y and compare them with an accuracy of 0.76

We can adjust then_estimators,learning_rate , random_statTo improve the accuracy, take the following values.

n_estimators: This parameter specifies the number of weak classifiers to be used.n_estimatorsA larger value of means that more weak classifiers will be used to build strong classifiers. This may lead to better performance, but it also increases the complexity and training time of the model.
learning_rate: this parameter is the contribution of each weak classifier to the final prediction. In AdaBoost algorithm, the weight of each weak classifier is proportional to its accuracy.learning_rateparameter controls the size of this scale. The smallerThe `learning_rate` implies that each weak classifier has less impact on the final result and more weak classifiers may be needed to achieve the same result.
random_state: this parameter is used to control the seeding of the random number generator to ensure reproducible results. SettingThe `random_state` ensures that each time the code is run, the same results are obtained as long as the input data remains the same. This is important for debugging and experimentation as it allows researchers to compare the effects of different models or parameter settings.

consultation

/codelady_g/article/details/122571189
/code/jagadeeshkasaraneni/orlfacerecognition/notebook