Exploring PCA And EM Clustering In Unsupervised Learning Essay.

Here we are interested in finding the d-dimensional subspace spanned by

Explain, using diagrams if necessary, why the objective of problem (2) takes this form(you may find it useful to recall slide 15 of the lecture on ‘Dimensionality Reduction’,and to consult the ‘Extra Reading’ for that week).

In each analysis we were careful to centre our input data by effectively subtracting off the mean. Why is it important to centre the data in this way?

Show that we can re-write the objective of problem (1) as follows, and provide an expression for the matrix S:

If we were to replace S with the sample correlation matrix and then proceed to perform PCA with this objective, under what circumstances would this form of PCA differ from the covariance matrix version?

We wish to maximise this expression subject to the usual constraints using the EM algorithm.

Explain why, as the solution we generate from the spherical GMM EM clustering algorithm will tend towards the solution we would generate from the k-means clustering algorithm.

Describe and explain the form of the boundary which discriminates between clusters in this case.

Describe and explain the form of the boundary which discriminates between clusters in this case in general.

Explain the importance of the Representer Theorem?

Objective of PCA

1a)

Let us assume the unlabelled dataset on ,        

Mean values is

Orthonormal basis set values is ,

Let us consider the D dimensional of PCA values is,

The unlabelled dataset of the dimensional reduction can be used for the data mining. Its goal of this problem we can use the informative features of input data, so that they can be used further in predictive algorithms (Perner, 2015). The dimensional reduction of the covariance matrix has a special property that they point towards the directions of the most variance within the data. The 1st dimension of the vector points can be specified that direction of the highest variance (Unsupervised monitoring of an elderly person's activities of daily living using Kinect sensors and a power meter, 2017).2 dimensional can be used for orthogonal of the highest variance on 1st vector. Let us consider the capturing more variance means capturing information to analyzed. For examples,

1b)

Show that problem of PCA values is,

   Argmax

  u[i].u[j]= 

   Argmin

u[i].u[j]= 

Argmin (-x[i].

Argmax

 The given values of the equation 1 and 2 are equivalents.

1c)

The analyzing of the input data that can be subtracting on the values is X[i]. The X[i] values is to be higher values of x[n], can be introduce in the new variables are H is should be positive (PCA ADMINISTRATIVE COMMITTEE., 2015) . Mean values of the input data dropped a level, the value would be minus 1 (-1) (PCA ADMINISTRATIVE COMMITTEE., 2015)

1d)

The rewrite the equation of formulation on PCA problem of the matrix expression is denoted as S values is,

Despite the values of the equation of input data is depending on the parameters can be follow on equation(1) is the values to be consider on the formulation is, u[i].u[j]= δij

With s values either 0 or 1, that can be assume the rewrite of matrix expression is cos (ξ + λu),

        Cos(u[i].u[j]+)=

 Let us assume the new variable is X,

     =-X (cos (η) cos h(ϑ) cos() − (−1)a sin(η) sinh(ϑ) sin()

     S =-[h e ϑ cos (η + (−1)^a) + e −ϑ cos (η − (−1)^a]

1e)

The rewiring value of the S value can be considered on correlation of the matrix of PCA circumstances (MARKS, 2011). The covariance of input data of the unlabelled dataset of PCA is,

S =-[h e ϑ cos (η + (−1) ^a) + e −ϑ cos (η − (−1) ^a]

The equation of the factor and to be calculates the weighted average of the original variables (Nikolov and Petrov, 2011). The weights S, are constructed of the variance of a) is maximized. The weighted of the matrix is S to be calculate the variance using the this formula,  

The decomposition of the matrix S that provides the PCA problem. Can be defined as,

The correlation of the matrix is S, is used instead of the covariance matrix, S the equation for Y can be modified (Uprichard and Byrne, 2012). The PCA equation is,

 y=W'D U^’S

Question 2

2a)

The unlabelled points on k cluster is

Importance of Centering Data

Gaussian random variable, X is,                                       

The expresion of the likelihood dataset of

= {µ [j] , π[j]} k j=1.

The likelihood of the various reasons can be used for computation convenience, we work with the loglikehood formula can be used for given equation is

l(θ|x)=logL(θ|x)

 {µ [j] , π[j]} k j=1

Which is defined up to an arbitrary additive constant? For example, the binomial log likelihood is

Pz(z=j)=, l(π|x)=xlog+(n−)log(1−).

In my problem of interest, we will derive our log likelihood from a sample rather than from a single observation, let us observe an likelihood is π1,π2,.............πn of f(π^([j])), then the overall likelihood the product of the individuals likelihoods,
 L(π |x)=n∏i=1f(xi| π)=n∏i=1L(π |xi)

∑i=1logf(xi| π)=n∑i=( π |xi).

2b)

The EM algorithm can be used for the iterative methods to find the maximum likelihood or maximum a posteriori estimates of parameters is statistical models, where the models can depends on unobserved latent variables values is γ i[j] (Lyon and Kinney,

2c)

Feature extraction:

The goal of the future work is extraction block is to extract the classification and to eliminate the rest. The statiscal model can be used for represent the statiscal of the each class, which allows the classes to be separated from each others. The statistical model usually has some probability justification (such as the GMM) but sometimes it might just be used for additional tasks such as data compression of the decision component output (Kiviet and Feng, 2014).

The specific class can be denoted as the Gaussian values of,

2d)

Minimum discrimination information can be used for generalization likelihood methodology for the classification and clustering of multivariate time series (Cole, Chu and Greenland, 2013). Discrimination between the class of multivariate time series that can be characterized by differing covariance or spectral structure is of non Gaussian case (Krishnan, 2016).

2e)

Initialized the value of UK and k is 1.2....k. Set the counter of iterative values is t=1,

Repeat the following steps are,

Given value of X we can evaluate the corresponding posterior probabilities, called responsibilities (Chambers, 2012). That can follow the each step,

3a)

The represented Theorem we can used for the any of several related results stating that minimize of the regularized empirical risk function defined over a reproducing kernel Hibert spaces cab be presented as a finite linear combination of kernel products evaluated on the input points sets that are includes the,       

3b)

The hold for all values is   > 0. Take the limit as   → 0 and use the fact that F is donoted the unlabelled dataset of the continous values are  limn Fn(x) = F(x). 

The variables are donoted as,

{x (i) , y(i)} n i=1 represents a set of training data, where x ∈ R m are input attributes, while y ∈ {0, 1} is the output label.

w ∈ R m is the weight vector of the linear discriminate which we seek, and; λ > 0 is some constant.

3d)

Let us consider the 2 dimensional input attribute are x = [x1, x2]

T that can be mapped on values is

   K (xi,xj) = (1 + xiTxj)2

Need to show that K (xi,xj) = ? (xi) T?(xj):

K (xi,xj) = (1 + xiT xj)2

 = 1+ xi12xj12 + 2 xi1xj1 xi2xj2+ xi22xj22 + 2xi1xj1 + 2xi2xj2

= [1 xi12 √2 xi1xi2 xi22 √2xi1 √2xi2]T [1 xj12 √2 xj1xj2 xj22 √2xj1 √2xj2]

= ?(xi)T?(xj), where ?(x) = [1 x12 √2 x1x2 x22 √2x1 √2x]                   

3e )

let us consider the one dimensional attributes values of feature map is φ : x 7→ φ(x),

They can follow the given values are,

References

Chambers, R. (2012). Maximum Likelihood Estimation for Sample Surveys. Hoboken: CRC Press.

Cole, S., Chu, H. and Greenland, S. (2013). Maximum Likelihood, Profile Likelihood, and Penalized Likelihood: A Primer. American Journal of Epidemiology, 179(2), pp.252-260.

Kiviet, J. and Feng, Q. (2014). Efficiency gains by modifying GMM estimation in linear models under heteroskedasticity. Munich: Casinos.

Krishnan, T. (2016). EM Algorithm and Extensions, The. Wiley Series in Probability and Statistics. Wiley.

Lyon, P. and Kinney, D. (2012). Convenience and choice for consumers: the domestic acceptability of canned food between the 1870s and 1930s. International Journal of Consumer Studies, 37(2), pp.130-135.

MARKS, M. (2011). Minimum Wages, Employer-Provided Health Insurance, and the Non-discrimination Law. Industrial Relations: A Journal of Economy and Society, 50(2), pp.241-262.

Nikola, P. and Petrov, N. (2011). Dimensional Reduction of Invariant Fields and Differential Operators. I. Reduction of Invariant Fields. Annales Henri Poincaré, 13(3), pp.449-480.

PCA ADMINISTRATIVE COMMITTEE. (2015). NEITHER JEW NOR GREEK. [Place of publication not identified]: DOULOS RESOURCES.

Perner, P. (2015). Machine Learning and Data Mining in Pattern Recognition. Cham: Springer International Publishing.

Tuomi, M. and Jones, H. (2012). Probabilities of exoplanet signals from posterior samplings. Astronomy & Astrophysics, 544, p.A116.

Unsupervised monitoring of an elderly person's activities of daily living using Kinect sensors and a power meter. (2017). Edith Cowan University, Edith Cowan University, Research Online, Perth, Western Australia, Perth, Western Australia.

Uprichard, E. and Byrne, D. (2012). Cluster analysis. London: SAGE.