In short: First, use unsupervised learning techniques to determine Kmeans and hierarchal clustering. Next, use random forest classification to predict cluster number and calculate accuracy. Last, write Kmeans clustering from scratch using Python [No packages can be used for this part]. Compare results.
First, use dataset1 (csv) with unsupervised learning techniques. Use Kmeans to determine the # of clusters. Tabulate the # of clusters from 1 – 40 and total within-cluster variance. Plot the scree plot. Using hierarchal clustering, calculate the pairwise distance. Create various dendrograms using complete and average linkage. Cut dendogram into groups of 5-7. Which is the most appropriate # of groups?
Next, do the same exercise above except this time we use random forest classification.
Last, using dataset2 (csv), write a function (without using Kmeans related packages) that calculates the withincluster variance, aggregates the data by the cluster number, and plots “total within cluster vs.number of cluster”. Add the cluster number to the last column of dataset, predict the cluster number using logisticregression and calculate the accuracy of your model.
Briefly compare results from each method.