Page | 1 COMP809: Data Mining & Machine Learning Assignment 1 (weight: 40%) Semester 2 2022 Data Mining Data Exploration and Analysis This is an individual assignment. Submission: A soft copy needs to be submitted through Canvas. Include your actual clean code (no screenshot) in an Appendix with appropriate comments for each task. Due date: Sunday 28 August 2022 at midnight NZ time. Late penalty: maximum late submission time is 24 hours after the due date. In this case, a 5% late penalty will be applied. Page | 2 AIMS This assignment provides an opportunity to solve two real-world data mining problems using the machine learning workbench. In the two questions given below justification of your answers carries a high proportion of the marks awarded. You are required to conduct experiments for both case studies and report them according to the specified requirements. Your answers below need to be supported by suitable evidence, wherever appropriate. Some examples of suitable evidence are the Confusion Matrices, Model Visualizations and Summary Statistics. Study Area I ( bank.csv use the Bank.zip) This application is concerned with predicting the outcome of direct bank marketing campaigns (phone calls) of Portuguese banking. The dataset contains 17 attributes for which outcomes of subscribing to a term deposit (yes/no) on a term deposit are known. You are required to build a model using the Decision Tree Classifier and answer the following questions based on the model built. Use the data segment on the subscriptions whose outcomes are known. In building the model, use the 10-fold cross-validation option for testing. a) Describe the pre-processing steps you have performed to prepare your data and perform initial data exploration by analysing the summary statistics of the dataset attributes. [5 marks] b) Using an appropriate method to identify the most influential features in classifying this dataset. Explain the process of the chosen feature selection method and use the top four features for building your model. Use a ‘breakdown’ analysis for selected features by the class and describe their distribution using appropriate plot(s). [10 marks] c) Now build a model using the Decision Tree algorithm. By adjusting two suitable parameters (one at a time) reduce the size of the tree to not more than 15 nodes to improve the interpretability of the model generated. Analyse your findings and discuss the results. Visualise the final generated decision tree and describe it. [10 marks] d) Describe the role of the two parameters in the model building that you used in c) above. Do you expect that manipulating the parameter, in the same way, will improve accuracy for other types of datasets? Justify your answer. [5 marks] e) Provide and carefully examine the confusion matrix. Generate and provide a classification report, showing precision, recall, F1 and overall accuracy, to evaluate your model performance. Describe your findings, is there any significant finding? [5 marks] Page | 3 Study Area II (Autism-Child-Data) This application is from the medical domain and is concerned with the diagnosis of childhood Autistic Spectrum Disorder Screening (ASDS) for a collection of individuals from whom relevant medical data has been obtained. The dataset contains 10 behavioural features (AQ-10-Child), 10 individual characteristics, and the outcome (effectiveness of detection). The objective is to predict whether the given individual characteristics are effective in detecting ASD cases. The effectiveness of ASDS detection is labelled as ‘Yes’ or ‘No’ in this dataset. For this dataset, you are required to use both the Naïve Bayes (NB) and Decision Tree classifier algorithms to build a predictive model for the ASDS. a) Describe what is an autism spectrum disorder (ASD) and discuss the significance of the early diagnosis of ASD. Briefly describe the Autism-Spectrum Quotient (AQ) and include two recent references to support your answer (no more than one page). [5 Marks] b) Describe the pre-processing steps and perform initial data exploration. Use an appropriate method of feature selection to identify the top five significant features. State the method used and list the features produced and explain why this feature selection method was used. Use a ‘breakdown’ analysis for selected features by the class and describe their distribution using appropriate plot(s). [15 marks] c) Discuss the independence assumption between the features in Naïve Bayes (NB) algorithm and support your answer concerning the selected features. [5 marks] d) Run the Naïve Bayes algorithm with the GaussianNB implementation for the selected features. Provide the evaluation metrics including the confusion matrix showing the performance of the NB model. Discuss the results. [10 marks] e) Run the Decision Tree Classifier algorithm and compare the top five features produced by the Decision Tree model with the list selected in part (b). Identify similarities and differences. Discuss any differences. [10 marks] f) Provide the evaluation metrics including the confusion matrix showing the performance of the Decision Tree model. Compare the performance of your models (NB and Decision Tree) and discuss your findings. [5 marks] There will be 5 marks for the presentation of the assignment including spelling and grammar, layout, formatting, and readability of the figures. Good luck!
欢迎咨询51作业君