STAT3006 Assignment 4—High-Dimensional Inference Due Date: 15th November 2021 Weighting: 25% Instructions • The assignment consists of three (3) problems; Problems 1 and 2 are worth 10 Marks each, and Problem 3 is worth 5 Mark. Each Mark is equally weighted and is worth 1% of the overall course grade. • The mathematical elements of the assignment can be completed by hand, in LaTeX (prefer- ably), or in Word (or other typesetting software). The mathematical derivations and ma- nipulations should be accompanied by clear explanations in English regarding necessary information required to interpret the mathematical exposition. • Computation problems can be answered using your programming language of choice, al- though R is generally recommended, or Python if you are uncomfortable with R. As with the mathematical exposition, you may choose to typeset your answers to the problems in whatever authoring or word processing software that you wish. You should also maintain a copy of any codes that you have produced. • Computer generated plots and hand drawn graphs should be included together with the text where problems are answered. • The assignment will require four (4) files containing data, that you can can download from the Assignment 4 section on Blackboard. These files are: zip.txt, golub_genes.csv, golub_labels.csv, and prostate.csv. • Submission files should include the following (which ever applies to you): – Scans of handwritten mathematical exposition. – Typeset mathematical exposition, outputted as a pdf file. – Typeset answers to computational problems, outputted as a pdf file. 1 – Program code/scripts that you wish to submit, outputted as a txt file. • All submission files should be labeled with your name and student number and archived together in a zip file and submitted at the TurnItIn link on Blackboard. We suggest naming using the convention: FirstName_LastName_STAT3006A4_[Problem_XX/Problem_XX_Part_YY].[FileExtension]. • As per my.uq.edu.au/information-and-services/manage-my-program/student-in tegrityand-conduct/academic-integrity-and-student-conduct, what you submit should be your own work. Even where working from sources, you should endeavour to write in your own words. You should use consistent notation throughout your assignment and define whatever is required. Problem 1 [10 Marks] Consider the data set zip.txt, which contains n = 7291 rows of data, where each row is an observation Z>i = ( Yi,X > i ) ∈ R1+q, with q = 256. Here Yi ∈ {0, 1, . . . , 9} is a label indicating the digit that is represented by the vectorized 16× 16 matrix (image) Xi ∈ Rq. Part 1 [1 Mark] Select one of the digits y ∈ {0, 1, . . . , 9} and plot m = 9 unique images with Yi = y, in the same plot, as characterized by Xi . Part 2 [2 Marks] Using all n = 7291 observations of the sample X˜n = ( X˜1, . . . , X˜n ) , where X˜i = Xi−X¯n (i ∈ [n]) and X¯n = n−1 ∑n i=1Xi, obtain a solution to the optimization problem: Fˆ, Rˆ = arg min F∈Rs×q ,R∈Rq×s n∑ i=1 ∥∥∥X˜i −RFX˜i∥∥∥2 2 , for s = 4 and report the minimum value: n∑ i=1 ∥∥∥X˜i − RˆFˆX˜i∥∥∥2 2 . 2 Part 3 [2 Marks] Using the results from Part 2, plot the forward mappings Wˆ1, . . . , Wˆn, where Wˆi = FˆX˜i, for each i ∈ [n], colored by the labels Yi. Discuss whether there appears to be differences in the distributions of the forward mapped observations Wˆ1, . . . , Wˆn, corresponding to different values of the labels Yi. Part 4 [1 Mark] Via a spectral decomposition of the Grammian G˜ = n∑ i=1 X˜iX˜ > i report the proportion of total variance that is explained by s = 4 eigenvectors corresponding to the first s largest eigenvalues. Part 5 [2 Marks] Using all n = 7291 observations of the sample Xn = (X1, . . . ,Xn), obtain a solution θˆ =( σˆ2, µˆ, Rˆ ) to the optimization problem arg max θ logLn (θ) , where logLn (θ) = n∑ i=1 log φ ( Xi;µ,RR > + σ2Iq ) = −n 2 q log (2pi)− n 2 log ∣∣RR> + σ2Iq∣∣−−1 2 trace ([ RR> + σ2Iq ]−1 S (µ) ) , S (µ) = n−1 ∑n i=1 (Xi − µ) (Xi − µ)>, and θ = (σ2,µ,R), with σ2 > 0, µ ∈ Rq, and R ∈ Rq×s, for s = 4. Part 6 [1 Marks] Using the results from Part 5, estimate the posterior expectations of the latent variables Wi: E (Wi|Xi) = ( R>R+ σ2Is )−1 R> (Xi − µ) for each i ∈ [n], and plot the estimated posterior expectations, colored by the labels Yi. 3 Part 7 [1 Marks] Using all n = 7291 observations of the sample Xn = (X1, . . . ,Xn), use 3-layer autoencoder with some activation function a : R → R (of your choice) to obtain an s-dimensional nonlinear dimensionality reduction Wi = F (Xi) , for each i ∈ [n] and for s = 4, where F : Rq → Rs, x 7→ (a (f1x+ c1) , . . . , a (fsx+ cs)) , for some vectors f1, . . . , fs ∈ Rq and scalars c1, . . . , cs ∈ R. Then, plot W1, . . . ,Wn, colored by the labels Yi. Problem 2 [10 Marks] Consider the data set golub_genes.csv , which contains q = 3571 rows of data corresponding observations Xi ∈ Rq (i ∈ [n]), in each of the n = 72 columns. Here, each row corresponds to the expression levels of a gene j across the n cells, corresponding to the columns. The data set golub_labels.csv then contains the corresponding label corresponding to the cell type of each of the n columns of golub_genes.csv, where the cells are either labeled as “ALL” or “AML”, where ALL stands for Acute Lymphoblastic Leukemia, and AML stands for Acute Myeloid Leukemia. For each i ∈ [n], we will write Yi = 1 if cell i is ALL and Yi = 2 if cell i is AML. Part 1 [2 Mark] Let f1 (x) = f (x|Y = 1) and f2 (x) = f (x|Y = 2) be the probability density functions of the gene expression levels for ALL and AML cells, respectively. Using an maximum mean discrepancy statistic with kernel of the form: κg (x,y) = g (‖x− y‖22) , where g (t) = exp {−βt} for β = 2−28, and using the data X1, . . . ,Xn and Y1, . . . , Yn, (assumed to be independent and identically distributed), test the hypotheses H0 : f1 = f2 versus H0 : f1 6= f2, at the α = 0.1 significance level. That is, report the test statistic, critical value, and decision that is made. If your decisions is to not reject the null hypothesis H0, then comment on whether 4 or not you believe that the test was powerful enough to reject the null hypothesis, based on the sample size. Part 2 [2 Mark] Let f1j (x) = f (x1|Y = 1) and f2j (x) = f (x2|Y = 2) be the marginal probability density functions of the gene expression levels of the jth gene for ALL and AML cells, respectively. Let P (f) = ∫ R xf (x) dx be the mean of univariate probability density function f : R → R≥0. For each j ∈ [q], compute a p-value Pj for a test of the hypotheses H0 :P (f1j) =P (f2j) versus H0 :P (f1j) 6=P (f2j) , using the data X1, . . . ,Xn and Y1, . . . , Yn. Plot the p-values Pq = (P1, . . . , Pq) using a his- togram. Part 3 [2 Mark] Following from Part 2, plot the empirical cumulative distribution function (ECDF) for the sample of p-values: F (p;Pq) = 1 q q∑ j=1 Jp ≤ PjK , along with the cumulative distribution function of the uniform distribution on the domain [0, 1]. Comment on whether the distribution of p-values is sub-uniform or not, and whether or not this observation conforms with the conclusions made in Part 1. Part 4 [2 Mark] Using the Benjamini–Hochberg and Benjamini–Yekutieli methods, identify sets of genes j ∈ [q] that are significant at the false discovery rate controlled level of αFD = 0.05. Report how many of the genes are significant under each method, and report the largest p-value that was rejected under each method. Part 5 [1 Mark] Using exploratory techniques and the samples X1, . . . ,Xn and Y1, . . . , Yn, explain whether you believe that the outcomes from either of the methods applied in Part 4 are valid by way of a discussion of the required assumptions. 5 Part 6 [1 Mark] Prove that ∞∑ j=1 δ (min {j,m}) j (j + 1) ≤ 1, for δ (k) = (2m)−1 k (k + 1), and provide a false discovery rate step-up rejection proce- dure based on this observation. Problem 3 [5 Marks] Let (X1, Y1) , . . . , (Xn, Yn) be an independent and identically distributed sample of pairs of covari- ates and responses, where Xi ∈ Rq and Yi ∈ R, where q may be larger than n. Suppose that E [Yi|Xi = xi] = β>xi, (1) for some potentially sparse β ∈ Rq (here, we take sparse to mean that many of the coordinates βj = 0 for many j ∈ [q], where β> = (β1, . . . , βq). We wish to estimate β via the so-called elastic net-penalized least squares estimator: βˆ = arg min β∈Rq 1 2n n∑ i=1 ( Yi − β>Xi )2 + λ {‖β‖1 + ‖β‖22} , (2) for some λ ≥ 0. Part 1 [1 Mark] Argue that Problem (2) is equivalent to the problem: βˆ = arg min β∈B(γ) 1 2n n∑ i=1 ( Yi − β>Xi )2 , (3) where B (γ) = { β ∈ Rq : ‖β‖1 + ‖β‖22 ≤ γ } , for some γ > 0. Part 2 [1 Mark] Plot the set B (γ) = { β ∈ Rq : ‖β‖1 + ‖β‖22 ≤ γ } for some value of γ and discuss whether you believe that elastic net-penalized least squares estimator can be sparse. Part 3 [2 Mark] Devise an algorithm for solving either Problem 2 or Problem 3. 6 Part 4 [1 Mark] The rows of the file prostate.csv contains n = 98 pairs of covariates Xi ∈ Rq (q = 8; in the first 8 columns) and response Yi (in the 9th column), for i ∈ [n]. Assuming that these data admit the relationship (2), compute estimates of β for these data using the elastic net-penalized least squares estimator for some increasing sequence of γ (or decreasing sequence of λ ≥ 0). Plot the trajectory of the sequence of estimates as γ increases (or λ decreases). 7
欢迎咨询51作业君