Deep learning, Multitask Learning and Causal Graph Learning

Abstract: The Population-based HIV Impact Assessment (PHIA) is an ongoing project that conducts nationally representative HIV-focused surveys for measuring national and regional progress toward UNAIDS’90-90-90 targets, the primary strategy to end the HIV epidemic. We believe the PHIA survey offers a unique opportunity to better understand the key factors that drive the HIV epidemics in the most affected countries in sub-Saharan Africa. In this article, we propose a novel causal structural learning algorithm to discover important covariates and potential causal pathways for 90-90-90 targets. Existing constrained-based causal structural learning algorithms are quite aggressive in edge removal. The proposed algorithm preserves more information about important features and potential causal pathways. It is applied to the Malawi PHIA (MPHIA) data set and leads to interesting results. We further compare and validate the proposed algorithm using BIC and using Monte Carlo simulations, and show that the proposed algorithm achieves improvement in true positive rates in important feature discovery over existing algorithms.

Abstract: 深度学习在诸如图像处理、自然语言处理、运筹、博弈等领域取得了巨大的成功。但其成功的原因依然缺乏严格的理论解释与验证。在这种未知性下，学术界与业界掀起了深度学习浪潮， 试图用深度神经网络去处理所有学习问题。很显然，在某些应用上，效果不如预期。 该报告将从数学上（统计学习的角度）揭露深度神经网络的学习能力并在一定程度阐明深度学习的适用范围。特别地，该报告聚焦如下四个基本问题：1. 深度网是否一定比单层网好? 2. 在什么情况下用深度学习会更有效？ 3. 为什么深度网在大数据时代取得这么大成功？4. 过参数化深度神经网络为何可规避过拟合现象?

Abstract: We study the target model with the help of auxiliary models from different but possibly related groups. Inspired by transfer learning, we propose a method called joint estimation transferred from strata. To obtain a sparse solution, JETS constructs a penalized framework combining a term that penalizes the target model and an additional term that penalizes the differences between auxiliary and target models. In this way, JETS overcomes the challenge caused by the limited samples in high-dimensional settings and obtains stable and accurate estimates regardless of whether auxiliary samples contain noisy information. We demonstrate that this method enjoys the computational advantage of traditional methods. During simulations and applications, the proposed method is compared with several existing methods and JETS outperforms others.

Abstract: Multi-task learning has attracted much attention due to growing multi-purpose research with multiple related data sources. Moreover, transduction with matrix completion is a useful method in multi-label learning. In this paper, we propose a transductive matrix completion algorithm that incorporates a calibration constraint for the features under the multi-task learning framework. The proposed algorithm recovers the incomplete feature matrix and target matrix simultaneously. Fortunately, the calibration information improves the completion results. In particular, we provide a statistical guarantee for the proposed algorithm, and the theoretical improvement induced by calibration information is also studied. Moreover, the proposed algorithm enjoys a sub-linear convergence rate. Several synthetic data experiments are conducted, which show the proposed algorithm out-performs other methods, especially when the target matrix is associated with the feature matrix in a nonlinear way.