Optimizing Simulation and Machine Learning

Abstract: We consider bandwidth selection for kernel density estimation. The performance of kernel density estimator heavily relies on the quality of the bandwidth. In this study, we propose an efficient plug-in kernel density estimator which first perturbs the bandwidth to estimate the optimal bandwidth, followed by applying a kernel density estimator with the estimated optimal bandwidth. The proposed method utilizes the zeroth-order information of kernel function and has a faster convergence rate than other plug-in methods in existing literature. Simulation results demonstrate superior finite sample performance and robustness of the proposed method.

Abstract: Backfitting is an iterative method widely used for training general additive models, with applications in various domains including LASSO, Bayesian inference, and neural networks. However, the convergence behavior in terms of its iteration number remains unknown. This study establishes a novel connection between backfitting and the finite-difference method within the framework of Gaussian processes. Leveraging this connection, we prove that backfitting necessitates a minimum of O(n log n) iterations to converge, where n is the data size. In spired by algebraic multigrid in finite difference, we modify backfitting by adding a sparse regression step in each iteration. This simple modification can be computed in O(n) time and significantly reduces the required iteration to O(log n).

Abstract: Evaluation of portfolio loss often requires time-consuming simulation, making the measurement of portfolio risk computationally challenging. In this paper, we construct lower and upper bounds for a widely used risk measure, the so-called conditional Value-at-Risk (CVaR), by exploiting its structural properties. These bounds serve as useful complements to the plug-in estimates in the literature, by providing information on the magnitudes of their biases. By leveraging existing methods for approximating portfolio loss function, we propose efficient estimators for the proposed lower and upper bounds. Moreover, we establish two confidence intervals for the two bounds. Combine them leading to a confidence interval for CVaR. Numerical results suggest that this confidence interval works quite well.

Abstract: We propose methods based on batching, sectioning, and generalized likelihood ratios (GLRs) for computing confidence intervals (CIs) and confidence regions (CRs) for quantiles in finite-horizon and steady-state simulations. We establish the uniform consistency of the GLR estimators and the asymptotic validity of the respective CIs and CRs for quantiles. We also establish the asymptotic validity of CIs and CRs for quantiles by batching and sectioning methods for both finite-horizon and steady-state simulations. Numerical experiments demonstrate the validity of the aforementioned methods as the coverage rates of the CIs and CRs approach the target levels for appropriately large sample sizes.