arXiv subjects

Vadim Sokolov

Publications and source records attributed to Vadim Sokolov.

At least 55 records · Page 3Linked to original sources

arXiv2022

Bayesian Calibration for Activity Based Models

We consider the problem of calibration and uncertainty analysis for activity-based transportation simulators. Activity-Based Models (ABMs) rely on statistical modeling of individual travelers' behavior to predict higher-order travel patterns in metropolitan areas. Input parameters are typically estimated from traveler surveys using maximum likelihood. We develop an approach that uses a Gaussian Process emulator to calibrate those parameters using traffic flow data. Our approach extends traditional emulators to handle the high-dimensional and non-stationary nature of transportation simulators. We introduce a deep learning dimensionality reduction model that is jointly estimated with Gaussin Process model to approximate the simulator. We demonstrate the methodology using several simulated examples as well as by calibrating key parameters of the Bloomington, Illinois model.

stat.AP↗

arXiv2022

Deep Learning Gaussian Processes For Computer Models with Heteroskedastic and High-Dimensional Outputs

Deep Learning Gaussian Processes (DL-GP) are proposed as a methodology for analyzing (approximating) computer models that produce heteroskedastic and high-dimensional output. Computer simulation models have many areas of applications, including social-economic processes, agriculture, environmental, biology, engineering and physics problems. A deterministic transformation of inputs is performed by deep learning and predictions are calculated by traditional Gaussian Processes. We illustrate our methodology using a simulation of motorcycle accidents and simulations of an Ebola outbreak. Finally, we conclude with directions for future research.

stat.AP↗

arXiv2022

On the Probability of Magnus Carlsen reaching 2900

How likely is it that Magnus Carlsen will achieve an Elo rating of $2900$? This has been a goal of Magnus and is of great current interest to the chess community. Our paper uses probabilistic methods to address this question. The probabilistic properties of Elo's rating system have long been studied, and we provide an application of such methods. By applying a Brownian motion model of Stern as a simple tool we provide answers. Our research also has fundamental bearing on the choice of the $K$-factor used in Elo's system for GrandMaster (GM) chess play. Finally, we conclude with a discussion of policy issues involved with the choice of $K$-factor.

stat.AP↗

arXiv2022

Housing Market Forecasting using Home Showing Events

Both buyers and sellers face uncertainty in real estate transactions in about when to time a transaction and at what cost. Both buyers and sellers make decisions without knowing the present and future state of the large and dynamic real estate market. Current approaches rely on analysis of historic transactions to price a property. However, as we show in this paper, the transaction data alone cannot be used to forecast demand. We develop a housing demand index based on microscopic home showings events data that can provide decision-making support for buyers and sellers on a very granular time and spatial scale. We use statistical modeling to develop a housing market demand forecast up to twenty weeks using high-volume, high-velocity data on home showings, listing events, and historic sales data. We demonstrate our analysis using data from seven million individual records sourced from a unique, proprietary dataset that has not previously been explored in application to the real estate market. We then employ a series of predictive models to estimate current and forecast future housing demand. A housing demand index provides insight into the level of demand for a home on the market and to what extent current demand represents future expectation. As a result, these indices provide decision-making support into important questions about when to sell or buy, or the elasticity present in the housing demand market, which impact price negotiations, price-taking and price-setting expectations. This forecast is especially valuable because it helps buyers and sellers to know on a granular and timely basis if they should engage in a home transaction or adjust their home price both in current and future states based on our forecasted housing demand index.

stat.AP↗

arXiv2021

Bayesian Learning: A Selective Overview

This paper presents an overview of some of the concepts of Bayesian Learning. The number of scientific and industrial applications of Bayesian learning has been growing in size rapidly over the last few decades. This process has started with the wide use of Markov Chain Monte Carlo methods that emerged as a dominant computational technique for Bayesian in the early 1990's. Since then Bayesian learning has spread well across several fields from robotics and machine learning to medical applications. This paper provides an overview of some of the widely used concepts and shows several applications. This is a paper based on the series of seminars given by students of a PhD course on Bayesian Learning at George Mason University. The course was taught in the Fall of 2021. Thus, the topics covered in the paper reflect the topics students selected to study.

stat.ME↗

arXiv2021

Deep Generative Models for Vehicle Speed Trajectories

Generating realistic vehicle speed trajectories is a crucial component in evaluating vehicle fuel economy and in predictive control of self-driving cars. Traditional generative models rely on Markov chain methods and can produce accurate synthetic trajectories but are subject to the curse of dimensionality. They do not allow to include conditional input variables into the generation process. In this paper, we show how extensions to deep generative models allow accurate and scalable generation. Proposed architectures involve recurrent and feed-forward layers and are trained using adversarial techniques. Our models are shown to perform well on generating vehicle trajectories using a model trained on GPS data from Chicago metropolitan area.

cs.LG↗

arXiv2021

Merging Two Cultures: Deep and Statistical Learning

Merging the two cultures of deep and statistical learning provides insights into structured high-dimensional data. Traditional statistical modeling is still a dominant strategy for structured tabular data. Deep learning can be viewed through the lens of generalized linear models (GLMs) with composite link functions. Sufficient dimensionality reduction (SDR) and sparsity performs nonlinear feature engineering. We show that prediction, interpolation and uncertainty quantification can be achieved using probabilistic methods at the output layer of the model. Thus a general framework for machine learning arises that first generates nonlinear features (a.k.a factors) via sparse regularization and stochastic gradient optimisation and second uses a stochastic output layer for predictive uncertainty. Rather than using shallow additive architectures as in many statistical models, deep learning uses layers of semi affine input transformations to provide a predictive rule. Applying these layers of transformations leads to a set of attributes (a.k.a features) to which predictive statistical methods can be applied. Thus we achieve the best of both worlds: scalability and fast predictive rule construction together with uncertainty quantification. Sparse regularisation with un-supervised or supervised learning finds the features. We clarify the duality between shallow and wide models such as PCA, PPR, RRR and deep but skinny architectures such as autoencoders, MLPs, CNN, and LSTM. The connection with data transformations is of practical importance for finding good network architectures. By incorporating probabilistic components at the output level we allow for predictive uncertainty. For interpolation we use deep Gaussian process and ReLU trees for classification. We provide applications to regression, classification and interpolation. Finally, we conclude with directions for future research.

stat.ME↗

arXiv2021

Deep Learning Partial Least Squares

High dimensional data reduction techniques are provided by using partial least squares within deep learning. Our framework provides a nonlinear extension of PLS together with a disciplined approach to feature selection and architecture design in deep learning. This leads to a statistical interpretation of deep learning that is tailor made for predictive problems. We can use the tools of PLS, such as scree-plot, bi-plot to provide model diagnostics. Posterior predictive uncertainty is available using MCMC methods at the last layer. Thus we achieve the best of both worlds: scalability and fast predictive rule construction together with uncertainty quantification. Our key construct is to employ deep learning within PLS by predicting the output scores as a deep learner of the input scores. As with PLS our X-scores are constructed using SVD and applied to both regression and classification problems and are fast and scalable. Following Frank and Friedman 1993, we provide a Bayesian shrinkage interpretation of our nonlinear predictor. We introduce a variety of new partial least squares models: PLS-ReLU, PLS-Autoencoder, PLS-Trees and PLS-GP. To illustrate our methodology, we use simulated examples and the analysis of preferences of orange juice and predicting wine quality as a function of input characteristics. We also illustrate Brillinger's estimation procedure to provide the feature selection and data dimension reduction. Finally, we conclude with directions for future research.

stat.ME↗

arXiv2020

Analyzing Stochastic Computer Models: A Review with Opportunities

In modern science, computer models are often used to understand complex phenomena, and a thriving statistical community has grown around analyzing them. This review aims to bring a spotlight to the growing prevalence of stochastic computer models -- providing a catalogue of statistical methods for practitioners, an introductory view for statisticians (whether familiar with deterministic computer models or not), and an emphasis on open questions of relevance to practitioners and statisticians. Gaussian process surrogate models take center stage in this review, and these, along with several extensions needed for stochastic settings, are explained. The basic issues of designing a stochastic computer experiment and calibrating a stochastic computer model are prominent in the discussion. Instructive examples, with data and code, are used to describe the implementation of, and results from, various methods.

stat.ME↗

arXiv2019

Strategic Bayesian Asset Allocation

Strategic asset allocation requires an investor to select stocks from a given basket of assets. The perspective of our investor is to maximize risk-adjusted alpha returns relative to a benchmark index. Historical returns are used to provide inputs into an optimization algorithm. Our approach uses Bayesian regularization to not only provide stock selection but also optimal sequential portfolio weights. By incorporating investor preferences with a number of different regularization penalties we extend the approaches of Black (1992) and Puelz (2015). We tailor standard sparse MCMC algorithms to calculate portfolio weights and perform selection. We illustrate our methodology on stock selection from the SP100 stock index and from the top fifty holdings of two hedge funds Renaissance Technologies and Viking Global. Finally, we conclude with directions for future research.

stat.AP↗

arXiv2019

Deep Learning: Computational Aspects

In this article we review computational aspects of Deep Learning (DL). Deep learning uses network architectures consisting of hierarchical layers of latent variables to construct predictors for high-dimensional input-output models. Training a deep learning architecture is computationally intensive, and efficient linear algebra libraries is the key for training and inference. Stochastic gradient descent (SGD) optimization and batch sampling are used to learn from massive data sets.

cs.LG↗

arXiv2019

Solving Large-Scale 0-1 Knapsack Problems and its Application to Point Cloud Resampling

0-1 knapsack is of fundamental importance in computer science, business, operations research, etc. In this paper, we present a deep learning technique-based method to solve large-scale 0-1 knapsack problems where the number of products (items) is large and/or the values of products are not necessarily predetermined but decided by an external value assignment function during the optimization process. Our solution is greatly inspired by the method of Lagrange multiplier and some recent adoptions of game theory to deep learning. After formally defining our proposed method based on them, we develop an adaptive gradient ascent method to stabilize its optimization process. In our experiments, the presented method solves all the large-scale benchmark KP instances in a minute whereas existing methods show fluctuating runtime. We also show that our method can be used for other applications, including but not limited to the point cloud resampling.

cs.LG↗

arXiv2019

Deep Learning for Energy Markets

Deep Learning is applied to energy markets to predict extreme loads observed in energy grids. Forecasting energy loads and prices is challenging due to sharp peaks and troughs that arise due to supply and demand fluctuations from intraday system constraints. We propose deep spatio-temporal models and extreme value theory (EVT) to capture theses effects and in particular the tail behavior of load spikes. Deep LSTM architectures with ReLU and $\tanh$ activation functions can model trends and temporal dependencies while EVT captures highly volatile load spikes above a pre-specified threshold. To illustrate our methodology, we use hourly price and demand data from 4719 nodes of the PJM interconnection, and we construct a deep predictor. We show that DL-EVT outperforms traditional Fourier time series methods, both in-and out-of-sample, by capturing the observed nonlinearities in prices. Finally, we conclude with directions for future research.

stat.ML↗

arXiv2019

Bayesian Regularization: From Tikhonov to Horseshoe

Bayesian regularization is a central tool in modern-day statistical and machine learning methods. Many applications involve high-dimensional sparse signal recovery problems. The goal of our paper is to provide a review of the literature on penalty-based regularization approaches, from Tikhonov (Ridge, Lasso) to horseshoe regularization.

stat.ME↗

arXiv2019

Practical Bayesian Optimization for Transportation Simulators

We provide a method to solve optimization problem when objective function is a complex stochastic simulator of an urban transportation system. To reach this goal, a Bayesian optimization framework is introduced. We show how the choice of prior and inference algorithm effect the outcome of our optimization procedure. We develop dimensionality reduction techniques that allow for our optimization techniques to be applicable for real-life problems. We develop a distributed, Gaussian Process Bayesian regression and active learning models that allow parallel execution of our algorithms and enable usage of high performance computing. We present a fully Bayesian approach that is more sample efficient and reduces computational budget. Our framework is supported by theoretical analysis and an empirical study. We demonstrate our framework on the problem of calibrating a multi-modal transportation network of city of Bloomington, Illinois. Finally, we discuss directions for further research.

stat.CO↗

arXiv2018

Effects of Predictive Real-Time Traffic Signal Information

This paper analyzes the impact of providing car drivers with predictive information on traffic signal timing in real-time, including time-to-green and green-wave speed recommendations. Over a period of six months, the behavior of these 121 drivers in everyday urban driving was analyzed with and without access to live traffic signal information. In a first period, drivers had the information providing service disabled in order to establish a baseline behavior; after that initial phase, the service was activated. In both cases, data from smartphone and vehicle sensors was collected, including speed, acceleration, fuel rate, acceleration and brake pedal positions. We estimated the changes in the driving behavior which result from drivers' receiving the traffic signal timing information by carefully comparing distributions of acceleration/deceleration patterns through statistical analysis. Our analysis demonstrates that there is a positive effect of providing traffic signal information timing to the drivers.

stat.AP↗

arXiv2018

Deep Reinforcement Learning for Dynamic Urban Transportation Problems

We explore the use of deep learning and deep reinforcement learning for optimization problems in transportation. Many transportation system analysis tasks are formulated as an optimization problem - such as optimal control problems in intelligent transportation systems and long term urban planning. Often transportation models used to represent dynamics of a transportation system involve large data sets with complex input-output interactions and are difficult to use in the context of optimization. Use of deep learning metamodels can produce a lower dimensional representation of those relations and allow to implement optimization and reinforcement learning algorithms in an efficient manner. In particular, we develop deep learning models for calibrating transportation simulators and for reinforcement learning to solve the problem of optimal scheduling of travelers on the network.

stat.ML↗

arXiv2018

Clusters of Driving Behavior from Observational Smartphone Data

Understanding driving behaviors is essential for improving safety and mobility of our transportation systems. Data is usually collected via simulator-based studies or naturalistic driving studies. Those techniques allow for understanding relations between demographics, road conditions and safety. On the other hand, they are very costly and time consuming. Thanks to the ubiquity of smartphones, we have an opportunity to substantially complement more traditional data collection techniques with data extracted from phone sensors, such as GPS, accelerometer gyroscope and camera. We developed statistical models that provided insight into driver behavior in the San Francisco metro area based on tens of thousands of driver logs. We used novel data sources to support our work. We used cell phone sensor data drawn from five hundred drivers in San Francisco to understand the speed of traffic across the city as well as the maneuvers of drivers in different areas. Specifically, we clustered drivers based on their driving behavior. We looked at driver norms by street and flagged driving behaviors that deviated from the norm.

cs.AI↗