arXiv subjects

Yue Song

Publications and source records attributed to Yue Song.

At least 73 records · Page 4Linked to original sources

arXiv2023

Covariate Adjustment in Randomized Clinical Trials with Missing Covariate and Outcome Data

When analyzing data from randomized clinical trials, covariate adjustment can be used to account for chance imbalance in baseline covariates and to increase precision of the treatment effect estimate. A practical barrier to covariate adjustment is the presence of missing data. In this paper, in the light of recent theoretical advancement, we first review several covariate adjustment methods with incomplete covariate data. We investigate the implications of the missing data mechanism on estimating the average treatment effect in randomized clinical trials with continuous or binary outcomes. In parallel, we consider settings where the outcome data are fully observed or are missing at random; in the latter setting, we propose a full weighting approach that combines inverse probability weighting for adjusting missing outcomes and overlap weighting for covariate adjustment. We highlight the importance of including the interaction terms between the missingness indicators and covariates as predictors in the models. We conduct comprehensive simulation studies to examine the finite-sample performance of the proposed methods and compare with a range of common alternatives. We find that conducting the proposed adjustment methods generally improves the precision of treatment effect estimates regardless of the imputation methods when the adjusted covariate is associated with the outcome. We apply the methods to the Childhood Adenotonsillectomy Trial to assess the effect of adenotonsillectomy on neurocognitive functioning scores.

stat.ME↗

arXiv2023

Optimal Transmission Switching with Uncertainties from both Renewable Energy and N-k Contingencies

This paper focuses on the N-k security-constrained optimal transmission switching (OTS) problem for variable renewable energy (VRE) penetrated power grids. A new three-stage stochastic and distributionally robust OTS model is proposed. The first stage has the primary purpose to schedule the power generation and network topology based on the forecast of VRE. The second stage controls the power generation and voltage magnitudes of voltage-controlled buses in response to VRE uncertainty, and the third stage reacts to N-k contingencies additionally by line switching and load shedding. The VRE and N-k contingencies, considering different availability of their probability distributions, are tackled by stochastic and distributionally robust optimization, respectively. By adopting stage-wise realization of uncertainties in VRE and contingencies, the associated corrective controls with different mechanisms can be handled separately and properly, which makes the proposed OTS model more realistic than existing two-stage ones. For solving the proposed OTS model, its tractable reformulation is derived, and a solution approach that combines the nested column-and-constraint generation algorithm and Dantzig-Wolfe procedure is developed. Finally, case studies include a simple IEEE network for illustrative purposes and then real system networks to demonstrate the efficacy of the proposed approach.

math.OC↗

arXiv2023

Thermodynamic Instabilities of Conformal Gravity Holography in Four Dimensions

Recently, a conjecture has been proposed, which indicates a correlation between super-entropy black holes and the thermodynamic instability \cite{Cong:2019bud}. W.Cong et al. suggested that the $C_{V}$ (specific heat capacity at constant volume) and $C_{P}$ (specific heat capacity at constant pressure) of a super-entropy black hole could not be greater than zero simultaneously as set in the extended phase space, which implies that the black hole is unstable in extended thermodynamics. This conjecture is intriguing and meaningful. Therefore, we did a study on that. After deriving the equations of specific heat capacities as well as plotting the relevant curves of the four-dimensional conformal black holes (a kind of super-entropy black hole), we obtained regions on the graphs where $C_{V}$ and $C_{P}$ are simultaneously greater than zero, which contradicts the conjecture that super-entropy black hole corresponds to its thermodynamic instability. Thus far, we have provided a counterexample to the hypothesis in \cite{Cong:2019bud}.

gr-qc↗

arXiv2023

Stability Constrained OPF in Microgrids: A Chance Constrained Optimization Framework with Non-Gaussian Uncertainty

To figure out the stability issues brought by renewable energy sources (RES) with non-Gaussian uncertainties in isolated microgrids, this paper proposes a chance constrained stability constrained optimal power flow (CC-SC-OPF) model. Firstly, we propose a bi-level optimization problem, of which the upper level aims to minimize the expected generation cost without violating the stability chance constraint; the lower level concerns about the stability index given by a semi-definite program (SDP). Secondly, we apply the Gaussian mixture model (GMM) to handle the non-Gaussian RES uncertainties and introduce analytical sensitivity analysis to reformulate chance constraints with respect to stability index and operational variables into linear deter-ministic versions. By incorporating linearized constraints, the bi-level model can be efficiently solved by Benders decomposition-based approach. Thirdly, we design a supplementary corrective countermeasure to compensate the possible control error caused by the linear approximation. Simulation results on the 33-bus microgrid reveal that compared to benchmarking approaches, the proposed model converges 30 times faster with more accurate solutions.

eess.SY↗

arXiv2023

An Analytical Formula for Stability Sensitivity Using SDP Dual

In this letter, we analytically investigate the sensitivity of stability index to its dependent variables in general power systems. Firstly, we give a small-signal model, the stability index is defined as the solution to a semidefinite program (SDP) based on the related Lyapunov equation. In case of stability, the stability index also characterizes the convergence rate of the system after disturbances. Then, by leveraging the duality of SDP, we deduce an analytical formula of the stability sensitivity to any entries of the system Jacobian matrix in terms of the SDP primal and dual variables. Unlike the traditional numerical perturbation method, the proposed sensitivity evaluation method is more accurate with a much lower computational burden. This letter applies a modified microgrid for comparative case studies. The results reveal the significant improvements on the accuracy and computational efficiency of stability sensitivity evaluation.

math.OC↗

arXiv2022

Impasse Surface of Differential-Algebraic Power System Models: An Interpretation Based on Admittance Matrices

The impasse surface is an important concept in the differential-algebraic equation (DAE) model of power systems, which is associated with short-term voltage collapse. This paper establishes a necessary condition for a system trajectory hitting the impasse surface. The condition is in terms of admittance matrices regarding the power network, generators and loads, which specifies the pattern of interaction between those system components that can induce voltage collapse. It applies to generic DAE models featuring high-order synchronous generators, static loads, induction motor loads and lossy power networks. We also identify a class of static load parameters that prevent power systems from hitting the impasse surface; this proves a conjecture made by Hiskens that has been unsolved for decades. Moreover, the obtained results lead to an early indicator of voltage collapse and a novel viewpoint that inductive compensation to the power network has a positive effect on preventing short-term voltage collapse, which are verified via numerical simulations.

math.OC↗

arXiv2022

Orthogonal SVD Covariance Conditioning and Latent Disentanglement

Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned, which could harm the model in the training stability and generalization abilities. In this paper, we systematically study how to improve the covariance conditioning by enforcing orthogonality to the Pre-SVD layer. Existing orthogonal treatments on the weights are first investigated. However, these techniques can improve the conditioning but would hurt the performance. To avoid such a side effect, we propose the Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR). The effectiveness of our methods is validated in two applications: decorrelated Batch Normalization (BN) and Global Covariance Pooling (GCP). Extensive experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization. The combinations with orthogonal weight can further boost the performance. Moreover, we show that our orthogonality techniques can benefit generative models for better latent disentanglement through a series of experiments on various benchmarks. Code is available at: \href{https://github.com/KingJamesSong/OrthoImproveCond}{https://github.com/KingJamesSong/OrthoImproveCond}.

cs.CV↗

arXiv2022

Chaos bound of charged particles around phantom AdS black hole

In this paper, we investigated the chaos of the phantom AdS black hole in near-horizon regions and at a certain distance from the black hole, where the Lyapunov exponent was calculated by the Jacobian determinant. We found that the angular momenta of charged particles nearby the black hole affect the position of the equilibrium orbit as well as the Lyapunov exponent. When the charge is large enough and the angular momenta take particular values, the bound is violated at a certain distance from the event horizon. Concurrently, we compared the differences by analyzing tables and figures after taking different values of $η$, where $η$ is a constant indicating the nature of the electromagnetic field. When $η=1$, it corresponds to the Reissner-Nordstr$\ddot{o}$m-AdS black hole (RN-AdS black hole) and it represents the phantom AdS black hole when $η=-1$. In this way, we can draw the following conclusions. Under different values of $η$, the variation trends of curves related to $λ^{2}-κ^{2}$ could be different, where it can be judged whether the constraint is violated by determining if $λ^{2}-κ^{2}$ is greater than zero. In addition, the corresponding numerical values of angular momenta violating the bound in the phantom AdS black hole are much smaller than the case of RN-AdS black hole. But cases in these different black holes have the same rules to violate the bound. The large absolute values of charge and $Λ$ are more likely to violate the bound.

gr-qc↗

arXiv2022

Fast Differentiable Matrix Square Root and Inverse Square Root

Computing the matrix square root and its inverse in a differentiable manner is important in a variety of computer vision tasks. Previous methods either adopt the Singular Value Decomposition (SVD) to explicitly factorize the matrix or use the Newton-Schulz iteration (NS iteration) to derive the approximate solution. However, both methods are not computationally efficient enough in either the forward pass or the backward pass. In this paper, we propose two more efficient variants to compute the differentiable matrix square root and the inverse square root. For the forward propagation, one method is to use Matrix Taylor Polynomial (MTP), and the other method is to use Matrix Padé Approximants (MPA). The backward gradient is computed by iteratively solving the continuous-time Lyapunov equation using the matrix sign function. A series of numerical tests show that both methods yield considerable speed-up compared with the SVD or the NS iteration. Moreover, we validate the effectiveness of our methods in several real-world applications, including de-correlated batch normalization, second-order vision transformer, global covariance pooling for large-scale and fine-grained recognition, attentive covariance pooling for video recognition, and neural style transfer. The experimental results demonstrate that our methods can also achieve competitive and even slightly better performances. The Pytorch implementation is available at https://github.com/KingJamesSong/FastDifferentiableMatSqrt

cs.CV↗

arXiv2022

GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Model

High-concurrency asynchronous training upon parameter server (PS) architecture and high-performance synchronous training upon all-reduce (AR) architecture are the most commonly deployed distributed training modes for recommendation models. Although synchronous AR training is designed to have higher training efficiency, asynchronous PS training would be a better choice for training speed when there are stragglers (slow workers) in the shared cluster, especially under limited computing resources. An ideal way to take full advantage of these two training modes is to switch between them upon the cluster status. However, switching training modes often requires tuning hyper-parameters, which is extremely time- and resource-consuming. We find two obstacles to a tuning-free approach: the different distribution of the gradient values and the stale gradients from the stragglers. This paper proposes Global Batch gradients Aggregation (GBA) over PS, which aggregates and applies gradients with the same global batch size as the synchronous training. A token-control process is implemented to assemble the gradients and decay the gradients with severe staleness. We provide the convergence analysis to reveal that GBA has comparable convergence properties with the synchronous training, and demonstrate the robustness of GBA the recommendation models against the gradient staleness. Experiments on three industrial-scale recommendation tasks show that GBA is an effective tuning-free approach for switching. Compared to the state-of-the-art derived asynchronous training, GBA achieves up to 0.2% improvement on the AUC metric, which is significant for the recommendation models. Meanwhile, under the strained hardware resource, GBA speeds up at least 2.4x compared to synchronous training.

cs.LG↗

arXiv2022

Bumpless Topology Transition

The topology transition problem of transmission networks is becoming increasingly crucial with topological flexibility more widely leveraged to promote high renewable penetration. This paper proposes a novel methodology to address this problem. Aiming at achieving a bumpless topology transition regarding both static and dynamic performance, this methodology utilizes various eligible control resources in transmission networks to cooperate with the optimization of line-switching sequence. Mathematically, a composite formulation is developed to efficiently yield bumpless transition schemes with AC feasibility and stability both ensured. With linearization of all non-convexities involved and tractable bumpiness metrics, a convex mixed-integer program firstly optimizes the line-switching sequence and partial control resources. Then, two nonlinear programs recover AC feasibility, and optimize the remaining control resources by minimizing the $\mathcal{H}_2$-norm of associated linearized systems, respectively. The final transition scheme is selected by accurate evaluation including stability verification using time-domain simulations. Finally, numerical studies demonstrate the effectiveness and superiority of the proposed methodology to achieve bumpless topology transition.

eess.SY↗

arXiv2022

RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection

The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings. In this paper, we observe that the singular value distributions of the in-distribution (ID) and OOD features are quite different: the OOD feature matrix tends to have a larger dominant singular value than the ID feature, and the class predictions of OOD samples are largely determined by it. This observation motivates us to propose \texttt{RankFeat}, a simple yet effective \texttt{post hoc} approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature (\emph{i.e.,} $\mathbf{X}{-} \mathbf{s}_{1}\mathbf{u}_{1}\mathbf{v}_{1}^{T}$). \texttt{RankFeat} achieves the \emph{state-of-the-art} performance and reduces the average false positive rate (FPR95) by 17.90\% compared with the previous best method. Extensive ablation studies and comprehensive theoretical analyses are presented to support the empirical results.

cs.LG↗

arXiv2022

Optimal Topology Transition

Network topology has significant impacts on operational performance of power systems. While extensive research efforts have been devoted to optimization of network topology for improving various system performances, the problem of how to transition from the initial topology to the desired optimal topology requires study. To address this problem, we propose the concept of optimal topology transition (OTT). This aims to find the topology transition trajectory from an initial topology to a desired terminal topology, which optimizes certain transition performance and satisfies operational constraints. The OTT problem is further formulated as a mixed-integer program under certain assumptions. Next, we propose the formulation of transition-embedded topology optimization that is capable of optimizing network topology and its transition trajectory simultaneously. Considering the time complexity of directly solving the mixed-integer programs, an efficient problem-specific solution algorithm is developed. Finally, numerical studies demonstrate the effectiveness of the proposed OTT and transition-embedded topology optimization models, as well as the superiority of the obtained optimal transition trajectories compared to ad hoc transition trajectories.

eess.SY↗

arXiv2022

Batch-efficient EigenDecomposition for Small and Medium Matrices

EigenDecomposition (ED) is at the heart of many computer vision algorithms and applications. One crucial bottleneck limiting its usage is the expensive computation cost, particularly for a mini-batch of matrices in the deep neural networks. In this paper, we propose a QR-based ED method dedicated to the application scenarios of computer vision. Our proposed method performs the ED entirely by batched matrix/vector multiplication, which processes all the matrices simultaneously and thus fully utilizes the power of GPUs. Our technique is based on the explicit QR iterations by Givens rotation with double Wilkinson shifts. With several acceleration techniques, the time complexity of QR iterations is reduced from $O{(}n^5{)}$ to $O{(}n^3{)}$. The numerical test shows that for small and medium batched matrices (\emph{e.g.,} $dim{<}32$) our method can be much faster than the Pytorch SVD function. Experimental results on visual recognition and image generation demonstrate that our methods also achieve competitive performances.

cs.CV↗

arXiv2022

Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality

Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned, which could harm the model in the training stability and generalization abilities. In this paper, we systematically study how to improve the covariance conditioning by enforcing orthogonality to the Pre-SVD layer. Existing orthogonal treatments on the weights are first investigated. However, these techniques can improve the conditioning but would hurt the performance. To avoid such a side effect, we propose the Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR). The effectiveness of our methods is validated in two applications: decorrelated Batch Normalization (BN) and Global Covariance Pooling (GCP). Extensive experiments on visual recognition demonstrate that our methods can simultaneously improve the covariance conditioning and generalization. Moreover, the combinations with orthogonal weight can further boost the performances.

cs.CV↗

arXiv2022

On the Eigenvalues of Global Covariance Pooling for Fine-grained Visual Recognition

The Fine-Grained Visual Categorization (FGVC) is challenging because the subtle inter-class variations are difficult to be captured. One notable research line uses the Global Covariance Pooling (GCP) layer to learn powerful representations with second-order statistics, which can effectively model inter-class differences. In our previous conference paper, we show that truncating small eigenvalues of the GCP covariance can attain smoother gradient and improve the performance on large-scale benchmarks. However, on fine-grained datasets, truncating the small eigenvalues would make the model fail to converge. This observation contradicts the common assumption that the small eigenvalues merely correspond to the noisy and unimportant information. Consequently, ignoring them should have little influence on the performance. To diagnose this peculiar behavior, we propose two attribution methods whose visualizations demonstrate that the seemingly unimportant small eigenvalues are crucial as they are in charge of extracting the discriminative class-specific features. Inspired by this observation, we propose a network branch dedicated to magnifying the importance of small eigenvalues. Without introducing any additional parameters, this branch simply amplifies the small eigenvalues and achieves state-of-the-art performances of GCP methods on three fine-grained benchmarks. Furthermore, the performance is also competitive against other FGVC approaches on larger datasets. Code is available at \href{https://github.com/KingJamesSong/DifferentiableSVD}{https://github.com/KingJamesSong/DifferentiableSVD}.

cs.CV↗

arXiv2022

Joule-Thomson expansion of d-dimensional charged AdS black holes with cloud of strings and quintessence

Herein, we focus on the study of Joule-Thomson expansion corresponding to a d-dimensional charged AdS black hole with cloud of strings and quintessence. Then its relevant solution and some thermodynamic properties are investigated. Specifically, we evaluate its Joule-Thomson expansion from four important aspects, including the Joule-Thomson coefficient, inversion curve, isenthalpic curve, and ratio $\frac{T_{i}^{min}}{T_{c}}$. After analysis, different dimensions with strings of cloud and quintessence parameters have different effects on the Joule-Thomson coefficient (the same situation are found for the inversion curve, isenthalpic curve, and ratio $\frac{T_{i}^{min}}{T_{c}}$).

gr-qc↗

arXiv2022

PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems

The development of personalized recommendation has significantly improved the accuracy of information matching and the revenue of e-commerce platforms. Recently, it has 2 trends: 1) recommender systems must be trained timely to cope with ever-growing new products and ever-changing user interests from online marketing and social network; 2) SOTA recommendation models introduce DNN modules to improve prediction accuracy. Traditional CPU-based recommender systems cannot meet these two trends, and GPU- centric training has become a trending approach. However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in CV and NLP areas. This issue can be explained by two characteristics of these recommendation models: First, they contain up to a thousand input feature fields, introducing fragmentary and memory-intensive operations; Second, the multiple constituent feature interaction submodules introduce substantial small-sized compute kernels. To remove this roadblock to the development of recommender systems, we propose a novel framework named PICASSO to accelerate the training of recommendation models on commodity hardware. Specifically, we conduct a systematic analysis to reveal the bottlenecks encountered in training recommendation models. We leverage the model structure and data distribution to unleash the potential of hardware through our packing, interleaving, and caching optimization. Experiments show that PICASSO increases the hardware utilization by an order of magnitude on the basis of SOTA baselines and brings up to 6x throughput improvement for a variety of industrial recommendation models. Using the same hardware budget in production, PICASSO on average shortens the walltime of daily training tasks by 7 hours, significantly reducing the delay of continuous delivery.

cs.DC↗