arXiv subjects

Jun Ma

Publications and source records attributed to Jun Ma.

At least 307 records · Page 17Linked to original sources

arXiv2022

Neural-iLQR: A Learning-Aided Shooting Method for Trajectory Optimization

Iterative linear quadratic regulator (iLQR) has gained wide popularity in addressing trajectory optimization problems with nonlinear system models. However, as a model-based shooting method, it relies heavily on an accurate system model to update the optimal control actions and the trajectory determined with forward integration, thus becoming vulnerable to inevitable model inaccuracies. Recently, substantial research efforts in learning-based methods for optimal control problems have been progressing significantly in addressing unknown system models, particularly when the system has complex interactions with the environment. Yet a deep neural network is normally required to fit substantial scale of sampling data. In this work, we present Neural-iLQR, a learning-aided shooting method over the unconstrained control space, in which a neural network with a simple structure is used to represent the local system model. In this framework, the trajectory optimization task is achieved with simultaneous refinement of the optimal policy and the neural network iteratively, without relying on the prior knowledge of the system model. Through comprehensive evaluations on two illustrative control tasks, the proposed method is shown to outperform the conventional iLQR significantly in the presence of inaccuracies in system models.

cs.LG↗

arXiv2022

Robust Fixed-Order Controller Design for Uncertain Systems with Generalized Common Lyapunov Strictly Positive Realness Characterization

This paper investigates the design of a robust fixed-order controller for single-input-single-output (SISO) polytopic systems with interval uncertainties, with the aim that the closed-loop stability is appropriately ensured and the performance specifications on sensitivity shaping are conformed in a specific finite frequency range. Utilizing the notion of generalized common Lyapunov strictly positive realness (CL-SPRness), the equivalence between strictly positive realness (SPRness) and strictly bounded realness (SBRness) is established; and then the specifications on robust stability and performance are transformed into the SPRness of newly constructed systems and further characterized in the framework of linear matrix inequality (LMI) conditions. The proposed methodology avoids the tedious yet mandatory evaluations of the specifications on all vertices of the uncertain polytopic system in an explicit form. Instead, solving five LMIs exclusively suffices for ensuring the robust stability and performance regardless of the number of vertices, and thus the typically heavy computational burden is considerably alleviated. It is also noteworthy that the proposed methodology additionally provides the necessary and sufficient conditions for this robust controller design with the consideration of a prescribed finite frequency range, and therefore significantly less conservatism is attained in the system performance.

eess.SY↗

arXiv2022

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results

Deep learning (DL) models have provided state-of-the-art performance in various medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder translating DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties could enable clinical review of the most uncertain regions, thereby building trust and paving the way toward clinical translation. Several uncertainty estimation methods have recently been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentage of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, highlighting the need for uncertainty quantification in medical image analyses. Finally, in favor of transparency and reproducibility, our evaluation code is made publicly available at: https://github.com/RagMeh11/QU-BraTS.

eess.IV↗

arXiv2022

A Two-Phase Paradigm for Joint Entity-Relation Extraction

An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task. However, these models sample a large number of negative entities and negative relations during the model training, which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance. In order to address the above issues, we propose a two-phase paradigm for the span-based joint entity and relation extraction, which involves classifying the entities and relations in the first phase, and predicting the types of these entities and relations in the second phase. The two-phase paradigm enables our model to significantly reduce the data distribution gap, including the gap between negative entities and other entities, as well as the gap between negative relations and other relations. In addition, we make the first attempt at combining entity type and entity distance as global features, which has proven effective, especially for the relation extraction. Experimental results on several datasets demonstrate that the spanbased joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-of-the-art span-based models for the joint extraction task, establishing a new standard benchmark. Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.

cs.CL↗

arXiv2022

Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypes

Few-shot named entity recognition (NER) enables us to build a NER system for a new domain using very few labeled examples. However, existing prototypical networks for this task suffer from roughly estimated label dependency and closely distributed prototypes, thus often causing misclassifications. To address the above issues, we propose EP-Net, an Entity-level Prototypical Network enhanced by dispersedly distributed prototypes. EP-Net builds entity-level prototypes and considers text spans to be candidate entities, so it no longer requires the label dependency. In addition, EP-Net trains the prototypes from scratch to distribute them dispersedly and aligns spans to prototypes in the embedding space using a space projection. Experimental results on two evaluation tasks and the Few-NERD settings demonstrate that EP-Net consistently outperforms the previous strong models in terms of overall performance. Extensive analyses further validate the effectiveness of EP-Net.

cs.CL↗

arXiv2022

A Context-Aware Approach for Textual Adversarial Attack through Probability Difference Guided Beam Search

Textual adversarial attacks expose the vulnerabilities of text classifiers and can be used to improve their robustness. Existing context-aware methods solely consider the gold label probability and use the greedy search when searching an attack path, often limiting the attack efficiency. To tackle these issues, we propose PDBS, a context-aware textual adversarial attack model using Probability Difference guided Beam Search. The probability difference is an overall consideration of all class label probabilities, and PDBS uses it to guide the selection of attack paths. In addition, PDBS uses the beam search to find a successful attack path, thus avoiding suffering from limited search space. Extensive experiments and human evaluation demonstrate that PDBS outperforms previous best models in a series of evaluation metrics, especially bringing up to a +19.5% attack success rate. Ablation studies and qualitative analyses further confirm the efficiency of PDBS.

cs.CL↗

arXiv2022

On Symmetric Gauss-Seidel ADMM Algorithm for $\mathcal{H}_\infty$ Guaranteed Cost Control with Convex Parameterization

This paper involves the innovative development of a symmetric Gauss-Seidel ADMM algorithm to solve the H-infinity guaranteed cost control problem. In the presence of parametric uncertainties, the H-infinity guaranteed cost control problem generally leads to the large-scale optimization. This is due to the exponential growth of the number of the extreme systems involved with respect to the number of parametric uncertainties. In this work, through a variant of the Youla-Kucera parameterization, the stabilizing controllers are parameterized in a convex set; yielding the outcome that the H-infinity guaranteed cost control problem is converted to a convex optimization problem. Based on an appropriate re-formulation using the Schur complement, it then renders possible the use of the ADMM algorithm with symmetric Gauss-Seidel backward and forward sweeps. Significantly, this approach alleviates the often-times prohibitively heavy computational burden typical in many H-infinity optimization problems while exhibiting good convergence guarantees, which is particularly essential for the related large-scale optimization procedures involved. With this approach, the desired robust stability is ensured, and the disturbance attenuation is maintained at the minimum level in the presence of parametric uncertainties. Rather importantly too, with the attained effectiveness, the methodology thus evidently possesses extensive applicability in various important controller synthesis problems, such as decentralized control, sparse control, and output feedback control problems.

math.OC↗

arXiv2022

Alternating Direction Method of Multipliers for Constrained Iterative LQR in Autonomous Driving

In the context of autonomous driving, the iterative linear quadratic regulator (iLQR) is known to be an efficient approach to deal with the nonlinear vehicle model in motion planning problems. Particularly, the constrained iLQR algorithm has shown noteworthy advantageous outcomes of computation efficiency in achieving motion planning tasks under general constraints of different types. However, the constrained iLQR methodology requires a feasible trajectory at the first iteration as a prerequisite when the logarithmic barrier function is used. Also, the methodology leaves open the possibility for incorporation of fast, efficient, and effective optimization methods to further speed up the optimization process such that the requirements of real-time implementation can be successfully fulfilled. In this paper, a well-defined motion planning problem is formulated under nonlinear vehicle dynamics and various constraints, and an alternating direction method of multipliers (ADMM) is utilized to determine the optimal control actions leveraging the iLQR. The approach is able to circumvent the feasibility requirement of the trajectory at the first iteration. An illustrative example of motion planning for autonomous vehicles is then investigated. A noteworthy achievement of high computation efficiency is attained with the proposed development; comparing with the constrained iLQR algorithm based on the logarithmic barrier function, our proposed method reduces the average computation time by 31.93%, 38.52%, and 44.57% in the three driving scenarios; compared with the optimization solver IPOPT, our proposed method reduces the average computation time by 46.02%, 53.26%, and 88.43% in the three driving scenarios. As a result, real-time computation and implementation can be realized through our proposed framework, and thus it provides additional safety to the on-road driving tasks.

cs.RO↗

arXiv2022

Sequential Convex Programming for Collaboration of Connected and Automated Vehicles

This paper investigates the collaboration of multiple connected and automated vehicles (CAVs) in different scenarios. In general, the collaboration of CAVs can be formulated as a nonlinear and nonconvex model predictive control (MPC) problem. Most of the existing approaches available for utilization to solve such an optimization problem suffer from the drawback of considerable computational burden, which hinders the practical implementation in real time. This paper proposes the use of sequential convex programming (SCP), which is a powerful approach to solving the nonlinear and nonconvex MPC problem in real time. To appropriately deploy the methodology, as a first stage, SCP requires linearization and discretization when addressing the nonlinear dynamics of the system model adequately. Based on the linearization and discretization, the original MPC problem can be transformed into a quadratically constrained quadratic programming (QCQP) problem. Besides, SCP also involves convexification to handle the associated nonconvex constraints. Thus, the nonconvex QCQP can be reduced to a quadratic programming (QP) problem that can be solved rather quickly. Therefore, the computational efficiency is suitably improved despite the existence of nonlinear and nonconvex characteristics, whereby the implementation is realized in real time. Furthermore, simulation results in three different scenarios of autonomous driving are presented to validate the effectiveness and efficiency of our proposed approach.

math.OC↗

arXiv2022

Boosting Span-based Joint Entity and Relation Extraction via Squence Tagging Mechanism

Span-based joint extraction simultaneously conducts named entity recognition (NER) and relation extraction (RE) in text span form. Recent studies have shown that token labels can convey crucial task-specific information and enrich token semantics. However, as far as we know, due to completely abstain from sequence tagging mechanism, all prior span-based work fails to use token label in-formation. To solve this problem, we pro-pose Sequence Tagging enhanced Span-based Network (STSN), a span-based joint extrac-tion network that is enhanced by token BIO label information derived from sequence tag-ging based NER. By stacking multiple atten-tion layers in depth, we design a deep neu-ral architecture to build STSN, and each atten-tion layer consists of three basic attention units. The deep neural architecture first learns seman-tic representations for token labels and span-based joint extraction, and then constructs in-formation interactions between them, which also realizes bidirectional information interac-tions between span-based NER and RE. Fur-thermore, we extend the BIO tagging scheme to make STSN can extract overlapping en-tity. Experiments on three benchmark datasets show that our model consistently outperforms previous optimal models by a large margin, creating new state-of-the-art results.

cs.CL↗

arXiv2022

Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition

For Named Entity Recognition (NER), sequence labeling-based and span-based paradigms are quite different. Previous research has demonstrated that the two paradigms have clear complementary advantages, but few models have attempted to leverage these advantages in a single NER model as far as we know. In our previous work, we proposed a paradigm known as Bundling Learning (BL) to address the above problem. The BL paradigm bundles the two NER paradigms, enabling NER models to jointly tune their parameters by weighted summing each paradigm's training loss. However, three critical issues remain unresolved: When does BL work? Why does BL work? Can BL enhance the existing state-of-the-art (SOTA) NER models? To address the first two issues, we implement three NER models, involving a sequence labeling-based model--SeqNER, a span-based NER model--SpanNER, and BL-NER that bundles SeqNER and SpanNER together. We draw two conclusions regarding the two issues based on the experimental results on eleven NER datasets from five domains. We then apply BL to five existing SOTA NER models to investigate the third issue, consisting of three sequence labeling-based models and two span-based models. Experimental results indicate that BL consistently enhances their performance, suggesting that it is possible to construct a new SOTA NER system by incorporating BL into the current SOTA system. Moreover, we find that BL reduces both entity boundary and type prediction errors. In addition, we compare two commonly used labeling tagging methods as well as three types of span semantic representations.

cs.CL↗

arXiv2022

Topic-Grained Text Representation-based Model for Document Retrieval

Document retrieval enables users to find their required documents accurately and quickly. To satisfy the requirement of retrieval efficiency, prevalent deep neural methods adopt a representation-based matching paradigm, which saves online matching time by pre-storing document representations offline. However, the above paradigm consumes vast local storage space, especially when storing the document as word-grained representations. To tackle this, we present TGTR, a Topic-Grained Text Representation-based Model for document retrieval. Following the representation-based matching paradigm, TGTR stores the document representations offline to ensure retrieval efficiency, whereas it significantly reduces the storage requirements by using novel topicgrained representations rather than traditional word-grained. Experimental results demonstrate that compared to word-grained baselines, TGTR is consistently competitive with them on TREC CAR and MS MARCO in terms of retrieval accuracy, but it requires less than 1/10 of the storage space required by them. Moreover, TGTR overwhelmingly surpasses global-grained baselines in terms of retrieval accuracy.

cs.IR↗

arXiv2022

SummScore: A Comprehensive Evaluation Metric for Summary Quality Based on Cross-Encoder

Text summarization models are often trained to produce summaries that meet human quality requirements. However, the existing evaluation metrics for summary text are only rough proxies for summary quality, suffering from low correlation with human scoring and inhibition of summary diversity. To solve these problems, we propose SummScore, a comprehensive metric for summary quality evaluation based on CrossEncoder. Firstly, by adopting the original-summary measurement mode and comparing the semantics of the original text, SummScore gets rid of the inhibition of summary diversity. With the help of the text-matching pre-training Cross-Encoder, SummScore can effectively capture the subtle differences between the semantics of summaries. Secondly, to improve the comprehensiveness and interpretability, SummScore consists of four fine-grained submodels, which measure Coherence, Consistency, Fluency, and Relevance separately. We use semi-supervised multi-rounds of training to improve the performance of our model on extremely limited annotated data. Extensive experiments show that SummScore significantly outperforms existing evaluation metrics in the above four dimensions in correlation with human scoring. We also provide the quality evaluation results of SummScore on 16 mainstream summarization models for later research.

cs.CL↗

arXiv2022

Knowledge-aware Neural Collective Matrix Factorization for Cross-domain Recommendation

Cross-domain recommendation (CDR) can help customers find more satisfying items in different domains. Existing CDR models mainly use common users or mapping functions as bridges between domains but have very limited exploration in fully utilizing extra knowledge across domains. In this paper, we propose to incorporate the knowledge graph (KG) for CDR, which enables items in different domains to share knowledge. To this end, we first construct a new dataset AmazonKG4CDR from the Freebase KG and a subset (two domain pairs: movies-music, movie-book) of Amazon Review Data. This new dataset facilitates linking knowledge to bridge within- and cross-domain items for CDR. Then we propose a new framework, KG-aware Neural Collective Matrix Factorization (KG-NeuCMF), leveraging KG to enrich item representations. It first learns item embeddings by graph convolutional autoencoder to capture both domain-specific and domain-general knowledge from adjacent and higher-order neighbours in the KG. Then, we maximize the mutual information between item embeddings learned from the KG and user-item matrix to establish cross-domain relationships for better CDR. Finally, we conduct extensive experiments on the newly constructed dataset and demonstrate that our model significantly outperforms the best-performing baselines.

cs.IR↗

arXiv2022

60 candidate high-velocity stars originating from the Sagittarius dwarf spheroidal galaxy in Gaia EDR3

Using proper motions from Gaia Early Data Release 3 (Gaia EDR 3) and radial velocities from several surveys, we identify 60 candidate high-velocity stars with total velocity greater than 75\% escape velocity that probably origin from Sagittarius dwarf spheroidal galaxy (Sgr) by orbital analysis. Sgr's gravity has little effect on the results and the Large Magellanic Cloud's gravity has non-negligible effect on only a few stars. The closest approach of these stars to the Sgr occurs when the Sgr passed its pericenter ($\sim$ 38.2 Myr ago), which suggest they were tidally stripped from the Sgr. The positions of these stars in the HR diagram and the chemical properties of 19 of them with available [Fe/H] are similar with the Sgr stream member stars. This is consistent with the assumption of their accretion origin. Two of the 60 are hypervelocity stars, which may also be produced by Hills mechanism.

astro-ph.GA↗

arXiv2022

Efficient and effective training of language and graph neural network models

Can we combine heterogenous graph structure with text to learn high-quality semantic and behavioural representations? Graph neural networks (GNN)s encode numerical node attributes and graph structure to achieve impressive performance in a variety of supervised learning tasks. Current GNN approaches are challenged by textual features, which typically need to be encoded to a numerical vector before provided to the GNN that may incur some information loss. In this paper, we put forth an efficient and effective framework termed language model GNN (LM-GNN) to jointly train large-scale language models and graph neural networks. The effectiveness in our framework is achieved by applying stage-wise fine-tuning of the BERT model first with heterogenous graph information and then with a GNN model. Several system and design optimizations are proposed to enable scalable and efficient training. LM-GNN accommodates node and edge classification as well as link prediction tasks. We evaluate the LM-GNN framework in different datasets performance and showcase the effectiveness of the proposed approach. LM-GNN provides competitive results in an Amazon query-purchase-product application.

cs.LG↗

arXiv2022

First experience with He conditioning of an SRF photoinjector

The recent achievements in the performance of superconducting RF (SRF) photoinjectors have opened a new era in the development of the reliable high-brightness CW electron sources. While the SRF guns become one of the most promising technologies, the compatibility of SRF environment with the complex photocathodes remains on the forefront of the modern accelerator science. The SRF cavities operate at cryogenic temperatures providing the ultra-high vacuum (UHV) environment highly beneficial for the photocathode performance. However, the necessity to keep the photocathodes at room temperature while being surrounded by the cavity walls that are kept at cryogenic temperatures creates an additional complexity for the SRF gun design and operation. The complex and volatile chemical compounds used for photocathodes have a high chance of contaminating the surfaces of an SRF cavity. When deposited, such compounds could create centers for cold electron emission which degrade performance of the SRF guns. Such a circumstance would require development of the in-situ processing techniques for restoring the SRF cavity performance. This paper presents the results of the successful implementation and application of the He conditioning method for cavity restoration using the existing SRF photoinjector at Brookhaven National Laboratory (BNL). The method has proven to be extremely effective and resulted in a dramatic improvement of the BNL gun performance.

physics.acc-ph↗

arXiv2022

Searching Extra-tidal Features around the Globular Cluster Whiting 1

Whiting 1 is a faint and young globular cluster in the halo of the Milky Way, and was suggested to have originated in the Sagittarius spherical dwarf galaxy (Sgr dSph). In this paper, we use the deep DESI Legacy Imaging Surveys to explore tentative spatial connection between Whiting 1 and the Sgr dSph. We redetermine the fundamental parameters of Whiting 1 and use the best-fitting isochrone (age $τ$=6.5 Gyr, metalicity Z=0.005 and $\rm d_{\odot}$=26.9 kpc) to construct a theoretical matched filter for the extra-tidal features searching. Without any smooth technique to the matched filter density map, we detect a round-shape feature with possible leading and trailing tails on either side of the cluster. This raw image is not totally new compared to old discoveries, but confirms that no more large-scale features can be detected under a depth of r<=22.5 mag. In our results, the whole feature stretches 0.1-0.2 degree along the orbit of Whiting 1, which gives a much larger area than the cluster core. The tails on both sides of the cluster align along the orbital direction of the Sgr dSph as well as the cluster itself, which implies that these debris are probably stripped remnants of Whiting 1 by the Milky Way.

astro-ph.GA↗