Searcharxiv⌕ Search

arXiv subjects

Heng Li

Publications and source records attributed to Heng Li.

At least 163 records · Page 9Linked to original sources
arXiv2023

What Causes Exceptions in Machine Learning Applications? Mining Machine Learning-Related Stack Traces on Stack Overflow

Machine learning (ML), including deep learning, has recently gained tremendous popularity in a wide range of applications. However, like traditional software, ML applications are not immune to the bugs that result from programming errors. Explicit programming errors usually manifest through error messages and stack traces. These stack traces describe the chain of function calls that lead to an anomalous situation, or exception. Indeed, these exceptions may cross the entire software stack (including applications and libraries). Thus, studying the patterns in stack traces can help practitioners and researchers understand the causes of exceptions in ML applications and the challenges faced by ML developers. To that end, we mine Stack Overflow (SO) and study 11,449 stack traces related to seven popular Python ML libraries. First, we observe that ML questions that contain stack traces gain more popularity than questions without stack traces; however, they are less likely to get accepted answers. Second, we observe that recurrent patterns exists in ML stack traces, even across different ML libraries, with a small portion of patterns covering many stack traces. Third, we derive five high-level categories and 25 low-level types from the stack trace patterns: most patterns are related to python basic syntax, model training, parallelization, data transformation, and subprocess invocation. Furthermore, the patterns related to subprocess invocation, external module execution, and remote API call are among the least likely to get accepted answers on SO. Our findings provide insights for researchers, ML library providers, and ML application developers to improve the quality of ML libraries and their applications.

cs.LG↗
arXiv2023

De novo reconstruction of satellite repeat units from sequence data

Satellite DNA are long tandemly repeating sequences in a genome and may be organized as high-order repeats (HORs). They are enriched in centromeres and are challenging to assemble. Existing algorithms for identifying satellite repeats either require the complete assembly of satellites or only work for simple repeat structures without HORs. Here we describe Satellite Repeat Finder (SRF), a new algorithm for reconstructing satellite repeat units and HORs from accurate reads or assemblies without prior knowledge on repeat structures. Applying SRF to real sequence data, we showed that SRF could reconstruct known satellites in human and well-studied model organisms. We also found satellite repeats are pervasive in various other species, accounting for up to 12% of their genome contents but are often underrepresented in assemblies. With the rapid progress on genome sequencing, SRF will help the annotation of new genomes and the study of satellite DNA evolution even if such repeats are not fully assembled.

q-bio.GN↗
arXiv2023

Black-box Adversarial Example Attack towards FCG Based Android Malware Detection under Incomplete Feature Information

The function call graph (FCG) based Android malware detection methods have recently attracted increasing attention due to their promising performance. However, these methods are susceptible to adversarial examples (AEs). In this paper, we design a novel black-box AE attack towards the FCG based malware detection system, called BagAmmo. To mislead its target system, BagAmmo purposefully perturbs the FCG feature of malware through inserting "never-executed" function calls into malware code. The main challenges are two-fold. First, the malware functionality should not be changed by adversarial perturbation. Second, the information of the target system (e.g., the graph feature granularity and the output probabilities) is absent. To preserve malware functionality, BagAmmo employs the try-catch trap to insert function calls to perturb the FCG of malware. Without the knowledge about feature granularity and output probabilities, BagAmmo adopts the architecture of generative adversarial network (GAN), and leverages a multi-population co-evolution algorithm (i.e., Apoem) to generate the desired perturbation. Every population in Apoem represents a possible feature granularity, and the real feature granularity can be achieved when Apoem converges. Through extensive experiments on over 44k Android apps and 32 target models, we evaluate the effectiveness, efficiency and resilience of BagAmmo. BagAmmo achieves an average attack success rate of over 99.9% on MaMaDroid, APIGraph and GCN, and still performs well in the scenario of concept drift and data imbalance. Moreover, BagAmmo outperforms the state-of-the-art attack SRL in attack success rate.

cs.SE↗
arXiv2023

3D Former: Monocular Scene Reconstruction with 3D SDF Transformers

Monocular scene reconstruction from posed images is challenging due to the complexity of a large environment. Recent volumetric methods learn to directly predict the TSDF volume and have demonstrated promising results in this task. However, most methods focus on how to extract and fuse the 2D features to a 3D feature volume, but none of them improve the way how the 3D volume is aggregated. In this work, we propose an SDF transformer network, which replaces the role of 3D CNN for better 3D feature aggregation. To reduce the explosive computation complexity of the 3D multi-head attention, we propose a sparse window attention module, where the attention is only calculated between the non-empty voxels within a local window. Then a top-down-bottom-up 3D attention network is built for 3D feature aggregation, where a dilate-attention structure is proposed to prevent geometry degeneration, and two global modules are employed to equip with global receptive fields. The experiments on multiple datasets show that this 3D transformer network generates a more accurate and complete reconstruction, which outperforms previous methods by a large margin. Remarkably, the mesh accuracy is improved by 41.8%, and the mesh completeness is improved by 25.3% on the ScanNet dataset. Project page: https://weihaosky.github.io/sdfformer.

cs.CV↗
arXiv2023

Analysis and optimization of a multicascade method for the size fractionation of poly-dispersed particle systems via sedimentation or centrifugation

Sedimentation and centrifugation can be used to sort particles by size, using a multistep (multicascade) method in which particles in the sediment are removed and the content of the supernatant is processed again, repeating the cycle several times. This paper proposes a theoretical analysis of this process, based on a one-dimensional model, with a view to identify parameters that are optimal to obtain a relatively monodispersed suspension, starting from a log-normal particle size distribution. We found that a rational choice of the sedimentation/centrifugation time enables to control the amount of particles outside of the desired size range (impurities). Surprisingly, the multistep method does not converge, as multiple steps are worse than 2 steps. Band sedimentation, in which a particle-rich layer is overlaid on clear fluid, offers substantial benefits in terms of impurity reduction with respect to starting from a completely mixed situation. An application to graphene fractionation is discussed.

physics.flu-dyn↗
arXiv2023

Hydrodynamic interactions change the buckling threshold of parallel flexible sheets in shear flow

Buckling induced by viscous flow changes the shape of sheet-like nanomaterial particles suspended in liquids. This instability at the particle scale affects collective behavior of suspension flows and has many technological and biological implications. Here, we investigated the effect of viscous hydrodynamic interactions on the morphology of flexible sheets. By analyzing a model experiment using thin sheets suspended in a shear cell, we found that a pair of sheets can bend for a shear rate ten times lower than the buckling threshold defined for a single sheet. This effect is caused by a lateral hydrodynamic force that arises from the disturbance flow field induced by the neighboring sheet. The lateral hydrodynamic force removes the buckling instability but massively enhances the bending deformation. For small separations between sheets, lubrication forces prevail and prevent deformation. Those two opposing effects result in a non-monotonic relation between distances and shear rate for bending. Our study suggests that the morphology of sheet-like particles in suspensions is not purely a material property, but also depends on particle concentration and microstructure.

physics.flu-dyn↗
arXiv2023

A step towards a general density Corrádi--Hajnal Theorem

For a nondegenerate $r$-graph $F$, large $n$, and $t$ in the regime $[0, c_{F} n]$, where $c_F>0$ is a constant depending only on $F$, we present a general approach for determining the maximum number of edges in an $n$-vertex $r$-graph that does not contain $t+1$ vertex-disjoint copies of $F$. In fact, our method results in a rainbow version of the above result and includes a characterization of the extremal constructions. Our approach applies to many well-studied hypergraphs (including graphs) such as the edge-critical graphs, the Fano plane, the generalized triangles, hypergraph expansions, the expanded triangles, and hypergraph books. Our results extend old results of Simonovits~\cite{SI68} and Moon~\cite{Moon68} on complete graphs and can be viewed as a step towards a general density version of the classical Corrádi--Hajnal Theorem~\cite{CH63}.

math.CO↗
arXiv2023

Dense RGB SLAM with Neural Implicit Maps

There is an emerging trend of using neural implicit functions for map representation in Simultaneous Localization and Mapping (SLAM). Some pioneer works have achieved encouraging results on RGB-D SLAM. In this paper, we present a dense RGB SLAM method with neural implicit map representation. To reach this challenging goal without depth input, we introduce a hierarchical feature volume to facilitate the implicit map decoder. This design effectively fuses shape cues across different scales to facilitate map reconstruction. Our method simultaneously solves the camera motion and the neural implicit map by matching the rendered and input video frames. To facilitate optimization, we further propose a photometric warping loss in the spirit of multi-view stereo to better constrain the camera pose and scene geometry. We evaluate our method on commonly used benchmarks and compare it with modern RGB and RGB-D SLAM systems. Our method achieves favorable results than previous methods and even surpasses some recent RGB-D SLAM methods.The code is at poptree.github.io/DIM-SLAM/.

cs.CV↗
arXiv2023

Studying Logging Practice in Machine Learning-based Applications

Logging is a common practice in traditional software development. Several research works have been done to investigate the different characteristics of logging practices in traditional software systems (e.g., Android applications, JAVA applications, C/C++ applications). Nowadays, we are witnessing more and more development of Machine Learning-based applications (ML-based applications). Today, there are many popular libraries that facilitate and contribute to the development of such applications, among which we can mention: Pytorch, Tensorflow, Theano, MXNet, Scikit-Learn, Caffe, and Keras. Despite the popularity of ML, we don't have a clear understanding of logging practices in ML applications. In this paper, we aim to fill this knowledge gap and help ML practitioners understand the characteristics of logging in ML-based applications. In particular, we conduct an empirical study on 110 open-source ML-based applications. Through a quantitative analysis, we find that logging practice in ML-based applications is less pervasive than in traditional applications including Android, JAVA, and C/C++ applications. Furthermore, the majority of logging statements in ML-based applications are in info and warn levels, compared to traditional applications where info is the majority of logging statement in C/C++ application and debug, error levels constitute the majority of logging statement in Android application. We also perform a quantitative and qualitative analysis of a random sample of logging statements to understand where ML developers put most of logging statements and examine why and how they are using logging. These analyses led to the following observations: (i) ML developers put most of the logging statements in model training, and in non-ML components. (ii) Data and model management appear to be the main reason behind the introduction of logging statements in ML-based applications.

cs.SE↗
arXiv2022

Protein-to-genome alignment with miniprot

Motivation: Protein-to-genome alignment is critical to annotating genes in non-model organisms. While there are a few tools for this purpose, all of them were developed over ten years ago and did not incorporate the latest advances in alignment algorithms. They are inefficient and could not keep up with the rapid production of new genomes and quickly growing protein databases. Results: Here we describe miniprot, a new aligner for mapping protein sequences to a complete genome. Miniprot integrates recent techniques such as k-mer sketch and SIMD-based dynamic programming. It is tens of times faster than existing tools while achieving comparable accuracy on real data. Availability and implementation: https://github.com/lh3/miniprot

q-bio.GN↗
arXiv2022

RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging

This paper proposes a deep recurrent Rotation Averaging Graph Optimizer (RAGO) for Multiple Rotation Averaging (MRA). Conventional optimization-based methods usually fail to produce accurate results due to corrupted and noisy relative measurements. Recent learning-based approaches regard MRA as a regression problem, while these methods are sensitive to initialization due to the gauge freedom problem. To handle these problems, we propose a learnable iterative graph optimizer minimizing a gauge-invariant cost function with an edge rectification strategy to mitigate the effect of inaccurate measurements. Our graph optimizer iteratively refines the global camera rotations by minimizing each node's single rotation objective function. Besides, our approach iteratively rectifies relative rotations to make them more consistent with the current camera orientations and observed relative rotations. Furthermore, we employ a gated recurrent unit to improve the result by tracing the temporal information of the cost graph. Our framework is a real-time learning-to-optimize rotation averaging graph optimizer with a tiny size deployed for real-world applications. RAGO outperforms previous traditional and deep methods on real-world and synthetic datasets. The code is available at https://github.com/sfu-gruvi-3dv/RAGO

cs.CV↗
arXiv2022

Spin/momentum properties of the paraxial optical beams

Spin angular momentum, an elementary dynamical property of classical electromagnetic fields, plays an important role in spin-orbit and light-matter interactions, especially in near-field optics. The research on optical spins has led to the discovery of phenomena such as optical spin-momentum locking and photonic topological quasiparticles, as well as applications in high-precision detection and nanometrology. Here, we investigate spin-momentum relations in paraxial optical systems and show that the optical spin angular momentum contains transverse and longitudinal spin components simultaneously. The transverse spin originates from inhomogeneities of field and governed by the vorticity of the kinetic momentum density, whereas the longitudinal spin parallel to the local canonical momentum is proportional to the polarization ellipticity of light. Moreover, the skyrmionlike spin textures arise from the optical transverse spin can be observed in paraxial beams, and their topologies are maintained free from the influence of the Gouy phase during propagation. Interestingly, the optical singularities, including both phase and polarization singularities, can also affect the spin-momentum properties significantly. Our findings describe the intrinsic spin-momentum properties in paraxial optical systems and apply in the analysis of the properties of spin-momentum in optical focusing, imaging, and scattering systems.

physics.optics↗
arXiv2022

Towards complete representation of bacterial contents in metagenomic samples

Background: In the metagenome assembly of a microbiome community, we may think abundant species would be easier to assemble due to their deeper coverage. However, this conjucture is rarely tested. We often do not know how many abundant species we are missing and do not have an approach to recover these species. Results: Here we proposed k-mer based and 16S RNA based methods to measure the completeness of metagenome assembly. We showed that even with PacBio High-Fidelity (HiFi) reads, abundant species are often not assembled as high strain diversity may lead to fragmented contigs. We developed a novel algorithm to recover abundant metagenome-assembled genomes (MAGs) by identifying circular assembly subgraphs. Our algorithm is reference-free and complement to standard metagenome binning. Evaluated on 14 real datasets, it rescued many abundant species that would be missing with existing methods. Conclusions: Our work stresses the importance of metagenome completeness which is often overlooked before. Our algorithm generates more circular MAGs and moves a step closer to the complete representation of microbiome communities.

q-bio.GN↗
arXiv2022

Extremal graphs for the suspension of edge-critical graphs

The Turán number of a graph $H$, $\text{ex}(n,H)$, is the maximum number of edges in an $n$-vertex graph that does not contain $H$ as a subgraph. For a vertex $v$ and a multi-set $\mathcal{F}$ of graphs, the suspension $\mathcal{F}+v$ of $\mathcal{F}$ is the graph obtained by connecting the vertex $v$ to all vertices of $F$ for each $F\in \mathcal{F}$. For two integers $k\ge1$ and $r\ge2$, let $H_i$ be a graph containing a critical edge with chromatic number $r$ for any $i\in\{1,\ldots,k\}$, and let $H=\{H_1,\ldots,H_k\}+v$. In this paper, we determine $\text{ex}(n, H)$ and characterize all the extremal graphs for sufficiently large $n$. This generalizes a result of Chen, Gould, Pfender and Wei on intersecting cliques. We also obtain a stability theorem for $H$, extending a result of Roberts and Scott on graphs containing a critical edge.

math.CO↗
arXiv2022

Two stability theorems for $\mathcal{K}_{\ell + 1}^{r}$-saturated hypergraphs

An $\mathcal{F}$-saturated $r$-graph is a maximal $r$-graph not containing any member of $\mathcal{F}$ as a subgraph. Let $\mathcal{K}_{\ell + 1}^{r}$ be the collection of all $r$-graphs $F$ with at most $\binom{\ell+1}{2}$ edges such that for some $\left(\ell+1\right)$-set $S$ every pair $\{u, v\} \subset S$ is covered by an edge in $F$. Our first result shows that for each $\ell \geq r \geq 2$ every $\mathcal{K}_{\ell+1}^{r}$-saturated $r$-graph on $n$ vertices with $t_{r}(n, \ell) - o(n^{r-1+1/\ell})$ edges contains a complete $\ell$-partite subgraph on $(1-o(1))n$ vertices, which extends a stability theorem for $K_{\ell+1}$-saturated graphs given by Popielarz, Sahasrabudhe and Snyder. We also show that the bound is best possible. Our second result is motivated by a celebrated theorem of Andrásfai, Erdős and Sós which states that for $\ell \geq 2$ every $K_{\ell+1}$-free graph $G$ on $n$ vertices with minimum degree $δ(G) > \frac{3\ell-4}{3\ell-1}n$ is $\ell$-partite. We give a hypergraph version of it. The \emph{minimum positive co-degree} of an $r$-graph $\mathcal{H}$, denoted by $δ_{r-1}^{+}(\mathcal{H})$, is the maximum $k$ such that if $S$ is an $(r-1)$-set contained in a edge of $\mathcal{H}$, then $S$ is contained in at least $k$ distinct edges of $\mathcal{H}$. Let $\ell\ge 3$ be an integer and $\mathcal{H}$ be a $\mathcal{K}_{\ell+1}^3$-saturated $3$-graph on $n$ vertices. We prove that if either $\ell \ge 4$ and $δ_{2}^{+}(\mathcal{H}) > \frac{3\ell-7}{3\ell-1}n$; or $\ell = 3$ and $δ_{2}^{+}(\mathcal{H}) > 2n/7$, then $\mathcal{H}$ is $\ell$-partite; and the bound is best possible. This is the first stability result on minimum positive co-degree for hypergraphs.

math.CO↗
arXiv2022

Degradation-invariant Enhancement of Fundus Images via Pyramid Constraint Network

As an economical and efficient fundus imaging modality, retinal fundus images have been widely adopted in clinical fundus examination. Unfortunately, fundus images often suffer from quality degradation caused by imaging interferences, leading to misdiagnosis. Despite impressive enhancement performances that state-of-the-art methods have achieved, challenges remain in clinical scenarios. For boosting the clinical deployment of fundus image enhancement, this paper proposes the pyramid constraint to develop a degradation-invariant enhancement network (PCE-Net), which mitigates the demand for clinical data and stably enhances unknown data. Firstly, high-quality images are randomly degraded to form sequences of low-quality ones sharing the same content (SeqLCs). Then individual low-quality images are decomposed to Laplacian pyramid features (LPF) as the multi-level input for the enhancement. Subsequently, a feature pyramid constraint (FPC) for the sequence is introduced to enforce the PCE-Net to learn a degradation-invariant model. Extensive experiments have been conducted under the evaluation metrics of enhancement and segmentation. The effectiveness of the PCE-Net was demonstrated in comparison with state-of-the-art methods and the ablation study. The source code of this study is publicly available at https://github.com/HeverLaw/PCENet-Image-Enhancement.

eess.IV↗
arXiv2022

Spacecraft Attitude Pointing Control under Pointing Forbidden Constraints with Guaranteed Accuracy

This paper focuses on the attitude pointing control problem under pointing-forbidden constraints and performance constraints. The spacecraft is expected to align its sensor's boresight to a desired direction, while the terminal control accuracy and the attitude adjustment rapidity should also be guaranteed simultaneously. To resolve this problem, a switching controller structure is proposed in this paper based on the reduced-attitude representation, fusing the artificial potential field (APF) methodology and the Prescribed Performance Control (PPC) scheme together. Firstly, a novel artificial potential field is presented, and a particular function is designed for the mollification of the switching process, aiming at providing a smooth transition for the system status. Subsequently, we propose a special performance function, which can freeze the PPC part when necessary. In this way, the intrinsic contradictory between the fast attitude maneuver and forbidden direction avoidance is tackled Further, an asynchronous switching strategy is designed, guarantees the system's stability. Based on these proposed issues, a switching backstepping controller is developed, and a tracking differentiator(TD) is employed to generate a smooth approximation of differential signals. Numerical simulation results are illustrated to show the effectiveness of the proposed scheme.

eess.SY↗
arXiv2022

An Empirical Study on the Usage of Automated Machine Learning Tools

The popularity of automated machine learning (AutoML) tools in different domains has increased over the past few years. Machine learning (ML) practitioners use AutoML tools to automate and optimize the process of feature engineering, model training, and hyperparameter optimization and so on. Recent work performed qualitative studies on practitioners' experiences of using AutoML tools and compared different AutoML tools based on their performance and provided features, but none of the existing work studied the practices of using AutoML tools in real-world projects at a large scale. Therefore, we conducted an empirical study to understand how ML practitioners use AutoML tools in their projects. To this end, we examined the top 10 most used AutoML tools and their respective usages in a large number of open-source project repositories hosted on GitHub. The results of our study show 1) which AutoML tools are mostly used by ML practitioners and 2) the characteristics of the repositories that use these AutoML tools. Also, we identified the purpose of using AutoML tools (e.g. model parameter sampling, search space management, model evaluation/error-analysis, Data/ feature transformation, and data labeling) and the stages of the ML pipeline (e.g. feature engineering) where AutoML tools are used. Finally, we report how often AutoML tools are used together in the same source code files. We hope our results can help ML practitioners learn about different AutoML tools and their usages, so that they can pick the right tool for their purposes. Besides, AutoML tool developers can benefit from our findings to gain insight into the usages of their tools and improve their tools to better fit the users' usages and needs.

cs.SE↗