arXiv subjects

Jin Li

Publications and source records attributed to Jin Li.

At least 163 records · Page 9Linked to original sources

arXiv2024

Understanding Missingness in Time-series Electronic Health Records for Individualized Representation

With the widespread of machine learning models for healthcare applications, there is increased interest in building applications for personalized medicine. Despite the plethora of proposed research for personalized medicine, very few focus on representing missingness and learning from the missingness patterns in time-series Electronic Health Records (EHR) data. The lack of focus on missingness representation in an individualized way limits the full utilization of machine learning applications towards true personalization. In this brief communication, we highlight new insights into patterns of missingness with real-world examples and implications of missingness in EHRs. The insights in this work aim to bridge the gap between theoretical assumptions and practical observations in real-world EHRs. We hope this work will open new doors for exploring directions for better representation in predictive modelling for true personalization.

cs.LG↗

arXiv2024

Entropies of the various components of the universe

In this work, we study the entropies of photons, dust (baryonic matter), dark matter, and dark energy in the context of cosmology. When these components expand freely with the universe, we calculate the entropy and specific entropy of each component from the perspective of statistics. Under specific assumptions and conditions, the entropies of these components can satisfy the second law of thermodynamics independently. Our calculations show that the specific entropy of matter cannot be a constant during the expansion of the universe except for photons. When these components interact with the space-time background, there could exist the phenomenon of particle production (annihilation). We study the influence of the interaction on the entropies of these components, and obtain the conditions guaranteeing that the entropy of each component satisfies the second law of thermodynamics.

gr-qc↗

arXiv2024

Observational appearance and additional photon rings of the asymmetric thin-shell wormhole in Horndeski theory

In this paper, we study the observational appearance of the asymmetric thin-shell wormhole (ATW) in Horndeski theory by employing the ray-tracing method. We first calculate the effective potential and null geodesic of the ATW, and then we obtain the deflection angle of the photon in the ATW spacetime. Based on the impact parameter of the photon, the trajectory of the photon can be classified into three cases. Two typical emission models of the thin accretion disk are considered to analyze the observational appearance of the ATW. By comparing the observational appearances of the ATW and a black hole with the same mass parameter, we find additional features in the observational appearance of the ATW, such as the ``lensing band" and ``photon ring group".

gr-qc↗

arXiv2024

Understanding the predication mechanism of deep learning through error propagation among parameters in strong lensing case

The error propagation among estimated parameters reflects the correlation among the parameters. We study the capability of machine learning of "learning" the correlation of estimated parameters. We show that machine learning can recover the relation between the uncertainties of different parameters, especially, as predicted by the error propagation formula. Gravitational lensing can be used to probe both astrophysics and cosmology. As a practical application, we show that the machine learning is able to intelligently find the error propagation among the gravitational lens parameters (effective lens mass $M_{L}$ and Einstein radius $θ_{E}$) in accordance with the theoretical formula for the singular isothermal ellipse (SIE) lens model. The relation of errors of lens mass and Einstein radius, (e.g. the ratio of standard deviations $\mathcal{F}=σ_{\hat{ M_{L}}}/ σ_{\hat{θ_{E}}}$) predicted by the deep convolution neural network are consistent with the error propagation formula of SIE lens model. As a proof-of-principle test, a toy model of linear relation with Gaussian noise is presented. We found that the predictions obtained by machine learning indeed indicate the information about the law of error propagation and the distribution of noise. Error propagation plays a crucial role in identifying the physical relation among parameters, rather than a coincidence relation, therefore we anticipate our case study on the error propagation of machine learning predictions could extend to other physical systems on searching the correlation among parameters.

astro-ph.GA↗

arXiv2023

GanFinger: GAN-Based Fingerprint Generation for Deep Neural Network Ownership Verification

Deep neural networks (DNNs) are extensively employed in a wide range of application scenarios. Generally, training a commercially viable neural network requires significant amounts of data and computing resources, and it is easy for unauthorized users to use the networks illegally. Therefore, network ownership verification has become one of the most crucial steps in safeguarding digital assets. To verify the ownership of networks, the existing network fingerprinting approaches perform poorly in the aspects of efficiency, stealthiness, and discriminability. To address these issues, we propose a network fingerprinting approach, named as GanFinger, to construct the network fingerprints based on the network behavior, which is characterized by network outputs of pairs of original examples and conferrable adversarial examples. Specifically, GanFinger leverages Generative Adversarial Networks (GANs) to effectively generate conferrable adversarial examples with imperceptible perturbations. These examples can exhibit identical outputs on copyrighted and pirated networks while producing different results on irrelevant networks. Moreover, to enhance the accuracy of fingerprint ownership verification, the network similarity is computed based on the accuracy-robustness distance of fingerprint examples'outputs. To evaluate the performance of GanFinger, we construct a comprehensive benchmark consisting of 186 networks with five network structures and four popular network post-processing techniques. The benchmark experiments demonstrate that GanFinger significantly outperforms the state-of-the-arts in efficiency, stealthiness, and discriminability. It achieves a remarkable 6.57 times faster in fingerprint generation and boosts the ARUC value by 0.175, resulting in a relative improvement of about 26%.

cs.CR↗

arXiv2023

Subtraction of the confusion foreground and parameter uncertainty of resolvable galactic binaries on the networks of space-based gravitational-wave detectors

There are tens of millions of compact binary systems in the Milky Way, called galactic binaries (GBs), most of which are unresolved, and the Gravitational waves (GWs) emitted overlap to form foreground confusion. By simulating such foreground confusion, we have studied how LISA, Taiji and TianQin, including their alternative orbital configurations, subtract resolvable GBs when they combine as some networks. The results of our research indicate that the order of detected number for a single detector from high to low is: Taiji-m, Taiji-p (c), LISA, TianQin I, TianQin II. For detector combinations on the network, the foreground confusion is effectively reduced as the number of detectors grows, and the optimal combinations with different numbers are: Taiji-m, LISA+Taiji-m, LISA+Taiji-m+TianQin I, and LISA+Taiji-m+TianQin I+II. The sensitivity curve is optimized as the number of detectors increases, which renders it possible to detect other gravitational wave sources more precisely and decrease the resolvable GBs parameter uncertainty. Based on this, we discuss the parameter uncertainty of resolvable GBs detected by the combinations above and find that GW detection can promote electromagnetic (EM) detection. On the contrary, we discovered that by utilizing EM detection, determining the inclination angle can reduce the uncertainty of GW strain amplitude by $\sim$93%, and determining the sky position can reduce the uncertainty of the phase by $\sim$30%, further strengthening the connection between GW detection and EM detection, and contributing to the research of Multi-messenger astronomy.

gr-qc↗

arXiv2023

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs

Despite Graph neural networks' significant performance gain over many classic techniques in various graph-related downstream tasks, their successes are restricted in shallow models due to over-smoothness and the difficulties of optimizations among many other issues. In this paper, to alleviate the over-smoothing issue, we propose a soft graph normalization method to preserve the diversities of node embeddings and prevent indiscrimination due to possible over-closeness. Combined with residual connections, we analyze the reason why the method can effectively capture the knowledge in both input graph structures and node features even with deep networks. Additionally, inspired by Curriculum Learning that learns easy examples before the hard ones, we propose a novel label-smoothing-based learning framework to enhance the optimization of deep GNNs, which iteratively smooths labels in an auxiliary graph and constructs many gradual non-smooth tasks for extracting increasingly complex knowledge and gradually discriminating nodes from coarse to fine. The method arguably reduces the risk of overfitting and generalizes better results. Finally, extensive experiments are carried out to demonstrate the effectiveness and potential of the proposed model and learning framework through comparison with twelve existing baselines including the state-of-the-art methods on twelve real-world node classification benchmarks.

cs.LG↗

arXiv2023

Deep learning forecasts of cosmic acceleration parameters from DECi-hertz Interferometer Gravitational-wave Observatory

Validating the accelerating expansion of the universe is an important issue for understanding the evolution of the universe. By constraining the cosmic acceleration parameter $X_H$, we can discriminate between the $Λ\mathrm{CDM}$ (cosmological constant plus cold dark matter) model and LTB (the Lemaître-Tolman-Bondi) model. In this paper, we explore the possibility of constraining the cosmic acceleration parameter with the inspiral gravitational waveform of neutron star binaries (NSBs) in the frequency range of 0.1Hz-10Hz, which can be detected by the second-generation space-based gravitational wave detector DECIGO. We use a convolutional neural network (CNN), a long short-term memory (LSTM) network combined with a gated recurrent unit (GRU), and Fisher information matrix to derive constraints on the cosmic acceleration parameter $X_H$. Based on the simulated gravitational wave data with a time duration of 1 month, we conclude that CNN can limit the relative error to 14.09%, while LSTM network combined with GRU can limit the relative error to 13.53%. Additionally, using Fisher information matrix for gravitational wave data with a 5-year observation can limit the relative error to 32.94%. Compared with the Fisher information matrix method, deep learning techniques will significantly improve the constraints on the cosmic acceleration parameters at different redshifts. Therefore, DECIGO is expected to provide direct measurements of the acceleration of the universe, by observing the chirp signals of coalescing binary neutron stars.

astro-ph.GA↗

arXiv2023

Parameter Estimation for Intermediate-Mass Binary Black Holes through Gravitational Waves Observed by DECIGO

With the anticipated launch of space-based gravitational wave detectors, including LISA, TaiJi, TianQin, and DECIGO, expected around 2030, the detection of gravitational waves generated by intermediate-mass black hole binaries (IMBBHs) becomes a tangible prospect. However, due to the detector's reception of a substantial amount of non-Gaussian, non-stationary data, employing traditional Bayesian inference methods for parameter estimation would result in significant resource demands and limitations in the waveform template library. Therefore, in this paper, we simulated foreground noise induced by stellar-origin binary black holes (SOBBHs), which is non-Gaussian and non-stationary, and we explore the use of Gaussian process regression (GPR) and deep learning for parameter estimation of Intermediate Mass Binary Black Holes (IMBBHs) in the presence of such non-Gaussian, non-stationary background noise. By comparing these results from deep learning and GPR, we demonstrate that deep learning can offer improved precision in parameter estimation compared to traditional GPR. Furthermore, compared to GPR, deep learning can provide posterior distributions of the sample parameters faster.

gr-qc↗

arXiv2023

The ability of Lisa, Taiji, and their networks to detect the stochastic gravitational wave background generated by Cosmic Strings

The cosmic string contributes to our understanding and revelation of the fundamental structure and evolutionary patterns of the universe, unifying our knowledge of the cosmos and unveiling new physical laws and phenomena. Therefore, we anticipate the detection of Stochastic Gravitational Wave Background (SGWB) signals generated by cosmic strings in space-based detectors. We have analyzed the detection capabilities of individual space-based detectors, Lisa and Taiji, as well as the joint space-based detector network, Lisa-Taiji, for SGWB signals produced by cosmic strings, taking into account other astronomical noise sources. The results indicate that the Lisa-Taiji network exhibits superior capabilities in detecting SGWB signals generated by cosmic strings and can provide strong evidence. The Lisa-Taiji network can achieve an uncertainty estimation of $ΔGμ/Gμ<0.5$ for cosmic string tension $Gμ\sim4\times10^{-17}$, and can provide evidence for the presence of SGWB signals generated by cosmic strings at $Gμ\sim10^{-17}$, and strong evidence at $Gμ\sim10^{-16}$. Even in the presence of only SGWB signals, it can achieve a relative uncertainty of $ΔGμ/Gμ<0.5$ for cosmic string tension $Gμ<10^{-18}$, and provide strong evidence at $Gμ\sim10^{-17}$.

astro-ph.CO↗

arXiv2023

AiluRus: A Scalable ViT Framework for Dense Prediction

Vision transformers (ViTs) have emerged as a prevalent architecture for vision tasks owing to their impressive performance. However, when it comes to handling long token sequences, especially in dense prediction tasks that require high-resolution input, the complexity of ViTs increases significantly. Notably, dense prediction tasks, such as semantic segmentation or object detection, emphasize more on the contours or shapes of objects, while the texture inside objects is less informative. Motivated by this observation, we propose to apply adaptive resolution for different regions in the image according to their importance. Specifically, at the intermediate layer of the ViT, we utilize a spatial-aware density-based clustering algorithm to select representative tokens from the token sequence. Once the representative tokens are determined, we proceed to merge other tokens into their closest representative token. Consequently, semantic similar tokens are merged together to form low-resolution regions, while semantic irrelevant tokens are preserved independently as high-resolution regions. This strategy effectively reduces the number of tokens, allowing subsequent layers to handle a reduced token sequence and achieve acceleration. We evaluate our proposed method on three different datasets and observe promising performance. For example, the "Segmenter ViT-L" model can be accelerated by 48% FPS without fine-tuning, while maintaining the performance. Additionally, our method can be applied to accelerate fine-tuning as well. Experimental results demonstrate that we can save 52% training time while accelerating 2.46 times FPS with only a 0.09% performance drop. The code is available at https://github.com/caddyless/ailurus/tree/main.

cs.CV↗

arXiv2023

Electromagnetic counterparts of high-frequency gravitational waves in a rotating laboratory frame system and their detection

We consider the perturbative photon flows (PPFs, i.e., electromagnetic (EM) counterparts) generated by the EM resonance response to high-frequency gravitational waves (HFGWs) with additional polarization states in a rotating laboratory frame system. It is found that when the propagating direction of HFGWs and the symmetrical axis of the laboratory frame system are the same, the PPFs have the maximum value. In this case, using the rotation (the rotation of the azimuth $ϕ$) of the EM detection system, all six possible polarization states of HFGWs can be separated and displayed. For the current experimental conditions, it is quite prospective to detect the PPFs generated by the HFGWs predicted in the braneworld models, the primordial black hole theories, the interaction mechanism between astrophysical plasma and intense EM radiation, etc., due to the large amplitudes (or high spectral densities) and spectral characteristics of these HFGWs. Detecting the primordial HFGWs from inflation faces great challenges at present, but it is not impossible.

gr-qc↗

arXiv2023

Testing the spatial geometry of the universe with TianQin: the prospect of using supermassive black hole binaries

The determination of the spatial geometry of the universe plays an important role in modern cosmology. Any deviation from the cosmic curvature $Ω_K=0$ would have a profound impact on the primordial inflation paradigm and fundamental physics. In this paper, we carry out a systematic study of the prospect of measuring cosmic curvature with the inspiral signal of supermassive black hole binaries (SMBHBs) that could be detected with TianQin. The study is based on a cosmological-model-independent method that extended the application of gravitational wave (GW) standard sirens in cosmology. By comparing the distances from future simulated GW events and simulated $H(z)$ data, we evaluate if TianQin would produce robust constraints on the cosmic curvature parameter $Ω_{k}$. More specifically, we consider 3-yr to 10-yr observations of supermassive black hole binaries with total masses ranging from $10^{3}M_\odot$ to $10^{7}M_\odot$. Our results show that in the future, with the synergy of 10-yr high-quality observations, we can tightly constrain the curvature parameter at the level of $1σ$ $Ω_k=-0.002\pm0.061$. Moreover, our findings indicate that the total mass of SMBHB does influence the estimation of cosmic curvature, implied by the analysis performed on different subsamples of gravitational wave data. Therefore, TianQin is expected to provide a powerful and competitive probe of the spatial geometry of the universe, compared to future spaced-based detectors such as DECIGO.

astro-ph.CO↗

arXiv2023

Two-stage MR Image Segmentation Method for Brain Tumors based on Attention Mechanism

Multimodal magnetic resonance imaging (MRI) can reveal different patterns of human tissue and is crucial for clinical diagnosis. However, limited by cost, noise and manual labeling, obtaining diverse and reliable multimodal MR images remains a challenge. For the same lesion, different MRI manifestations have great differences in background information, coarse positioning and fine structure. In order to obtain better generation and segmentation performance, a coordination-spatial attention generation adversarial network (CASP-GAN) based on the cycle-consistent generative adversarial network (CycleGAN) is proposed. The performance of the generator is optimized by introducing the Coordinate Attention (CA) module and the Spatial Attention (SA) module. The two modules can make full use of the captured location information, accurately locating the interested region, and enhancing the generator model network structure. The ability to extract the structure information and the detailed information of the original medical image can help generate the desired image with higher quality. There exist some problems in the original CycleGAN that the training time is long, the parameter amount is too large, and it is difficult to converge. In response to this problem, we introduce the Coordinate Attention (CA) module to replace the Res Block to reduce the number of parameters, and cooperate with the spatial information extraction network above to strengthen the information extraction ability. On the basis of CASP-GAN, an attentional generative cross-modality segmentation (AGCMS) method is further proposed. This method inputs the modalities generated by CASP-GAN and the real modalities into the segmentation network for brain tumor segmentation. Experimental results show that CASP-GAN outperforms CycleGAN and some state-of-the-art methods in PSNR, SSMI and RMSE in most tasks.

eess.IV↗

arXiv2023

Dynamic Selection in Algorithmic Decision-making

This paper identifies and addresses dynamic selection problems in online learning algorithms with endogenous data. In a contextual multi-armed bandit model, a novel bias (self-fulfilling bias) arises because the endogeneity of the data influences the choices of decisions, affecting the distribution of future data to be collected and analyzed. We propose an instrumental-variable-based algorithm to correct for the bias. It obtains true parameter values and attains low (logarithmic-like) regret levels. We also prove a central limit theorem for statistical inference. To establish the theoretical properties, we develop a general technique that untangles the interdependence between data and actions.

econ.EM↗

arXiv2023

Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation

In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are often unlabelled. In this case, identifying infrequent events, such as anomalies, poses a great challenge. This problem becomes even more difficult in non-stationary environments, which can cause deterioration of the predictive performance of a model. To address the above challenges, the paper proposes an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.

cs.LG↗

arXiv2023

Vertical Ferroelectricity in Van der Waals Materials: Models and Devices

Ferroelectricity has a wide range of applications in functional electronics and is extremely important for the development of next-generation information storage technology, but it is difficult to achieve due to its special symmetry requirements. In this letter, based on van derWaals stacking, a generic model is proposed for realizing ferroelectric devices, where a freely movable center layer is packaged in two fixed and symmetrically stacked layers. In this model, the ferroelectric phase transition can be realized between the two equivalent and eccentric ground stacking-states with opposite polarizations. By means of first-principles calculations, taking the h-BN/h-BN/h-BN and h-BN/Graphene/h-BN as feasible models, we carefully evaluate the magnitude of ferroelectricity. The corresponding polarizations are estimated as 1.83 and 1.35 pC/m, respectively, which are comparable to the sliding ferroelectricity. Such a new tri-layer model of vertical ferroelectricity can be constructed by arbitrary van derWaals semiconducting materials, and usually holds low switching barrier. Optimized material combinations with remarkable polarization are highly expectable to be discovered from the huge candidate set for future information storage.

cond-mat.mtrl-sci↗

arXiv2023

Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners

Representation learning has been evolving from traditional supervised training to Contrastive Learning (CL) and Masked Image Modeling (MIM). Previous works have demonstrated their pros and cons in specific scenarios, i.e., CL and supervised pre-training excel at capturing longer-range global patterns and enabling better feature discrimination, while MIM can introduce more local and diverse attention across all transformer layers. In this paper, we explore how to obtain a model that combines their strengths. We start by examining previous feature distillation and mask feature reconstruction methods and identify their limitations. We find that their increasing diversity mainly derives from the asymmetric designs, but these designs may in turn compromise the discrimination ability. In order to better obtain both discrimination and diversity, we propose a simple but effective Hybrid Distillation strategy, which utilizes both the supervised/CL teacher and the MIM teacher to jointly guide the student model. Hybrid Distill imitates the token relations of the MIM teacher to alleviate attention collapse, as well as distills the feature maps of the supervised/CL teacher to enable discrimination. Furthermore, a progressive redundant token masking strategy is also utilized to reduce the distilling costs and avoid falling into local optima. Experiment results prove that Hybrid Distill can achieve superior performance on different benchmarks.

cs.CV↗