arXiv subjects

Jin Song

Publications and source records attributed to Jin Song.

18 recordsLinked to original sources

arXiv2026

PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent

Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.

cs.CV↗

arXiv2026

HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes

Recently, zero-shot object customization generation methods have rapidly developed and shown tremendous potential for applications. For instance, in the e-commerce domain, consumers can observe the visual effect of furniture placed within their personal living spaces or clothes worn on their own bodies. Many existing approaches perform object customization generation based on diffusion models and extracted reference object features. However, the generated object significantly diverges from the original reference object in details such as patterns and curves. Particularly for asymmetrical reference objects, the absence of comprehensive multi-viewpoint information prevents the generation of object poses that harmonize with the background scene. To address these shortcomings, we have constructed a novel dataset comprising multi-angle images of furniture and indoor scenes. Based on diffusion models, we introduce HomeDiffusion, which can leverage multi-viewpoint images of the same reference object to accurately generate visually harmonious object poses within specified areas of the background scene. During the diffusion process, we further extract high-fidelity details of the reference object and perform cross-attention with the noise latents in the latent space, thereby ensuring the preservation of details in the customized object generation. Extensive qualitative and quantitative experiments demonstrate that our method achieves superior performance over other existing zero-shot as well as few-shot object customization approaches.

cs.CV↗

arXiv2026

Home3D 1.0: A High-Fidelity Image-to-3D Asset Generation System for Interior Design

We present Home3D 1.0, a modular image-to-3D generation system that produces high-quality 3D assets from a single reference image, targeting interior design and e-commerce applications. Given a photograph of a furniture or decor item, the system outputs a mesh with physically-based rendering (PBR) materials, and the mesh can be decomposed into material-specific components. The pipeline is organized into four tightly coupled modules: Geometry reconstructs a watertight mesh through latent SDF modelling with a geometry VAE and a coarse-to-fine flow-matching DiT; Texture predicts multiview albedo observations, reprojects them onto the mesh, and completes unseen surface regions with a 3D texture field; Material uses MatWeaver to obtain component masks through video-based segmentation and UV-space voting, then retrieves and bakes PBR maps from a curated material library through hierarchical multi-modal matching; and Parts generates material-editable semantic part meshes with a PartVAE and PartDiT, decoding multi-head part-specific SDF fields in one pass. Each module is evaluated independently with dedicated metrics, highlighting both the current system capability and the remaining gaps toward broader deployment.

cs.CV↗

arXiv2026

Semi-Supervised Noise Adaptation: Transferring Knowledge from Noise Domain

Transfer learning aims to facilitate the learning of a target domain by transferring knowledge from a source domain. The source domain typically contains semantically meaningful samples (*e.g.*, images) to facilitate effective knowledge transfer. However, a recent study observes that the noise domain constructed from simple distributions (*e.g.*, Gaussian distributions) can serve as a surrogate source domain in the semi-supervised setting, where only a small proportion of target samples are labeled while most remain unlabeled. Based on this surprising observation, we formulate a novel problem termed *Semi-Supervised Noise Adaptation* (SSNA), which aims to leverage a synthetic noise domain to improve the generalization of the target domain. To address this problem, we first establish a generalization bound characterizing the effect of the noise domain on generalization, based on which we propose a Noise Adaptation Framework (NAF). Extensive experiments demonstrate that NAF effectively leverages the noise domain to tighten the generalization bound of the target domain, leading to improved performance. The codes are available at https://github.com/AIResearch-Group/SSNA.

cs.LG↗

arXiv2026

SkeleGuide: Explicit Skeleton Reasoning for Context-Aware Human-in-Place Image Synthesis

Generating realistic and structurally plausible human images into existing scenes remains a significant challenge for current generative models, which often produce artifacts like distorted limbs and unnatural poses. We attribute this systemic failure to an inability to perform explicit reasoning over human skeletal structure. To address this, we introduce SkeleGuide, a novel framework built upon explicit skeletal reasoning. Through joint training of its reasoning and rendering stages, SkeleGuide learns to produce an internal pose that acts as a strong structural prior, guiding the synthesis towards high structural integrity. For fine-grained user control, we introduce PoseInverter, a module that decodes this internal latent pose into an explicit and editable format. Extensive experiments demonstrate that SkeleGuide significantly outperforms both specialized and general-purpose models in generating high-fidelity, contextually-aware human images. Our work provides compelling evidence that explicitly modeling skeletal structure is a fundamental step towards robust and plausible human image synthesis.

cs.CV↗

arXiv2025

Beyond Pixels: Benchmarking and Reward-Based Assessing Framework for Visual Spatial Aesthetics

In recent years, Image Quality Assessment (IQA) for AI-generated images (AIGI) has advanced rapidly; however, existing methods primarily target portraits and artistic images, lacking a systematic evaluation of interior scenes. We introduce Spatial Aesthetics, a paradigm that assesses the aesthetic quality of interior images along four dimensions: layout, harmony, lighting, and distortion. We construct SA-BENCH, the first benchmark for spatial aesthetics, comprising 18,000 images and 50,000 precise annotations. Employing SA-BENCH, we systematically evaluate current IQA methodologies and develop SA-IQA, through MLLM fine-tuning and a multidimensional fusion approach, as a comprehensive reward framework for assessing spatial aesthetics. We apply SA-IQA to two downstream tasks: (1) serving as a reward signal integrated with GRPO reinforcement learning to optimize the AIGC generation pipeline, and (2) Best-of-N selection to filter high-quality images and improve generation quality. Experiments indicate that SA-IQA significantly outperforms existing methods on SA-BENCH, setting a new standard for spatial aesthetics evaluation. Code is available at https://github.com/AlibabaResearch/SA-IQA.

cs.CV↗

arXiv2025

FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning

Federated learning (FL) enables collaborative training across clients while preserving privacy. While most existing FL methods assume homogeneous model architectures, client heterogeneity in both data and resources makes this assumption impractical, thus motivating model-heterogeneous FL. To address this problem, we propose Federated Representation Entanglement (FedRE), a framework built upon a novel form of client knowledge termed entangled representation. Specifically, each client aggregates its local representations into a single entangled representation using normalized random weights, and then applies the same weights to integrate the corresponding one-hot label encodings into an entangled-label encoding. Both are subsequently uploaded to the server to train a global classifier. During training, each entangled representation is supervised across categories via its entangled-label encoding, while random weights are re-sampled at each round to introduce diversity, alleviating overconfidence in the global classifier and yielding smoother decision boundaries. Moreover, each client uploads a single entangled representation along with its entangled-label encoding, mitigating the risk of representation inversion attacks and reducing communication overhead. Extensive experiments demonstrate that FedRE achieves an effective trade-off among model performance, privacy protection, and communication overhead. The codes are available at https://github.com/AIResearch-Group/FedRE.

cs.LG↗

arXiv2025

Mantis: A Versatile Vision-Language-Action Model with Disentangled Visual Foresight

Recent advances in Vision-Language-Action (VLA) models demonstrate that visual signals can effectively complement sparse action supervisions. However, letting VLA directly predict high-dimensional visual states can distribute model capacity and incur prohibitive training cost, while compressing visual states into more compact supervisory signals inevitably incurs information bottlenecks. Moreover, existing methods often suffer from poor comprehension and reasoning capabilities due to the neglect of language supervision. This paper introduces Mantis, a novel framework featuring a Disentangled Visual Foresight (DVF) to tackle these issues. Specifically, Mantis decouples visual foresight prediction from the backbone with the combination of meta queries and a diffusion Transformer (DiT) head. With the current visual state provided to the DiT via a residual connection, a simple next-state prediction objective enables the meta queries to automatically capture the latent actions that delineate the visual trajectory, and hence boost the learning of explicit actions. The disentanglement reduces the burden of the VLA backbone, enabling it to maintain comprehension and reasoning capabilities through language supervision. Empirically, pretrained on human manipulation videos, robot demonstrations, and image-text pairs, Mantis achieves a 96.7% success rate on LIBERO benchmark after fine-tuning, surpassing powerful baselines while exhibiting high convergence speed. Real-world evaluations show that Mantis outperforms $\pi_{0.5}$, a leading open-source VLA model, particularly in instruction-following capability, generalization to unseen instructions, and reasoning ability. Code and weights are released to support the open-source community.

cs.CV↗

arXiv2025

Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking

As valuable digital assets, deep neural networks necessitate robust ownership protection, positioning neural network watermarking (NNW) as a promising solution. Among various NNW approaches, weight-based methods are favored for their simplicity and practicality; however, they remain vulnerable to forging and overwriting attacks. To address those challenges, we propose NeuralMark, a robust method built around a hashed watermark filter. Specifically, we utilize a hash function to generate an irreversible binary watermark from a secret key, which is then used as a filter to select the model parameters for embedding. This design cleverly intertwines the embedding parameters with the hashed watermark, providing a robust defense against both forging and overwriting attacks. An average pooling is also incorporated to resist fine-tuning and pruning attacks. Furthermore, it can be seamlessly integrated into various neural network architectures, ensuring broad applicability. Theoretically, we analyze its security boundary. Empirically, we verify its effectiveness and robustness across 13 distinct Convolutional and Transformer architectures, covering five image classification tasks and one text generation task. The source codes are available at https://github.com/AIResearch-Group/NeuralMark.

cs.CR↗

arXiv2025

PMNO: A novel physics guided multi-step neural operator predictor for partial differential equations

Neural operators, which aim to approximate mappings between infinite-dimensional function spaces, have been widely applied in the simulation and prediction of physical systems. However, the limited representational capacity of network architectures, combined with their heavy reliance on large-scale data, often hinder effective training and result in poor extrapolation performance. In this paper, inspired by traditional numerical methods, we propose a novel physics guided multi-step neural operator (PMNO) architecture to address these challenges in long-horizon prediction of complex physical systems. Distinct from general operator learning methods, the PMNO framework replaces the single-step input with multi-step historical data in the forward pass and introduces an implicit time-stepping scheme based on the Backward Differentiation Formula (BDF) during backpropagation. This design not only strengthens the model's extrapolation capacity but also facilitates more efficient and stable training with fewer data samples, especially for long-term predictions. Meanwhile, a causal training strategy is employed to circumvent the need for multi-stage training and to ensure efficient end-to-end optimization. The neural operator architecture possesses resolution-invariant properties, enabling the trained model to perform fast extrapolation on arbitrary spatial resolutions. We demonstrate the superior predictive performance of PMNO predictor across a diverse range of physical systems, including 2D linear system, modeling over irregular domain, complex-valued wave dynamics, and reaction-diffusion processes. Depending on the specific problem setting, various neural operator architectures, including FNO, DeepONet, and their variants, can be seamlessly integrated into the PMNO framework.

cs.LG↗

arXiv2024

Two-dimensional fractional discrete NLS equations: dispersion relations, rogue waves, fundamental and vortex solitons

We introduce physically relevant new models of two-dimensional (2D) fractional lattice media accounting for the interplay of fractional intersite coupling and onsite self-focusing. Our approach features novel discrete fractional operators based on an appropriately modified definition of the continuous Riesz fractional derivative. The model of the 2D isotropic lattice employs the discrete fractional Laplacian, whereas the 2D anisotropic system incorporates discrete fractional derivatives acting independently along orthogonal directions with different L\'evy indices (LIs). We derive exact linear dispersion relations (DRs), and identify spectral bands that permit linear modes to exist, finding them to be similar to their continuous counterparts, apart from differences in the wavenumber range. Additionally, the modulational instability in the discrete models is studied in detail, and, akin to the linear DRs, it is found to align with the situation in continuous models. This consistency highlights the nature of our newly defined discrete fractional derivatives. Furthermore, using Gaussian inputs, we produce a variety of rogue-wave structures. By means of numerical methods, we systematically construct families of 2D fundamental and vortex solitons, and examine their stability. Fundamental solitons maintain the stability due to the discrete nature of the interactions, preventing the onset of the critical and supercritical collapse. On the other hand, vortex solitons are unstable in the isotropic lattice model. However, the anisotropic one -- in particular, its symmetric version with equal LIs acting in both directions -- maintains stable vortex solitons with winding numbers $S=1$ and $S=3$. The detailed results stress the robustness of the newly defined discrete fractional Laplacian in supporting well-defined soliton modes in the 2D lattice media.

nlin.PS↗

arXiv2024

Data-driven 2D stationary quantum droplets and wave propagations in the amended GP equation with two potentials via deep neural networks learning

In this paper, we develop a systematic deep learning approach to solve two-dimensional (2D) stationary quantum droplets (QDs) and investigate their wave propagation in the 2D amended Gross-Pitaevskii equation with Lee-Huang-Yang correction and two kinds of potentials. Firstly, we use the initial-value iterative neural network (IINN) algorithm for 2D stationary quantum droplets of stationary equations. Then the learned stationary QDs are used as the initial value conditions for physics-informed neural networks (PINNs) to explore their evolutions in the some space-time region. Especially, we consider two types of potentials, one is the 2D quadruple-well Gaussian potential and the other is the PT-symmetric HO-Gaussian potential, which lead to spontaneous symmetry breaking and the generation of multi-component QDs. The used deep learning method can also be applied to study wave propagations of other nonlinear physical models.

cs.LG↗

arXiv2024

Two-stage initial-value iterative physics-informed neural networks for simulating solitary waves of nonlinear wave equations

We propose a new two-stage initial-value iterative neural network (IINN) algorithm for solitary wave computations of nonlinear wave equations based on traditional numerical iterative methods and physics-informed neural networks (PINNs). Specifically, the IINN framework consists of two subnetworks, one of which is used to fit a given initial value, and the other incorporates physical information and continues training on the basis of the first subnetwork. Importantly, the IINN method does not require any additional data information including boundary conditions, apart from the given initial value. Corresponding theoretical guarantees are provided to demonstrate the effectiveness of our IINN method. The proposed IINN method is efficiently applied to learn some types of solutions in different nonlinear wave equations, including the one-dimensional (1D) nonlinear Schr\"odinger equations (NLS) equation (with and without potentials), the 1D saturable NLS equation with PT -symmetric optical lattices, the 1D focusing-defocusing coupled NLS equations, the KdV equation, the two-dimensional (2D) NLS equation with potentials, the 2D amended GP equation with a potential, the (2+1)-dimensional KP equation, and the 3D NLS equation with a potential. These applications serve as evidence for the efficacy of our method. Finally, by comparing with the traditional methods, we demonstrate the advantages of the proposed IINN method.

physics.comp-ph↗

arXiv2023

Data-driven localized waves and parameter discovery in the massive Thirring model via extended physics-informed neural networks with interface zones

In this paper, we study data-driven localized wave solutions and parameter discovery in the massive Thirring (MT) model via the deep learning in the framework of physics-informed neural networks (PINNs) algorithm. Abundant data-driven solutions including soliton of bright/dark type, breather and rogue wave are simulated accurately and analyzed contrastively with relative and absolute errors. For higher-order localized wave solutions, we employ the extended PINNs (XPINNs) with domain decomposition to capture the complete pictures of dynamic behaviors such as soliton collisions, breather oscillations and rogue-wave superposition. In particular, we modify the interface line in domain decomposition of XPINNs into a small interface zone and introduce the pseudo initial, residual and gradient conditions as interface conditions linked adjacently with individual neural networks. Then this modified approach is applied successfully to various solutions ranging from bright-bright soliton, dark-dark soliton, dark-antidark soliton, general breather, Kuznetsov-Ma breather and second-order rogue wave. Experimental results show that this improved version of XPINNs reduce the complexity of computation with faster convergence rate and keep the quality of learned solutions with smoother stitching performance as well. For the inverse problems, the unknown coefficient parameters of linear and nonlinear terms in the MT model are identified accurately with and without noise by using the classical PINNs algorithm.

nlin.PS↗

arXiv2023

Deep learning soliton dynamics and complex potentials recognition for 1D and 2D PT-symmetric saturable nonlinear Schrödinger equations

In this paper, we firstly extend the physics-informed neural networks (PINNs) to learn data-driven stationary and non-stationary solitons of 1D and 2D saturable nonlinear Schrödinger equations (SNLSEs) with two fundamental PT-symmetric Scarf-II and periodic potentials in optical fibers. Secondly, the data-driven inverse problems are studied for PT-symmetric potential functions discovery rather than just potential parameters in the 1D and 2D SNLSEs. Particularly, we propose a modified PINNs (mPINNs) scheme to identify directly the PT potential functions of the 1D and 2D SNLSEs by the solution data. And the inverse problems about 1D and 2D PT -symmetric potentials depending on propagation distance z are also investigated using mPINNs method. We also identify the potential functions by the PINNs applied to the stationary equation of the SNLSE. Furthermore, two network structures are compared under different parameter conditions such that the predicted PT potentials can achieve the similar high accuracy. These results illustrate that the established deep neural networks can be successfully used in 1D and 2D SNLSEs with high accuracies. Moreover, some main factors affecting neural networks performance are discussed in 1D and 2D PT Scarf-II and periodic potentials, including activation functions, structures of the networks, and sizes of the training data. In particular, twelve different nonlinear activation functions are in detail analyzed containing the periodic and non-periodic functions such that it is concluded that selecting activation functions according to the form of solution and equation usually can achieve better effect.

nlin.PS↗

arXiv2023

Symmetry breaking bifurcations and excitations of solitons in linearly coupled NLS equations with PT-symmetric potentials

We address symmetry breaking bifurcations (SBBs) in the ground-state (GS) and dipole-mode (DM) solitons of the 1D linearly coupled NLS equations, modeling the propagation of light in a dual-core planar waveguide with the Kerr nonlinearity and two types of PT-symmetric potentials. The PT-symmetric potential is employed to obtained different types of solutions. A supercritical pitchfork bifurcation occurs in families of symmetric solutions of both the GS and DM types. A novel feature of the system is interplay between breakings of the PT and inter-core symmetries. Stability of symmetric GS and DM modes and their asymmetric counterparts, produced by SBBs of both types, is explored via the linear-stability analysis and simulations. It is found that the instability of PT-symmetric solutions takes place prior to the inter-core symmetry breaking. Surprisingly, stable inter-core-symmetric GS solutions may remain stable while the PT symmetry is broken. Fully asymmetric GS and DM solitons are only partially stable. Moreover, we construct symmetric and asymmetric GS solitons under the action of a pure imaginary localized potential, for which the SBB is subcritical. These results exhibit that stable solitons can still be found in dissipative systems. Finally, excitations of symmetric and asymmetric GS solitons are investigated by making the potential's parameters or the system's coupling constant functions, showing that GS solitons can be converted from an asymmetric shape onto a symmetric one under certain conditions. These results may pave the way for the study of linear and nonlinear phenomena in a dual-core planar waveguide with PT potential and related experimental designs.

nlin.PS↗

arXiv2023

Formations and dynamics of two-dimensional spinning asymmetric quantum droplets controlled by a PT-symmetric potential

In this paper, vortex solitons are produced for a variety of 2D spinning quantum droplets (QDs) in a PT-symmetric potential, modeled by the amended Gross-Pitaevskii equation with Lee-Huang-Yang corrections. In particular, exact QD states are obtained under certain parameter constraints, providing a guide to finding the respective generic family. In a parameter region of the unbroken PT symmetry, different families of QDs originating from the linear modes are obtained in the form of multipolar and vortex droplets at low and high values of the norm, respectively, and their stability is investigated. In the spinning regime, QDs become asymmetric above a critical rotation frequency, most of them being stable. The effect of the PT -symmetric potential on the spinning and nonspinning QDs is explored by varying the strength of the gain-loss distribution. Generally, spinning QDs trapped in the PT -symmetric potential exhibit asymmetry due to the energy flow affected by the interplay of the gain-loss distribution and rotation. Finally, interactions between spinning or nonspinning QDs are explored, exhibiting elastic collisions under certain conditions.

quant-ph↗

arXiv2023

Superconductivity above 30 K achieved in dense scandium

Superconductivity is one of most intriguing quantum phenomena, and the quest for elemental superconductors with high critical temperature (Tc) is of great scientific significance due to their relatively simple material composition and the underlying mechanism. Here we report the experimental discovery of densely compressed scandium (Sc) becoming the first elemental superconductor with Tc breaking into 30 K range, which is comparable to the Tc values of the classic La-Ba-Cu-O or LaFeAsO superconductors. Our results show that Tconset of Sc increases from ~3 K at around 43 GPa to ~32 K at about 283 GPa (Tczero ~ 31 K), which is well above liquid neon temperature. Interestingly measured Tc shows no sign of saturation up to the maximum pressure achieved in our experiments, indicating that Tc might be even higher upon further compression.

cond-mat.supr-con↗