arXiv subjects

Wang Zhang

Publications and source records attributed to Wang Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2024

HybridFlow: A Flexible and Efficient RLHF Framework

Reinforcement Learning from Human Feedback (RLHF) is widely used in Large Language Model (LLM) alignment. Traditional RL can be modeled as a dataflow, where each node represents computation of a neural network (NN) and each edge denotes data dependencies between the NNs. RLHF complicates the dataflow by expanding each node into a distributed LLM training or generation program, and each edge into a many-to-many multicast. Traditional RL frameworks execute the dataflow using a single controller to instruct both intra-node computation and inter-node communication, which can be inefficient in RLHF due to large control dispatch overhead for distributed intra-node computation. Existing RLHF systems adopt a multi-controller paradigm, which can be inflexible due to nesting distributed computation and data communication. We propose HybridFlow, which combines single-controller and multi-controller paradigms in a hybrid manner to enable flexible representation and efficient execution of the RLHF dataflow. We carefully design a set of hierarchical APIs that decouple and encapsulate computation and data dependencies in the complex RLHF dataflow, allowing efficient operation orchestration to implement RLHF algorithms and flexible mapping of the computation onto various devices. We further design a 3D-HybridEngine for efficient actor model resharding between training and generation phases, with zero memory redundancy and significantly reduced communication overhead. Our experimental results demonstrate 1.53$\times$~20.57$\times$ throughput improvement when running various RLHF algorithms using HybridFlow, as compared with state-of-the-art baselines. HybridFlow source code will be available at https://github.com/volcengine/verl.

cs.LG↗

arXiv2024

SEMDR: A Semantic-Aware Dual Encoder Model for Legal Judgment Prediction with Legal Clue Tracing

Legal Judgment Prediction (LJP) aims to form legal judgments based on the criminal fact description. However, researchers struggle to classify confusing criminal cases, such as robbery and theft, which requires LJP models to distinguish the nuances between similar crimes. Existing methods usually design handcrafted features to pick up necessary semantic legal clues to make more accurate legal judgment predictions. In this paper, we propose a Semantic-Aware Dual Encoder Model (SEMDR), which designs a novel legal clue tracing mechanism to conduct fine-grained semantic reasoning between criminal facts and instruments. Our legal clue tracing mechanism is built from three reasoning levels: 1) Lexicon-Tracing, which aims to extract criminal facts from criminal descriptions; 2) Sentence Representation Learning, which contrastively trains language models to better represent confusing criminal facts; 3) Multi-Fact Reasoning, which builds a reasons graph to propagate semantic clues among fact nodes to capture the subtle difference among criminal facts. Our legal clue tracing mechanism helps SEMDR achieve state-of-the-art on the CAIL2018 dataset and shows its advance in few-shot scenarios. Our experiments show that SEMDR has a strong ability to learn more uniform and distinguished representations for criminal facts, which helps to make more accurate predictions on confusing criminal cases and reduces the model uncertainty during making judgments. All codes will be released via GitHub.

cs.CL↗

arXiv2024

Applications of Multi-Agent Deep Reinforcement Learning Communication in Network Management: A Survey

With the advancement of artificial intelligence technology, the automation of network management, also known as Autonomous Driving Networks (ADN), is gaining widespread attention. The network management has shifted from traditional homogeneity and centralization to heterogeneity and decentralization. Multi-agent deep reinforcement learning (MADRL) allows agents to make decisions based on local observations independently. This approach is in line with the needs of automation and has garnered significant attention from academia and industry. In a distributed environment, information interaction between agents can effectively address the non-stationarity problem of multiple agents and promote cooperation. Therefore, in this survey, we first examined the application of MADRL in network management, including specific application fields such as traffic engineering, wireless network access, power control, and network security. Then, we conducted a detailed analysis of communication behavior between agents, including communication schemes, communication content construction, communication object selection, message processing, and communication constraints. Finally, we discussed the open issues and future research directions of agent communication in MADRL for future network management and ADN applications.

cs.NI↗

arXiv2024

Next-Generation Simulation Illuminates Scientific Problems of Organised Complexity

As artificial intelligence becomes increasingly prevalent in scientific research, data-driven methodologies appear to overshadow traditional approaches in resolving scientific problems. In this Perspective, we revisit a classic classification of scientific problems and acknowledge that a series of unresolved problems remain. Throughout the history of researching scientific problems, scientists have continuously formed new paradigms facilitated by advances in data, algorithms, and computational power. To better tackle unresolved problems, especially those of organised complexity, a novel paradigm is necessitated. While recognising that the strengths of new paradigms have expanded the scope of resolvable scientific problems, we aware that the continued advancement of data, algorithms, and computational power alone is hardly to bring a new paradigm. We posit that the integration of paradigms, which capitalises on the strengths of each, represents a promising approach. Specifically, we focus on next-generation simulation (NGS), which can serve as a platform to integrate methods from different paradigms. We propose a methodology, sophisticated behavioural simulation (SBS), to realise it. SBS represents a higher level of paradigms integration based on foundational models to simulate complex systems, such as social systems involving sophisticated human strategies and behaviours. NGS extends beyond the capabilities of traditional mathematical modelling simulations and agent-based modelling simulations, and therefore, positions itself as a potential solution to problems of organised complexity in complex systems.

cs.AI↗

arXiv2024

A Light-weight Transformer-based Self-supervised Matching Network for Heterogeneous Images

Matching visible and near-infrared (NIR) images remains a significant challenge in remote sensing image fusion. The nonlinear radiometric differences between heterogeneous remote sensing images make the image matching task even more difficult. Deep learning has gained substantial attention in computer vision tasks in recent years. However, many methods rely on supervised learning and necessitate large amounts of annotated data. Nevertheless, annotated data is frequently limited in the field of remote sensing image matching. To address this challenge, this paper proposes a novel keypoint descriptor approach that obtains robust feature descriptors via a self-supervised matching network. A light-weight transformer network, termed as LTFormer, is designed to generate deep-level feature descriptors. Furthermore, we implement an innovative triplet loss function, LT Loss, to enhance the matching performance further. Our approach outperforms conventional hand-crafted local feature descriptors and proves equally competitive compared to state-of-the-art deep learning-based methods, even amidst the shortage of annotated data.

cs.CV↗

arXiv2024

Graphs Generalization under Distribution Shifts

Traditional machine learning methods heavily rely on the independent and identically distribution assumption, which imposes limitations when the test distribution deviates from the training distribution. To address this crucial issue, out-of-distribution (OOD) generalization, which aims to achieve satisfactory generalization performance when faced with unknown distribution shifts, has made a significant process. However, the OOD method for graph-structured data currently lacks clarity and remains relatively unexplored due to two primary challenges. Firstly, distribution shifts on graphs often occur simultaneously on node attributes and graph topology. Secondly, capturing invariant information amidst diverse distribution shifts proves to be a formidable challenge. To overcome these obstacles, in this paper, we introduce a novel framework, namely Graph Learning Invariant Domain genERation (GLIDER). The goal is to (1) diversify variations across domains by modeling the potential seen or unseen variations of attribute distribution and topological structure and (2) minimize the discrepancy of the variation in a representation space where the target is to predict semantic labels. Extensive experiment results indicate that our model outperforms baseline methods on node-level OOD generalization across domains in distribution shift on node features and topological structures simultaneously.

cs.LG↗

arXiv2023

One step closer to unbiased aleatoric uncertainty estimation

Neural networks are powerful tools in various applications, and quantifying their uncertainty is crucial for reliable decision-making. In the deep learning field, the uncertainties are usually categorized into aleatoric (data) and epistemic (model) uncertainty. In this paper, we point out that the existing popular variance attenuation method highly overestimates aleatoric uncertainty. To address this issue, we propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method.

cs.LG↗

arXiv2023

3D Printed Multilayer Structures for High Numerical Aperture Achromatic Metalenses

Flat optics consisting of nanostructures of high-refractive-index materials produce lenses with thin form factors that tend to operate only at specific wavelengths. Recent attempts to achieve achromatic lenses uncover a trade-off between the numerical aperture (NA) and bandwidth, which limits performance. Here we propose a new approach to design high NA, broadband and polarization-insensitive multilayer achromatic metalenses (MAM). We combine topology optimization and full wave simulations to inversely design MAMs and fabricate the structures in low-refractive-index materials by two-photon polymerization lithography. MAMs measuring 20 micrometer in diameter operating in the visible range of 400-800 nm with 0.5 NA and 0.7 NA were achieved with efficiencies of up to 42%. We demonstrate broadband imaging performance of the fabricated MAM under white light, and RGB narrowband illuminations. These results highlight the potential of the 3D printed multilayer structures for realizing broadband and multi-functional meta-devices with inverse design.

physics.optics↗

arXiv2023

ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction

Deep neural networks (DNN) have shown great capacity of modeling a dynamical system; nevertheless, they usually do not obey physics constraints such as conservation laws. This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling to endow the invariant properties. ConCerNet consists of two steps: (i) a contrastive learning method to automatically capture the system invariants (i.e. conservation properties) along the trajectory observations; (ii) a neural projection layer to guarantee that the learned dynamics models preserve the learned invariants. We theoretically prove the functional relationship between the learned latent representation and the unknown system invariant function. Experiments show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics by a large margin. With neural network based parameterization and no dependence on prior knowledge, our method can be extended to complex and large-scale dynamics by leveraging an autoencoder.

cs.LG↗

arXiv2022

Colorful Optical Vortices with White Light Illumination

The orbital angular momentum (OAM) of light holds great promise for applications in optical communication, super-resolution imaging, and high-dimensional quantum computing. However, the spatio-temporal coherence of the light source has been essential for generating OAM beams, as incoherent ambient light would result in polychromatic and obscured OAM beams in the visible spectrum. Here, we extend the applications of OAM to ambient lighting conditions. By miniaturizing spiral phase plates and integrating them with structural color filters, we achieve spatio-temporal coherence using only an incoherent white light source. These optical elements act as building blocks that encode both color and OAM information in the form of colorful optical vortices. Thus, pairs of transparent substrates that contain matching positions of these vortices constitute a reciprocal optical lock and key system. Due to the multiple helical eigenstates of OAM, the pairwise coupling can be further extended to form a one-to-many matching and validation scheme. Generating and decoding colorful optical vortices with broadband white light could find potential applications in anti-counterfeiting, optical metrology, high-capacity optical encryption, and on-chip 3D photonic devices.

physics.optics↗

arXiv2022

Tri-Functional Metasurface for Phase, Amplitude, and Luminescence Control

In optical anti-counterfeiting, several distinct optically variable devices (OVDs) are often concurrently employed to compensate for the insufficient security level of constituent OVDs. Alternatively, metasurfaces that exhibit multiple optical responses effectively combine multiple OVDs into one, thus significantly enhancing their security and hindering fraudulent replication. This work demonstrates the simultaneous control of three separate optical responses, i.e., phase, amplitude, and luminescence, using anisotropic gap-plasmon metasurfaces. Due to the incorporated geometric anisotropy, the designed structure exhibits distinct responses under x- and y-polarized light, revealing either a color image, or a holographic projection in the far field. Furthermore, inserting upconversion nanoparticles (UCNPs) into the dielectric gaps of the structures, the designed metasurface is able to generate a third luminescent image upon illumination with the near-infrared light. The stochastic distribution of the UCNPs constitutes a unique fingerprint, achieving a physically unclonable function (PUF) layer. Crucially, our triple-mode metasurface requires only readily attainable equipment such as a macro-lens/camera and a laser pointer to read most of the channels, thus paving the way towards highly secure and easy-to-authenticate metasurface-driven OVDs (mOVDs).

physics.optics↗

arXiv2022

Overcoming Van der Waals Forces in reconfigurable nanostructures

Reconfigurable metamaterials require constituent nanostructures to demonstrate switching of shapes with external stimuli. For generality, such nanostructures would touch and stick to other surfaces in one of its configurations. Yet, a longstanding challenge is in overcoming this stiction caused by Van der Waals forces, which impedes shape recovery. Here, we introduce a stiff yet self-recovering material system based on acrylic acid, and tested it in high-aspect ratio structures, where recovery is weak. This designer material has a storage modulus of ~5.2 GPa at room temperature and ~90 MPa in the rubbery state at 150 Celsius, an order of magnitude higher than previous reports. A high-resolution resin for two-photon lithography was developed based on this polymer system, enabling 3D printing of nanopillars with diameters of ~400 nm and aspect ratio as high as ~10. Experimentally, we observed self-recovery as collapsed and touching structures overcome stiction to stand back up. We developed a theoretical model to explain the recoverability of these sub-micron structures. Reconfigurable structural colour prints and holograms were demonstrated, indicating potential applications of the material system as a shape memory polymer suitable for sub-micron reconfigurable metamaterials.

physics.app-ph↗

arXiv2022

Reconfiguring colours of single relief structures by directional stretching

Colour changes can be achieved by straining photonic crystals or gratings embedded in stretchable materials. However, the multiple repeat units and the need for a volumetric assembly of nanostructures limit the density of information content. Inspired by surface reliefs on oracle bones and music records as means of information archival, here we endow surface-relief elastomers with multiple sets of information that are accessible by mechanical straining along in-plane axes. Distinct from Bragg diffraction effects from periodic structures, we report trenches that generate colour due to variations in trench depth, enabling individual trench segments to support a single colour. Using 3D printed cuboids, we replicated trenches of varying geometric parameters in elastomers. These parameters determine the initial colour (or lack thereof), the response to capillary forces, and the appearance when strained along or across the trenches. Strain induces modulation in trench depth or the opening and closure of a trench, resulting in surface reliefs with up to six distinct states, and an initially featureless surface that reveals two distinct images when stretched along different axes. The highly reversible structural colours are promising in optical data archival, anti-counterfeiting, and strain-sensing applications.

physics.optics↗

arXiv2022

High-order Photonic Cavity Modes Enabled 3D Structural Color

It remains a challenge to directly print three-dimensional arbitrary shapes that exhibit structural colors at the micrometer scale. Woodpile photonic crystals (WPCs) fabricated via two-photon lithography (TPL) are promising as building blocks to produce 3D geometries that generate structural colors due to their ability to exhibit either omnidirectional or anisotropic photonic stopbands. However, existing approaches have focused on achieving structural colors when illuminating WPCs from the top, which necessitates print resolutions beyond the limit of commercial TPL and/or post-processing techniques. Here, we devised a new strategy to support high-order photonic cavity modes upon side-illumination on WPCs that surprisingly generate large reflectance peaks in the visible spectrum. Based on that, we demonstrate one-step printing of 3D photonic structural colors without requiring post-processing or subwavelength features. Vivid colors with reflectance peaks exhibiting a full width at half maximum of ~25 nm, a maximum reflectance of 50%, gamut of ~85% of sRGB, and large viewing angles, were achieved. In addition, we also demonstrated voxel-level manipulation and control of colors in arbitrary-shaped 3D objects constituted with WPCs as unit cells, which has great potential for applications in dynamic color displays, colorimetric sensing, anti-counterfeiting, and light-matter interaction platforms.

physics.optics↗

arXiv2022

Representation Learning on Heterostructures via Heterogeneous Anonymous Walks

Capturing structural similarity has been a hot topic in the field of network embedding recently due to its great help in understanding the node functions and behaviors. However, existing works have paid very much attention to learning structures on homogeneous networks while the related study on heterogeneous networks is still a void. In this paper, we try to take the first step for representation learning on heterostructures, which is very challenging due to their highly diverse combinations of node types and underlying structures. To effectively distinguish diverse heterostructures, we firstly propose a theoretically guaranteed technique called heterogeneous anonymous walk (HAW) and its variant coarse HAW (CHAW). Then, we devise the heterogeneous anonymous walk embedding (HAWE) and its variant coarse HAWE in a data-driven manner to circumvent using an extremely large number of possible walks and train embeddings by predicting occurring walks in the neighborhood of each node. Finally, we design and apply extensive and illustrative experiments on synthetic and real-world networks to build a benchmark on heterostructure learning and evaluate the effectiveness of our methods. The results demonstrate our methods achieve outstanding performance compared with both homogeneous and heterogeneous classic methods, and can be applied on large-scale networks.

cs.SI↗

arXiv2021

Robust Deep Reinforcement Learning through Adversarial Loss

Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs, which raises concerns about deploying such agents in the real world. To address this issue, we propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against $l_p$-norm bounded adversarial attacks. Our framework is compatible with popular deep reinforcement learning algorithms and we demonstrate its performance with deep Q-learning, A3C and PPO. We experiment on three deep RL benchmarks (Atari, MuJoCo and ProcGen) to show the effectiveness of our robust training algorithm. Our RADIAL-RL agents consistently outperform prior methods when tested against attacks of varying strength and are more computationally efficient to train. In addition, we propose a new evaluation method called Greedy Worst-Case Reward (GWC) to measure attack agnostic robustness of deep RL agents. We show that GWC can be evaluated efficiently and is a good estimate of the reward under the worst possible sequence of adversarial attacks. All code used for our experiments is available at https://github.com/tuomaso/radial_rl_v2.

cs.LG↗

arXiv2021

Generation of robust spatiotemporal optical vortices with transverse orbital angular momentum beyond $10^2$

Recently, photons have been observed to possess transverse orbital angular momentum (OAM); however, it is unclear as whether they can hold a transverse OAM higher than 1. Here, we theoretically and experimentally demonstrate that high-order spatiotemporal Bessel optical vortices (STBOVs) can stably carry transverse OAM even beyond $10^2$. Through the inverse design of the spiral phase, an STBOV of any order can be controllably generated using a 4f pulse shaper. In contrast to conventional longitudinal OAM, the vector direction of the transverse OAM can be distinguished by the unique time-symmetrical evolution of STBOVs. More interestingly, the stability of STBOVs improves with their increasing orders owing to enhanced space-time coupling, making these beams particularly suitable for the generation of ultra-high transverse OAM. Our work paves the way for further research and application of this unique OAM of photons.

physics.optics↗

arXiv2021

A Survey on Role-Oriented Network Embedding

Recently, Network Embedding (NE) has become one of the most attractive research topics in machine learning and data mining. NE approaches have achieved promising performance in various of graph mining tasks including link prediction and node clustering and classification. A wide variety of NE methods focus on the proximity of networks. They learn community-oriented embedding for each node, where the corresponding representations are similar if two nodes are closer to each other in the network. Meanwhile, there is another type of structural similarity, i.e., role-based similarity, which is usually complementary and completely different from the proximity. In order to preserve the role-based structural similarity, the problem of role-oriented NE is raised. However, compared to community-oriented NE problem, there are only a few role-oriented embedding approaches proposed recently. Although less explored, considering the importance of roles in analyzing networks and many applications that role-oriented NE can shed light on, it is necessary and timely to provide a comprehensive overview of existing role-oriented NE methods. In this review, we first clarify the differences between community-oriented and role-oriented network embedding. Afterwards, we propose a general framework for understanding role-oriented NE and a two-level categorization to better classify existing methods. Then, we select some representative methods according to the proposed categorization and briefly introduce them by discussing their motivation, development and differences. Moreover, we conduct comprehensive experiments to empirically evaluate these methods on a variety of role-related tasks including node classification and clustering (role discovery), top-k similarity search and visualization using some widely used synthetic and real-world datasets...

cs.SI↗