Searcharxiv⌕ Search

arXiv subjects

Wenjia Zhang

Publications and source records attributed to Wenjia Zhang.

29 records · Page 2Linked to original sources
arXiv2023

Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL

Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets without interacting with the environment. However, the performance of existing offline RL algorithms heavily depends on the scale and state-action space coverage of datasets. Real-world data collection is often expensive and uncontrollable, leading to small and narrowly covered datasets and posing significant challenges for practical deployments of offline RL. In this paper, we provide a new insight that leveraging the fundamental symmetry of system dynamics can substantially enhance offline RL performance under small datasets. Specifically, we propose a Time-reversal symmetry (T-symmetry) enforced Dynamics Model (TDM), which establishes consistency between a pair of forward and reverse latent dynamics. TDM provides both well-behaved representations for small datasets and a new reliability measure for OOD samples based on compliance with the T-symmetry. These can be readily used to construct a new offline RL algorithm (TSRL) with less conservative policy constraints and a reliable latent space data augmentation procedure. Based on extensive experiments, we find TSRL achieves great performance on small benchmark datasets with as few as 1% of the original samples, which significantly outperforms the recent offline RL algorithms in terms of data efficiency and generalizability.Code is available at: https://github.com/pcheng2/TSRL

cs.LG↗
arXiv2023

NarrativePlay: Interactive Narrative Understanding

In this paper, we introduce NarrativePlay, a novel system that allows users to role-play a fictional character and interact with other characters in narratives such as novels in an immersive environment. We leverage Large Language Models (LLMs) to generate human-like responses, guided by personality traits extracted from narratives. The system incorporates auto-generated visual display of narrative settings, character portraits, and character speech, greatly enhancing user experience. Our approach eschews predefined sandboxes, focusing instead on main storyline events extracted from narratives from the perspective of a user-selected character. NarrativePlay has been evaluated on two types of narratives, detective and adventure stories, where users can either explore the world or improve their favorability with the narrative characters through conversations.

cs.CL↗
arXiv2023

General Spatial Photonic Ising Machine Based on Interaction Matrix Eigendecomposition Method

The spatial photonic Ising machine has achieved remarkable advancements in solving combinatorial optimization problems. However, it still remains a huge challenge to flexibly mapping an arbitrary problem to Ising model. In this paper, we propose a general spatial photonic Ising machine based on interaction matrix eigendecomposition method. Arbitrary interaction matrix can be configured in the two-dimensional Fourier transformation based spatial photonic Ising model by using values generated by matrix eigendecomposition. The error in the structural representation of the Hamiltonian decreases substantially with the growing number of eigenvalues utilized to form the Ising machine. In combination with the optimization algorithm, as low as 65% of the eigenvalues is required by intensity modulation to guarantee the best probability of optimal solution for a 20-vertex graph Max-cut problem, and this probability decreases to below 20% for zero best chance. Our work provides a viable approach for spatial photonic Ising machines to solve arbitrary combinatorial optimization problems with the help of multi-dimensional optical property.

cs.ET↗
arXiv2023

NewsQuote: A Dataset Built on Quote Extraction and Attribution for Expert Recommendation in Fact-Checking

To enhance the ability to find credible evidence in news articles, we propose a novel task of expert recommendation, which aims to identify trustworthy experts on a specific news topic. To achieve the aim, we describe the construction of a novel NewsQuote dataset consisting of 24,031 quote-speaker pairs that appeared on a COVID-19 news corpus. We demonstrate an automatic pipeline for speaker and quote extraction via a BERT-based Question Answering model. Then, we formulate expert recommendations as document retrieval task by retrieving relevant quotes first as an intermediate step for expert identification, and expert retrieval by directly retrieving sources based on the probability of a query conditional on a candidate expert. Experimental results on NewsQuote show that document retrieval is more effective in identifying relevant experts for a given news topic compared to expert retrieval

cs.IR↗
arXiv2023

Discriminator-Guided Model-Based Offline Imitation Learning

Offline imitation learning (IL) is a powerful method to solve decision-making problems from expert demonstrations without reward labels. Existing offline IL methods suffer from severe performance degeneration under limited expert data. Including a learned dynamics model can potentially improve the state-action space coverage of expert data, however, it also faces challenging issues like model approximation/generalization errors and suboptimality of rollout data. In this paper, we propose the Discriminator-guided Model-based offline Imitation Learning (DMIL) framework, which introduces a discriminator to simultaneously distinguish the dynamics correctness and suboptimality of model rollout data against real expert demonstrations. DMIL adopts a novel cooperative-yet-adversarial learning strategy, which uses the discriminator to guide and couple the learning process of the policy and dynamics model, resulting in improved model performance and robustness. Our framework can also be extended to the case when demonstrations contain a large proportion of suboptimal data. Experimental results show that DMIL and its extension achieve superior performance and robustness compared to state-of-the-art offline IL methods under small datasets.

cs.LG↗
arXiv2022

A Manifold View of Adversarial Risk

The adversarial risk of a machine learning model has been widely studied. Most previous works assume that the data lies in the whole ambient space. We propose to take a new angle and take the manifold assumption into consideration. Assuming data lies in a manifold, we investigate two new types of adversarial risk, the normal adversarial risk due to perturbation along normal direction, and the in-manifold adversarial risk due to perturbation within the manifold. We prove that the classic adversarial risk can be bounded from both sides using the normal and in-manifold adversarial risks. We also show with a surprisingly pessimistic case that the standard adversarial risk can be nonzero even when both normal and in-manifold risks are zero. We finalize the paper with empirical studies supporting our theoretical results. Our results suggest the possibility of improving the robustness of a classifier by only focusing on the normal adversarial risk.

cs.LG↗
arXiv2021

Supervised Contrastive Learning for Multimodal Unreliable News Detection in COVID-19 Pandemic

As the digital news industry becomes the main channel of information dissemination, the adverse impact of fake news is explosively magnified. The credibility of a news report should not be considered in isolation. Rather, previously published news articles on the similar event could be used to assess the credibility of a news report. Inspired by this, we propose a BERT-based multimodal unreliable news detection framework, which captures both textual and visual information from unreliable articles utilising the contrastive learning strategy. The contrastive learner interacts with the unreliable news classifier to push similar credible news (or similar unreliable news) closer while moving news articles with similar content but opposite credibility labels away from each other in the multimodal embedding space. Experimental results on a COVID-19 related dataset, ReCOVery, show that our model outperforms a number of competitive baseline in unreliable news detection.

cs.CL↗
arXiv2021

Quadrature Photonic Spatial Ising Machine

The mining in physics and biology for accelerating the hardcore algorithm to solve non-deterministic polynomial (NP) hard problems has inspired a great amount of special-purpose ma-chine models. Ising machine has become an efficient solver for various combinatorial optimizationproblems. As a computing accelerator, large-scale photonic spatial Ising machine have great advan-tages and potentials due to excellent scalability and compact system. However, current fundamentallimitation of photonic spatial Ising machine is the configuration flexibility of problem implementationin the accelerator model. Arbitrary spin interactions is highly desired for solving various NP hardproblems. Moreover, the absence of external magnetic field in the proposed photonic Ising machinewill further narrow the freedom to map the optimization applications. In this paper, we propose anovel quadrature photonic spatial Ising machine to break through the limitation of photonic Isingaccelerator by synchronous phase manipulation in two and three sections. Max-cut problem solutionwith graph order of 100 and density from 0.5 to 1 is experimentally demonstrated after almost 100iterations. We derive and verify using simulation the solution for Max-cut problem with more than1600 nodes and the system tolerance for light misalignment. Moreover, vertex cover problem, modeled as an Ising model with external magnetic field, has been successfully implemented to achievethe optimal solution. Our work suggests flexible problem solution by large-scale photonic spatialIsing machine.

cs.ET↗
arXiv2021

Photonic Convolution Neural Network Based on Interleaved Time-Wavelength Modulation

Convolution neural network (CNN), as one of the most powerful and popular technologies, has achieved remarkable progress for image and video classification since its invention in 1989. However, with the high definition video-data explosion, convolution layers in the CNN architecture will occupy a great amount of computing time and memory resources due to high computation complexity of matrix multiply accumulate operation. In this paper, a novel integrated photonic CNN is proposed based on double correlation operations through interleaved time-wavelength modulation. Micro-ring based multi-wavelength manipulation and single dispersion medium are utilized to realize convolution operation and replace the conventional optical delay lines. 200 images are tested in MNIST datasets with accuracy of 85.5% in our photonic CNN versus 86.5% in 64-bit computer.We also analyze the computing error of photonic CNN caused by various micro-ring parameters, operation baud rates and the characteristics of micro-ring weighting bank. Furthermore, a tensor processing unit based on 4x4 mesh with 1.2 TOPS (operation per second when 100% utilization) computing capability at 20G baud rate is proposed and analyzed to form a paralleled photonic CNN.

cs.ET↗
arXiv2021

Stability of SGD: Tightness Analysis and Improved Bounds

Stochastic Gradient Descent (SGD) based methods have been widely used for training large-scale machine learning models that also generalize well in practice. Several explanations have been offered for this generalization performance, a prominent one being algorithmic stability [18]. However, there are no known examples of smooth loss functions for which the analysis can be shown to be tight. Furthermore, apart from the properties of the loss function, data distribution has also been shown to be an important factor in generalization performance. This raises the question: is the stability analysis of [18] tight for smooth functions, and if not, for what kind of loss functions and data distributions can the stability analysis be improved? In this paper we first settle open questions regarding tightness of bounds in the data-independent setting: we show that for general datasets, the existing analysis for convex and strongly-convex loss functions is tight, but it can be improved for non-convex loss functions. Next, we give a novel and improved data-dependent bounds: we show stability upper bounds for a large class of convex regularized loss functions, with negligible regularization parameters, and improve existing data-dependent bounds in the non-convex setting. We hope that our results will initiate further efforts to better understand the data-dependent setting under non-convex loss functions, leading to an improved understanding of the generalization abilities of deep networks.

cs.LG↗