arXiv subjects

Sida Zhou

Publications and source records attributed to Sida Zhou.

3 recordsLinked to original sources

arXiv2024

WebCanvas: Benchmarking Web Agents in Online Environments

For web agents to be practically useful, they must adapt to the continuously evolving web environment characterized by frequent updates to user interfaces and content. However, most existing benchmarks only capture the static aspects of the web. To bridge this gap, we introduce WebCanvas, an innovative online evaluation framework for web agents that effectively addresses the dynamic nature of web interactions. WebCanvas contains three main components to facilitate realistic assessments: (1) A novel evaluation metric which reliably capture critical intermediate actions or states necessary for task completions while disregarding noise caused by insignificant events or changed web-elements. (2) A benchmark dataset called Mind2Web-Live, a refined version of original Mind2Web static dataset containing 542 tasks with 2439 intermediate evaluation states; (3) Lightweight and generalizable annotation tools and testing pipelines that enables the community to collect and maintain the high-quality, up-to-date dataset. Building on WebCanvas, we open-source an agent framework with extensible modules for reasoning, providing a foundation for the community to conduct online inference and evaluations. Our best-performing agent achieves a task success rate of 23.1% and a task completion rate of 48.8% on the Mind2Web-Live test set. Additionally, we analyze the performance discrepancies across various websites, domains, and experimental environments. We encourage the community to contribute further insights on online agent evaluation, thereby advancing this field of research.

cs.CL↗

arXiv2019

Empirical effect of graph embeddings on fraud detection/ risk mitigation

Graph embedding technics are studied with interest on public datasets, such as BlogCatalog, with the common practice of maximizing scoring on graph reconstruction, link prediction metrics etc. However, in the financial sector the important metrics are often more business related, for example fraud detection rates. With our privileged position of having large amount of real-world non-public P2P-lending social data, we aim to study empirically whether recent advances in graph embedding technics provide a useful signal for metrics more closely related to business interests, such as fraud detection rate.

cs.SI↗

arXiv2016

Magnetic Trapping of Cold Methyl Radicals

We have demonstrated that a supersonic beam of methyl radicals (CH$_3$) in the ground rotational state of both $para$ and $ortho$ species has been slowed down to standstill with a magnetic molecular decelerator, and successfully captured spatially in an anti-Helmholtz magnetic trap for $>$ 1 s. The trapped CH$_3$ radicals have a mean translational temperature of about 200 mK with an estimated density of $>5.0\times10^7$ cm$^{-3}$. The methyl radical is an ideal system for the study of cold molecules not only because of its high reactivities at low temperatures, but also because further cooling below 1 mK is plausible via sympathetic cooling with ultracold atoms. The demonstrated trapping capability of methyl radicals opens up various possibilities for realizing ultracold ensembles of molecules towards Bose-Einstein condensation of polyatomic molecules and investigations of reactions governed by quantum statistics.

physics.chem-ph↗