SearcharxivSearch

arXiv subjects

Lu Deng

Publications and source records attributed to Lu Deng.

At least 19 recordsLinked to original sources

Doubly Robust Estimation of Causal Effect on CVR with Targeted Regularization

Post-click conversion rate (CVR) is a key metric in various scenarios including e-commerce and advertising, reflecting the efficiency and user experience in the second stage of the conversion process. Estimating the causal effect on CVR is therefore of great practical importance. However, directly applying existing causal inference methods to clicked samples introduces sample selection bias and increased variance due to the exclusion of non-click data. Recent studies on CVR prediction introduce "ideal loss", which optimizes model parameters using an unbiased estimate of the loss over the full sample. Nevertheless, there is no guarantee that unbiasedness of the loss implies unbiasedness of the final estimator. We revisit this challenge from the perspective of semiparametric theory. Specifically, we develop a new doubly robust causal effect estimator for chain-structured outcomes such as CVR, and derive its theoretical properties in detail. It achieves a faster convergence rate compared to nuisance parameters estimation and is therefore more robust when using flexible nonparametric estimators, including neural networks. Based on these theoretical findings, we further design a framework based on targeted regularization to improve numerical stability and practical applicability. Extensive experiments on synthetic and real-world data demonstrate the effectiveness and robustness of our method. In addition, we find that naively combining loss debiasing with standard causal estimators underperforms our method, highlighting the necessity of developing the new estimator tailored to this CVR-style objective with solid theoretical guarantees.

cs.LG

Design-based edge-level causal inference with machine learning assisted covariate adjustment

We study design-based causal inference for edge-level outcomes in directed networks under dyadic interference. In this setting, outcomes are defined on directed edges and depend on the joint treatment assignments of pairs of units, inducing a complex dependence structure that invalidates standard estimation and inference procedures developed for node-level data. We construct Horvitz--Thompson estimators for a general class of edge-level causal effects and establish their asymptotic normality under mild regularity conditions. To enable valid inference, we develop variance estimators that exploit identifiable components of network dependence, yielding substantially less conservative bounds than classical approaches. To improve efficiency, we incorporate auxiliary covariates through a sample splitting and cross-fitting procedure. A key technical challenge is that standard two-fold sample splitting fails in the presence of edge-level outcomes due to the dependence induced by shared units. To address this issue, we introduce a three-fold sample splitting and cross-fitting scheme that restores the conditional independence required for unbiased estimation. Under a stability condition, the resulting covariate-adjusted estimator is asymptotically normal and accommodates both linear adjustment and flexible machine learning methods. We further introduce a calibration step that guarantees no asymptotic efficiency loss relative to the unadjusted estimator. Simulation studies and a real-data application confirm the theoretical results and demonstrate substantial efficiency gains.

stat.ME

Journey to the Centre of Cluster: Harnessing Interior Nodes for A/B Testing under Network Interference

A/B testing on platforms often faces challenges from network interference, where a unit's outcome depends not only on its own treatment but also on the treatments of its network neighbors. To address this, cluster-level randomization has become standard, enabling the use of network-aware estimators. These estimators typically trim the data to retain only a subset of informative units, achieving low bias under suitable conditions but often suffering from high variance. In this paper, we first demonstrate that the interior nodes - units whose neighbors all lie within the same cluster - constitute the vast majority of the post-trimming subpopulation. In light of this, we propose directly averaging over the interior nodes to construct the mean-in-interior (MII) estimator, which circumvents the delicate reweighting required by existing network-aware estimators and substantially reduces variance in classical settings. However, we show that interior nodes are often not representative of the full population, particularly in terms of network-dependent covariates, leading to notable bias. We then augment the MII estimator with a counterfactual predictor trained on the entire network, allowing us to adjust for covariate distribution shifts between the interior nodes and full population. By rearranging the expression, we reveal that our augmented MII estimator embodies an analytical form of the point estimator within prediction-powered inference framework. This insight motivates a semi-supervised lens, wherein interior nodes are treated as labeled data subject to selection bias. Extensive and challenging simulation studies demonstrate the outstanding performance of our augmented MII estimator across various settings.

stat.ME

Distilling Future Temporal Knowledge with Masked Feature Reconstruction for 3D Object Detection

Camera-based temporal 3D object detection has shown impressive results in autonomous driving, with offline models improving accuracy by using future frames. Knowledge distillation (KD) can be an appealing framework for transferring rich information from offline models to online models. However, existing KD methods overlook future frames, as they mainly focus on spatial feature distillation under strict frame alignment or on temporal relational distillation, thereby making it challenging for online models to effectively learn future knowledge. To this end, we propose a sparse query-based approach, Future Temporal Knowledge Distillation (FTKD), which effectively transfers future frame knowledge from an offline teacher model to an online student model. Specifically, we present a future-aware feature reconstruction strategy to encourage the student model to capture future features without strict frame alignment. In addition, we further introduce future-guided logit distillation to leverage the teacher's stable foreground and background context. FTKD is applied to two high-performing 3D object detection baselines, achieving up to 1.3 mAP and 1.3 NDS gains on the nuScenes dataset, as well as the most accurate velocity estimation, without increasing inference cost.

cs.CV

HMVLM: Multistage Reasoning-Enhanced Vision-Language Model for Long-Tailed Driving Scenarios

We present HaoMo Vision-Language Model (HMVLM), an end-to-end driving framework that implements the slow branch of a cognitively inspired fast-slow architecture. A fast controller outputs low-level steering, throttle, and brake commands, while a slow planner-a large vision-language model-generates high-level intents such as "yield to pedestrian" or "merge after the truck" without compromising latency. HMVLM introduces three upgrades: (1) selective five-view prompting with an embedded 4s history of ego kinematics, (2) multi-stage chain-of-thought (CoT) prompting that enforces a Scene Understanding -> Driving Decision -> Trajectory Inference reasoning flow, and (3) spline-based trajectory post-processing that removes late-stage jitter and sharp turns. Trained on the Waymo Open Dataset, these upgrades enable HMVLM to achieve a Rater Feedback Score (RFS) of 7.7367, securing 2nd place in the 2025 Waymo Vision-based End-to-End (E2E) Driving Challenge and surpassing the public baseline by 2.77%.

cs.CV

Causal inference with dyadic data in randomized experiments

Estimating treatment effects in networked settings is a central challenge in online controlled experiments, particularly on social media platforms. We investigate a scenario where the unit-level outcome of interest comprises a series of dyadic outcomes that record pairwise interactions between units, spanning from point-to-point messaging at the microscale to bilateral trade flows at the macroscale. Because the response is defined at the dyadic level, the treatment assigned to one unit can affect the outcomes of all dyads that involve it, inducing a form of network interference. We propose a design-based causal inference framework for randomized experiments with dyadic outcomes. Within this framework, we propose estimators of the global average treatment effect under Bernoulli, complete, and cluster randomization, derive the convergence rates, and establish a central limit theorem for Bernoulli randomization. We further construct a class of variance estimators that are asymptotically conservative under transparent degree conditions. Numerical studies show that the proposed estimators can reduce bias and mean squared error relative to estimators based on unit-level outcomes in a range of finite-sample settings. We illustrate the methods using two large-scale experiments on WeChat, evaluating the impact of a recommendation algorithm and a calling feature.

stat.ME

CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Driving

While end-to-end autonomous driving has advanced significantly, prevailing methods remain fundamentally misaligned with human cognitive principles in both perception and planning. In this paper, we propose CogAD, a novel end-to-end autonomous driving model that emulates the hierarchical cognition mechanisms of human drivers. CogAD implements dual hierarchical mechanisms: global-to-local context processing for human-like perception and intent-conditioned multi-mode trajectory generation for cognitively-inspired planning. The proposed method demonstrates three principal advantages: comprehensive environmental understanding through hierarchical perception, robust planning exploration enabled by multi-level planning, and diverse yet reasonable multi-modal trajectory generation facilitated by dual-level uncertainty modeling. Extensive experiments on nuScenes and Bench2Drive demonstrate that CogAD achieves state-of-the-art performance in end-to-end planning, exhibiting particular superiority in long-tail scenarios and robust generalization to complex real-world driving conditions.

cs.RO

Causal Inference in Social Platforms Under Approximate Interference Networks

Estimating the total treatment effect (TTE) of a new feature in social platforms is crucial for understanding its impact on user behavior. However, the presence of network interference, which arises from user interactions, often complicates this estimation process. Experimenters typically face challenges in fully capturing the intricate structure of this interference, leading to less reliable estimates. To address this issue, we propose a novel approach that leverages surrogate networks and the pseudo inverse estimator. Our contributions can be summarized as follows: (1) We introduce the surrogate network framework, which simulates the practical situation where experimenters build an approximation of the true interference network using observable data. (2) We investigate the performance of the pseudo inverse estimator within this framework, revealing a bias-variance trade-off introduced by the surrogate network. We demonstrate a tighter asymptotic variance bound compared to previous studies and propose an enhanced variance estimator outperforming the original estimator. (3) We apply the pseudo inverse estimator to a real experiment involving over 50 million users, demonstrating its effectiveness in detecting network interference when combined with the difference-in-means estimator. Our research aims to bridge the gap between theoretical literature and practical implementation, providing a solution for estimating TTE in the presence of network interference and unknown interference structures.

stat.AP

Unbiased Estimation for Total Treatment Effect Under Interference Using Aggregated Dyadic Data

In social media platforms, user behavior is often influenced by interactions with other users, complicating the accurate estimation of causal effects in traditional A/B experiments. This study investigates situations where an individual's outcome can be broken down into the sum of multiple pairwise outcomes, a reflection of user interactions. These outcomes, referred to as dyadic data, are prevalent in many social network contexts. Utilizing a Bernoulli randomized design, we introduce a novel unbiased estimator for the total treatment effect (TTE), which quantifies the difference in population mean when all individuals are assigned to treatment versus control groups. We further explore the bias of our estimator in scenarios where it is impractical to include all individuals in the experiment, a common constraint in online control experiments. Our numerical results reveal that our proposed estimator consistently outperforms some commonly used estimators, underscoring its potential for more precise causal effect estimation in social media environments.

cs.SI

Ego Group Partition: A Novel Framework for Improving Ego Experiments in Social Networks

Estimating the average treatment effect in social networks is challenging due to individuals influencing each other. One approach to address interference is ego cluster experiments, where each cluster consists of a central individual (ego) and its peers (alters). Clusters are randomized, and only the effects on egos are measured. In this work, we propose an improved framework for ego cluster experiments called ego group partition (EGP), which directly generates two groups and an ego sub-population instead of ego clusters. Under specific model assumptions, we propose two ego group partition algorithms. Compared to the original ego clustering algorithm, our algorithms produce more egos, yield smaller biases, and support parallel computation. The performance of our algorithms is validated through simulation and real-world case studies.

cs.SI

Optimized Covariance Design for AB Test on Social Network under Interference

Online A/B tests have become increasingly popular and important for social platforms. However, accurately estimating the global average treatment effect (GATE) has proven to be challenging due to network interference, which violates the Stable Unit Treatment Value Assumption (SUTVA) and poses a great challenge to experimental design. Existing network experimental design research was mostly based on the unbiased Horvitz-Thompson (HT) estimator with substantial data trimming to ensure unbiasedness at the price of high resultant estimation variance. In this paper, we strive to balance the bias and variance in designing randomized network experiments. Under a potential outcome model with 1-hop interference, we derive the bias and variance of the standard HT estimator and reveal their relation to the network topological structure and the covariance of the treatment assignment vector. We then propose to formulate the experimental design problem to optimize the covariance matrix of the treatment assignment vector to achieve the bias and variance balance by minimizing a well-crafted upper bound of the mean squared error (MSE) of the estimator, which allows us to decouple the unknown interference effect component and the experimental design component. An efficient projected gradient descent algorithm is presented to implement the desired randomization scheme. Finally, we carry out extensive simulation studies 2 to demonstrate the advantages of our proposed method over other existing methods in many settings, with different levels of model misspecification.

stat.ME

Efficacy of Wolbachia-mediated sterility to suppress dengue: a synthetic control study

In a study conducted in Singapore, a country prone to dengue outbreaks due to its climate and urban population, researchers examined the effectiveness of releasing male Aedes aegypti mosquitoes infected with Wolbachia (wAlbB strain) to reduce dengue transmission. These infected males, when mating with wild-type females, produced non-viable eggs, leading to vector suppression. Extensive field trials involving over 600,000 residents in four townships were conducted from 2018 to 2022. The results showed a 57% decline in total dengue incidence and a 64% decline in clustered dengue incidence. This approach offers promise for large-scale dengue control in regions facing rising dengue cases, providing a critical solution in combating the disease.

q-bio.PE

6DOF Pose Estimation of a 3D Rigid Object based on Edge-enhanced Point Pair Features

The point pair feature (PPF) is widely used for 6D pose estimation. In this paper, we propose an efficient 6D pose estimation method based on the PPF framework. We introduce a well-targeted down-sampling strategy that focuses more on edge area for efficient feature extraction of complex geometry. A pose hypothesis validation approach is proposed to resolve the symmetric ambiguity by calculating edge matching degree. We perform evaluations on two challenging datasets and one real-world collected dataset, demonstrating the superiority of our method on pose estimation of geometrically complex, occluded, symmetrical objects. We further validate our method by applying it to simulated punctures.

cs.CV

Colliding-probe bi-atomic magnetometers via energy circulation: Breaking symmetry-enforced magneto-optical rotation blockade

We have developed an inelastic wave scattering based colliding-probe bi-atomic magnetometer theory. We show a propagation growth blockade in single probe based magnetic field sensing schemes, revealing the root cause of strong suppression of nonlinear magneto-optical rotation effect (NMORE) in single probe based atomic magnetometers. We further show, both experimentally and theoretically, a colliding probe bi-atomic magnetometer that lifts this NMORE blockade. The directional energy circulation in this new atomic magnetometry technique results in more than two orders of magnitude increase in NMORE signal as well as greater than 6dB increase of magnetic field detection sensitivity. The new technique may have broad applications in photon gates and switching operations.

quant-ph

Magneto-optical rotation: Accurate approximated analytical solutions for single probe atomic magnetometers

We report an approximated analytical solution for a single-probe four-state atomic magnetometer where no analytical solution exists. This approximated analytical solution demonstrates excellent accuracy in broad probe power and detuning ranges when compared with the numerical solution obtained using a 4th order Runge-Kutta differential equation solver on MATLAB. The theoretical framework and results also encompass widely applied single-probe three-state atomic magnetometers for which no analytical solution, even approximated, is available to date in small detuning regions.

quant-ph

TriVoC: Efficient Voting-based Consensus Maximization for Robust Point Cloud Registration with Extreme Outlier Ratios

Correspondence-based point cloud registration is a cornerstone in robotics perception and computer vision, which seeks to estimate the best rigid transformation aligning two point clouds from the putative correspondences. However, due to the limited robustness of 3D keypoint matching approaches, outliers, probably in large numbers, are prone to exist among the correspondences, which makes robust registration methods imperative. Unfortunately, existing robust methods have their own limitations (e.g. high computational cost or limited robustness) when facing high or extreme outlier ratios, probably unsuitable for practical use. In this paper, we present a novel, fast, deterministic and guaranteed robust solver, named TriVoC (Triple-layered Voting with Consensus maximization), for the robust registration problem. We decompose the selecting of the minimal 3-point sets into 3 consecutive layers, and in each layer we design an efficient voting and correspondence sorting framework on the basis of the pairwise equal-length constraint. In this manner, the 3-point sets can be selected independently from the reduced correspondence sets according to the sorted sequence, which can significantly lower the computational cost and meanwhile provide a strong guarantee to achieve the largest consensus set (as the final inlier set) as long as a probabilistic termination condition is fulfilled. Varied experiments show that our solver TriVoC is robust against up to 99% outliers, highly accurate, time-efficient even with extreme outlier ratios, and also practical for real-world applications, showing performance superior to other state-of-the-art competitors.

cs.CV

Accelerating single-crystal growth by stimulated and self-guided channeling

We report a self-guided and "stimulated" single-crystal growth acceleration effect in static super-saturated aqueous solutions, producing inorganic (KH$_2$PO$_4$) and organic (tetraphenyl-phosphonium-family) nonlinear optical single-crystals with novel morphologies. The extraordinarily fast unidirectional growth in the presence of complete lateral growth suppression defies all current impurity, defect and dislocation based crystal growth inhibition mechanisms. We propose a self-channeling-stimulated accelerated growth theory that can satisfactorily explain all experimental results. Using molecular dynamics analysis and a modified two-component crystal growth model that includes microscopic surface molecular selectivity we show the lateral growth arrest is the combined result of the self-channeling and a self-shielding effect. These single-crystals exhibit remarkable mechanical flexibility in winding and twisting, demonstrating their unique advantages for chip-size quantum and biomedical applications, as well as for production of high-yield/high-potency pharmaceutical materials.

physics.chem-ph

Hybrid level anharmonicity and interference induced photon blockade in a two-qubit cavity QED system with dipole-dipole interaction

We theoretically study a quantum destructive interference (QDI) induced photon blockade in a two-qubit driven cavity QED system with dipole-dipole interaction (DDI). In the absence of dipole-dipole interaction, we show that a QDI-induced photon blockade can be achieved only when the qubit resonance frequency is different from the cavity mode frequency. When DDI is introduced the condition for this photon blockade is strongly dependent upon the pump field frequency, and yet is insensitive to the qubit-cavity coupling strength. Using this tunability feature we show that the conventional energy-level-anharmonicity-induced photon blockade and this DDI-based QDI-induced photon blockade can be combined together, resulting in a hybrid system with substantially improved mean photon number and second order correlation function. Our proposal provides a non-conventional and experimentally feasible platform for generating single photons.

quant-ph