Searcharxiv⌕ Search

arXiv subjects

Zhaojian Li

Publications and source records attributed to Zhaojian Li.

At least 55 records · Page 3Linked to original sources
arXiv2023

A Unified Linear Speedup Analysis of Federated Averaging and Nesterov FedAvg

Federated learning (FL) learns a model jointly from a set of participating devices without sharing each other's privately held data. The characteristics of non-i.i.d. data across the network, low device participation, high communication costs, and the mandate that data remain private bring challenges in understanding the convergence of FL algorithms, particularly regarding how convergence scales with the number of participating devices. In this paper, we focus on Federated Averaging (FedAvg), one of the most popular and effective FL algorithms in use today, as well as its Nesterov accelerated variant, and conduct a systematic study of how their convergence scale with the number of participating devices under non-i.i.d. data and partial participation in convex settings. We provide a unified analysis that establishes convergence guarantees for FedAvg under strongly convex, convex, and overparameterized strongly convex problems. We show that FedAvg enjoys linear speedup in each case, although with different convergence rates and communication efficiencies. For strongly convex and convex problems, we also characterize the corresponding convergence rates for the Nesterov accelerated FedAvg algorithm, which are the first linear speedup guarantees for momentum variants of FedAvg in convex settings. Empirical studies of the algorithms in various settings have supported our theoretical results.

cs.LG↗
arXiv2023

SoybeanNet: Transformer-Based Convolutional Neural Network for Soybean Pod Counting from Unmanned Aerial Vehicle (UAV) Images

Soybeans are a critical source of food, protein and oil, and thus have received extensive research aimed at enhancing their yield, refining cultivation practices, and advancing soybean breeding techniques. Within this context, soybean pod counting plays an essential role in understanding and optimizing production. Despite recent advancements, the development of a robust pod-counting algorithm capable of performing effectively in real-field conditions remains a significant challenge This paper presents a pioneering work of accurate soybean pod counting utilizing unmanned aerial vehicle (UAV) images captured from actual soybean fields in Michigan, USA. Specifically, this paper presents SoybeanNet, a novel point-based counting network that harnesses powerful transformer backbones for simultaneous soybean pod counting and localization with high accuracy. In addition, a new dataset of UAV-acquired images for soybean pod counting was created and open-sourced, consisting of 113 drone images with more than 260k manually annotated soybean pods captured under natural lighting conditions. Through comprehensive evaluations, SoybeanNet demonstrated superior performance over five state-of-the-art approaches when tested on the collected images. Remarkably, SoybeanNet achieved a counting accuracy of $84.51\%$ when tested on the testing dataset, attesting to its efficacy in real-world scenarios. The publication also provides both the source code (\url{https://github.com/JiajiaLi04/Soybean-Pod-Counting-from-UAV-Images}) and the labeled soybean dataset (\url{https://www.kaggle.com/datasets/jiajiali/uav-based-soybean-pod-images}), offering a valuable resource for future research endeavors in soybean pod counting and related fields.

cs.CV↗
arXiv2023

High-Precision Fruit Localization Using Active Laser-Camera Scanning: Robust Laser Line Extraction for 2D-3D Transformation

Recent advancements in deep learning-based approaches have led to remarkable progress in fruit detection, enabling robust fruit identification in complex environments. However, much less progress has been made on fruit 3D localization, which is equally crucial for robotic harvesting. Complex fruit shape/orientation, fruit clustering, varying lighting conditions, and occlusions by leaves and branches have greatly restricted existing sensors from achieving accurate fruit localization in the natural orchard environment. In this paper, we report on the design of a novel localization technique, called Active Laser-Camera Scanning (ALACS), to achieve accurate and robust fruit 3D localization. The ALACS hardware setup comprises a red line laser, an RGB color camera, a linear motion slide, and an external RGB-D camera. Leveraging the principles of dynamic-targeting laser-triangulation, ALACS enables precise transformation of the projected 2D laser line from the surface of apples to the 3D positions. To facilitate laser pattern acquisitions, a Laser Line Extraction (LLE) method is proposed for robust and high-precision feature extraction on apples. Comprehensive evaluations of LLE demonstrated its ability to extract precise patterns under variable lighting and occlusion conditions. The ALACS system achieved average apple localization accuracies of 6.9 11.2 mm at distances ranging from 1.0 m to 1.6 m, compared to 21.5 mm by a commercial RealSense RGB-D camera, in an indoor experiment. Orchard evaluations demonstrated that ALACS has achieved a 95% fruit detachment rate versus a 71% rate by the RealSense camera. By overcoming the challenges of apple 3D localization, this research contributes to the advancement of robotic fruit harvesting technology.

cs.RO↗
arXiv2023

Cross-modal Generative Model for Visual-Guided Binaural Stereo Generation

Binaural stereo audio is recorded by imitating the way the human ear receives sound, which provides people with an immersive listening experience. Existing approaches leverage autoencoders and directly exploit visual spatial information to synthesize binaural stereo, resulting in a limited representation of visual guidance. For the first time, we propose a visually guided generative adversarial approach for generating binaural stereo audio from mono audio. Specifically, we develop a Stereo Audio Generation Model (SAGM), which utilizes shared spatio-temporal visual information to guide the generator and the discriminator to work separately. The shared visual information is updated alternately in the generative adversarial stage, allowing the generator and discriminator to deliver their respective guided knowledge while visually sharing. The proposed method learns bidirectional complementary visual information, which facilitates the expression of visual guidance in generation. In addition, spatial perception is a crucial attribute of binaural stereo audio, and thus the evaluation of stereo spatial perception is essential. However, previous metrics failed to measure the spatial perception of audio. To this end, a metric to measure the spatial perception of audio is proposed for the first time. The proposed metric is capable of measuring the magnitude and direction of spatial perception in the temporal dimension. Further, considering its function, it is feasible to utilize it instead of demanding user studies to some extent. The proposed method achieves state-of-the-art performance on 2 datasets and 5 evaluation metrics. Qualitative experiments and user studies demonstrate that the method generates space-realistic stereo audio.

cs.SD↗
arXiv2023

Active Laser-Camera Scanning for High-Precision Fruit Localization in Robotic Harvesting: System Design and Calibration

Robust and effective fruit detection and localization is essential for robotic harvesting systems. While extensive research efforts have been devoted to improving fruit detection, less emphasis has been placed on the fruit localization aspect, which is a crucial yet challenging task due to limited depth accuracy from existing sensor measurements in the natural orchard environment with variable lighting conditions and foliage/branch occlusions. In this paper, we present the system design and calibration of an Active LAser-Camera Scanner (ALACS), a novel perception module for robust and high-precision fruit localization. The hardware of ALACS mainly consists of a red line laser, an RGB camera, and a linear motion slide, which are seamlessly integrated into an active scanning scheme where a dynamic-targeting laser-triangulation principle is employed. A high-fidelity extrinsic model is developed to pair the laser illumination and the RGB camera, enabling precise depth computation when the target is captured by both sensors. A random sample consensus-based robust calibration scheme is then designed to calibrate the model parameters based on collected data. Comprehensive evaluations are conducted to validate the system model and calibration scheme. The results show that the proposed calibration method can detect and remove data outliers to achieve robust parameter computation, and the calibrated ALACS system is able to achieve high-precision localization with millimeter-level accuracy.

cs.RO↗
arXiv2023

Privacy-Preserving Data-Enabled Predictive Leading Cruise Control in Mixed Traffic

Data-driven predictive control of connected and automated vehicles (CAVs) has received increasing attention as it can achieve safe and optimal control without relying on explicit dynamical models. However, employing the data-driven strategy involves the collection and sharing of privacy-sensitive vehicle information, which is vulnerable to privacy leakage and might further lead to malicious activities. In this paper, we develop a privacy-preserving data-enabled predictive control scheme for CAVs in a mixed traffic environment, where human-driven vehicles and CAVs coexist. We tackle external eavesdroppers and honest-but-curious central unit eavesdroppers who wiretap the communication channel of the mixed traffic system and intend to infer the CAVs' state and input information. An affine masking-based privacy protection method is designed to conceal the true state and input signals, and an extended form of the data-enabled predictive leading cruise control under different matrix structures is derived to achieve privacy-preserving optimal control for CAVs. Numerical simulations demonstrate that the proposed scheme can protect the privacy of CAVs against attackers without affecting control performance or incurring heavy computations.

eess.SY↗
arXiv2023

Dimension Reduction for Efficient Data-Enabled Predictive Control

In this letter, we propose a simple yet effective singular value decomposition (SVD) based strategy to reduce the optimization problem dimension in data-enabled predictive control (DeePC). Specifically, in the case of linear time-invariant systems, the excessive input/output measurements can be rearranged into a smaller data library for the non-parametric representation of system behavior. Based on this observation, we develop an SVD-based strategy to pre-process the offline data that achieves dimension reduction in DeePC. Numerical experiments confirm that the proposed method significantly enhances the computation efficiency without sacrificing the control performance.

eess.SY↗
arXiv2023

Challenges and Opportunities for Second-life Batteries: A Review of Key Technologies and Economy

Due to the increasing volume of Electric Vehicles in automotive markets and the limited lifetime of onboard lithium-ion batteries (LIBs), the large-scale retirement of LIBs is imminent. The battery packs retired from Electric Vehicles still own 70%-80% of the initial capacity, thus having the potential to be utilized in scenarios with lower energy and power requirements to maximize the value of LIBs. However, spent batteries are commonly less reliable than fresh batteries due to their degraded performance, thereby necessitating a comprehensive assessment from safety and economic perspectives before further utilization. To this end, this paper reviews the key technological and economic aspects of second-life batteries (SLBs). Firstly, we introduce various degradation models for first-life batteries and identify an opportunity to combine physics-based theories with data-driven methods to establish explainable models with physical laws that can be generalized. However, degradation models specifically tailored to SLBs are currently absent. Therefore, we analyze the applicability of existing battery degradation models developed for first-life batteries in SLB applications. Secondly, we investigate fast screening and regrouping techniques and discuss the regrouping standards for the first time to guide the classification procedure and enhance the performance and safety of SLBs. Thirdly, we scrutinize the economic analysis of SLBs and summarize the potentially profitable applications. Finally, we comprehensively examine and compare power electronics technologies that can substantially improve the performance of SLBs, including high-efficiency energy transformation technologies, active equalization technologies, and technologies to improve reliability and safety.

eess.SY↗
arXiv2023

A Unified Framework for Online Data-Driven Predictive Control with Robust Safety Guarantees

Despite great successes, model predictive control (MPC) relies on an accurate dynamical model and requires high onboard computational power, impeding its wider adoption in engineering systems, especially for nonlinear real-time systems with limited computation power. These shortcomings of MPC motivate this work to make such a control framework more practically viable for real-world applications. Specifically, to remove the required accurate dynamical model and reduce the computational cost for nonlinear MPC (NMPC), this paper develops a unified online data-driven predictive control pipeline to efficiently control a system with guaranteed safety without incurring large computational complexity. The new aspect of this idea is learning not only the real system but also the control policy, which results in a reasonable computational cost for the data-driven predictive controllers. More specifically, we first develop a spatial temporal filter (STF)-based concurrent learning scheme to systematically identify system dynamics for general nonlinear systems. We then develop a robust control barrier function (RCBF) for safety guarantees in the presence of model uncertainties and learn the RCBF-based NMPC policy. Furthermore, to mitigate the performance degradation due to the existing model uncertainties, we propose an online policy correction scheme through perturbation analysis and design of an ancillary feedback controller. Finally, extensive simulations on two applications, cart-inverted pendulum and automotive powertrain control, are performed to demonstrate the efficacy of the proposed framework, which shows comparable performance with much lower computational cost in comparison with several benchmark algorithms.

eess.SY↗
arXiv2023

Overtaking-enabled Eco-approach Control at Signalized Intersections for Connected and Automated Vehicles

Preceding vehicles typically dominate the movement of following vehicles in traffic systems, thereby significantly influencing the efficacy of eco-driving control that concentrates on vehicle speed optimization. To potentially mitigate the negative effect of preceding vehicles on eco-driving control at the signalized intersection, this paper proposes an overtakingenabled eco-approach control (OEAC) strategy. It combines driving lane planning and speed optimization for connected and automated vehicles to relax the first-in-first-out queuing policy at the signalized intersection, minimizing the target vehicle's energy consumption and travel delay. The OEAC adopts a receding horizon two-stage control framework to derive optimal driving trajectories for adapting to dynamic traffic conditions. In the first stage, the driving lane optimization problem is formulated as a Markov decision process and solved using dynamic programming, which takes into account the uncertain disturbance from preceding vehicles. In the second stage, the vehicle's speed trajectory with the minimal driving cost is optimized rapidly using Pontryagin's minimum principle to obtain the closed-form analytical optimal solution. Extensive simulations are conducted to evaluate the effectiveness of the OEAC. The results show that the OEAC is excellent in driving cost reduction over constant speed and regular eco-approach and departure strategies in various traffic scenarios, with an average improvement of 20.91% and 5.62%, respectively.

eess.SY↗
arXiv2023

Extended Neighboring Extremal Optimal Control with State and Preview Perturbations

Optimal control schemes have achieved remarkable performance in numerous engineering applications. However, they typically require high computational cost, which has limited their use in real-world engineering systems with fast dynamics and/or limited computation power. To address this challenge, Neighboring Extremal (NE) has been developed as an efficient optimal adaption strategy to adapt a pre-computed nominal control solution to perturbations from the nominal trajectory. The resulting control law is a time-varying feedback gain that can be pre-computed along with the original optimal control problem, and it takes negligible online computation. However, existing NE frameworks only deal with state perturbations while in modern applications, optimal controllers (e.g., predictive controllers) frequently incorporate preview information. Therefore, a new NE framework is needed to adapt to such preview perturbations. In this work, an extended NE (ENE) framework is developed to systematically adapt the nominal control to both state and preview perturbations. We show that the derived ENE law is two time-varying feedback gains on the state perturbation and the preview perturbation. We also develop schemes to handle nominal non-optimal solutions and large perturbations to retain optimal performance and constraint satisfaction. Case study on nonlinear model predictive control is presented due to its popularity but it can be easily extended to other optimal control schemes. Promising simulation results on the cart inverted pendulum problem demonstrate the efficacy of the ENE algorithm.

eess.SY↗
arXiv2023

Label-Efficient Learning in Agriculture: A Comprehensive Review

The past decade has witnessed many great successes of machine learning (ML) and deep learning (DL) applications in agricultural systems, including weed control, plant disease diagnosis, agricultural robotics, and precision livestock management. Despite tremendous progresses, one downside of such ML/DL models is that they generally rely on large-scale labeled datasets for training, and the performance of such models is strongly influenced by the size and quality of available labeled data samples. In addition, collecting, processing, and labeling such large-scale datasets is extremely costly and time-consuming, partially due to the rising cost in human labor. Therefore, developing label-efficient ML/DL methods for agricultural applications has received significant interests among researchers and practitioners. In fact, there are more than 50 papers on developing and applying deep-learning-based label-efficient techniques to address various agricultural problems since 2016, which motivates the authors to provide a timely and comprehensive review of recent label-efficient ML/DL methods in agricultural applications. To this end, we first develop a principled taxonomy to organize these methods according to the degree of supervision, including weak supervision (i.e., active learning and semi-/weakly- supervised learning), and no supervision (i.e., un-/self- supervised learning), supplemented by representative state-of-the-art label-efficient ML/DL methods. In addition, a systematic review of various agricultural applications exploiting these label-efficient algorithms, such as precision agriculture, plant phenotyping, and postharvest quality assessment, is presented. Finally, we discuss the current problems and challenges, as well as future research directions. A well-classified paper list can be accessed at https://github.com/DongChen06/Label-efficient-in-Agriculture.

cs.CV↗
arXiv2023

Data-Enabled Predictive Control for Fast Charging of Lithium-Ion Batteries with Constraint Handling

Fast charging of lithium-ion batteries has gained extensive research interests, but most of existing methods are either based on simple rule-based charging profiles or require explicit battery models that are non-trivial to identify accurately. In this paper, instead of relying on parametric battery models that are costly to derive and calibrate, we employ a novel data-enabled predictive control (DeePC) paradigm to perform safe and optimal fast charging for lithium-ion batteries. The developed DeePC methodology is based on behavioral system theory and directly utilizes the input-output measurements from the battery system to predict the future trajectory and compute the optimal control policy. Constraints on input current and battery states are incorporated in the DeePC formulation to ensure battery fast charging with safe operations. Furthermore, we propose a principal component analysis based scheme to reduce the dimension of the optimization variables in the DeePC algorithm, which significantly enhances the computation efficiency without compromising the charging performance. Numerical simulations are performed on a high-fidelity battery simulator to validate the efficacy of the proposed fast charging strategy.

eess.SY↗
arXiv2023

ABatRe-Sim: A Comprehensive Framework for Automated Battery Recycling Simulation

With the rapid surge in the number of on-road Electric Vehicles (EVs), the amount of spent lithium-ion (Li-ion) batteries is also expected to explosively grow. The spent battery packs contain valuable metal and materials that should be recovered, recycled, and reused. However, only less than 5% of the Li-ion batteries are currently recycled, due to a multitude of challenges in technology, logistics and regulation. Existing battery recycling is performed manually, which can pose a series of risks to the human operator as a consequence of remaining high voltage and chemical hazards. Therefore, there is a critical need to develop an automated battery recycling system. In this paper, we present ABatRe-sim, an open-source robotic battery recycling simulator, to facilitate the research and development in efficient and effective battery recycling au-omation. Specifically, we develop a detailed CAD model of the battery pack (with screws, wires, and battery modules), which is imported into Gazebo to enable robot-object interaction in the robot operating system (ROS) environment. It also allows the simulation of battery packs of various aging conditions. Furthermore, perception, planning, and control algorithms are developed to establish the benchmark to demonstrate the interface and realize the basic functionalities for further user customization. Discussions on the utilization and future extensions of the simulator are also presented.

cs.RO↗
arXiv2023

O2RNet: Occluder-Occludee Relational Network for Robust Apple Detection in Clustered Orchard Environments

Automated apple harvesting has attracted significant research interest in recent years due to its potential to revolutionize the apple industry, addressing the issues of shortage and high costs in labor. One key technology to fully enable efficient automated harvesting is accurate and robust apple detection, which is challenging due to complex orchard environments that involve varying lighting conditions and foliage/branch occlusions. Furthermore, clustered apples are common in the orchard, which brings additional challenges as the clustered apples may be identified as one apple. This will cause issues in localization for subsequent robotic operations. In this paper, we present the development of a novel deep learning-based apple detection framework, Occluder-Occludee Relational Network (O2RNet), for robust detection of apples in such clustered environments. This network exploits the occuluder-occludee relationship modeling head by introducing a feature expansion structure to enable the combination of layered traditional detectors to split clustered apples and foliage occlusions. More specifically, we collect a comprehensive apple orchard image dataset under different lighting conditions (overcast, front lighting, and back lighting) with frequent apple occlusions. We then develop a novel occlusion-aware network for apple detection, in which a feature expansion structure is incorporated into the convolutional neural networks to extract additional features generated by the original network for occluded apples. Comprehensive evaluations are performed, which show that the developed O2RNet outperforms state-of-the-art models with a higher accuracy of 94\% and a higher F1-score of 0.88 on apple detection.

cs.CV↗
arXiv2022

Algorithm Design and Integration for a Robotic Apple Harvesting System

Due to labor shortage and rising labor cost for the apple industry, there is an urgent need for the development of robotic systems to efficiently and autonomously harvest apples. In this paper, we present a system overview and algorithm design of our recently developed robotic apple harvester prototype. Our robotic system is enabled by the close integration of several core modules, including visual perception, planning, and control. This paper covers the main methods and advancements in deep learning-based multi-view fruit detection and localization, unified picking and dropping planning, and dexterous manipulation control. Indoor and field experiments were conducted to evaluate the performance of the developed system, which achieved an average picking rate of 3.6 seconds per apple. This is a significant improvement over other reported apple harvesting robots with a picking rate in the range of 7-10 seconds per apple. The current prototype shows promising performance towards further development of efficient and automated apple harvesting technology. Finally, limitations of the current system and future work are discussed.

cs.RO↗
arXiv2022

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

On-ramp merging is a challenging task for autonomous vehicles (AVs), especially in mixed traffic where AVs coexist with human-driven vehicles (HDVs). In this paper, we formulate the mixed-traffic highway on-ramp merging problem as a multi-agent reinforcement learning (MARL) problem, where the AVs (on both merge lane and through lane) collaboratively learn a policy to adapt to HDVs to maximize the traffic throughput. We develop an efficient and scalable MARL framework that can be used in dynamic traffic where the communication topology could be time-varying. Parameter sharing and local rewards are exploited to foster inter-agent cooperation while achieving great scalability. An action masking scheme is employed to improve learning efficiency by filtering out invalid/unsafe actions at each step. In addition, a novel priority-based safety supervisor is developed to significantly reduce collision rate and greatly expedite the training process. A gym-like simulation environment is developed and open-sourced with three different levels of traffic densities. We exploit curriculum learning to efficiently learn harder tasks from trained models under simpler settings. Comprehensive experimental results show the proposed MARL framework consistently outperforms several state-of-the-art benchmarks.

eess.SY↗
arXiv2022

Deep Data Augmentation for Weed Recognition Enhancement: A Diffusion Probabilistic Model and Transfer Learning Based Approach

Weed management plays an important role in many modern agricultural applications. Conventional weed control methods mainly rely on chemical herbicides or hand weeding, which are often cost-ineffective, environmentally unfriendly, or even posing a threat to food safety and human health. Recently, automated/robotic weeding using machine vision systems has seen increased research attention with its potential for precise and individualized weed treatment. However, dedicated, large-scale, and labeled weed image datasets are required to develop robust and effective weed identification systems but they are often difficult and expensive to obtain. To address this issue, data augmentation approaches, such as generative adversarial networks (GANs), have been explored to generate highly realistic images for agricultural applications. Yet, despite some progress, those approaches are often complicated to train or have difficulties preserving fine details in images. In this paper, we present the first work of applying diffusion probabilistic models (also known as diffusion models) to generate high-quality synthetic weed images based on transfer learning. Comprehensive experimental results show that the developed approach consistently outperforms several state-of-the-art GAN models, representing the best trade-off between sample fidelity and diversity and highest FID score on a common weed dataset, CottonWeedID15. In addition, the expanding dataset with synthetic weed images can apparently boost model performance on four deep learning (DL) models for the weed classification tasks. Furthermore, the DL models trained on CottonWeedID15 dataset with only 10% of real images and 90% of synthetic weed images achieve a testing accuracy of over 94%, showing high-quality of the generated weed samples. The codes of this study are made publicly available at https://github.com/DongChen06/DMWeeds.

cs.CV↗