arXiv subjects

Mark Lawford

Publications and source records attributed to Mark Lawford.

17 recordsLinked to original sources

arXiv2026

Scene Graph-based Driving Scenario Extraction for Automotive Egocentric Datasets

Extracting scenarios from unlabelled real-world sensor data streams is a critical but challenging task in the development process of automated driving systems (ADS). Automatically sifting through large datasets to spatially and temporally locate critical scenarios can enable scenario-based coverage analysis of ADS datasets. In this paper, we present a method for extracting scenarios from egocentric datasets using scene graphs and Linear Temporal Logic (LTL). We first process egocentric sensor data and HD maps to generate a sequence of scene graphs representing a driving scenario. Next, we use LTL to formally specify driving scenarios of interest, then extract all instances of the scenarios from the dataset using an off-the-shelf model checker, which evaluates the LTL formula against the sequence of scene graphs. Our approach can be used on both simulated and real world datasets. We evaluate the method on the training and validation datasets from Argoverse 2 consisting of 850 15-second real-world driving logs, and several videos of dashcam footage. We demonstrate the effectiveness of our approach for extracting and querying scenarios by evaluating against a rule-based benchmark based on track annotations and HD maps.

cs.RO↗

arXiv2026

Engineering Resource-constrained Software Systems with DNN Components: a Concept-based Pruning Approach

Deep Neural Networks (DNNs) are widely used by engineers to solve difficult problems that require predictive modeling from data. However, these models are often massive, with millions or billions of parameters, and require substantial computational power, RAM, and storage. This becomes a limitation in practical scenarios where strict size and resource constraints must be respected. In this paper, we present a novel concept-based pruning technique for DNNs that guides pruning decisions using human-interpretable concepts, such as features, colors, and classes. This is particularly important in a software engineering context, as DNNs are integrated into systems and must be pruned according to specific system requirements. Our concept-based pruning solution analyzes neuron activations to identify important neurons from a system requirements viewpoint and uses this information to guide the DNN pruning. We assess our solution using the VGG-19 network and a dataset of 26'384 RGB images, focusing on its ability to produce small, effective pruned DNNs and on the computational complexity and performance of these pruned DNNs. We also analyzed the pruning efficiency of our solution and compared alternative configurations. Our results show that concept-based pruning efficiently generates much smaller, effective pruned DNNs. Pruning greatly improves the computational efficiency and performance of DNNs, properties that are particularly useful for practical applications with stringent memory and computational time constraints. Finally, alternative configuration options enable engineers to identify trade-offs adapted to different practical situations.

cs.SE↗

arXiv2026

Ensembles-based Feature Guided Analysis

Recent Deep Neural Networks (DNN) applications ask for techniques that can explain their behavior. Existing solutions, such as Feature Guided Analysis (FGA), extract rules on their internal behaviors, e.g., by providing explanations related to neurons activation. Results from the literature show that these rules have considerable precision (i.e., they correctly predict certain classes of features), but the recall (i.e., the number of situations these rule apply) is more limited. To mitigate this problem, this paper presents Ensembles-based Feature Guided Analysis (EFGA). EFGA combines rules extracted by FGA into ensembles. Ensembles aggregate different rules to increase their applicability depending on an aggregation criterion, a policy that dictates how to combine rules into ensembles. Although our solution is extensible, and different aggregation criteria can be developed by users, in this work, we considered three different aggregation criteria. We evaluated how the choice of the criterion influences the effectiveness of EFGA on two benchmarks (i.e., the MNIST and LSC datasets), and found that different aggregation criteria offer alternative trade-offs between precision and recall. We then compare EFGA with FGA. For this experiment, we selected an aggregation criterion that provides a reasonable trade-off between precision and recall. Our results show that EFGA has higher train recall (+28.51% on MNIST, +33.15% on LSC), and test recall (+25.76% on MNIST, +30.81% on LSC) than FGA, with a negligible reduction on the test precision (-0.89% on MNIST, -0.69% on LSC).

cs.LG↗

arXiv2025

Search-based Software Testing Driven by Domain Knowledge: Reflections and New Perspectives

Search-based Software Testing (SBST) can automatically generate test cases to search for requirements violations. Unlike manual test case development, it can generate a substantial number of test cases in a limited time. However, SBST does not possess the domain knowledge of engineers. Several techniques have been proposed to integrate engineers' domain knowledge within existing SBST frameworks. This paper will reflect on recent experimental results by highlighting bold and unexpected results. It will help re-examine SBST techniques driven by domain knowledge from a new perspective, suggesting new directions for future research.

cs.SE↗

arXiv2025

Feature-Guided Analysis of Neural Networks: A Replication Study

Understanding why neural networks make certain decisions is pivotal for their use in safety-critical applications. Feature-Guided Analysis (FGA) extracts slices of neural networks relevant to their tasks. Existing feature-guided approaches typically monitor the activation of the neural network neurons to extract the relevant rules. Preliminary results are encouraging and demonstrate the feasibility of this solution by assessing the precision and recall of Feature-Guided Analysis on two pilot case studies. However, the applicability in industrial contexts needs additional empirical evidence. To mitigate this need, this paper assesses the applicability of FGA on a benchmark made by the MNIST and LSC datasets. We assessed the effectiveness of FGA in computing rules that explain the behavior of the neural network. Our results show that FGA has a higher precision on our benchmark than the results from the literature. We also evaluated how the selection of the neural network architecture, training, and feature selection affect the effectiveness of FGA. Our results show that the selection significantly affects the recall of FGA, while it has a negligible impact on its precision.

cs.LG↗

arXiv2025

Engineering Automotive Digital Twins on Standardized Architectures: A Case Study

Digital twin (DT) technology has become of interest in the automotive industry. There is a growing need for smarter services that utilize the unique capabilities of DTs, ranging from computer-aided remote control to cloud-based fleet coordination. Developing such services starts with the software architecture. However, the scarcity of DT architectural guidelines poses a challenge for engineering automotive DTs. Currently, the only DT architectural standard is the one defined in ISO 23247. Though not developed for automotive systems, it is one of the few feasible starting points for automotive DTs. In this work, we investigate the suitability of the ISO 23247 reference architecture for developing automotive DTs. Through the case study of developing an Adaptive Cruise Control DT for a 1/10th-scale autonomous vehicle, we identify some strengths and limitations of the reference architecture and begin distilling future directions for researchers, practitioners, and standard developers.

cs.RO↗

arXiv2025

Search-based Testing of Simulink Models with Requirements Tables

Search-based software testing (SBST) of Simulink models helps find scenarios that demonstrate that the system can reach a state that violates one of its requirements. However, many SBST techniques for Simulink models rely on requirements being expressed in logical languages, limiting their adoption in industry. To help with the adoption, SBST methods and tools for Simulink models need to be integrated with tools used by engineers to specify requirements. This work presents the first black-box testing approach for Simulink models that supports Requirements Table (RT), a tool from Simulink Requirements Toolbox used by practitioners to express software requirements. We evaluated our solution by considering 60 model-RT combinations each made by a model and an RT. Our SBST framework returned a failure-revealing test case for 70% of the model-RT combinations. Remarkably, it identified a failure-revealing test case for three model-RT combinations for a cruise controller of an industrial simulator that other previously used tools were not able to find. The efficiency of our SBST solution is acceptable for practical applications and comparable with existing SBST tools that are not based on RT.

cs.SE↗

arXiv2024

Novel Fundus Image Preprocessing for Retcam Images to Improve Deep Learning Classification of Retinopathy of Prematurity

Retinopathy of Prematurity (ROP) is a potentially blinding eye disorder because of damage to the eye's retina which can affect babies born prematurely. Screening of ROP is essential for early detection and treatment. This is a laborious and manual process which requires trained physician performing dilated ophthalmological examination which can be subjective resulting in lower diagnosis success for clinically significant disease. Automated diagnostic methods can assist ophthalmologists increase diagnosis accuracy using deep learning. Several research groups have highlighted various approaches. Captured ROP Retcam images suffer from poor quality. This paper proposes the use of improved novel fundus preprocessing methods using pretrained transfer learning frameworks to create hybrid models to give higher diagnosis accuracy. Once trained and validated, the evaluations showed that these novel methods in comparison to traditional imaging processing contribute to better and in many aspects higher accuracy in classifying Plus disease, Stages of ROP and Zones in comparison to peer papers.

eess.IV↗

arXiv2024

Simulation-based Analysis of a Novel Loop-based Road Topology for Autonomous Vehicles

The challenges in implementing SAE Level 4/5 autonomous vehicles are manifold, with intersection navigation being a pervasive one. We analyze a novel road topology invented by a co-author of this paper, Xiayong Hu. The topology eliminates the need for traditional traffic control and cross-traffic at intersections, potentially improving the safety of autonomous driving systems. The topology, herein called the Zonal Road Topology, consists of unidirectional loops of road with traffic flowing either clockwise or counter-clockwise. Adjacent loops are directionally aligned with one another, allowing vehicles to transfer from one loop to another through a simple lane change. To evaluate the Zonal Road Topology, a one km2 pilot-track near Changshu, China is currently being set aside for testing. In parallel, traffic simulations are being performed. To this end, we conduct a simulation-based comparison between the Zonal Road Topology and a traditional road topology for a generic Electric Vehicle (EV) using the Simulation for Urban MObility (SUMO) platform and MATLAB/Simulink. We analyze the topologies in terms of their travel efficiency, safety, energy usage, and capacity. Drive time, number of halts, progress rate, and other metrics are analyzed across varied traffic levels to investigate the advantages and disadvantages of the Zonal Road Topology. Our results indicate that vehicles on the Zonal Road Topology have a lower, more consistent drive time with greater traffic throughput, while using less energy on average. These results become more prominent at higher traffic densities.

cs.RO↗

arXiv2023

Test Case Generation for Drivability Requirements of an Automotive Cruise Controller: An Experience with an Industrial Simulator

Automotive software development requires engineers to test their systems to detect violations of both functional and drivability requirements. Functional requirements define the functionality of the automotive software. Drivability requirements refer to the driver's perception of the interactions with the vehicle; for example, they typically require limiting the acceleration and jerk perceived by the driver within given thresholds. While functional requirements are extensively considered by the research literature, drivability requirements garner less attention. This industrial paper describes our experience assessing the usefulness of an automated search-based software testing (SBST) framework in generating failure-revealing test cases for functional and drivability requirements. Our experience concerns the VI-CarRealTime simulator, an industrial virtual modeling and simulation environment widely used in the automotive domain. We designed a Cruise Control system in Simulink for a four-wheel vehicle, in an iterative fashion, by producing 21 model versions. We used the SBST framework for each version of the model to search for failure-revealing test cases revealing requirement violations. Our results show that the SBST framework successfully identified a failure-revealing test case for 66.7% of our model versions, requiring, on average, 245.9s and 3.8 iterations. We present lessons learned, reflect on the generality of our results, and discuss how our results improve the state of practice.

cs.SE↗

arXiv2022

Simulation-based Testing of Simulink Models with Test Sequence and Test Assessment Blocks

Simulation-based software testing supports engineers in finding faults in Simulink models. It typically relies on search algorithms that iteratively generate test inputs used to exercise models in simulation to detect design errors. While simulation-based software testing techniques are effective in many practical scenarios, they are typically not fully integrated within the Simulink environment and require additional manual effort. Many techniques require engineers to specify requirements using logical languages that are neither intuitive nor fully supported by Simulink, thereby limiting their adoption in industry. This work presents HECATE, a testing approach for Simulink models using Test Sequence and Test Assessment blocks from Simulink Test. Unlike existing testing techniques, HECATE uses information from Simulink models to guide the search-based exploration. Specifically, HECATE relies on information provided by the Test Sequence and Test Assessment blocks to guide the search procedure. Across a benchmark of 16 Simulink models from different domains and industries, our comparison of HECATE with the state-of-the-art testing tool S-TALIRO indicates that HECATE is both more effective (more failure-revealing test cases) and efficient (less iterations and computational time) than S-TALIRO for ~94% and ~81% of benchmark models respectively. Furthermore, HECATE successfully generated a failure-revealing test case for a representative case study from the automotive domain demonstrating its practical usefulness.

cs.SE↗

arXiv2021

Is the Rush to Machine Learning Jeopardizing Safety? Results of a Survey

Machine learning (ML) is finding its way into safety-critical systems (SCS). Current safety standards and practice were not designed to cope with ML techniques, and it is difficult to be confident that SCSs that contain ML components are safe. Our hypothesis was that there has been a rush to deploy ML techniques at the expense of a thorough examination as to whether the use of ML techniques introduces safety problems that we are not yet adequately able to detect and mitigate against. We thus conducted a targeted literature survey to determine the research effort that has been expended in applying ML to SCS compared with that spent on evaluating the safety of SCSs that deploy ML components. This paper presents the (surprising) results of the survey.

cs.LG↗

arXiv2020

Literature Review of Computer Tools for the Visually Impaired: a focus on Search Engines

A sudden reliance on the internet has resulted in the global standardization of specific software and interfaces tailored for the average user. Whether it be web apps or dedicated software, the methods of interaction are seemingly similar. But when the computer tool is presented with unique users, specifically with a disability, the quality of interaction degrades, sometimes to a point of complete uselessness. This roots from one's focus on the average user rather than the development of a platform for all (a golden standard). This paper reviews published works and products that deal with providing accessibility to visually impaired online users. Due to the variety of tools that are available to computer users, the paper focuses on search engines as a primary tool for browsing the web. By analyzing the attributes discussed below, the reader is equipped with a set of references for existing applications, along with practical insight and recommendations for accessible design. Finally, the necessary considerations for future developments and summaries of important focal points are highlighted.

cs.HC↗

arXiv2020

Supporting Modularity in Simulink Models

Model-Based Development (MBD) is widely used for embedded controls development, with Matlab Simulink being one of the most used modelling environments in industry. As with all software, Simulink models are subject to evolution over their lifetime and must be maintained. Modularity is a fundamental software engineering principle facilitating the construction of complex software, and is used in textual languages such as C. However, as Simulink is a graphical modelling language, it is not currently well understood how modularity can be leveraged in development with Simulink, nor whether it can be supported with current Simulink modelling constructs. This paper presents an effective way of achieving modularity in Simulink by introducing the concept of a Simulink module. The effectiveness of the approach is measured using well-known indicators of modularity, including coupling and cohesion, cyclomatic complexity, and information hiding ability. A syntactic interface is defined in order to represent all data flow across the module boundary. Four modelling guidelines are also presented to encourage best practice. Also, a custom tool that supports the modelling of Simulink modules is described. Finally, this work is demonstrated and evaluated on a real-world example from the nuclear domain.

cs.SE↗

arXiv2019

Assurance via workflow+ modelling and conformance

We propose considering assurance as a model management enterprise: saying that a system is safe amounts to specifying three workflows modelling how the safety engineering process is defined and executed, and checking their conformance. These workflows are based on precise data modelling as in functional block diagrams, but their distinctive feature is the presence of relationships between the output data of a process and its input data; hence, the name ``WorkflowPlus'', WF+ . A typical WP^+ model comprises three layers: (i) process and control flow, (ii) dataflow (with input-output relationships), and (iii) argument flow or constraint derivation. Precise dataflow modelling signifies a crucial distinction of (WP+)-based and GSN-based assurance, in which the data layer is mainly implicit. We provide a detailed comparative analysis of the two formalisms and conclude that GSN does not fulfil its promises.

cs.SE↗

arXiv2019

Multiple Model Synchronization with Multiary Delta Lenses with Amendment and K-Putput

Multiple (more than 2) model synchronization is ubiquitous and important for model driven engineering, but its theoretical underpinning gained much less attention than the binary case. Specifically, the latter was extensively studied by the bx community in the framework of algebraic models for update propagation called lenses. Now we make a step to restore the balance and propose a notion of multiary delta lens. Besides multiarity, our lenses feature {\em reflective} updates, when consistency restoration requires some amendment of the update that violated consistency. We emphasize the importance of various ways of lens composition for practical applications of the framework, and prove several composition results.

cs.LO↗

arXiv2015

Formal Verification of Real-Time Function Blocks Using PVS

A critical step towards certifying safety-critical systems is to check their conformance to hard real-time requirements. A promising way to achieve this is by building the systems from pre-verified components and verifying their correctness in a compositional manner. We previously reported a formal approach to verifying function blocks (FBs) using tabular expressions and the PVS proof assistant. By applying our approach to the IEC 61131-3 standard of Programmable Logic Controllers (PLCs), we constructed a repository of precise specification and reusable (proven) theorems of feasibility and correctness for FBs. However, we previously did not apply our approach to verify FBs against timing requirements, since IEC 61131-3 does not define composite FBs built from timers. In this paper, based on our experience in the nuclear domain, we conduct two realistic case studies, consisting of the software requirements and the proposed FB implementations for two subsystems of an industrial control system. The implementations are built from IEC 61131-3 FBs, including the on-delay timer. We find issues during the verification process and suggest solutions.

cs.SE↗