Searcharxiv⌕ Search

arXiv subjects

Laurie Williams

Publications and source records attributed to Laurie Williams.

At least 55 records · Page 3Linked to original sources
arXiv2023

S3C2 Summit 2202-09: Industry Secure Suppy Chain Summit

Recent years have shown increased cyber attacks targeting less secure elements in the software supply chain and causing fatal damage to businesses and organizations. Past well-known examples of software supply chain attacks are the SolarWinds or log4j incidents that have affected thousands of customers and businesses. The US government and industry are equally interested in enhancing software supply chain security. We conducted six panel discussions with a diverse set of 19 practitioners from industry. We asked them open-ended questions regarding SBOMs, vulnerable dependencies, malicious commits, build and deploy, the Executive Order, and standards compliance. The goal of this summit was to enable open discussions, mutual sharing, and shedding light on common challenges that industry practitioners with practical experience face when securing their software supply chain. This paper summarizes the summit held on September 30, 2022.

cs.CR↗
arXiv2023

A Comparative Study of Software Secrets Reporting by Secret Detection Tools

Background: According to GitGuardian's monitoring of public GitHub repositories, secrets sprawl continued accelerating in 2022 by 67% compared to 2021, exposing over 10 million secrets (API keys and other credentials). Though many open-source and proprietary secret detection tools are available, these tools output many false positives, making it difficult for developers to take action and teams to choose one tool out of many. To our knowledge, the secret detection tools are not yet compared and evaluated. Aims: The goal of our study is to aid developers in choosing a secret detection tool to reduce the exposure of secrets through an empirical investigation of existing secret detection tools. Method: We present an evaluation of five open-source and four proprietary tools against a benchmark dataset. Results: The top three tools based on precision are: GitHub Secret Scanner (75%), Gitleaks (46%), and Commercial X (25%), and based on recall are: Gitleaks (88%), SpectralOps (67%) and TruffleHog (52%). Our manual analysis of reported secrets reveals that false positives are due to employing generic regular expressions and ineffective entropy calculation. In contrast, false negatives are due to faulty regular expressions, skipping specific file types, and insufficient rulesets. Conclusions: We recommend developers choose tools based on secret types present in their projects to prevent missing secrets. In addition, we recommend tool vendors update detection rules periodically and correctly employ secret verification mechanisms by collaborating with API vendors to improve accuracy.

cs.CR↗
arXiv2023

Do Software Security Practices Yield Fewer Vulnerabilities?

Due to the ever-increasing security breaches, practitioners are motivated to produce more secure software. In the United States, the White House Office released a memorandum on Executive Order (EO) 14028 that mandates organizations provide self-attestation of the use of secure software development practices. The OpenSSF Scorecard project allows practitioners to measure the use of software security practices automatically. However, little research has been done to determine whether the use of security practices improves package security, particularly which security practices have the biggest impact on security outcomes. The goal of this study is to assist practitioners and researchers making informed decisions on which security practices to adopt through the development of models between software security practice scores and security vulnerability counts. To that end, we developed five supervised machine learning models for npm and PyPI packages using the OpenSSF Scorecared security practices scores and aggregate security scores as predictors and the number of externally-reported vulnerabilities as a target variable. Our models found four security practices (Maintained, Code Review, Branch Protection, and Security Policy) were the most important practices influencing vulnerability count. However, we had low R^2 (ranging from 9% to 12%) when we tested the models to predict vulnerability counts. Additionally, we observed that the number of reported vulnerabilities increased rather than reduced as the aggregate security score of the packages increased. Both findings indicate that additional factors may influence the package vulnerability count. We suggest that vulnerability count and security score data be refined such that these measures may be used to provide actionable guidance on security practices.

cs.CR↗
arXiv2023

Trusting code in the wild: A social network-based centrality rating for developers in the Rust ecosystem

As modern software extensively uses open source packages, developers regularly pull in new upstream code through frequent updates. While a manual review of all upstream changes may not be practical, developers may rely on the authors' and reviewers' identities, among other factors, to decide what level of review the new code may require. The goal of this study is to help downstream project developers prioritize review efforts for upstream code by providing a social network-based centrality rating for the authors and reviewers of that code. To that end, we build a social network of 6,949 developers across the collaboration activity from 1,644 Rust packages. Further, we survey the developers in the network to evaluate if code coming from a developer with a higher centrality rating is likely to be accepted with lesser scrutiny by the downstream projects and, therefore, is perceived to be more trusted. Our results show that 97.7\% of the developers from the studied packages are interconnected via collaboration, with each developer separated from another via only four other developers in the network. The interconnection among developers from different Rust packages establishes the ground for identifying the central developers in the ecosystem. Our survey responses ($N=206$) show that the respondents are more likely to not differentiate between developers in deciding how to review upstream changes (60.2\% of the time). However, when they do differentiate, our statistical analysis showed a significant correlation between developers' centrality ratings and the level of scrutiny their code might face from the downstream projects, as indicated by the respondents.

cs.SE↗
arXiv2023

SecretBench: A Dataset of Software Secrets

According to GitGuardian's monitoring of public GitHub repositories, the exposure of secrets (API keys and other credentials) increased two-fold in 2021 compared to 2020, totaling more than six million secrets. However, no benchmark dataset is publicly available for researchers and tool developers to evaluate secret detection tools that produce many false positive warnings. The goal of our paper is to aid researchers and tool developers in evaluating and improving secret detection tools by curating a benchmark dataset of secrets through a systematic collection of secrets from open-source repositories. We present a labeled dataset of source codes containing 97,479 secrets (of which 15,084 are true secrets) of various secret types extracted from 818 public GitHub repositories. The dataset covers 49 programming languages and 311 file types.

cs.CR↗
arXiv2023

What Challenges Do Developers Face About Checked-in Secrets in Software Artifacts?

Throughout 2021, GitGuardian's monitoring of public GitHub repositories revealed a two-fold increase in the number of secrets (database credentials, API keys, and other credentials) exposed compared to 2020, accumulating more than six million secrets. To our knowledge, the challenges developers face to avoid checked-in secrets are not yet characterized. The goal of our paper is to aid researchers and tool developers in understanding and prioritizing opportunities for future research and tool automation for mitigating checked-in secrets through an empirical investigation of challenges and solutions related to checked-in secrets. We extract 779 questions related to checked-in secrets on Stack Exchange and apply qualitative analysis to determine the challenges and the solutions posed by others for each of the challenges. We identify 27 challenges and 13 solutions. The four most common challenges, in ranked order, are: (i) store/version of secrets during deployment; (ii) store/version of secrets in source code; (iii) ignore/hide of secrets in source code; and (iv) sanitize VCS history. The three most common solutions, in ranked order, are: (i) move secrets out of source code/version control and use template config file; (ii) secret management in deployment; and (iii) use local environment variables. Our findings indicate that the same solution has been mentioned to mitigate multiple challenges. However, our findings also identify an increasing trend in questions lacking accepted solutions substantiating the need for future research and tool automation on managing secrets.

cs.SE↗
arXiv2022

An Extended Model of Software Configuration

Feature toggles and configuration options are modern programmatic techniques to easily include or exclude functionality in a software product. The research contributions to these two techniques have most often been focused on either one of them separately. However, focusing on the similarities of these two techniques may enable a more fruitful combined family of research on software configuration, a term we use to encompass both techniques. Also, a common terminology may have enabled meta-analysis, a more practical application of the research on the two techniques, and prevented duplication of research effort. The goal of this research study is to aid researchers in conducting a family of research on software configuration by extending an existing model of software configuration that provides terminology for research studies. To achieve our goal, we started with Seigmund et al. Model of Software Configuration (MSC) which was developed based on interviews and publications on configuration options. We explicitly extend the MSC to include feature toggles and to add qualitative analysis of feature toggle-related resources. From our analysis, we proposed MSCv2 as an extended version of MSC and evaluated it through its application on five academic publications and the Chrome system. Our results indicate that multiple researchers studying the same system may provide different definitions of software configuration in their publications. Also, similar research questions may be answered on feature toggles and configuration options repeatedly because of a lack of a clear definition of software configuration. These observations indicate that having a model for defining software configuration may enable more clear and generalized research on the software configuration family of research. Practitioners benefit MSCv2 in their systems to better knowledge transfer to other practitioners and researchers.

cs.SE↗
arXiv2022

Investigating co-occurrences of MITRE ATT\&CK Techniques

Cyberattacks use adversarial techniques to bypass system defenses, persist, and eventually breach systems. The MITRE ATT\&CK framework catalogs a set of adversarial techniques and maps between adversaries and their used techniques and tactics. Understanding how adversaries deploy techniques in conjunction is pivotal for learning adversary behavior, hunting potential threats, and formulating a proactive defense. The goal of this research is to aid cybersecurity practitioners and researchers in choosing detection and mitigation strategies through co-occurrence analysis of adversarial techniques reported in MITRE ATT&CK. We collect the adversarial techniques of 115 cybercrime groups and 484 malware from the MITRE ATT\&CK. We apply association rule mining and network analysis to investigate how adversarial techniques co-occur. We identify that adversaries pair T1059: Command and scripting interface and T1105: Ingress tool transfer techniques with a relatively large number of ATT\&CK techniques. We also identify adversaries using the T1082: System Information Discovery technique to determine their next course of action. We observe adversaries deploy the highest number of techniques from the TA0005: Defense evasion and TA0007: Discovery tactics. Based on our findings on co-occurrence, we identify six detection, six mitigation strategies, and twelve adversary behaviors. We urge defenders to prioritize primarily the detection of TA0007: Discovery and mitigation of TA0005: Defense evasion techniques. Overall, this study approximates how adversaries leverage techniques based on publicly reported documents. We advocate organizations investigate adversarial techniques in their environment and make the findings available for a more precise and actionable understanding.

cs.CR↗
arXiv2022

An investigation of security controls and MITRE ATT\&CK techniques

Attackers utilize a plethora of adversarial techniques in cyberattacks to compromise the confidentiality, integrity, and availability of the target organizations and systems. Information security standards such as NIST, ISO/IEC specify hundreds of security controls that organizations can enforce to protect and defend the information systems from adversarial techniques. However, implementing all the available controls at the same time can be infeasible and security controls need to be investigated in terms of their mitigation ability over adversarial techniques used in cyberattacks as well. The goal of this research is to aid organizations in making informed choices on security controls to defend against cyberthreats through an investigation of adversarial techniques used in current cyberattacks. In this study, we investigated the extent of mitigation of 298 NIST SP800-53 controls over 188 adversarial techniques used in 669 cybercrime groups and malware cataloged in the MITRE ATT\&CK framework based upon an existing mapping between the controls and techniques. We identify that, based on the mapping, only 101 out of 298 control are capable of mitigating adversarial techniques. However, we also identify that 53 adversarial techniques cannot be mitigated by any existing controls, and these techniques primarily aid adversaries in bypassing system defense and discovering targeted system information. We identify a set of 20 critical controls that can mitigate 134 adversarial techniques, and on average, can mitigate 72\% of all techniques used by 98\% of the cataloged adversaries in MITRE ATT\&CK. We urge organizations, that do not have any controls enforced in place, to implement the top controls identified in the study.

cs.CR↗
arXiv2022

Are your dependencies code reviewed?: Measuring code review coverage in dependency updates

As modern software extensively uses free open source packages as dependencies, developers have to regularly pull in new third-party code through frequent updates. However, without a proper review of every incoming change, vulnerable and malicious code can sneak into the codebase through these dependencies. The goal of this study is to aid developers in securely accepting dependency updates by measuring if the code changes in an update have passed through a code review process. We implement Depdive, an update audit tool for packages in Crates.io, npm, PyPI, and RubyGems registry. Depdive first (i) identifies the files and the code changes in an update that cannot be traced back to the package's source repository, i.e., \textit{phantom artifacts}; and then (ii) measures what portion of changes in the update, excluding the phantom artifacts, has passed through a code review process, i.e., \textit{code review coverage}. Using Depdive, we present an empirical study across the latest ten updates of the most downloaded 1000 packages in each of the four registries. We further evaluated our results through a maintainer agreement survey. We find the updates are typically only partially code-reviewed (52.5\% of the time). Further, only 9.0\% of the packages had all their updates in our data set fully code-reviewed, indicating that even the most used packages can introduce non-reviewed code in the software supply chain. We also observe that updates either tend to have high \textit{CRC} or low \textit{CRC}, suggesting that packages at the opposite end of the spectrum may require a separate set of treatments.

cs.SE↗
arXiv2022

From Threat Reports to Continuous Threat Intelligence: A Comparison of Attack Technique Extraction Methods from Textual Artifacts

The cyberthreat landscape is continuously evolving. Hence, continuous monitoring and sharing of threat intelligence have become a priority for organizations. Threat reports, published by cybersecurity vendors, contain detailed descriptions of attack Tactics, Techniques, and Procedures (TTP) written in an unstructured text format. Extracting TTP from these reports aids cybersecurity practitioners and researchers learn and adapt to evolving attacks and in planning threat mitigation. Researchers have proposed TTP extraction methods in the literature, however, not all of these proposed methods are compared to one another or to a baseline. \textit{The goal of this study is to aid cybersecurity researchers and practitioners choose attack technique extraction methods for monitoring and sharing threat intelligence by comparing the underlying methods from the TTP extraction studies in the literature.} In this work, we identify ten existing TTP extraction studies from the literature and implement five methods from the ten studies. We find two methods, based on Term Frequency-Inverse Document Frequency(TFIDF) and Latent Semantic Indexing (LSI), outperform the other three methods with a F1 score of 84\% and 83\%, respectively. We observe the performance of all methods in F1 score drops in the case of increasing the class labels exponentially. We also implement and evaluate an oversampling strategy to mitigate class imbalance issues. Furthermore, oversampling improves the classification performance of TTP extraction. We provide recommendations from our findings for future cybersecurity researchers, such as the construction of a benchmark dataset from a large corpus; and the selection of textual features of TTP. Our work, along with the dataset and implementation source code, can work as a baseline for cybersecurity researchers to test and compare the performance of future TTP extraction methods.

cs.CR↗
arXiv2022

What are the Practices for Secret Management in Software Artifacts?

Throughout 2021, GitGuardian's monitoring of public GitHub repositories revealed a two-fold increase in the number of secrets (database credentials, API keys, and other credentials) exposed compared to 2020, accumulating more than six million secrets. A systematic derivation of practices for managing secrets can help practitioners in secure development. The goal of our paper is to aid practitioners in avoiding the exposure of secrets by identifying secret management practices in software artifacts through a systematic derivation of practices disseminated in Internet artifacts. We conduct a grey literature review of Internet artifacts, such as blog articles and question and answer posts. We identify 24 practices grouped in six categories comprised of developer and organizational practices. Our findings indicate that using local environment variables and external secret management services are the most recommended practices to move secrets out of source code and to securely store secrets. We also observe that using version control system scanning tools and employing short-lived secrets are the most recommended practices to avoid accidentally committing secrets and limit secret exposure, respectively.

cs.SE↗
arXiv2022

Do I really need all this work to find vulnerabilities? An empirical case study comparing vulnerability detection techniques on a Java application

CONTEXT: Applying vulnerability detection techniques is one of many tasks using the limited resources of a software project. OBJECTIVE: The goal of this research is to assist managers and other decision-makers in making informed choices about the use of software vulnerability detection techniques through an empirical study of the efficiency and effectiveness of four techniques on a Java-based web application. METHOD: We apply four different categories of vulnerability detection techniques \textendash~ systematic manual penetration testing (SMPT), exploratory manual penetration testing (EMPT), dynamic application security testing (DAST), and static application security testing (SAST) \textendash\ to an open-source medical records system. RESULTS: We found the most vulnerabilities using SAST. However, EMPT found more severe vulnerabilities. With each technique, we found unique vulnerabilities not found using the other techniques. The efficiency of manual techniques (EMPT, SMPT) was comparable to or better than the efficiency of automated techniques (DAST, SAST) in terms of Vulnerabilities per Hour (VpH). CONCLUSIONS: The vulnerability detection technique practitioners should select may vary based on the goals and available resources of the project. If the goal of an organization is to find "all" vulnerabilities in a project, they need to use as many techniques as their resources allow.

cs.SE↗
arXiv2022

Dazzle: Using Optimized Generative Adversarial Networks to Address Security Data Class Imbalance Issue

Background: Machine learning techniques have been widely used and demonstrate promising performance in many software security tasks such as software vulnerability prediction. However, the class ratio within software vulnerability datasets is often highly imbalanced (since the percentage of observed vulnerability is usually very low). Goal: To help security practitioners address software security data class imbalanced issues and further help build better prediction models with resampled datasets. Method: We introduce an approach called Dazzle which is an optimized version of conditional Wasserstein Generative Adversarial Networks with gradient penalty (cWGAN-GP). Dazzle explores the architecture hyperparameters of cWGAN-GP with a novel optimizer called Bayesian Optimization. We use Dazzle to generate minority class samples to resample the original imbalanced training dataset. Results: We evaluate Dazzle with three software security datasets, i.e., Moodle vulnerable files, Ambari bug reports, and JavaScript function code. We show that Dazzle is practical to use and demonstrates promising improvement over existing state-of-the-art oversampling techniques such as SMOTE (e.g., with an average of about 60% improvement rate over SMOTE in recall among all datasets). Conclusion: Based on this study, we would suggest the use of optimized GANs as an alternative method for security vulnerability data class imbalanced issues.

cs.CR↗
arXiv2022

Reducing the Cost of Training Security Classifier (via Optimized Semi-Supervised Learning)

Background: Most of the existing machine learning models for security tasks, such as spam detection, malware detection, or network intrusion detection, are built on supervised machine learning algorithms. In such a paradigm, models need a large amount of labeled data to learn the useful relationships between selected features and the target class. However, such labeled data can be scarce and expensive to acquire. Goal: To help security practitioners train useful security classification models when few labeled training data and many unlabeled training data are available. Method: We propose an adaptive framework called Dapper, which optimizes 1) semi-supervised learning algorithms to assign pseudo-labels to unlabeled data in a propagation paradigm and 2) the machine learning classifier (i.e., random forest). When the dataset class is highly imbalanced, Dapper then adaptively integrates and optimizes a data oversampling method called SMOTE. We use the novel Bayesian Optimization to search a large hyperparameter space of these tuning targets. Result: We evaluate Dapper with three security datasets, i.e., the Twitter spam dataset, the malware URLs dataset, and the CIC-IDS-2017 dataset. Experimental results indicate that we can use as low as 10% of original labeled data but achieve close or even better classification performance than using 100% labeled data in a supervised way. Conclusion: Based on those results, we would recommend using hyperparameter optimization with semi-supervised learning when dealing with shortages of labeled security data.

cs.CR↗
arXiv2022

What are Weak Links in the npm Supply Chain?

Modern software development frequently uses third-party packages, raising the concern of supply chain security attacks. Many attackers target popular package managers, like npm, and their users with supply chain attacks. In 2021 there was a 650% year-on-year growth in security attacks by exploiting Open Source Software's supply chain. Proactive approaches are needed to predict package vulnerability to high-risk supply chain attacks. The goal of this work is to help software developers and security specialists in measuring npm supply chain weak link signals to prevent future supply chain attacks by empirically studying npm package metadata. In this paper, we analyzed the metadata of 1.63 million JavaScript npm packages. We propose six signals of security weaknesses in a software supply chain, such as the presence of install scripts, maintainer accounts associated with an expired email domain, and inactive packages with inactive maintainers. One of our case studies identified 11 malicious packages from the install scripts signal. We also found 2,818 maintainer email addresses associated with expired domains, allowing an attacker to hijack 8,494 packages by taking over the npm accounts. We obtained feedback on our weak link signals through a survey responded to by 470 npm package developers. The majority of the developers supported three out of our six proposed weak link signals. The developers also indicated that they would want to be notified about weak links signals before using third-party packages. Additionally, we discussed eight new signals suggested by package developers.

cs.CR↗
arXiv2021

Open or Sneaky? Fast or Slow? Light or Heavy?: Investigating Security Releases of Open Source Packages

Vulnerabilities in open source packages can be a security risk for the client projects that use these packages as dependencies. When a new vulnerability is discovered in a package, the package should quickly release a fix in a new version, referred to as security release in this study. The security release should be well-documented and require minimal migration effort to facilitate fast adoption by the client projects. However, to what extent the open source packages follow these recommendations is not known. The goal of this study is to aid software practitioners and researchers in understanding the current practice of releasing security fixes by open source packages and identifying areas for improvement through an empirical study of security releases. Specifically, in this paper, we study (1) the time lag between fix and release; (2) how security fixes are documented in the release notes; (3) code change characteristics (size and semantic versioning) of the release; and (4) the time lag between the release and an advisory publication on Snyk or NVD (two popular vulnerability databases) for security releases over a dataset of 4,377 security advisories across seven package ecosystems. We find that the median security release is available in under 4 days of the corresponding fix and contains 134 lines of code (LOC) change. Further, we find that 61.5% of the security releases come with a release note that documents the corresponding security fix. However, Snyk and NVD may take a median of 25 days (from the release) to publish an advisory for these security releases, possibly resulting in delayed notification to the client projects. Based on our findings, we make four recommendations for the package maintainers and the ecosystem administrators, such as using private fork for security fixes and standardizing the practice for announcing security releases.

cs.CR↗