Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2025
License: CC BY
Data sources: ZENODO
ZENODO
Preprint . 2025
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2025
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

Economic DORA: Practice-Level Analysis of DevOps Metrics in AI- Assisted Solo Development

Authors: Holford, Peter;

Economic DORA: Practice-Level Analysis of DevOps Metrics in AI- Assisted Solo Development

Abstract

I present an N=1 longitudinal case study of my own development practice using AI coding assistants (Claude Code, Claude Chat) over 57 days (276 commits) to build a production web application from zero to functional deployment. I introduce the Economic DORA framework, extending traditional DevOps Research and Assessment (DORA) metrics with token economics and granular method attribution. Through retrospective git commit analysis and development chat log inference, I identified five development practices with statistically large measured effect sizes: (1) proactive Architecture Decision Record creation before feature implementation (Φ = 0.89, p < 0.001), (2) structured problem agreement protocols (Φ = 0.63), (3) systematic documentation updates after incidents (Φ = 1.0 for recurrence prevention), (4) cross-session continuity tracking (estimated 20% time savings), and (5) Day 1 methodology setup (associated with 80% fewer failures over 60 days). A critical temporal finding emerged: methodology adoption timing mattered more than methodology existence—proactive adoption was associated with zero initial failures (0% failure rate across 3 features), while reactive adoption only prevented recurrence but not the costly original incidents. I estimate 63% token cost savings (1,600–2,100 tokens) and 71% time savings (2.5 hours) per feature when optimal practices are followed, though these figures are based on estimated token costs and require prospective validation. I introduce PRISM (Performance, Recovery, Investment, Stability, Method), a composite scoring system (0–100) extending traditional DORA by adding token economics (Investment) as a first-class dimension. The first four components (P, R, I, S) generate the scored composite, while Method Attribution serves as an explanatory layer revealing why scores change. PRISM’s Investment component (token cost per feature) provided leading indicators of degradation, declining from 20/25 (Elite) in September to 10/25 (Low) in November, one week before Change Failure Rate spiked from 2.4% to 31.6%. Method Attribution—a tagging layer tracking development approaches (PLANNED, QUICK, etc.)—revealed the rootcause: PLANNED commits dropped from 44% to 8%, correlating with PRISM degradation (r = −0.94). This demonstrates that token cost increases combined with methodology shifts can predict future failures, enabling proactive intervention before quality collapses. While limited to a single developer and greenfield project context, this work contributes: (1) a novel framework combining DevOps metrics with AI economics, (2) practice-level granular insights beyond aggregate method classification, (3) complete transparency with open dataset and classification criteria, and (4) a detailed replication protocol for N=20 validation studies. I openly acknowledge limitations including retrospective classification bias, estimated (not measured) token costs for the study period, and uncontrolled confounding variables, and I invite critical peer review and independent replication. I invite collaboration on prospective validation—contact me to participate in N=20 replication study.

Keywords

DevOps, Case Study, Software Engineering, Software Productivity, Token Economics, DORA metrics, AI-assisted development

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
Green