Software factories

The holy grail is a self-evolving system where a feature request is magically built in the existing system. Side-effects aside, whether and when we will reach this target remains to be seen. We’re certainly on path towards the target with teams who get a sense of this future reality in parts of their processes.

As teams work on codifying steps of the development process, we’re seeing more and more proof points for reusable workflows that accelerate release cycles. A single bug or feature request can now be designed, coded, and verified in an agentic loop, if supported with the right tools, platform, and guardrails. Teams that have well-engineered systems and high engineering rigour benefit the most. It’s all about low build times enabling fast iteration, modular architecture making code changes cheap and simple, and high test coverage enabling continuous delivery to users.

Better models enable multi-hour execution for increasing scope, triggering the move of agents from the local developer machines to micro VMs that host them. Leading teams can reverse-engineer existing code, design a replacement and the needed guardrails, and build loops that will eventually result in a running system that passes all checks. This approach accelerates migrations and codebase modernization. Guardrails also become more autonomous with highly specialized background agents that compensate for lower quality code generated by coding agents and that protect invariants or stability of the system. Examples include agents that refactor code, apply reliability patterns, write missing tests, reorganize test suites to reduce execution time, etc.

The key question remains where the developers are in this process. For the foreseeable time, they will build and operate the backbone that builds the system (thus the name software factory) and set the constraints under which the system is allowed to evolve. Depending on business needs and appetite for risk, teams can choose from a set of hand-off points where human interaction and sign-off is explicitly required:

• idea->plan: explicit decision whether to act on the idea or feature request with focus on keeping the scope of the product and key user journeys in place.
• plan->execute: humans review plans or specification, increasing the likelihood of understanding the system. After approval, the system takes over implementation, verification, and more.
• change request->deploy: final pass code review, if required by regulatory needs.
• incident->remediation: the on-call person takes a decision on remediation making the needed trade-offs taking their business knowledge and current state of the system into account.

These handoff points are not entirely new, yet these points seem to be a good anchor for explicitly keeping the software delivery in check. Each of the steps is prone to benefit from further automation supporting the decision making process. For change request analysis, the person can be provided with a measure of risk of the changes, a summary of the changes can be automatically tested against the specification or regulatory guardrails. To speed up incident response, the on-call person can receive an automatically collected summary of the state of the system and changes around the time of incident.

“You build it, you run it” vs. “They build it, you run it”

The “You build it, you run it” approach has promoted healthy practices in teams and increased reliability of the supported systems. With citizen developers (non-engineers who build apps with AI assistance) getting abilities to build applications, it’s tempting to drift back to “They build it, you run it”, placing the burden of ownership and operations on an existing engineering team.

A question arises how we can avoid creating a culture of throwing AI-generated apps over the fence, just to be operated by experts who never wrote a single line in any of these apps? Training citizen developers to have intuition on software and security risks is going to pay off in general. Yet, requiring everyone to acquire a baseline set of skills of software engineers is not the only path. Typically, it’s platforms that separate concerns, offering user-level abstractions that run on top of a solid foundation.

This foundation essentially needs to operate itself and be self-healing in terms of known failure modes, beyond simply recycling unhealthy compute nodes. For example, for applications with databases, it’s possible to add automatic volume increase to avoid running out of space or adding an index to speed up expensive queries. Effectively, the idea is to make these failure modes disappear (in most cases), reducing the burden on the user to know about these concerns from the get go and by setting healthy constraints (e.g. increase disk space max. n times). Existing vibe coding solutions from vendors typically fall short when it comes to accessing data and APIs from production systems. Usually, this leads to static data being embedded in applications, resulting in bypassing access control for data or public leaks in the worst case. The platform can and should ship with mechanisms protecting users, setting the right access and data guardrails.

The challenge remains on what the starting point of the citizen developer journey is. Developers start with a repo or clone an existing template with scaffolding. Citizen developers require to be guided through the creation process from the very first prompt they write. On top, this needs to happen with as little friction as possible so that they (and the agent) do not look for workaround. Defining tight constraints enables a progressive build out of the platform and sets clear expectations to the user on what’s possible for them to build, be it a static website, a data visualization dashboard, or a simple tool with limited access to data from internal systems.

Will we still need IDEs?

Traditional IDEs ship with tools that help navigate the code base and perform deterministic code transformation. Yet, popular plugins from coding agent providers reduce the role of the IDE to a mere terminal/chat window and rendering of large change sets. Thus, it’s easy and common to skip IDEs completely, favoring the popular coding CLIs where reviewing large plans/specs and preparing prompts is a very barebones and basic experience that surprisingly works well enough.

As IDE users migrate off to CLIs, the question is what features could help slow down this trend. The incumbent IDEs are clearly missing out on:

• exposing access to deterministic IDE tools for transformation and refactoring, lowering the token bills
• access to the IDE environment for compilation, running tests, reducing failure rate in coding loops
• annotating agent output with feedback (visual and text) to increase accuracy in prompting
• native support for agent skills with support for verification and (back)testing
• multi-session orchestration to avoid conflicting changes done by agents in parallel
• coding session insights with usage statistics and view of agent traces

Some of these features are already available via tools (e.g. codeburn, agentsview, kit), but bundling them into a (new) product is highly likely to improve the agentic engineering experience. Coding agents also ship with OpenTelemetry support, containing telemetry signals that can be captured and visualized. Visualizing agent session telemetry comes with new challenges related to data privacy as traditional observability tools are designed to provide broad access to application telemetry, rather than offering access to user-level data.

Growing gaps

It’s well known that agentic engineering amplifies both the bad and good practices in teams. Teams that ship in fast cycles will likely continuously widen the gap between themselves and their peers who are slow to adapt. It’s impressive to hear from new startups that invested in codifying processes early on and how they’re benefiting from the acceleration that agentic engineering offers.

Small companies benefit first as it’s much easier for one team member to build the set of guardrails in code and shared skills. Growing from this early base helps accelerate delivery at increased consistency.

Existing companies face the risk of increasing fragmentation on process and tech stack, reinforcing status quo (e.g. adopting microservice architectures with a high degree of freedom of choice for the tech stack and lack of platform support and/or application level SDKs). Common approaches are encouraging the build out of shared skill marketplaces and shared tooling that supports the engineering process, yet the fate of these efforts highly depends on organisational structures and culture of re-use.

A future where developer platform teams step up by one layer and provide a unified process of building software is likely. However, working styles and practices differ significantly on individual level, often being part of one’s identity and skill differentiation. Forcing large groups of engineers to adopt the same structure can become too constraining and limit innovation, even if sufficient extension points for customization are provided.

Summary

During the event, we discussed many proof points for future timelines, yet many questions still remain open. While it’s clearer where human judgement will be most needed when agentic engineering takes off and crunches through product backlogs, it’s less clear what organisational structures and shapes of teams will prevail or correlate with successful outcomes. Unclear terms reveal how young the discipline is with the added level of complexity of social media hyping up names, without triggering prior discussions on these amongst practitioners with decades of engineering expertise.

The future is certainly exciting and requires leadership from practitioners to influence and help understand trade-offs along the journey we’re on.

The sheer influx of information and day-to-day developments is really hard to follow. Just in January Steve Yegge released a new take on the IDE with Gastown, only to reflect a month later on being bitten by the AI vampire, highlighting the addictive side of building software with agents. The impact of FOMO and pressure on mental health is widely discussed in the industry as well, given agents can amplify undesired behavior. OpenClaw triggered euphoria, showing the power of agents to a wide audience, further accelerating the FOMO. In the meantime, its author Peter Steinberger was acquihired by OpenAI, with the project transitioning into a foundation (see State of the Claw for a recent update). Others claim not to be fooled again, recollecting MS-DOS times.

The key question is where things are going medium to long term, hence the title of the post. My goal is to highlight some of the constraints, mechanisms, or factors that will influence how Agentic Engineering evolves, putting structure into notes that I’ve been collecting for a while. Feed the text to your agent for a TL;DR or enjoy reading just like I enjoyed compiling the material and typing this post.

Leveling the playing field

Equipped with coding agents, non-experts can create software and debug problems they encounter without the need to rely on their Google-fu, know the right keywords, or seek support from colleagues. They can build tools and businesses end-to-end in a way that was previously inaccessible to them, benefiting from the drastically lower cost of creation. Surely, many non-experts will have a different objective in mind when coding, seeing it merely as a tool and a means to get the job done. The created software may never be used by more than a single user. When it does, though, it will require security hardening, productionization, and a safe place to run. Otherwise, it will immediately become a security liability.

The (software) craftsman among us are worried about skill atrophy in the age of LLMs. The bigger risk worth acknowledging is lack of skill formation as non-experts become dependent on tools without forming deep enough understanding of their internals. Provider downtime can feel like a blackout: work simply stops. On a positive note, we use things all the time that a single person cannot build from scratch on their own and we’re perfectly fine with this. “I, Pencil” retold by Milton Friedman offers a 2-minute lesson on the complexity and connectedness of our world.

Looking at implications for software teams, platforms and developer experience teams are challenged to accommodate an expanded range of software contributors, going even beyond product managers or designers. Teams will need to tune their assumptions about development environments, onboarding approaches, and create new safety nets, lowering contribution friction. It is also an opportunity to unify the tech stack and deployments of internal apps, forming portal-like marketplaces where apps can be easily adjusted, remixed, and integrated with existing APIs and tools. These managed apps could end up as a remixed experience of codepen.io, glitch.com, and Google’s AI Studio. Without these platforms, non-expert authors of software will be searching for a team to harden and operate their creation, directly breaking “You build it, you run it” principles that many teams follow today.

What bottlenecks will we hit?

All of them. Many times. Our engineering processes and the underlying platforms were built to scale with human activity. Agentic Engineering lowers the cost of producing changes faster than our existing systems and processes can adjust.

Systems built for human-driven workloads

When Netflix’s platform is spinning up cloud compute to stream a video, there is a human using a device to access this video. When a customer service agent picks up a phone, it’s because a human had an interaction with the business and something went south. When existing system constraints were challenged, there were usually clear incentives: marketing (spam), building influence (misinformation through media content), financial profit (sneaker bots, event ticket bots, HFT, fraud, DDoS, malware). In all these, we observed the effect of leverage with a human operator or human-in-the-loop somewhere in the process. Agents will bring these effects into a multitude of places, far beyond putting it into the hands of the next generation of script kiddies.

Lowering cost of contributions and need for improving verification

Lowering the cost of contributions results in more incoming changes. Many of these changes would not have existed before as the cost of doing them would be too high given the value. Now, a change is developed and submitted quicker than it would have taken to assess the need for this change in the first place. When going through existing processes, these code changes trigger code review requests, CI/CD runs with builds, tests, artifact uploads, security scans, etc. Any of these steps that are slow, cannot run in parallel, or require extensive human involvement will result in inefficiencies and frustration that will continue to pile up. These steps used to be to a large extent correlated with human activity and subject to human-level constraints. Dependency updates, if automated were largely expected to be non-breaking. Well-run engineering orgs or large orgs that needed to accommodate a high number of contributors, adopted practices that helped them scale (or reduce costs and lead time), such as Spotify’s fleetshift for fleet-wide refactoring. Other teams may have never seen the need for optimizations or assessed them as having a clearly negative ROI.

Agentic Engineering benefits from rapid verification cycles. A few minutes spent waiting for a PR build to complete or for a code review to come in directly affect the dopamine hits that operators of agents experience. To accelerate, various strategies are needed, such as splitting test suites, using multi-stage builds, being able to launch the application in parallel (locally or on a devbox), and test automation helps keep verification small enabling agentic coding or optimization loops. As release frequency increases, relying only on real-user A/B tests may become too slow for early iteration loops, so simulated traffic from synthetic personas may be used more often as an early signal. To manage incidents, we have established practices in SRE where automating runbooks is far from being novel, yet a practice that’s way more accessible than before. Literally all existing tools and practices need to be challenged, adjusted, or dropped. Early innovators’ products are likely to be absorbed and integrated into existing, established platforms to cope with the pace of development.

Drowning in code and loss of system understanding

When agents contribute code, change sets tend to increase not only in frequency, but also in size. Does it make sense to review the large PRs? Tools like devin help break down PRs into semantic chunks, thus lowering the burden on larger PRs. One can also ask the agent to commit in small chunks and leverage stacked diffs for reviews. However, given a high rate of rework from agents on the same files, what’s the ROI for code reviews on every PR? Maybe setting quality and security guardrails that when met result in an automatic merge are enough? Looking at human constraints on time, allocating fixed time chunks for the team to review changes from the current day, collectively reading and building a shared understanding of the codebase may be a smart tactic. Getting comfortable with agents writing code without supervision will require more platform support (sandboxing) and engineers getting comfortable with building without writing, focusing on agent coordination. A world not every craftsman will enjoy.

To cope with the increasing amount of code that needs to be understood in a structured way, we’re seeing foundations of new tools being built. Codebase size influences the ability of agents to reason about it and affects iteration speed as inference time is correlated with the number of input tokens. Projects like cased/kit, GitNexus, or GitLab Knowledge Graph aim at providing tools that index codebases and expose their symbols or structure in a more efficient way when compared to (rip)grep. Coding agents also support Language Server Protocol (LSP) servers to access IDE-like code navigation features and jump around the codebase.

Specs instead of code?

Spec-driven development can complement Architecture Decision Records (ADRs) by turning intent into something executable and verifiable. What’s new is that full applications can be built purely from the spec. First projects start shipping with disclaimers: “use an agent to make your own” based on the released spec with a tech stack of your choice. This approach brings us closer to a scenario where software can be rewritten on demand, with far less manual implementation effort. The key is in the verification stage: being able to verify the adjusted acceptance and verification criteria coming with the next iteration of the specification. I believe I’ve seen an ERP company have their product work this way, but I cannot find the reference anymore. An approach like this would also mean that framework upgrades and migrations can be executed in a similar fashion, addressing a large chunk of technical debt that exists today.

Cognitive load of change

Another bottleneck will be on the human side. We’re used to a certain pace of software development and delivery. Further acceleration increases the cognitive load and challenges our ability to reason about the changes across systems and codebases. Increasing the number of changesets, their span, and the number of changes still in flight leads to a significant explosion in scope and complexity. It’s not unlikely that the already observed differences between high performers and the rest of the teams, will start requiring structural changes to our teams. It remains to be seen which exact ones.

Effect on Open Source

Open Source is a prime example of bottlenecks, especially when critical projects are maintained by a single person. GitHub sees an influx of activity on their platform, on track for a 14x increase of commits. At the same time the platform has less than 90% uptime showing the pressure their SRE teams are under. Looking at repo creation stats, it’s ~331k repos per day, with clearly increasing momentum since the start of 2026 ². GitHub also published own statistics proving the surge of activity ³ and highlighting that planning for 10x load increase was insufficient, requiring a pivot towards 30x.

Diffusion of quality and loss of discoverability of new projects

The influx of new repos affects discoverability, for example for hot topics such as coding agent sandboxing. Try finding which of the ‘claude sandbox’ projects is good enough to be used safely. Sifting through the project list is more time-consuming than it used to be, because it’s much harder to assess whether a project just looks good or whether it actually works and what its quality level is. Large PRs break the UI, making it harder than needed to review incoming contributions. Faced with an influx of activity, spam PRs or comments, maintainers observe past incentives being put out of balance. The current training data for LLMs is built on the prior generation of OSS projects. Many of these were high quality and key dependencies across a large number of projects. As incentives shift, the question is how strong and resilient the ecosystem really is. Open Source used to be a way to tap into developer capacity, especially the most committed community members willing to contribute value. Supported by agentic coding, some projects consider closing down their projects to trusted contributors only, making more efficient use of their time. There is even a GitHub feature helping to limit PRs to contributors only and vouch as an experimental project for trust management and means to reinforce the strong links in the ecosystem.

AI contribution and attribution policy divergence

The community is torn on handling AI-assisted contributions. Linux Kernel contributions invite explicit authorship tags (Assisted-by) whereas Kubernetes explicitly bans them. Adrin Jalali, one of the core maintainers of scikit-learn, published a piece outlining different strategies for maintainers of open source projects, recommending creating agent guidance files (e.g. AGENTS.md) aiming to help increase quality of incoming AI contributions. Melissa Weber Mendonça maintains open-source-ai-contribution-policies with a collection of AI policies across the ecosystem.

Security triage burden

In addition to code contributions, projects now face more incoming issues and security advisories. Claude Code has a bug command that creates github issues. Looking through the types of issues discussed is a mix of amusing and frightening. curl is known for receiving an influx of low-quality vulnerability reports, struggling to sift through them. To adjust incentives, they stopped their bug bounty program. The problem here is the signal-to-noise ratio that hides the valuable reports.

In the State of the Claw talk, Peter mentioned that OpenClaw had 1142 security advisories since January 31 (>16 per day) with an acceptance rate of 41%. This was an estimated 5700 hours of work over 69 calendar days (5700h = 237 calendar days or >700 working days). This particular project is popular enough to attract enough attention from both ends: contributors and attackers. Not every single critical dependency in the software supply chain will be lucky to have enough hands on deck and a foundation structure to support governance. We see first signs of large-scale attacks with ripple effects from the attack on trivy propagating through the ecosystem.

Compute capacity shortages as a driver for innovation?

When limits are healthy

Constraints are great as they lead to reflection and innovation. As the AI datacenter build-out affects the whole supply chain with memory, storage prices, and availability of GPUs and CPUs (and other components), compute efficiency will hopefully matter more and more. We’ve become complacent and indifferent to resource usage as an industry as it’s become so easy to just bump memory or CPU in the cloud instead of spending the time to profile an application and understand reasons for performance bottlenecks. The local inference movement with llama.cpp is a great example of a successful movement aiming to bring inference to local machines, with their compute constraints. Same goes for on-device inference with ExecuTorch.

Limits on coding plans are often the only reason for a person to ask a question: Could I have used a cheaper model? or Was this really the right task for a coding agent? Without constraints, it’s too easy to just continuously run on the currently most capable and expensive model, an approach that is not sustainable long-term. Lack of limits reinforces bad behaviors. Too tight limits do not allow users to experience new capabilities in action.

GPU capacity constraints and hidden price hikes for models

GPU capacity shortages will also play a key role moving forward. They could be a possible explanation for features such as adaptive thinking, now enforced for new models and deprecating the feature for existing ones. Capacity limits would also make subtle bugs in prompt caching hurt much more (Anthropic since released a post mortem on the quality issues caused by changes shipped: default reasoning effort, cleaning thinking sessions, system prompt changes). While the length of tasks that LLMs are able to solve keeps growing exponentially, it remains to be seen whether token usage rises exponentially as well. Anthropic’s 10x growth rates would suggest that, yet we lack clear data on costs of AI agents.

What we definitely see is explicit price hikes with releases of OpenAI models: gpt-5.4 is 11% more expensive than gpt-5.2/5.3 which is 40% more expensive than gpt-5.1 - a total of 55%. Surely, the model is more capable, though one has to ask whether the models are really fed with incrementally more complex tasks by all users? Without clear, task-specific evals, engineering teams often opt to switch their coding model to the newest one as this is the (now) recommended one, feeding the FOMO on one end and filling the revenue hat on the other. Many users do not have the tools or the capacity to create their own, task-specific benchmarks (yet). Hopefully, they take the time to do so for customer-facing products as this has clearer ROI. If not, they will experience funny inference glitches, such as links being injected into LLM outputs where a single word was expected.

Anthropic’s price hikes are more subtle and hidden in features like adaptive thinking or system prompt that change the behavior of their harness or those that are more visible like the tokenizer change in Claude Opus 4.7 resulting in 38% higher request cost. On the other hand, they’re also more explicit with recent updates to the Enterprise plan, switching to usage-based pricing by introducing a $20 per seat price just to get access to tools with all interactions billed at API price level. No more free riding and no quotas. Reduces complexity, and challenges existing budgets.

GitHub feels the heat as well, adjusting their individual plans, pausing sign-ups, reducing model availability of Opus to the highest tier only as well as retiring previous Opus versions. GitHub’s move to offer Opus 4.7 with a promotional 7.5x premium request multiplier (vs. 3x for Opus 4.6) was a 2.5x price hike as well, a prelude to shifting the whole platform to usage-based billing. This just gives users more arguments to run with Opus-class open-source models like the newly released Kimi K2.6. Frequent changes to pricing models of established providers are prone to push more users toward open-weight and local alternatives, strengthening competitive pressure on proprietary vendors.

Pricing models, revisited

As products and platforms start getting agent-ready, they are forced to open up access to agents via official interfaces: CLIs, APIs, MCP, etc. Google’s NotebookLM is famous for not having an API, requiring hacks for agent access and possibly risking account bans if crawling will get classified as abuse. Many other platforms got away with not releasing programmatic access to their platforms, locking in their users and making migration between products unnecessarily difficult. With Salesforce announcing their Headless 360 initiative, other players will feel even more pressure to catch up. The result will be two-fold. Firstly, users will (hopefully) gain an ability to integrate and migrate between platforms with more ease. Secondly, existing pricing models of platforms will be challenged.

All the problems we’ve seen with coding agents will also show up here. Seat-based pricing will get challenged as activity on the accounts will get an additional component, not correlated with human activity anymore. Public platforms dealt with automated traffic via waiting rooms or bot protection products, incl. recent releases allowing to take in payments for access, giving http response code 402 Payment required a new life. That’s not going to fly for SaaS. The growing share of agent traffic will put a strain on margins, leading platforms to rethink their pricing models. An approach where “agents must buy seats just like human employees” is unlikely to succeed long-term as the usage patterns of agents will continuously evolve and become more complex. Assigning agents subsidized quotas would result in agents gaming the system, registering accounts to be used in parallel or just running at the full utilization of rate limits all the time (like crawlers do). The usage-based models that the coding plans are converging on are much more likely to be applied more broadly to API-based products.

Summary

Roughly 18 years ago, I had a course in Artificial Intelligence at university where the professor was explaining how an AI could compose computer programs through natural language analysis. What felt like a complete abstraction existing purely on the whiteboard is now a reality used by millions. Having witnessed both sides is fascinating and humbling at the same time.

The ecosystem around Agentic Engineering is evolving in parallel with an accelerated race to ship new and more capable models while dealing with constraints in GPU capacity. This mix of challenges in processes, tools, and infrastructure build-out results in fast-paced change with implications that cannot be easily foreseen. We will continue hitting various bottlenecks as progress is made, revisiting old approaches or adding new ones. All we can do is embrace the uncertainty and adjust course when needed. With the bottleneck shifting from generation to orchestration and verification, we have an opportunity to close some of the gaps in processes and systems that existed for a long time, thus having an opportunity to improve the industry as a whole. To achieve this, we have to be able to teach Agentic Engineering while new ways and approaches are being figured out.

More tools to come, more approaches to be tried out. As exhausting as it is to ride the wave of change, it’s highly rewarding as well.

Updated on Apr 24th: Referenced Anthropic post mortem on recent quality issues. Updated on Apr 29th: Referenced GitHub’s move to usage-based billing and added stats from update on GitHub availability.

The sprawl of tooling and practices along with viral content circulating through social media makes many people uncomfortable. Many look for direction, best practices, or clear instructions on what the state of the art is. The reality is that nobody has it all figured out yet and many of the shiny things shown on social media are demos showing the happy path. Mario Zechner, the creator of pi.dev (the harness used by OpenClaw), poured cold water on the hype, asking for more discipline and reason when using agents. Martin Fowler said recently that in XP and agile terms we’re at the stage comparable to the early 90s full of experimentation and new ideas. Thus, it’s too early for a manifesto.

Because it’s still early days, authors of tools have not yet figured out how to build their products with consistently high quality. We see this in the regressions in new versions of Claude Code. Non-determinism adds another layer of difficulty with new failure modes that need to be learned and mitigated. For example, compaction can result in lost instructions, which is non-intuitive to users. It takes time to build the intuition and mechanisms that help mitigate problems end-to-end. Early days of SRE were similar, though we were at a stage where companies with sufficient scale had a time advantage as they hit the limits quicker. This gave time to build up and codify some of the knowledge in articles or books.

Learning journey for Agentic Engineering

A pragmatic way of building organizational know-how is through guided experimentation where a small group of power users defines the set of experiments or practices to try out. They take on the risk of trying out something that won’t work out, but they also guide participants throughout the learning journey with existing knowledge they have. Oftentimes, their time advantage in using the tools or practices is measured in days. The guided experiments can be time-boxed (e.g. 4 hours) and done in pairs to facilitate peer learning, ideally across teams.

What is important when considering teaching agentic engineering is that everyone’s journey is different. Those who jumped on LLMs early on and kept following the evolution of models, coding assistants, and similar tools have a vastly different intuition about the capabilities than folks who have been resistant to trying them (or who could not make prior generations of tools work). Thus, the early adopters are also those who are most likely to thrive in the experimentation setup mentioned.

Looking for more structured approaches to learning, we find models with 8 levels of agentic engineering. A mental model that worked for me is slightly simpler, directly rooted in the level of autonomy and size of changes delegated to agents:

• Suggestion mode, aka. Tab Autocomplete where engineers get familiar with whole chunks of code being suggested instead of just function names and arguments.
• Interactive mode with Agents in IDEs allowing to interact with the codebase through prompting, asking simple questions and progressively requesting more and more complex code transformations of features.
• Supervised execution with Agents in CLIs with engineers starting to scope larger tasks and acceptance criteria leveraging the planning mode, but accepting every single command from the agent.
• Delegated, autonomous Agent Loops in CLIs where building up on the prompting skills, engineers build verification methods and safety nets allowing them to diligently plan large tasks and let the agent crunch through them autonomously.
• Autonomous Background Agents that run in the background and react to defined changes in the system, e.g. commits, pushes, or periodically scan the codebase or running software deciding when to activate and what procedure to run.

Meet users where they are on their learning journey

When teaching agentic tooling or practices, it helps to understand where a person is in their learning journey. This helps to present ideas new to them in ways that stay close to their current comfort zone. Seeing is believing, so make sure to add a healthy dose of live demos where people present what worked for them using real-life examples, not just fancy demos. When designing trainings, these levels also need to be accounted for, either in expressing clear preconditions for entry or in making sure that no single approach is favored.

The above list represents a spectrum of working modes rather than a strict maturity model. This means that there is no single right way, though there are many wrong ones. Certainly, the outcomes and token usage differs between them. However, it’s important that engineers find their own mix that feels right for them, their level of tolerance for risk and for their team/product as well as budget. This will challenge existing team structures, likely leading to some reshuffling to bring colleagues with similar development styles together (temporarily or permanently), so that they continue to raise the bar and to facilitate peer learning. In order to avoid teams getting stuck in local minima, the big question is how one can coach whole teams?

Teaching teams

Teaching teams used to be a task for the engineering managers who could rely on their past hands-on know-how and credibly advise their teams. Now, they’re faced with a situation where time for hands-on coding is limited and where they may need to learn new tools first, before they’re able to teach their team. They will need to be more intentional about their own learning, leveraging power users to acquire skills or hints on what to look at. Waiting for EMs to learn before they can coach their team is of course too slow.

Power users can help coach teams effectively. The key is to break down the agentic engineering cycles that experts do in one go into meaningful phases, adding inspections along the way. Inspections can be done individually (pairing, code review) or as a group (like in swarming). Let’s say we want to teach a team to author implementation plans in plan mode. While an experienced engineer would do plan and execute in one go, we can break this into two steps. First, the plan is prepared and checked in. Next, it’s reviewed along with the prompt used. This helps transfer the intuition that comes with experience and potentially capture some recommended practices (or constraints in agent instructions). Afterwards, the plan is implemented and results are reviewed. Similarly, one can teach prompting patterns, e.g. focusing on reviewing code for certain types of defects.

Scaling beyond teams

To scale further, we can learn from the OpenClaw moment in China where companies set up events that offered hands-on support for setting up the tool. The key is to meet users where they gather anyway to spark their initial interest or incentivizing them to stop at easy to access places. Setting up booths near office entrances, kitchen spaces or in cafeterias comes to mind here. Same goes for workshops and demo sessions at internal conferences or town hall meetings. While these formats can spark interest and drive attention, lasting effects depend on curiosity of the attendees and structure of workshops delivered.

When setting up formats for cross-team knowledge exchange, it’s important to provide a platform for early adopters to share and showcase their wins, enabling them to earn some peer credibility as experts and ensuring they get connected with other experts. To balance out the focus on experts, formats where leaders can discuss adoption friction are needed as a safe space to discuss organizational change and gaps for enablement. For further scaling, one would usually seek champions per organization who would be acting as multipliers. Forming a network of champions, they support each other in sharing lessons learned. Whether champions are chosen among early adopters who earned some visibility by sharing their knowledge or by role is highly dependent on the organizational culture and the seniority required for the change management process. Key to success is organizational backing by sponsors from leadership who can help remove organizational blockers.

The Global Day of Code Retreat is coming soon. Going down the memory lane of Coding Katas, Test-Driven-Development and the related workshops I had the opportunity to attend/host, I got reminded of the long-term value of these exercises.

This posts showcases strategies for shipping software in iterations, which build on the fundamental practices introduced in these workshops. Just like the small steps in coding katas, the strategies outlined here may seem obvious or too simple to leverage in a real-life project, until one encounters the right project to apply them in.

Introduction

Code Retreat workshops

The Code Retreat format promotes Test-Driven-Development through coding katas and injects added fun through pairing, learning from one another, and making the exercises harder through constraints (e.g. no loops). While Katas may feel simple and a repetitive exercise helping getting into flow or learning a new programming language/construct, there is an implicit concept they teach us. It’s all about the simple iterations and sequences of tests that one defines in order to complete the set of requirements in an iterative way. The important bit is that the complexity level may only increase in a gradual way. In the meantime, this concept has been codified as Transformation Priority Premises, which to the familiar eye will look like an extension of the intuitive evolution of null → single result → two results → loop, etc.

Elephant Carpaccio

The Elephant Carpaccio exercise takes the concept of developing in iterations to the next level. Its aim is to teach how to slice a feature into very small iterations spanning all layers. One is asked to define a detailed iteration plan and set of acceptance criteria for each increment. The caveat is that each iteration is to be delivered in minutes! (without Copilot / GPT-3 support ;-)) While tricky at first, with some practice during the workshop this gets rather fun quickly. Try it in your team if you haven’t yet!

Ship early and ship often! (single feature)

More often than not, developers have the tendency to ship features only when fully ready, delaying integration and increasing pull request size (and as a result the lead time). For the team, it’s hard to understand the quality and progress on the feature as the code isn’t used in production and never deployed. As most problems happen in production, this doesn’t sound like a good strategy, right?

A common technique in teams practicing trunk-based development is to hide the feature behind a feature flag. This allows for frequent code integration and avoids unintended production usage. It’s a step forward, but heavily missing out on the value the code may have from actually being used in production continuously, even when the feature is not fully finished (or not even a single line of code of logic is written).

Testing latency assumptions

Whenever new service calls or complex calculations are are added to an application, the overall response time of the application may increase. Typically, services will have a defined latency SLO for its operations, which clients rely on and use to define timeouts. In other cases, increased response time will have immediate business implications as users will interact with the software less frequently as it gets slower or abandon the application along the journey. It’s possible to use A/B experiments to validate the latency buffer a service has, but these may take weeks to reach statistical significance. Hence, there is value in learning early in a project that the pure added latency triggers undesired business impact for users.

Instead of waiting for the feature to be fully implemented, shipping code that’s a no-op in terms of business-logic, but such that simulates the processing time allows to verify latency impact of the new functions. When code runs in production with the added latency, impact on processes, end-user KPIs will become immediately visible, which helps to validate that the chosen design approach is viable in production early in the development cycle. As clients spend more CPU time to process the responses, this also provides early insights for capacity planning.

To prepare for adding business logic, we add an execution budget after which the function terminates automatically and returns a fallback value. Lastly, business logic can be added in multiple iterations, relying on the guaranteed execution time to cover for performance inefficiencies. At this last step, the added latency is optional and can be dropped as soon as the function returns values for all input combinations.

The diagram below demonstrates the iterative evolution of the function code added to an example application. Each iteration ends with a production deployment. The 4th step should have additional iterations to develop the business logic.

Verifying inputs and discovering edge cases

When faced with business logic needing to process many input parameters, it’s helpful to learn more about the input parameters and data access patterns. Frequently, this task cannot be done based on data dumps.

Shipping code that just processes the input parameters allows to calculate the distribution of input parameters, estimate the likelihood of certain edge cases to occur in production (and these tend to occur more frequently than expected), or to record the access patterns based on combination of inputs or frequency of use. This helps in selecting the right data structures for efficient processing, define strategies for populating caches, etc. Long-term, the already developed input verification procedures can help detect data skew or inform that new use cases were added to production, which may invalidate assumptions taken during development.

The function can either store the statistics for requests in memory and/or log these with every n-th call. To minimize impact on the main execution flow of the application, incoming traffic can be duplicated into a separate application that logs required statistics over time.

Shipping logic in iterations

Just like in TDD where the passing (acceptance) tests demonstrate real progress, the iterations of the code you shipped showcase progress that’s verified in production. This allows to understand when a certain iteration triggers problems in production. As soon as first use cases are ready for production use, the service can invoke the logic for these asynchronously and log results for offline verification.

For simple cases, it’s sufficient to log the input and output to perform offline verification of the results. For complex cases, like rewrites or migrations (see picture below) it’s advised to record the result of both operations and to perform additional asynchronous result comparison, helping to measure and log the correctness of the performed operations (often referred to as the Parallel Run Pattern). These comparisons also showcase the incremental progress of the development work, which is much needed in migrations.

Given sufficient confidence for a subset of use cases, the source of truth of the operation can be shifted to the new implementation, which is the essence of the Strangler Fig Pattern.

Being responsible!

Production deployments of the described patterns require high discipline. For high traffic use cases, it is advised to invoke early stage code infrequently, e.g. for a fraction of incoming requests or through capping the number of executions per time unit. The mentioned technique of leveraging latency budgets by ensuring maximum execution time of the executed code helps in ensuring that early stage code has predictable runtime implications. Impact on memory footprint needs to be managed carefully as well. This is easy to achieve through limiting the size of any intermediate results or statistics computed, stored, and logged. Unsurprisingly, these are practices that come in handy nonetheless and would be built up eventually over time, often as a result of production hardening or (worst case) in response to incidents.

Untangling dependencies! (cross-team projects)

The strategies described so far apply mostly to single functions or applications. In cross-team projects, dependencies are typically the limiting factor for successful project delivery on time. Dependencies in larger projects cannot be avoided and need to be managed. Most likely, you will be familiar with one of the following situations:

• Frontend teams waiting for a backend API to be ready, so that they can start building UI widgets and wiring it to the provided data.
• Backend teams waiting for dependencies on APIs providing the data/fields they need to calculate results.
• Analysts delaying building dashboards until the time when data is available in production. Unless, built-in in the UI frameworks, interaction data tends to be de-scoped until the very end.
• Teams waiting for UI components to be ready and shown in production, so that they can verify the end-to-end user journey by clicking a button to initiate an action as part of their tests.

The waiting time in each of the scenarios is the factor to focus on when improving project delivery speed. Here, lessons from the Elephant Carpaccio exercise come in handy and trigger thinking in end-to-end iterations. Coupled with addressing identified dependencies early on in the project it’s a powerful combo.

Shipping the simplest slice

The more dependencies, the more it’s important to manage scope and optimize for early integration. Just like the slices defined in the exercise, the value of small iterations shipped to production cannot be understated. More often than not, simplest will be counter-intuitive to what one is used to.

Building a complex UI widget with a CTA button? From a dependency point of view, it’s the button that really matters as it links data with the destination. Shipping a UI component with just the button unblocks teams building and testing the triggered action along with providing the click events/data. When looked at visually, the widget may feel odd as it would not be shown to your users, but reducing dependencies enables the team to focus on further iterations of the UI component itself. It’s also a great starting point to reserve screen estate for the full component and observe its incremental evolution. Naturally, such change requires to be hidden behind a feature toggle, so that this button is shown only when desired during testing.

API first

While widely practiced through IDLs or OpenAPI specifications that help auto-generate API clients, just the specs may not be enough. To untangle dependencies, beyond agreement on the API contracts, it’s essential to provide a set of sample responses when the code is invoked in production. Such responses can be then returned by API endpoints serving static responses, alternating through sample requests or switching responses based on input context. In case a new service needs to be built, a simple web server serving static files may be enough to start with (even if later replaced by a different application). Re-using any contract tests is helpful here as well, as long as the responses are used in early stage development, in production.

Early end-to-end integration

The earlier features built by multiple teams are integrated with one another, the better. It’s helpful to align on a small end-to-end deliverable across all teams early in the project. In this iteration, API endpoints with sample responses and clients for those APIs are developed to invoke APIs and process (at least) a few API parameters (it’s worth noting that events are also APIs).

When the integration in production succeeds, it ensures that every piece of the puzzle can be further iterated on with relaxed dependencies. Early deployment to production enables everyone to observe incremental progress, incl. interaction events in analytics systems. Moreover, when the features are treated as production-grade from the start, the team continuously learns about daily operations of the system.

Lastly, through simulation of system (end-user) activity, non-functional requirements can be verified uncovering performance bottlenecks. Applications generating such simulated traffic (be it UI interactions, API calls, or events pushed into the system) can be leveraged as part of CI/CD pipelines as smoke tests executed ahead of releasing a change. Additionally, such simulator can serve as an end-to-end probe continuously verifying that the system works in production.

Summary

The strategies outlined in this post can help in reducing risk of software delivery by shipping software to production early, in small, and well-defined iterations. Applied to the right problem, they will result in more frequent code integration and deployment to production. Teams are enabled to gather information about the complexity of the problem at hand at the design stage of the feature delivery, untangle dependencies, and provide more transparency on the progress of the project.

Shipping to production should be frequent and fun! If it’s not, do it more often and address the factors leading to toil or anxiety, be it through automation of lengthy and manual procedures, by adding gradual deployments with automatic rollbacks, or canary builds to reduce the blast radius of failures. In cloud-native environments, there are plenty of tools making it easy to adopt these deployment practices.