Goals and Responsibilities in the Cloud Native Journey

What are the boundaries of developer responsibility? One of the potential promises of cloud native development and the growing cloud native ecosystem of tools is “full life cycle” ownership for developers. But is this level of responsibility realistic or desirable for every organization and every developer? Does every developer need to be responsible for the full code, ship, run workflow of cloud native application development?

While we've long advocated for the idea that with a developer control plane, an organization can give any developer full ownership, we know, particularly from our experience talking to thought leaders, CTOs, SREs, field engineers, architects, and more, that no one size fits all. The complete complement of tools exists to support empowering developers with full ownership, but much of the software development world isn't built with a "developer-as-owner" mindset.

While we hear a lot about the buzzy nature of cloud native, we don't hear nearly as much about the daily reality for the 99% developer (the majority of developers working today). The 99% developer works in stable production environments with tried-and-tested solutions. Stability is critical, and experimentation with tooling and possibly unfamiliar cloud native environments, all falling under the developer's responsibility, isn't conducive to many organizations' mission-critical work.

So where should a developer's responsibility begin and end?

Software developer roles and responsibilities: It depends on who you ask

Depending on the person you ask and what role they play, as well as the organization they are a part of, very different answers are likely when asking the question, "How much responsibility for the full software life cycle does a developer have?" And as long as the organization has a clear grasp of what they are doing and what they are asking of their developers, there really is no wrong answer to this question.

From one extreme to another: Full code-ship-run developer responsibility versus pure code focus

Kasper Nissen, Lead Platform Architect at Lunar Bank, believes developers should "shift left" and encourages them to own the full life cycle. To support this, Lunar has invested in extensive onboarding and "paved path" developer control planes to ensure that developers start from day one with everything they need to build and run their own applications.

"Platform teams can't just put all this responsibility onto developers and say, 'Go figure it out and become experts in all these different systems'. Our job is to create an easy choice, but if that is not what the developer wants, they have the freedom to control it for themselves… but then it's their responsibility." -Kasper Nissen, Lead Platform Architect, Lunar

For Lunar the idea of "you build it, you run it" is the everyday reality, and because they've invested in this approach, they realize the benefits of fast feedback loops and faster, safer shipping of software.

Extending Kasper's viewpoint, Mario Loria, Senior Site Reliability Engineer at CartaX, echoed the sentiment that developers should be in the driver's seat: "It should not be up to me as an SRE to define how your application gets deployed or at what point it needs to be rolled back, or at what point it needs to be changed, or when its health check should be modified." For Loria, developers are the most capable of — and should be empowered — to make these decisions, which only becomes possible with developer autonomy, self-service and ownership.

SREs, in this scenario, should focus on infrastructure and core services to support the developer's primary goal: shipping software safely at speed. The developer doesn't necessarily need to care about what platforms and tools are used but does need to be able to use them to, for example, canary a service or get service metrics. In this case, it's a middle ground in which the developer's responsibility is to have enough understanding of the full life cycle to troubleshoot problems in their services.

Exploring the Google developer experience

Some organizations, as Cheryl Hung, formerly of Google, shared, support the idea of the pure developer even if they do offer some flexibility for developers who want more responsibility. Companies like Google strive to make the developer experience seamless, but this comes with a downside. If a developer never learns how infrastructure works with their code, it can create blind spots, particularly when the developer moves on to another organization.

The Google-like experience also shields developers from truly understanding the complexity of shipping and running code. Cheryl explained that even when a developer is free from worrying about infrastructure, they still have to focus on coding software that will actually work, meaning that the infrastructure needs to work. Usually a developer does not care what this infrastructure looks like but having some insight into its workings could be helpful in delivering software safely at speed.

Cheryl argued that a middle ground could be the clear path provided by a centralized control plane, which relieves a developer who doesn't want to take on full responsibility but does not lock them out if they do want to know and do more.

"Most of the time, you don't need the depths of everything to code. You need something like Backstage - a developer portal that lets you do what you need to from a dashboard. Give developers a UI from which they can do 95% of the things they need to do. This lessens the learning curve and creates a good developer experience. At the same time, if they need to access anything beyond that, empower them to learn the platform tools, to 'break the glass' to escape the default dev environment they've been given."

"Don't mess with Mr. In Between"; understand him instead

"Some organizations are set up to empower developers to take on as much as they want; others are siloed and prefer to 'contain' developers, ensuring that there are no deviations, no going off-piste," explained Nicki Watt, CTO/CEO of software consultancy, OpenCredo. This is largely, she continued, about organizational culture, how an organization plans to achieve its business goals, and to some extent, developer preference.

"Perhaps more important than actually being empowered to, and tasked with, running software end to end is understanding. That is, developers should possess multiple skills and understand exactly what they're working on, even if they are not responsible for running it. In many organizations, empowering developers with in-depth knowledge about how distributed applications work and the common challenges inherent in distributed systems provides more value to both the organization and the entire team (the developer, the SRE, the ops teams, etc.)."

Bake in mechanical sympathy and understanding: Identifying and tracking the ingredients in the developer's cake

Kelsey Hightower, longtime and vocal leader in the cloud space, has framed the discussion on developer responsibility as one of being responsible for the ingredients you put into your code, i.e., akin to a supply chain for your software, or more formally, a software bill of materials (SBOM).

"So, if you're the software developer, you're going to have to have some responsibility for the ingredients that go into it. And we're going to ask you more questions around, 'Do you really need that additional library that has 50 transient dependencies? Is there a better library that doesn't have any transient dependencies, that has only the functionality you need?' So, we just need everyone to be aware of their responsibility in the pipeline. And then, what we're going to do is we're going to make that whole process more transparent by having things like the SBOM, or software bill of materials that says, 'This is what's inside of this,' and if you're the developer you can look at that report and you say, "Wow, I put that in there. And that isn't the right ingredient for this particular software application.' And have responsibility for that."

If a developer knows what ingredients go into their code, they should be aware of what consequences and dependencies may result from using those ingredients and be able to account for them. A centralized platform can provide some visibility into how code performs and behaves, but there are additional considerations with regard to how different services interact with each other, for example. Here, a developer should have a complete overview of and responsibility for the ingredients that go into the "cake".

After all, as another podcast guest, Apostolis Apostolidis (Toli), Principal Practice Engineer at UK-based car-purchasing platform cinch, stated plainly, the one coding the application is the one best positioned to understand how it works, what could happen downstream, and potentially how to fix things.

Ultimately, regardless of how much a developer needs to do in the life cycle, they are the ones who have the relevant knowledge and understanding of the big picture. Here, "mechanical sympathy" comes to mind, i.e., the better a developer understands the hardware, infrastructure and how things run, the better they will be at coding and avoiding problems in the first place and helping to diagnose problems that arise.

Developer responsibility: It's not all or nothing

Most of the experts and practitioners we've talked to seem to agree: there is no single answer to the question as to whether or not a developer is responsible for the full code-ship-run equation. It is not an all-or-nothing question. Rather, depending on the organization, its culture, and what it aims to achieve, developers could be coding and doing little else (as Cheryl Hung described her Google experience), or they could run the show (as Kasper Nissen described of Lunar Bank). And in reality, developers exist everywhere on that spectrum.

What most experts do agree about is that it is valuable and important for developers to be able to account for their code and how it might behave in production – to justify why they've coded something the way they did, to help identify problems and causes faster when they arise, and ultimately learn from their experience and code more efficiently and effectively in the future.