The shift left fallacy
I am done with the whole shift left storyline.
When I started computer science in 1999 the professors stated that a lot of money and failure are spared with a good design. In 2000 the professors told us that starting early with the requirement elicitation would reduce costs. Early was defined as the moment preceding the software design process, this in contrast to it being part of the software design process.
The story of collecting requirements early was followed a year later by the teaching that iterative software development. Iterative development would reduce costs and development time in contrast to a singular sequential process (waterfall).
Iterative development would shift the whole software development cycle more to the left. The design, build, test and run activities all would be executed earlier in time, and therefore reducing big mistakes and thus reducing time and money.
The reasoning was always the same;
Think before you build, ask before you build, deliver quickly for feedback from your stakeholders resulting in saving money and time.
This reasoning is very reasonable.
Defects are created during the creation (coding), and are identified during the various testing/ validation activities. Correcting these defects is more costly when it is part of your product than in the design phase.
A defect is not “just” a programmer’s mistake, a defect in this context can also be an end-user/ stakeholder stating “This is not what I meant.”. A defect is the gap between what is created in reality and what is expected in the mind. The cost of this gap is the rationale behind all the shift left stories.
See the figure of Stickyminds (I) for an indication of the costs saving related to catching defect early in the software development process
Shift left fallacy during my life as a consultant
When I started in consulting in 2006 as an analyst, the story of my seniors was that requirement elicitation should start as early as possible. You as an requirement analist should start before the approval of the business case. Requirements were already present at the Why-level, as business requirements. Therefore business requirements were already relevant in the business impact analysis and even in the ideation phase. In that time we called ideation the problem identification phase.
In the following years my experience with analysts, testers, architects, service management and project manager is that they all teach their peers;
“The earlier you get involved and play a role,
the higher the quality and lower the cost of the project/change.”
Being involved in an earlier stage of the change process is what is called “Shift left”.
The rationale in the 2006-2014 (agile) period was that the waterfall and Prince2/PMP way of working stated that every step needs to have a clear beginning and end. Additionally the activities are organized in a clear sequential way. The upside is that it was clear who was responsible at a certain stage in the process. The downside of this is that it also resulted in a “I am done, now it is your problem” type of mentality (see figure [II] dzone).
RUP identified collaboration
One of my favourite change process definitions is the “Rational Unified Process” from 1999. RUP identified that software development was not a sequential process and identified that the involvement of different disciplines all are relevant in all the phases of iterative development (see III).
This “old-school” method of iterative multidisciplainry development rings true even today.
RUP identified in 1999 what we as a community have rediscovered in the Agile-way-of-working:
Product development is mostly a collaborative exercise.
DevOps wins from Ideation
RUP stated that software products have four phases with multiple iterations. We tried to reckonise in product development the three sub-domains: ideation, validate, produce [V].
It appears that we now embrace the focus on discover and deliver [VI].
BuzDevOps wins from DevOps
Today in 2021 the shift left rationale is used for the topics of BizDevOps [III] and Security. The famous devops Discover/Deliver cycle is extended with the input from the business.
This embraces the iterative way of product development and acknowledges that products now-a-days are most of the time software products. This is a very good development when the goal is customer and business centric software solutions delivered in an iterative continuous improvement way.
three’s a crowd
And here is the catch. All this increased effectiveness is adopted under the flag of being more efficient.
Remember that the storyline was that more “design” upfront would reduce the overall costs and time. This results in a reduced time to market (IX, X), but also in more “actors” and “professions” being active at the same moment in time, requiring a more sophisticated coordination.
We all know that three is a crowd and more than seven people in a group results in group-dynamics that cost a lot of time and effort. By involving Developer, Service Management and Operations, Tester, Security, Business, User Experience expert, Requirements/Business Analyst, Solution Architect, Enterprise Architect, Systems Architect etc to the table it is getting a bit crowded.
War on talent
The big challenges here are (A) when does a change deserve the cost and focus of a team, (B) which expertise do we seek in one person to keep the size of a multidisciplinary team within control?
These two main challenges do not have an easy answer. I even dare to state that there is no answer.
People are being educated more and more in a specific niche. When you are a student you select an education that is specific, when you want to re-school yourself you go for a specific set of skills that are in high demand in the marketplace.
People that are experienced in a broad spectrum of expertise are very scarce. The war for talent for software engineering expertise has been our reality for years. Let alone for someone that combines a business point of view, a feeling for user experience, the ability to oversee all the various (software) product components and is also a great software engineer.
Shifting everything to the left, is painting a large group of people in the corner of the room.
Security is adopting shift left
In security I see the same “shift left” development. In the beginning (1994 ~ 2016) security was the profession of a few specialists, nowadays security is seen as an organisational capability.
Driving forces behind this are (A) the complex IT landscape with complex integrations, (B) the non-stop exposure to cyberthreat and (C) the GDPR demanding security and privacy by design and (D) the zero-trust directive by the US government.
This implies that every IT professional needs to learn and understand how to apply security in their every day work and into their specific skills.
For non-IT employees the demand also keeps stacking up. The non-IT employees of an organisation are getting more IT-literate which is closing the business/ IT gap. The non-IT employees are also more and more educated in the added value of Enterprise Design. Now they also need to understand what security and risk implies for the design of their business organisation and products.
Efficiency vs Effectiveness and burnout
This is a hefty workload for the organisation as a whole.
How should we do this in a planned and coordinated way? I do not know.
Is it realistic to expect this of every company? I do not think so.
What does this then imply? I do not know.
While writing this blog David das Neves published his blog titled “The DevSecOps Paradox”. David approaches the shift left valacy from the point of view of the developers. Stating that it is not realistic to demand from developers to become security specialists.
His advice is to centralise by “Shifting your complete Governance lifecycle model left” and by “creating highly specialized enablement and platform teams” to counter the lack of experts.
I do feel for this approach but it is contrary to the decentralised, mastery and autonomy movement. We need to be resilient as an organisation and therefore we decentralise, but we need to meet a certain quality and to mitigate the scale issue by centralising. This is a pendulum. I look forward to more postings on this topic by David and you as reader.
Continous Improvement you
What I do know is that the profession of HR is becoming more and more important to support people in continuously improving themselves, without getting into a burn out.
I also know that the skill of storytelling is getting essential for domain specialists so that they can communicate with the people that are experts of a different domain, since it is impossible for everybody to understand each other on the same level of detail.
How to improve the quality of our products, stay resilient as an organisation, stay relevant to our stakeholders and not burn-out everybody, that is our challenge.
Resilience and Human-centered are the main topics for the next years.
I would like to encourage everybody reading this last sentence to send me a message to collaborate and/ or reply to David’s blog.
Together we will find a way to evolve in resilience byond DevOps and keep the quality of our products improving.
Xebia’s core values are: People First, Sharing Knowledge, Quality without Compromise and Customer Intimacy. That is why this blog entry is published under the License of Creative Commons Attribution-ShareAlike 4.0
Look at our consultancy services, training offers and careers below or contact us at firstname.lastname@example.org