When I first took on the role of a New Agile Leader, I felt stuck in the middle with my fellow leaders. I managed technical engineering teams that were embracing Agile practices to improve their delivery quality. While our executives were shifting their focuses to delivering value and meeting strategic objectives, as New Agile Leaders, we were merely playing an increased supportive role to those around us that ultimately limited the organization’s transformation and those above and below us. In this report I will explore why, with so much change going on all around them, New Agile Leaders miss the opportunity to adopt a new leadership mindset and practices necessary for guiding their organizations through a successful Agile transformation.
1. INTRODUCTION
How often have you felt that a transforming organization does not understand what is expected of a new Agile Leadernd the transformation support they require? It is very unlikely that these leaders will simply “figure it out”, “get with the program”, or learn from “trial by fire”. Although few people doubt the importance of the Agile Leaders’ role, many do not understand how it differs from traditional team management nor do many truly comprehend the specific coaching and executive support necessary for new Agile Leaders to successfully transform their teams.
While many factors lead to the success or failure of an Agile transformation, Agile Leaders are a key element. These new Agile Leaders, however, are in the unique situation of being “stuck in the middle”, as they feel pressure to support those working both above and below in significantly new ways. Many of those designated as Agile Leaders are experienced supervisors, managers, and mentors. Agile Leaders are chosen because of their ability to act as the human resource manager of cross-functional teams. Trusted to lead their team members to adopt a very different way of working, these leaders can no longer rely on their managerial authority and instead help their teams thrive by empowering the teams to experiment and to continuously improve their delivery practices. Likewise, executives will increase their reliance on Agile Leaders over what they had expected from traditional managers. They look to their Agile Leaders for guidance and understanding on how the transition is progressing, requiring their Agile Leaders’ assistance to gain a deep understanding of what their teams, stakeholders, and customers value and ultimately need from the transformation.
While true that these new Agile Leaders need to be ready to lead others in their new way of working, Agile Leaders also need guidance and care when tackling their own personal transformations as their organization adopts these new Agile practices. They need coaching specific to their new roles and responsibilities, relevant new metrics and reporting, and methods for leading their teams through the organizational changes that will take place. Each of these skills will be critical to the success of their teams.
In my story here, you will learn how I experienced this firsthand in my initial transformation at a Fortune 100 financial services firm. We knew we were stuck, but we did not know why or what was holding us back.
2. Background
Throughout my technology project management roles, I had often been the one driving process improvement, adoption of new technology, and shifting roles and responsibilities to solve problems. I was new, however, to changing mindsets—I had always been expected to adapt to the users’ attitudes and behaviors.
When I started my agile journey, I had been working as a Senior Engineering Manager specializing in hardware architecture and design. Leaving this prior role behind, I became a new Agile Leader. I was on the ground from day one, and my organization immediately sent the early adopters through Agile basics and leadership training. After many years of decentralized, siloed data center infrastructure delivery, our financial services organization had followed a series of technical and organizational changes.
In addition to insourcing our 100+ engineers and technicians, we virtualized our hardware and developed tiers of standard options, including enterprise software and services used by our banking and lending business lines. Our final change was to reorganize our oversized technology-centric teams into a dozen cross functional “two pizza” scrum teams that could deliver end to end solutions to the internal business customers. We shifted from leading a specific hardware resource team to being assigned to a specific line of business. Each team had a Product Owner that worked closely with their business partner to understand both their product line and what their prioritized needs were.
After years of project and technical team management, my five peers and I were now first-time Agile Leaders. Following our successful technical transformation, we were excited to be recognized as change-agents responsible for improving our overall delivery processes. I knew many of the team members that I would be leading well, and we gelled quickly. We had engaged customers and our executives worked tirelessly in support of our success. Starting out, I focused on how I needed to lead my teams differently and what it would take to convince them that this newfound methodology would solve our problems of governance-heavy and rule-laden processes. Our teams were highly skilled and familiar with one another. Our coaches formed Communities of Practice supporting our common technologies, and we rallied around a standard architecture, big room planning, and common metrics. We had well-qualified executives spearheading our efforts, and we all had been well-trained in the scaled agile module we would be following. Even our internal customers were excited about the next phase of our transformation.
3. Team Success
Our teams were quick to adapt to their new work practices. Psychological safety was evident in the ways they were willing to experiment and make mistakes, and they excelled at collaborating with each other and our business counterparts. Focusing on just one business area made purpose and intention on both sides clearer, and dedicated teams meant solid trust relationships were forming. One leadership highlight happened when team members who had previously kept their heads down as “follow-the-established-rules” type individuals now began being emboldened to learn by trial and error. They could test and take a few iterations to determine the best approach, while relying on teammates for guidance and feedback. Jack, a server deployment technician, was one such team member who had not been one to buck the system and attempt to work outside the established standards. Given a green light to innovate and suggest new server configurations and deployment practices, Jack led many successful experiments and drove innovative improvements throughout the team. In recognition of the value he was driving, Jack received a promotion within six months of the new team’s formation.
Successes like these and others were at times hard to identify even though our teams were following a standard practice of tracking, measuring, and publishing their delivery improvement measurements through our agile lifecycle management tool. Claire, a project manager turned ScrumMaster, dug into the tool with her years of Microsoft Project and Excel skills to develop custom reports that greatly increased the usability of delivery performance data. Instead of removing only impediments that team was raising, she was uncovering issues that had not yet surfaced as complaints around obvious process constraints. Our customers were pleased that we were becoming more predictable.
When we first standardized our hardware delivery options, our customers often pushed back to request exceptions that were costly both in time and additional expense. They were accustomed to relying on their software vendor recommendations that were not available in our offerings. As the teams consistently delivered the standard hardware with the correct configurations and thorough testing, our customers realized that we could deliver consistent, high-quality solutions within the architecture standard. Along with these achievements, we also improved in our ability to identify and manage risk and dependencies before they became blockers.
Support teams were able to drive their own improvements now that they could rely on our teams’ consistent and standardized practices. As an example, purchasing for our group was streamlined and capacity could be maintained without fear of excessive inventory. Changes to our standards were being shared earlier and passed along to our customers, which gave both groups more time to prepare and reduce any impacts caused by the changes. Shared services, such as database management, found it easier to work with our cross-functional teams. Instead of having to work with each type of technical resource, they were now able to engage with the teams as a whole, streamlining their requirements and delivery planning.
4. Executive Focus
While the solution delivery teams were mastering smaller incremental deliveries that improved capacity scaling and reduced overbuilding, capacity was not unlimited and at times there were conflicts around delivery prioritization. Our executives, with their focus on realizing strategic objectives, soon concluded that they could better assess priority based on expected outcomes for the broader business than the product owners who had a narrower, business unit view. The clarity provided by the executive oversight reduced conflict and brought order to what had previously been heated debate and hostility towards the Agile Leaders facing unwinnable situations.
Our executives worked closely with their counterparts in the business units and would push requests down to us as individual, high priority needs. Our executives did not expect the Agile Leaders to work together to identify or solve organizational problems that could be impacting more than one line of business. They were developing some improvements at their level, including policy changes, without a lot of input from the Agile Leaders. While they continued to provide us the latitude to implement the changes as each team saw fit to best serve their customer base, the Agile Leaders were not joining forces to push change up and throughout the lines of business.
Using this team approach, the executives were also joining forces with other parts of enterprise services to assess where the transformation could be extended into additional areas of the organization. Our teams’ lessons learned from utilizing agile practices to deliver infrastructure, not software or services, was shadowed and often repeated. We were proud of the impact our work was making and sharing our lessons learned from developing the capabilities of our new teams.
5. Stagnation
After the first wave of efficiency, teams continued to deliver improvements but at a slower pace. Lead Time metrics plateaued when innovation at the team level stagnated because they were constrained by our enterprise delivery processes. Initially, we had found creative ways to work around the roadblocks an enterprise rule could create, but we were not working across multiple teams to collaborate on ways to improve our work at the organizational level.
The reach of the teams’ experiments was also limited to the scope of the individual team and the requests that our specific business line made. Without looking for similar conditions across multiple teams, it was impossible to predict what improvements would serve the larger organization as a whole, so innovative solutions remained localized and experiment opportunities began to dry up. The teams had become very efficient within the confines of their work, but little innovation was occurring at a higher level.
As Agile Leaders, we were stuck between our teams that were serving our customers and our executives that were serving the organization. Beyond clearing impediments for our teams and acting as the voice of our customers at the organizational level, we were clearly uncertain of our true leadership purpose. Other than trying to outdo each other with bold experiments and sometimes unorthodox approaches—architecture standards are established for a reason and ignoring them does not always result in the most positive outcomes—we were failing to recognize broader organizational opportunities as they presented themselves.
Such an opportunity arose when the organization was facing a shared service crisis during which demand for a complex implementation collided with a platform upgrade, bringing productivity to a standstill for several weeks. Customers complained and the executives stepped in, as they were now accustomed, to solve for a lack of technical expertise. As Agile Leaders, we missed the chance to avert the crisis by both demonstrating the need for reinventing the service’s delivery process and by providing evidence that additional resources were required to meet increasing demand. Had we combined the knowledge we had for removing process inefficiencies (e.g. introducing generic intake forms for non-standard requests) and awareness that our teams could better assist the understaffed team (e.g. we could provide in depth requirements analysis), we could have strengthened this group and driven the organizational change ourselves.
Even as we began to stagnate, we continued to be adept at firefighting and creative problem solving to handle unplanned requests. Our agile practices made these responses appear to be even more efficient, but they were unrepeatable and often had low-value outcomes. Our team members were becoming increasingly frustrated with the lack of continued improvements. They wanted to expand their efforts, but we continued to keep them within the rails of their customer base. We thought this was the point of the new way of working—focus on the “product” that was right in front of us and under our control. Unwittingly, we missed the opportunity to relieve the constraints they were feeling by limiting the span of their problem-solving opportunity. Our lines of business were similar enough that these unplanned requests repeated themselves across the organization but with little coordinated effort to resolve them.
6. New Silos Formed
While the teams shared common metrics, we were not using these metrics to recognize the common threads in our delivery challenges. The commonality could have helped us find indicators for our greatest organizational issues. Instead, we continued comparing and contrasting our metrics as performance ratings in the competition to see who was doing the best in the new Agile working model. We assumed that our opportunities for improvement were unique, and we were never incentivized or coached to review the larger data set together to identify any common adverse conditions that we could all benefit from removing.
Treating our findings this way meant that similar challenges were addressed in a variety of ways, and the group was not benefiting from the individual lessons learned. In one instance my team recognized that the time associated with infrastructure builds was as low as it could go. They developed a very effective method for forecasting the virtual builds they would need based on the type of new request they were working on. Having the environments built in advance greatly reduced the wait time between when the solution was accepted, and final build could be delivered. This learning, however, was used as another incremental success for our silo. It was not shared with my fellow Agile Leaders to drive their own success and improvements.
Our team members also became increasingly frustrated with the limitations of our delivery impact. Often the grumblings were shared between teams, and many questioned why the teams were not working more closely together to solve problems. In essence, we had gone through an Agile transformation that unintentionally reformed new organizational silos like those we hoped to break down. Previously, we had been subjected to technology-based silos, required to work with each team to collectively implement our solutions. Now, our new organization had created silos around their customers and products.
The improvement stagnation was also noticed by our internal customers. They were expecting larger issues to be resolved, and they continued to raise them at the executive level, who continued to push them down to us to solve individually. We were focused so intently on our internal customers and their immediate concerns while also succumbing to the increasingly involved role of our executive team, that we did not notice that the stagnation was created in large part by those of us stuck in the middle and our own leadership shortcomings. We had limited our teams desire to push innovations further into the organization. When we heard their frustrations or saw opportunities visible from our position in the middle, we did not try to come together as a single voice to impact the executive team’s actions. Instead, we just accepted our limitations and the limited successes they produced.
Not knowing our need to truly become agilists ourselves, we were contributing to the problems of the organization as much as we helped it. Had we been leaving our egos at the door, identifying each other’s strengths, and leading each other to achieve great outcomes, we could have contributed much more as a team of Agile Leaders. As individuals we had grown to believe that shortcomings were weaknesses that needed to be avoided, refusing our own mindset shift to lean on those who had the strengths we lacked to achieve greater outcomes than we could individually. Had we formed our own agile team and managed these process improvements at the organizational level, the improvements could have continued. We also could have joined forces with the executive team, to provide them our insights and innovations to overcome challenges at the organizational level.
In addition to failing to notice the new silos, we were missing the opportunity to change our own leadership practices from a tiered approach to 360-degree leadership. We were leading our teams, we were being led by our executives, but we failed to notice the real opportunity that we had been given—for peer leadership that could have benefited all our teams and better served our executives and customers. There were many missed opportunities for the Agile Leaders to lead one another to solve a specific issue and drive the organization to new levels of success.
7. Moving Forward
As a result of our stagnating successes, the organization’s assessment was that the teams had made all the improvement they were going to make. In sharing common processes and standards, the number of Agile Leaders could be reduced, and the teams could be combined into larger units organized by similar customer/product lines. While each of the remaining leaders now had a larger set of teams to experiment with improvements, they had little time available to innovate because of the demands of a larger customer and product line base. They continued to practice their non-Agile leadership practices, adopting few behavior changes and treating the organization as a tiered group instead of an integrated system.
After the reorganization I moved on to two more Agile Leader roles that resulted in similar situations. I was brought on for my Agile and technical delivery experience, to share with my new peers how to best adapt to the changes brought on by a scaled transformation. The New Agile Leaders I worked with were technical managers, each responsible for a single internal product with a team of engineers that provided enhancements and support. While they had worked together before the transformation, they maintained their silos that prevented very little cross-team collaboration.
In both subsequent assignments, the executives did not play an active part of the transformation and their roles did not change. This experience led me to appreciate the positive role that my executives played in my first transformation. I recognized that they had adopted a new leadership mindset, valuing each other’s capabilities and insights while focusing on reaching strategic objectives. When explaining that prioritization should be based on strategic objectives and not funding or influence in my coaching engagements today, I often cite this executive team as the first that I witnessed making tough choices based on a team understanding of what mattered most to the organization.
As an outsider working with new teams, I began to recognize that the other leaders lacked any type of mindset shift leading to new relationships with their peers and shared influencing of their executives. The continuance of old attitudes and beliefs were again the underlying cause of the lack of improvement beyond the individual team level. I also began to identify these now-repeating behaviors:
- They favored their team’s capability over their peers.
- They had little interest in working together to improve overall performance at the organizational level.
- They failed to see commonalities in the product and customer lines, instead believing that each teams’ work was unique.
- These behaviors appeared in both high-performing and low-performing leaders, and their individual skillset did not change how they adapted to change at the leader level.
Beginning to understand what was thwarting Agile leadership performance did not make it any easier for me to drive change in my peers. As an experienced Agilist and an outsider, I had some success in implementing new practices and promoting leadership-level behaviors that would serve the overall organization, but they mainly fell on deaf ears that were only focused on change at the team level, lacking executive support for organizational improvements driven at the Agile Leader level.
Throughout my experience, I took away these lessons learned in how Agile Leaders need to work together on their transformation effort:
- Whether or not Agile Leaders have worked together in the past, they need to spend time at the beginning of the transformation defining their new role, expectations, and how they can work together to contribute to the transformation outcomes
- Just like their Agile teams, Agile Leaders can adopt Agile roles and follow Agile practices to manage their own shared backlog of work
- Agile Leaders need to relate their teams’ product goals to their organization’s transformation and strategic goals, finding shared opportunities to collaborate with other teams
- Agile leaders need to be proficient at assessing transformation success at their team and organizational levels, with the ability to transparently report on it to their stakeholders and executives
- Agile leaders need to understand the purpose and process of collecting and using Agile transformation and product delivery metrics, taking an active role in how their teams develop, use, and roll up their metrics so that organizational improvements can be discovered and measured
- Agile Leaders’ communications and demonstrations with executives serve to showcase the leaders’ ability to assess and solve problems at the organizational level
- Beyond training leaders in standard Agile practices with their team members, they also need training in Agile Leadership skills and Organizational Change Management
8. Conclusion
My role as a new Agile Leader provided me first-hand experience in how to help others in a similar position. It also generated within me a deep passion for Agile Leaders stuck in these middle layers of an organization who lack a vision and training to unlock their potential. Taking the learnings from my three attempts at corporate transformations, I made a career move into coaching. I brought what I had experienced as an Agile Leader into my coaching practice and developed a new approach to transforming Agile Leaders.
When I initially engage with new leaders, I am passionate about helping them get “unstuck” and recognize the leadership mindset changes they need to make for their organization, not just their teams, to become agile. These Agile Leaders need guidance in understanding their complex position as the middle layer of the transformation. In addition to leading their teams to adopt agile practices, they need to work with their peers to identify multi-team or multi-product opportunities and then jointly solve them together, across all the impacted teams, to achieve success thru organizational outcomes. New Agile Leaders need to identify one another’s strengths and contributions, relying on one another as an Agile team does. While their focus may be the work of the individual teams, working together in these efforts will increase their teams’ collective successes.
Agile Leaders also need to go beyond merely supporting their executives in their role of meeting transformation goals. By joining forces with their peers to collectively apply their new leadership mindset, they will capitalize on their vantage point to identify the difficult changes necessary to overcome transformation challenges. I wished I had known these lessons from the start when I moved from a traditional technology manager. I just might have rallied with my peers and helped all of us from getting stuck in the middle.
9. Acknowledgements
I would like to acknowledge the countless colleagues who have helped me on my Agile Leadership journey. Those that started us down the right path with my first transformation to those that patiently listened when I shared stories of my past successes that often sounded like crazy talk. The naysayers were likely my biggest champions, requiring us to go slowly and prove that the changes we were making were worthwhile, you taught me patience and perseverance. Also great thanks to my managers who have invested in my personal growth along the way thru training, mentoring, and coaching.
Thanks to my Agile 2021 Experience Report Track Chairs, David Kane and Rebecca Wirfs-Brock, for all their effort to bring this portion of the conference together. And thanks most of all to my shepherd, Kevin Cowart, for all that you taught me about experience report writing and the effort you put into editing the final product.
