Aug 10 ’09

The Service-Oriented Organization: People, Politics & Pitfalls

by Denise P. Kalm in Mainframe Executive

In a Service-Oriented Architecture (SOA)-enabled enterprise, services and servers are a common good; they aren’t “owned” by a line of business or technology team. However, it’s possible to solve the technological challenges of SOA and still fail because the organization isn’t in sync with SOA design; organizational dynamics matter. The share-everything compliance challenge is just the beginning; the structure of an organization probably will have to change. Learn how you can transform your company into a Service-Oriented Organization (SOO) to reap SOA success.

The Goal

SOA is more than the flavor of the month; it’s the fabric of success for innovative enterprises such as Amazon and eBay. But SOA is really just an architectural construct, a framework and vision of a way for IT to implement business requests. To be service-oriented, a business process is assembled from a library of services, which are repeatable business tasks.

This architecture provides a clear vision on which to build an IT infrastructure and new applications. But, in most cases, the effect of SOA on an organization—its people—is overlooked. The current hierarchical structured organization doesn’t map well to the open, dynamic nature of SOA. It isn’t flexible enough to respond to the rapidly changing business world. The hierarchy tends to lead to isolated operations in the organization. Work silos compete for prestige, recognition, leadership and projects, with the result being no cross-functional area teamwork. As leadership training coach Dr. Dennis Romig notes, “1-1=0.” This means that when teams compete, the result is often a negative; the efforts can cancel out each other. (See his article, “Leading Side by Side to Bust Silos and Cross-Functional Competitiveness,” Sidebyside.com, 2003.)

With a silo:

• Information is tightly held
• Relationships are constrained
• Resources aren’t shared
• Team goals supersede organizational goals.

SOA is inherently a share-everything world. Services and resources are a corporate common good, a significant departure from the way applications were managed in the past. But does the organization support a share-everything world? You need an SOO to match your SOA implementation.

The Challenges

Many companies have already overcome SOA technical problems, so let’s focus on the organizational challenges. Some of the issues apply to almost any change, but some are more SOA-specific.

Change is hard: Most change initiatives fail. According to one study, no more than 33 percent of all major initiatives survived beyond the first implementation (see Pat Morley, “Research Findings on Program Failure and Success,” Man in the Mirror “Weekly Briefing,” 2004). The SOO isn’t an easy design to accomplish; it changes everything and change is hard. People and organizations resist change. Yet, many viable strategies and architectures have failed because the organization wasn’t able to remodel itself to successfully fully exploit the new technologies. SOA is no different; as painful as change is, there’s no way to avoid it.

In studying change, a new cognitive science was created—a fusion of psychology and neuroscience. Studies have shown that change is actually painful; it causes physiological discomfort.

New information and new ideas require you to use working memory, which activates the prefrontal cortex. This is an energy-intensive part of the brain, which means that change also can produce mental exhaustion. The orbital frontal cortex sees changes in expectations as errors. Such an interpretation, which you have no control over, can trigger the amygdale portion of your brain, where fear resides. All this happens without engaging your conscious mind; it bypasses rational thought. Though you can’t overcome your physiology, you can understand it. Once you understand how an “amygdala hijack” works, you can then plan to manage your inevitable reaction.

The commons of SOA: SOA implies a share-everything world. In 1968, Garrett Hardin, a leading and controversial ecologist and author of “The Tragedy of the Commons,” noted a problem with resources held in common. He used the analogy of a shared grazing pasture—a common—where the value of increasing your own herd outweighs the “cost” to you of overgrazing. The cost of overgrazing is shared by everyone, where the extra cattle benefit only you. But when everyone behaves this way, the commons is exhausted; in IT terms, capacity is exhausted and performance suffers. A current analogy is the overfishing of the oceans. What incentive does a fisherman or a country have to hold back on fishing without assurance that everyone else will? The core of the problem is a conflict between individual interests and the common good.

Without organizational change, services can become an unmanaged “commons,” where benefit accrues to the line of business that really exploits the services (and more important, the resources backing them), perhaps skewing development to favor the line of business. Who manages and controls the services? How do you charge back? Who watches the watchers?

Long before Hardin formulated his First Law of Ecology theory, Aristotle noted one of the problems that can arise: “That which is common to the greatest number has the least care bestowed upon it.” Remember the problems that arose soon after CICS regions became a shared resource? This is a much bigger and more complex problem.

When CICS Multi-Region Option (MRO) took off, business units saw each other as competitors, vying for resources and development efforts. Problems occurring in the region could have an impact on everyone. A looping transaction could stop everyone in their tracks, absorbing all the virtual storage or CPU cycles. It’s human nature to act in one’s own best interest, which makes managing the commons so difficult.

Newtonian organizations: Organizations will have to change if SOA is to succeed. Brenda Michelson, principal of Elemental Links, an advisory and consulting practice focused on business-driven IT, defines SOA as “Siloed Organization Abolishment” and “Seizing Operational Agility,” which speaks to the need for change in this area. Business units can no more “own” a service than they can a CICS region or UNIX server. If they can’t own it, they can’t control it. Who pays? Who ensures performance?

Research with children indicates that silo-busting can begin by bringing groups together to work on cooperative, mutually beneficial projects. This might be the first step toward implementing a new organizational design with a new SOA project. Test the concept by pulling together individuals from different organizations and lines of business to create a new team.

From the beginning, organizations were structured along Newtonian, closed-loop principles—the organization as a machine. This isn’t surprising; the field of Organizational Development (OD) arose from the work of engineers and those who admired them. This model leads to a set of working principles:

• Manage by separating everything into component parts (tasks)
• Influence is a direct result of one person applying force to another
• Plan by assuming the future is predictable
• Assume there are objective measures to describe everything
• All creativity is bounded in the organization
• The world is defined by boundaries that separate one thing (person) from another
• Chaos and disorder are bad things.

These constructs may have worked in the past, but they won’t work now. The assumptions are no longer valid.

Process-driven leadership: Too often, processes drive people in an organization, not the other way around. Success is defined by a count of “things accomplished” rather than looking at the larger picture. When processes are statically defined with the goal of providing clarity and consistency across the company, adaptability and resiliency to change are impacted. People and their ingenuity and creativity are the soul of successful business. They should remain in the driver’s seat.

The Solutions

Making sense of change: Expectation shapes one’s reality, so this is where openness and information sharing help. Letting go of hidden agendas and the “right” answer helps people come to grips, in their own time, with a new idea. If we pay attention and focus, we can learn to manage our reaction to our normal brain chemistry over time. For many, understanding and expecting a hijack is enough to help them manage it. For example, having a clear plan and timely résumé can help tame the anxiety created by the chemicals triggered by the amygdala when a layoff is announced. But all this takes time and patience. One way managers can help is to sketch out a goal with a broad brush, so each team member can create for himself or herself a plan for changes they’re willing to make. This helps them take responsibility for their behaviors and reactions.

Just learning how to work in a new system isn’t enough. Marc Chouinard, principal partner, Strategy-Driven Performance, has helped many companies beat the statistical odds and succeed with change initiatives. He notes that, “Until employees absolutely believe they ‘need’ the new skill to be successful, they won’t use it—because it isn’t yet part of the success strategies they use, day in and day out, to be successful at what they do.” (See Chouinard’s article, “How to Eliminate the 70% Failure Rate of All Organizational Improvement Initiatives … and Ensure Desired Changes Take Place and are Sustainable,” Peers Think Tank, 2007.) The lack of the right organizational structure, vision, and processes can make it difficult, if not impossible, for change to occur.

Results-oriented, people-centered leadership: Chouinard explains that to become results-oriented, managers need to move beyond looking simply at processes and examine how “doing things the way we’ve always done them” is stifling new ideas that are part of sustaining innovation. In his article, “People-Centered Business Continuity and Resiliency as a Competitive Advantage” (Strategy-Driven Performance, 2008), Chouinard observes that to ensure an initiative such as SOA really works, employees need:

• A high level of self-efficacy: a belief they have the resources and capabilities to meet the demands of their job
• An opportunity to succeed on two levels: inner (belief that this change is essential to your personal survival) and outer (that the organization supports this; that processes and people don’t preclude the change)
• A clear understanding of the vision and objectives
• A clear understanding of their role
• The responsibility and power to actually accomplish these tasks.

To foster successful change, management must consider the organizational components (processes, systems, policies, reward systems, etc.) to see what might block transition to an SOO.

Quantum mechanical model of organizations: Writer and management consultant Margaret J. Wheatley studied both organizations and physics and discovered some strong parallels. (See her 1999 book, Leadership and the New Science—Discovering Order in a Chaotic World, Berrett-Koehler Publishers.) In Wheatley’s mind, the boxes and lines, hierarchies, and silos no longer work for organizations, if, in fact, they were ever the right answer. Today, in a rapidly changing world, the limits of this kind of structure are apparent; it works no better at creating successful organizations than do Newtonian mechanics explain subatomic behavior.

Wheatley believes quantum mechanics offer a better way to represent how organizations and people work best. Quantum mechanics look at entire systems, focusing primarily on relationships between entities, not the entities in isolation. It speaks not just to the subatomic world, but many other applications. Holistic health draws on it, as does Gaia theory (James Lovelock) where the earth is really a self-regulating system. In this model, chaos and disruption become allies to an organization. Planning expert T.J. Cartwright refers to chaos as “order without predictability.” Disequilibrium is the base condition necessary for growth. Balance and stability equal inertia; it takes more motive power to start an engine than to keep it running. Yet most companies are about maintaining order and control and responding to fear.

When a system is disturbed, it tends to reorganize into a new form of order, while retaining its identity. It can then become what it needs to be to function in the open system we call the world of business. An open system is merely a collection of processes that you can visualize only in temporary structures, which could represent work teams. These temporary structures flow and change to adapt to externalities they can’t control, but must live with. Just like a stream, which adapts and flows around rocks and trees, changing shape and speed, growing larger and calmer, then narrow and more turbulent, the organization must adapt and change, while retaining its identity. The stream is still a stream, just adapting to its environment and relationship with it.

An organization must know its purpose and goal, but be resilient to adapting because the future is unpredictable and not all relationships and connections are easily visible. As banks and governments found, a sudden drop in the value of the Thai baht was capable of rocking their world even if they didn’t trade in that currency; everything is interconnected now. A light touch of the spider web of international trade can ripple out and transform every node on the web.

This also speaks to the nature of reality. “We co-create our environments by our acts of observation; what we choose to notice and worry about,” notes American organizational theorist Karl Weick. But simultaneously, we change everything we notice by the simple act of observation. Everything we do affects connections we can’t see.

So what elements matter in the quantum mechanical world?

• Power generated by the quality of our relationships
• Fields—non-material forces, such as organizational culture, values, vision, ethics
• Openness
• The value of one—working anywhere in the system, one person can change everything
• Disequilibrium that allows reorganization and adaptation, then leads to growth.

A basic law of open systems is that while change takes energy, an open system can import energy; it doesn’t have to experience a net loss. Change is a good thing—a necessary and constant element that organizations must be able to manage well.

To make this work in an organization, certain characteristics are vital:

• Information as nourishment and energy for everyone. It must be universally available, but each person must consider the lens through which he or she interprets it. Collaboration solves the problem of a limited frame; together we approach the truth.
• Respecting uniqueness (and everyone is unique)
• Honing listening and conversing skills
• Honoring and rewarding collaboration
• Fostering relationships
• Evolving and changing organizational structures
• Non-static job roles that relate only to relationships or resources
• Curiosity
• Embracing disruption and change.

This last characteristic is the most challenging for companies, but all real growth requires a disruption of identity, assumptions, and reality. Maintaining the status quo simply won’t work; it denies the nature of the environment in which we live.

Making It Work

A noted writer of biological books, Roger Lewin, notes: “The best run companies function much like a flock of birds, in which individuals following fairly simple rules interact with each other to form a cohesive and dynamic whole.” (See Roger Lewin & Birute Regine, “Do People Matter?” Institute for Enterprise Architecture Developments, www.enterprise-architecture.info.) Unlike Newton, he sees organizations as “organisms, not machines.” The goal is to create a dynamic culture where there’s a balance between stability and anarchy.

Managers in this culture, according to Lewin and Regine, need to have these characteristics:

• Authenticity: If managers are honest, trust develops; they create a safe place in which to be creative.
• Attention: Listening is the most important factor. Ask questions.
• Acknowledgement: Immediately express appreciation. Value people beyond the tasks they complete or their role.
• Accountability: Managers must take responsibility.
• Alignment: This involves both crafting and communicating a vision, and then ensuring that employees buy into this purpose.
• Answers: The manager isn’t always the one who has them.

Teams must be able to spontaneously form and reform and be truly inclusive. The “matrix” organizational structure works best here; teams can be reformed and re-created at any time, and the organization can be responsive to the rapid rate of business change. Individuals feel loyalty and a sense of belonging to the organization, not to any individual team (or silo). The challenge is that it requires real leaders— managers who truly believe in empowered and independent employees. The reins must be loosely held, if at all. Hierarchical structures have too much inertia; they can’t respond quickly enough.

Dynamic teams mean the share-everything culture of SOA includes people and processes, not just services. But this also needs to extend up the organization. Without corporate-level buy-in, there’ll be a tendency for middle management to revert to silos and internal competition. They’ll hoard their resources, especially their best people, vying for an edge.

In most companies, business units compete more with each other than competitors. When no one owns the people, the processes, or the services, this alone might force a closer working arrangement, but it doesn’t bring their goals into alignment. Everyone must be clear on corporate goals and agree to them. They must share a vision and culture that overrides personal or business unit objectives. Culture is the core set of values and assumptions people share. Too often, this is assumed or unclear. In creating, communicating, and achieving buy-in for the culture, vision and values, executives can really lead, instead of managing and controlling.

But a challenge for all management is giving up control, power, and the use of fear. It has to be all about learning, and learning happens only in a safe environment. True collaboration is defined as “a mutually beneficial relationship between two or more who work toward common goals by sharing responsibility, authority, and accountability.” (See Roger Schwarz, The Skilled Facilitator: A Comprehensive Resource for Consultants, Managers, Trainers and Coaches, Jossey-Bass, 2002.)

Notice the emphasis on sharing. First, the goals must be common; next, learning must be more important than control. Roger Schwarz created the “Facilitated Leader” approach, which supports mutual learning. Anyone can be a facilitative leader, regardless of their title or position. Facilitative leaders collect and share all information and they listen without judgment until all views are heard. The goal is a free, open commitment to a decision from each team member. Such leaders are noted by their curiosity, transparency, and joint accountability.

To create a flexible team, each member must commit to:

• Testing all assumptions
• Sharing all the data
• Agreeing on definitions
• Being willing to explain both reasoning and intent
• A balance between inquiry and advocacy
• Jointly designing an approach and solution
• Believing that each team member is acting with integrity
• Being open to discussing the tough stuff.

Executives can support this by creating a new budget model that rewards facilitative leadership and increased collaboration. Business units can’t be competing for budget dollars, or the matrix teams won’t earn support. They’ll want to own the best people and will always figure out a way to get the most money they can for their own interests.

Summary

SOA without SOO is a recipe for disaster. The share-everything, people-centered organization offers each employee empowerment and fosters engagement and commitment. Whether applying matrix management or facilitated leadership, the organization must be an open and evolving organization, responding to unpredictable environmental change as needed for its own survival.