The Solution Architecture Journey at HSBC

CC&C is on an exciting journey with HSBC as described in this video from a recent Open Group conference.

Watch Kate Platanova, HSBC Chief Architect Technical Domains, and Dan Warfield, CC&C Europe Princiapal, u7h talk about HSBC’s commitment to the next generation of Solution Architects, and how CC&C’s Architecture Fundamentals training for teams is part of the worldwide journey.

Why I Still Like TOGAF

Why I Still Like TOGAF

In the Beginning

When I learned about TOGAF sometime in the early 21st Century, I recognised it immediately – even though it was new to me.

I guessed someone had observed my methods for the last 25 years and created a standardised way to apply them. That’s not exactly what happened, of course. But it’s partly true, in the sense that TOGAF aggregated an evolving set of best practices used to plan, design, understand and manage software, especially complex enterprise software at scale, and I was one of the practitioners in that mix in the 1990s.

puzzle piecesIn the decade before TOGAF was being born in 1995, I had led design and delivery of a half dozen first-of-a-kind enterprise systems. As the projects became increasingly large, the value of good holistic, well-documented design became increasingly obvious to me. That value scales up with complexity, and not in a linear manner. This was always the point of the thinking that created TOGAF and other architecture frameworks and methods – or at least the part I recognised as reflecting my own way of working to deliver business change based on IT and OT innovation. It is based partly on the recognition that in a nontrivial enterprise:

  • A holistic, abstract model of the fundamentals of the business as well as its transitory needs for change – and how these fit together – should be the starting point for investing in change and measuring its value.
  • This map of How Everything Fits Together in the enterprise, which is only partly about technology and which belongs to the business, is constantly changing in ways that can never completely align to changes in the technology environment.
  • New system designs that ignore this will create massive, ongoing integration challenges and significant technical debt, and thus will be inherently and increasingly more unstable than holistic design.
  • Almost every deployed system only exists in a matrix of other systems, all of which are changing constantly if only because the lower levels of their tech stacks keep going out of date.

An important job of the enterprise architect was and is to manage those business and IT maps as well as possible, to keep them reasonably aligned and to ensure a holistic, logical investment approach while everything is changing. TOGAF described a formal method for putting these insights into practice in a repeatable, understandable way, adding much value (along with others such as Zachman) as part of the community’s manifestation of the Holistic Architecture meme.

The Old and the New

By 1995 I was working out the IBM Labs in Hawthorne, where our team was trying to understand how a very large IBM data and process model called Insurance Application Architecture could be deployed effectively into the emerging world of OOA, OOD and very interestingly (and mostly disappointingly) OODB.

bunny in birdcage guarded by parrots

Controlling the Wild Coder

Why was our team focused on the transition to object orientation and model-based software design? Partly because OO solved some interesting technical problems that were very difficult and in some cases impossible in the world of procedural code and entity-relationship data modelling. We were fired up about Grady Booch and Hofstadter’s Godel, Escher, Bach: An Eternal Golden Braid and Design Patterns by Gamma, Helm, Johnson and Vlissides).

Partly for market reasons. OO and models and methodologies were important Shiny Objects for CIOs and developers, just like Agile and DevOps today. Vendors needed a story and, if possilble, products. These were popular topics energised by the idea that adoption would reduce the opportunity for developers to create untraceable bugs, integration nightmares, operational vibrations, and undecipherable code (OO code was going to be self-documenting, apparently). Part of the appeal of OO in particular was the idea that by encapsulating code and data, you could somehow encapsulate bad programming in a way that protected everyone else, sort of like imprisoning an immortal demon in a perpetual force field. And OO advocates persuaded many that we could transcend the expensive, slow, rigid, value-blind behaviour of the older generation of developers by freeing developers to just get on with the job and deliver great code overnight.

In the background of this debate was an explosion of development tools, methodology evangelists, and new ways of modelling that could perhaps be plugged into code generators, or at least that could provide traceability between imagined and delivered systems.

The Never-Ending Debate

Two people arguingTwenty years on, we have agile and DevOps and Continuous Integration and Continuous Deployment. We have cloud and more cloud. Low-code and no-code platforms such as SalesForce and ServiceNow and Pega have great traction.

And we have TOGAF 9.2, still part of the conversation. All still attack the same problem – striking the balance between urgently needed quick-and-dirty delivery on the one hand and on the other hand, effective management of security and technical debt and integration and operational stability. Some are fond of saying that Enterprise Architecture is a dead letter and TOGAF is finished.

I disagree. Most of those saying this loudest are making the old mistake of equating EA with software architecture, which is not the same thing. It’s very instructive that fewer EAs and Business Architects report the CIO. This is because while the application of TOGAF-style thinking should govern important IT investments, and can radically improve the ROI of software investments, TOGAF and EA are not ultimately about software development. They are about business value. Without the sort of formal methods and agreed semantics represented by things like TOGAF, there is no real hope of reducing chaos, cost and risk in large-scale IT. This must somehow be done while enabling innovation and artistry in areas where it’s actually valuable and enabling the quick delivery cycles promised by the Agilists.

Many enterprises try to dodge this bullet by outsourcing a lot (or all) of IT. They are outsourcing problems they couldn’t solve themselves (encapsulation) but doing nothing to reduce the world’s supply of friction between the need for agility and the need for governance. And you can’t outsource responsibility for your own bottom line. Increasing the variety in suppliers only makes understanding the holistic design of your own organisation more important. Now you must replace complex communications once based on tribal knowledge with formalised communications codified in contracts. TOGAF still has a lot to offer to all the parties in this game as a way of managing a disciplined holistic view of how things work, how the fit together, and how they might best be changed.

Still in the Game

Man looking at papers on boardThe effects on business strategy and governance of digitization, outsourcing, cloud, mobile, IoT and IT/OT convergence are creating a new context in which it clearer than ever that Enterprise Architecture, using methods like TOGAF, is a fundamental business discipline for managing disruption and value. There are flaws in TOGAF and its extended family (MODAF, PEAF and all the rest). And not all EA efforts succeed. EA practitioners sometimes make themselves irrelevant by embracing absurdly rigid practices or boil-the-ocean-before-we-start approaches. Sometimes organisational culture is so anti-design and anti-systems thinking that the EA has no chance.

But on balance, TOGAF has done much more good than harm, can be used effectively in spite of flaws and is needed in the modern world. So I still like TOGAF, in spite of its weaknesses (yes, some are significant). We need this kind of thing to help us make sense not just of IT but of most businesses. I liked it before I knew about it, and still do.

What is TOGAF Essentials 2018?

What is TOGAF Essentials 2018?

Be Sure Your TOGAF 9 Certification Is Up To Date

The release of the TOGAF® Standard, Version 9.2 left people asking whether a TOGAF 9 certification earned many years ago is still valid. The answer is yes. But employers and customers may well ask whether the practitioner is actually up to date. That’s where the TOGAF Essentials credential comes in. The Open Group‘s TOGAF® Essentials 2018.credential is designed to let you answer that question in the affirmative. The half-day training is a quick bridge to TOGAF v9.2 knowledge. It is available to all current TOGAF 9 certified individuals and covers the key differences between versions 9.1 and 9.2. After the training you receive a link to the Open Group’s 20-question online assessment. A passing score earns the TOGAF® Essentials 2018 badge, demonstrating your commitment to staying current with TOGAF. The badge shows you are up to date on newer TOGAF features, and adding the badge to your CV, LinkedIn profile, etc. demonstrates your achievement. TOGAF Essentials certification supports career advancement, whether in your own firm or in others. Instructors for accredited TOGAF 9 training are required have the TOGAF® Essentials 2018 credential.

TOGAF Essentials Covers New TOGAF v9.2 Features

These include a wide range of changes including:

  • Refactoring the TOGAF Body of Knowledge structure
  • Changes to support modern DevOps, agile, cloud and platform-based environments.
  • Significant extensions (such as Capability, Value Stream and more) and updates to the content metamodel
  • Updated and simplified definitions of key terms, more aligned with other EA standards
  • Restructure of the TOGAF Body of Knowledge

CC&C’s TOGAF® Essentials 2018 training is live online instruction.that includes the standard course backed up by professional insights from one of our experienced architects. CC&C training also includes after-course support from the facilitator should you have questions about the content or the certification process. All of our TOGAF Essentials students have passed the exam and now have the updated credential. More importantly they are up to date when using TOGAF on the job, aligned with current tools, definitions and ADM changes.

Changes to Risk and Security Guidance

This includes much-expanded and updated Security Architecture guidance, which is found in the new TOGAF Series Guide,Integrating Risk and Security within a TOGAF® Enterprise Architecture.

Enterprise and Security Architectures

TOGAF 9.2 view of Enterprise Architecture, InfoSec and Risk Management (c) The Open Group

Value Streams and Business Capabilities

TOGAF 9.2 also fully supports the use of business capabilities and value streams in Business Architecture modeling. This moves TOGAF in line with modern practice, which often prefers this approach to process modeling as a starting point. TOGAF value stream stages This illustration is an example of mapping business capabilities to value stream stages. It adds a heat map overlay as a way of showing which capabilities need attention in order to support the value stream. The TOGAF Essentials training explains how these concepts fit into the updated TOGAF ADM, and where to go for more in-depth information on using these in business architecture work.

 

A CIO-Level View of IT4IT™

CC&C IT Transformation Lead Dan Warfield at The Open Group Conference

Dan spoke on a CIO-Level View of IT4IT.

IT4IT matters at C-level because it is a key tool for transitioning from project-based planning to a service-based digital product investment model. This is a key stage in the Digital Transformation journey.

 

7 Reasons Why IT4IT is Good News for Enterprise IT

IT Can Be Just As Well-Instrumented as the Rest of the Business

Most big companies wish they were a lot better at running enterprise IT more like a business and less like an artists’ colony.

It’s the norm today for every stage of value creation in large IT shops to suffer from too much friction in teams and between processes, inadequate control metrics, brittle handoffs and unsatisfactory operational practices.

At the same time there is immense pressure to leap ahead on value, cost and speed, and to have the right position on current hot buttons such as cloud, security, DevOps and so on.

How can IT4IT help a CIO show real progress? Here are seven reasons why it’s actually a big deal:

1. People get it

When I show this model to pretty much anyone living in Enterprise-IT-Land, they quickly find themselves somewhere on the map. That’s a key starting point. This instant recognition is in stark contrast to esoteric models like ITIL and COBIT, which are robust, complex, very useful to experienced practitioners – and incomprehensible to most people, including most IT professionals.

Yet IT4IT is completely mappable to the more formal models needed by auditors, process engineers and tool builders. It bridges a critical semantic gap that can be the key obstacle to implementing best practice.

Corollary: The IT4IT Forum must resist the temptation to turn IT4IT into another complex, impenetrable standard. Its fundamental clarity is a great achievement.

2. It isn’t proprietary

The Open Group IT4IT Forum certainly includes vendors with a marketing agenda, such as IBM, HP, Microsoft and ServiceNow, to name a few. But none of them own it. That’s important.

The Forum also includes thought leaders from industry and government – enterprise IT experts who have a vested interest in keeping Open Group work open.

This matters because other big best practice IT models, valuable as they may be, are skewed to commercial or professional agendas, resulting in uneven depth and coverage.

At worst they are specifically designed to help salesmen lead inexperienced users down an attractive path to a particular vendor’s offering.

At best they are designed for a narrower audience than the CIO, and often gloss over anything not deeply interesting to their target group.

Try asking an ITIL practitioner which metadata should be packaged with a software release in order to facilitate automated closed-loop defect management. Ask how exactly that might be achieved in a typical application development shop – or what packaging rules are needed for a typical infrastructure engineering release. They probably won’t find the answer in their ITIL manual.

Similarly, COBIT does a fantastic job of describing every process that matters to enterprise IT, and is pretty complete when it comes to naming the important inputs and outputs – but it doesn’t tell us what they look like. It’s a great resource for anyone trying to improve IT value delivery at scale. But it is intractably skewed to the concerns of auditors.

IT4T has the right design point for IT executives charged with consistently delivering real, measurable business value.

3. It is based on value chains

The secret sauce in IT4IT is that it starts with value chains – a tried and true way of thinking about how value is created in all types of business.

By using a conceptual starting point familiar to every non-IT executive in a large enterprise, IT4IT from the very beginning makes sense to that audience. As IT organizations create tools, processes and investment plans based on this model, it remains easy (if done correctly) to map these directly back to enterprise strategy.

This means we can start to measure Enterprise Architecture outcomes using the only important metric for commercial organizations – dollars.

We can know what we’re talking about when we claim our IT investments have something to do with the actual purpose of the enterprise (which is never making back-office IT operations more elegant).

4. It is about service delivery

A recurring wistful meme in enterprise IT has been the idea of the Service.

There are lots of variations on this theme. IBM’s 20-year-old Service Oriented Architecture is a premier example of this attractive idea. The work done to implement SOA in large organizations is also a brilliant example of how hard it is to move the needle in the real world. Trying to migrate the status quo to a new paradigm is a difficult endeavor.

IT4IT gives us a fresh, comprehensible template for understanding services in the large – and in the cloud as well.

When I first showed the service delivery components of IT4IT to a seasoned traditional architect, he raised his eyebrows and said “That looks a lot like cloud,” to which I said “Yes it does.”

He wasn’t sure non-cloud apps could be delivered in that model. Actually the point is that non-cloud apps are missing something, and at worst will have to be encapsulated in a service oriented wrapper to be useful in the virtualized, cloud-centric world of modern enterprise IT.

We need a way of understanding that our work that is all about services, not about infrastructure ops examples like “provision a server.” If that’s all we needed to think about, we wouldn’t need service catalogs or IT4IT – or architecture, for that matter. We need to think about everything consumed by IT customers as a service.

Then the world starts to change.

5. It requires a common operational data model

To make all this stuff work — things like automated closed-loop defect management, DevOps, effective metering and usage monitoring, real complexity management, traceability from demand signal to runtime stack – we need a single language to describe all the information exposed across different parts of the value chain.

Only if all those bits of data fit into a single, well-ordered conceptual model, can we start to replace artistry and tribal knowledge with consistent automated processes.

Most big IT shops don’t have this today.

Why is it so hard to know how the production environment maps back to business users and their interests? One reason is that our language changes so much as the original idea wends its way through the transformations along the value chain.

The logic of IT4IT demands a common data model across the whole span of IT value delivery. By adding a consistent horizontal semantics to the vertical semantics of things like ITSM, we really can begin to get control of the whole business of IT.

The fact that IT4IT also is inherently tied to enterprise value chains and strategies means that the semantics of the operating model, even as we drill down into the details, should remain mappable back to real business value.

6. It demands standardized APIs across diverse tool sets

A common data model connecting a standard set of processes across the IT landscape also means that a standard set of APIs must emerge to support it.

Standards are funny things. They make operational life easier and cheaper for almost everyone by commoditizing what were once unique interfaces, metrics, and technologies. They make commercial life harder for vendors by commoditizing what were once Unique Selling Points.

In the IT tools space there will be stubborn reluctance to let go of hard-won competitive positioning, fear of lost margins and declining commercial advantage, disruption of complacency, defensive moves against existential threats to tiny empires and once-solid revenue streams.

But is it really to the advantage of tool vendors to push back on standards? Sometimes it’s essential. At other times you have to let go and move on.

IT4IT was partly developed as a roadmap for vendors who understand that they need to shift their strong resources to work more complex than building a better Service Desk.

Why? Because the world is changing: However much you love the serenity of the elves, they are still marching relentlessly to the Grey Havens and will not return.

Whose Service Desk isn’t good enough, when good enough is all you need? How does a proprietary code library survive the age of Git?

There are plenty of new opportunities for tool innovators who focus on customer value. There are enough unsolved challenges to keep them busy — and prosperous.

7. It can be the lens for envisioning strategic transformation

In the age of Cloud, DevOps, open source, virtualization and all that stuff, enterprise IT has no chance of standing still, no matter how firmly one’s feet are nailed to the data center floor.

Your competitors always seem to be cheaper, faster, better. The shadow IT factories are busy day and night. The CEO wants to know why you aren’t keeping up. It’s not an easy question.

How do you make well-governed enterprise IT cheaper, faster, more responsive – how do you make it provide better outcomes, better services, better value?

IT4IT suggests that part of the answer is to step back from the daily uproar, to apply a systems thinking / value chain lens to an undertaking that needs more than ever to be redefined primarily as an effective business service delivery engine.

It’s essential to understand and to be able to describe the transformational flow through the whole organization, to understand where the pinch points are – ineffective processes, poor signal-to-noise ratios along communication channels, lost data, lost meaning.

IT4IT is not the only template for making an IT service delivery heat map that shows the best opportunities to ease process flow, guide IT transformation, focus investments. But it has a lot to offer:

  • By being non-proprietary, IT4IT reduces the chance that heat maps will be skewed to produce outcomes more favorable to salesmen than to the enterprise.
  • By pushing toward standard, cross-functional language for describing functional components and information flows, IT4IT makes it easier to have a cross-functional discussion about what the heat map should look like.
  • By being comprehensible to almost everyone engaged in the IT management, IT4IT makes it more likely that the outcome of mapping exercises will make sense to people who weren’t in the workshop.
  • By being derived in the first instance from a value-chain way of looking at business problems, IT4IT increases the possibility that proposed IT transformation investments will align with something that is actually important to the shareholders.

In the hands of a skilled practitioner, IT4IT is a valuable addition to the transformation tool kit.

It’s worth a look.