Security Architecture – moving forward with an approach to outline a framework

It is a key success criteria in system development and architecture to improve and extend models, procedures and underlying frameworks. This is especially needed when it comes to cyber security of complex systems. I started recently to improve my framework for a robust security architecture. Many stakeholders tend to start with the details in such complex systems which may result in missing overall requirements and ramifications. Security in the scope of vast, distributed systems needs to be specified, designed, implemented and operated based on a solid framework – let’s call it a Security Architecture. I have seen many approaches in order to cover this tricky task. Many of them tend to be too complex. Unfortunately, complexity is not a driver for security (in contrast to simplicity). On the other hand, it’s a tough job to keep the Security Architecture for huge systems simple. Beside the need for a simple approach, transparency and clearness in the scope of Security Architecture are important attributes that should be addresses as key-objective. Security controls need to be structured and encapsulated in the relevant components of the Security Architecture in a clear and traceable manner. I prefer a structure consisting of the following main components:

1. Security Infrastructure [ Communication and Network Security, Perimeter Security, …]
2. System Security Services [ Access Control, Identity Management, Credential Management, Audit, Backup and Recovery, …]
3. Application Security [ Operation Systems, Databases, Web and Application Server, SaaS, Enterprise Applications, Collaboration, and Messaging, … ]
4. Service Security [ System Maintenance, System Operation, Change Management, Incident Management, Event Management and Forensics, Stakerholder & User Feedback, ...]
5. Security Management [ Policies and Roles, Risk Management, Training and Awareness, Secure Coding, Design Principles, Algorithms]
   

The components 1-4 are the basic layers of the Security Architecture. A more vertical component is Security Management which covers and affects all the other 4 essential parts of the Security Architecture.

Security must be based on a solid (security) architecture

We can read a lot about vulnerabilities, malicious code and horrifying threat scenarios these days. And, we can also learn from all these experts how to fight this. Actually, there is nothing about war and weapons (that could help anyhow). Everything is about solid requirement management (covering security from the very beginning), a decent architecture as well as a design which addresses security seriously. Sure, the team must be qualified to handle this. Just some thoughts: A sustainable architecture is composed of discrete elements, called components. Components are the core parts of architecture. Their design and composition is essential to meet the requirement for a sustainable architecture. Beside these factors, security is another success criterion. Components must be secured in accordance with industry recommended practices. Design and implementation must adhere to security principles, design patterns and coding rules. They must be configured according to the security policies of the organization. This must apply for all components the architecture consists of. Remember the weakest link paradigm; one weak component could compromise the security of the whole architecture. Components which expose interfaces to the “outside world”, like user or communication interfaces are especially under attack or even the entry point for an intruder. This must be considered when specifying, designing and developing these entities. And, interfaces must be well-defined to support an integrative approach in order to achieve end-to-end security. The idea behind this is that a system that is composed of components must assure security when sending or receiving message from on or more component to another and even beyond the system.

Divide and Protect

Divide and conquer is a well-know strategy in software design and architecture. In terms of OOA/OOD it is not really my favorite approach, but this is not the topic of this post.
Divide and protect is one option to secure large distributed systems. The concept of Divide and Protect is about the compartmentation of a system into functional blocks with identical requirements in terms of security and privacy. It supports a defense in depth strategy, and it helps to handle the complexity of large installations. The compartmentation of a given system leads to security zones with different levels of trust that should be outlined in a digram. Based on such diagram (red = not trusted, .., green = trusted), the system architecture can be developed in a comprehensive manner. This is especially true for the communication architecture and the selection of appropriate protocols. By using this approach, non-functional requirements can be addressed in the early beginning of the product development process which means no change (requests) later on.

Failure is an option

I read about a conference on failing in the Silicon Valley last year (and I was fascinated by this way of thinking immideately). Well, I think this approach to tackle issues and mistakes is one success criterion which makes this exceptional high-tech valley that successful. Ten years ago, I was lucky to work for one year in Santa Clara / Sunnyvale. At this time, the Silicon Valley was the epicentre of the internet boom. I learned a lot about trying out new things and to be innovative in thinking and developing systems. I need to mention from time to time that software development is still a heuristic process. There are no tools to produce exactly that result in terms of code which is intended in the phase of project initialization. It (the mother of all tools) has been promised for years but it has not arrived yet. Sure, there are patterns, models, code fragments, IDE’s and many other helpful things but in the end it is up to a human beeing (the engineer) to compose and develop the solution. And this still works by trial and error in many cases. The most important lesson is to accept failure and to learn from it. This sounds easier than done and means a learning process for the whole team. But this is absolutely necessary in order to handle the complexity of computer science these days in a professional manner. One of my methods to take care of this is to document alternative solutions (to the choosen way of implementing it) for a given project and to outline the reasons for discarding. As an easy rule of thumb: In order to develop solutions successfully, we need to learn how to fail in the right ways.

Success criterions for development projects

Many factors do impact the success of development projects these days, much more than a couple of years ago. I have learned to focus on a very limited number (3-4) of key objectives to achieve in large scale projects in order to stay on track. Key objectives are the principles of a project and should be well-know, understood and accepted within the project team. To miss even one of them means failure. If it comes to decision making, the key objectives build a good foundation to move forward. This is a proven approach for projects but also for the development process itself. Here are my four (4) key objectives for successful system development:

1. Precise, high quality requirements based on strong user involvement
2. A motivated and skilled team with the ability to learn constantly
3. Flexible and realistic project management
4. An open and innovative environment which understands the fact that software development is a heuristic process and which accepts failure
   

I don’t want to simplify system development. I do know that there are many other factors and ramifications. But people tend to get lost in too many details and micro-management tasks. Concentrating on the essentials does lead to successful development in a very complex world.

Security Architecture – an approach to outline a framework

Security in the scope of vast, distributed systems needs to be specified, designed, implemented and operated based on a solid framework – let’s call it a Security Architecture. I have seen many approaches in order to cover this tricky task. Many of them tend to be too complex. Unfortunately, complexity is not a driver for security (in contrast to simplicity). On the other hand, it’s a tough job to keep the Security Architecture for huge systems simple. Beside the need for a simple approach, transparency and clearness in the scope of Security Architecture are important attributes that should be addresses as key-objective. Security controls need to be structured and encapsulated in the relevant components of the Security Architecture in a clear and traceable manner. I prefer a structure consisting of the following main components:

1. Security Infrastructure [ Communication and Network Security, Perimeter Security, …]
2. System Security Services [ Access Control, Identity Management, Credential Management, Audit, Backup and Recovery, …]
3. Application Security [ Operation Systems, Databases, Web and Application Server, SaaS, Enterprise Applications, Collaboration, and Messaging, … ]
4. Service Security [ System Maintenance, System Operation, Change Management, Incident Management, Event Management and Forensics, …]
5. Security Management [ Policies and Roles, Risk Management, Training and Awareness, … ]
   

The components 1-4 are the basic layers of the Security Architecture. A more vertical component is Security Management which covers and affects all the other 4 essential parts of the Security Architecture.

Quality of Non-Functional Requirements

As already outlined, non-functional requirements are a crucial success criterion in distributed systems (and in software development in general). These requirements need to be prioritized in order to focus on the main use-cases of the system. Beside prioritization, using a clear syntax is important as well because non-functional requirements tend to be fuzzy. This limits the acceptance during development. Like their functional siblings, non-functional requirements should adhere to the following criterions as well:

• Clear and non-ambiguous
• Described by using simple and consistent terminology which is well-known by all stakeholders
• Testable at the end of the day in order to achieve a measurable outcome
• Traceable from the beginning until the end (architecture, design, implementation, test, roll-out)
• Technical feasible considering the tools and systems that are part of development and deployment scenario
• Realistic in realization which depends on the planning horizon, the skill-set and location(s) of the team, the infrastructure and the development environment
  

Ideally, a designated requirement manager and a software architect are the perfect team members to make this happen. All stakeholders should agree on this proceeding in the beginning and are asked to monitor the adherence over the whole lifetime. “Lessons learned” are a good approach to refine this process. Good and bad examples should be used to tune a successful requirement management to perfection.

More on Software Architecture – Architectural Styles

Our domain of software architecture is improving constantly and gets specified in more detail. This is not bad thing because it helps to avoid misconceptions about their place in software development. Architectural styles are one building block of the process in striving for the best fitting solution. Looking for analogies, ‘real architecture’ in terms of brick and mortar has a long history in architectural styles like Gothic, Tudor, Art Nouveau, or Postmodern. These styles are influenced by social, political, cultural and other trends. Of course, technical factors like material, technologies, infrastructure and appliances play an additional role (and money, of course). In software architecture, these factors are predominant. But trends and hypes play their role as well. SOA is a good example to illustrate this. In general, architectural styles can be clustered based on the part of the system they describe like structure [Component, Layered], deployment [Client-Server, n-Tier, Peer-to-Peer], communication patterns [Message Bus], and others. Sometimes it is hard to assign a style properly to cluster. Styles to handle user interaction [MVC] might be a part of the structure or could be seen as a separate cluster. But this is not important. More relevant is the process of combining styles in order to achieve the best solution. Some styles might be determined by the requirements directly. Hardware, integration of legacy system and other infrastructure related preconditions might limit the choice. Moreover, the skill set of the team is another factor. In general, functional and non-functional requirements should influence the architectural styles and their combination used to build the system (hypes should not btw.). If this is done properly, the benefits are proven patterns, a common understanding, interoperability and reuse. In other words, the combination of the best fitting architectural styles is a key success criterions in the process of building a software system.

Security in the scope of Software Architecture

Security must be addressed in the development very early. Taken existing and well-defined security requirements for granted, the system and software architecture (an artifact called “Architecture Specification”) must consider and reflect security. I do favor an approach based on a set of essential building blocks in order to achieve the expected level of security. Parts of the building set are:

• Secure Components
• Secure Infrastructure and Services
• Secure Execution Environment
• Secure Network Environment (zones, compartments, sandboxes)
• End-to-End Security (supported by services like identity, authentication, authorization, auditing)
• Secure Operation (Logging, Import/Export, Backup/Restore) and Security Appliances
  

The approach addresses common security paradigms like “Layered Defense”, “Security in Depth” as well as general design objectives (modularity, consistency, extensibility, robustness). These building blocks are the foundation for a security architecture where security controls can be applied. Just to drill a little bit down. Secure components can be characterized as in the following:

Design and composition of components are essential steps to meet the requirement for a sustainable architecture. Components must be secured in accordance with recommended practices. Design and implementation must adhere to security principles, design patterns and coding rules. They must be configured according to the security policies of the organization. Remember the weakest link paradigm; one weak component could compromise the security of the whole architecture. Components that expose interfaces to the “outside world”, like user or communication interfaces are especially under attack or even the entry point for an intruder. This must be considered when specifying, designing and developing these entities. And, interfaces must be well-defined to support an integrative approach in order to achieve end-to-end security. The overall security requirements for the component design should be derived from general security objectives such confidentiality, integrity, availability, and accountability.

Software Architecture and Requirement Management…

… should be close friends that like to communicate and interact. Why? Software Architecture is the main and first interface between the requirements coming from different stakeholders and the development team. Based on the requirements, the “Architecture Specification” will be developed. Requirements are fetched from very different sources, depending on the domain. The subsequent bullets list just some of them, clustered as Functional (FR) and Non-Functional Requirements (NFR):
- Customers (FR)
- Existing Platforms, Mainline (FR), (NFR)
- General Market Requirements (NFR)
- Standards and Regulations (FR)
- Best Practice and Patterns (NFR)
- Quality Attributes, preferably prioritized, utility trees are recommended (NFR)
As a result, the “Architecture Specification” should reflect all requirements as well as their importance and emphasis in the project. Any mismatch (or even missing requirement) can be detected in the scope of a review or even a architecture test. This is good news because it avoids very expensive changes in later steps of the software development process.

Software Architecture (is alive and kicking)

I’m about going back a little bit to the roots, trying to write and blog more on software and system architecture. It might be necessary because it’s the foundation for all the distributed and web applications as well as for security architecture. It’s ez to get lost in all the details that come with these challenging topics. And, of course, it is my core business and field of expertise.
Software Architecture is the highest level in the area of software development (but it is not superficial or shallow, not at all). Software Architecture is the foundation for all the other more detailed development steps that will follow in the life cycle of a system. Because of its early position in this process, Software Architecture is an important success criterion. And because of this fact, it should be tested, at least by a very detailed review. In order to be testable, Software Architecture must be documented, preferable in a single document called the “Architecture Specification” based on well-defined views. Diagrams and figures are mandatory. The quality of the Software Architecture affects the quality of the whole system in creation predominantly. A well documented and widely teached Software Architecture is a perfect guidance for the development team. Project management needs it to make parallel development on components happen. It is highly recommended to communicate the Software Architecture to all other relevant stakeholders: Customers, 3rd Parties, Marketing, Operations & Services, and Test Teams. More is about to come …

Software Architecture and balancing stakeholder needs

I recognized recently that I blog a lot about computer security (because of several reasons). This is definitely not a boring topic. On the contrary, it’s complex, fascinating and a fast moving target. You need a decent understanding in computer science to keep up with all the things going on out there. On the other hand, the area of system security (comprising computer and network security) is full of misconceptions. Too many people still believe that a firewall is the silver bullet to keep out attackers, worms and other malware. I don’t comment on that anymore. But this never ending discussion leads me to one of my core competences – software architecture and all the team, communication and development related aspects. A software architect needs to understand stakeholder needs. Typical stakeholders are end user, developer, test stuff, marketing folks, project manager, just to name a couple of them. But understanding and documenting is not enough. A stable and successful architecture balances stakeholder needs and reflects this in all their lifecycle stages, from requirement management until testing, delivery and maintenance. I know this happens just in theory, in an ideal world somewhere in a software glasshouse. But we should strive to come closer. Achieving tradeoffs is an important success criterion in the process of creating a stable and lasting architecture. These tradeoffs should be the result of negotiations with the ultimate goal to come to a win-win situation for all participants (àstakeholders). From the technical perspective, most of the tradeoffs must be achieved between functional and non-functional requirements (aka quality attributes). From this we see that there is s strong link to security as an essential quality attribute in a connected world of ubiquitous computing and routed protocols. Many design decisions in the scope of security architecture are in marked contrast to the ideas of usability folks and the needs of project management. But these contradictions must be addressed and resolved, which might be a tough job. Anyhow, for the sake of a successful product achieving broad market acceptance, failing in balancing stakeholders needs is not option.

(These Days) Development Skills

I do remember times very well when software development was envisioned as a process of combining components (either ActiveX or some kind of Beans), and smart people started to perceive software as a kinda utility (like water, electricity or network access). I did not comment on this. Talking about the technical aspects, not much has changed. Writing (beautiful) code is still a creative and a heuristic process. And, beside patterns and practice, it needs strong skills and a lot of experience. Again, there is nothing new about this (since 19** ?). Just two examples from today’s problem spaces: parallelization and distributed apps on a large scale. To cope with today’s hardware platforms (multi-core) needs a lot of knowledge and diligence in developing applications running on them. There is no magic tool, workbench, compiler, etc. to solve such problems implicitly, not yet. You need a decent understanding of processors, cashing, threads and shared memory. Sure, testing this stuff comes with another steep learning curve. Secondly, largely distributed systems (take upcoming clouds as an example) do need a different approach pertaining to availability and consistency (see my posts on BASE, CAP, etc.). It’s up to the development team to hide the trade-off between data consistency, system availability, and tolerance to network partitions. This is not a piece of cake. It needs new approaches and thinking models, pretty similar as when we moved from the client server-paradigm to more or less simple distributed- application-patterns. In essence, the heuristic part of software development is alive and kicking.

ACID, BASE, CAP and the Clouds

This is not about chemistry, medication or transcendent music. But it is a fussy topic. I assume the ACID (Atomicity Consistency Isolation Durability) paradigm is well known and understood from working with databases or in the scope of transactional logic. But this is about to change in large distributed applications running in the clouds (of whatever flavor) with distributed data repositories (e.g.: partitioned databases on distributed nodes). Running in clouds implies the utilization of web services predominantly. This environment leads to the CAP theorem which is about Consistency, Availability and Partition Tolerance with the restriction that only two goals (of CAP) can be achieved in a real world implementation. So it is up to application development to decide between availability and consistency and to conceal this conflict by different means, for instance by using a smart user interface logic and design as well as sophisticated update and messaging strategies in order to achieve consistency in the second step. - I apologize for oversimplifying this whole topic for the sake of a concise post. – Anyhow, this is already in place in large online-stores. To bring the third acronym into the game, an approach to make this happen is called BASE. It stands for Basically Available, Soft state, Eventually consistent). An architecture based on decoupled subsystems is an important prerequisite to achieve this.
There is nothing new about these acronyms and the paradigm change. Great articles can be found on the web (for instance Werner Vogel’s “Eventually consistent”) or other stuff especially available at ACM QUEUE. I just wanted to emphasize the need to check on this from the architectural perspective in order to answer the question: Is this application the appropriate choice to be deployed in the clouds or not? Or say it the other way around: Is BASE good enough for you? Or, is ACID essential for your system and the underlying requirements that are basically quality attributes? For many systems in the industrial domain (where data integrity and consistency is king) the answer might be not (or probably not yet). To answer these question correctly in order to adhere to such paradigms (or not), is essential because the clouds are already on the horizon.

Key Success Criterions in Software Development

Software Development is still a heuristic process affected by many internal and external parameters. Many books are written in order to make this sort of craftsmanship more deterministic. The outcome may vary. I really like to define 4 (four) key objectives at the beginning of a project, either software or system development. Our brains are filled up with too much information; so four (4) might be feasible to keep in mind:

• Tame complexity: complexity kills any system; over-engineering leads to a system that is not maintainable any more and just extensible by wrappers and such nasty type of things
• Requirement Management: make sure that a proper requirement management is in place; either by using the well-known (and rarely used thoroughly) principles [complete, traceable, testable, …] or by newer, agile processes
• Software & System Architecture: a well defined and described architecture must be in place and communicated to the whole team
• Plan for change and failure: we live in a world of constant change, and the same is true for a software projects; this must be addressed in our way of creating software; in addition, failure is an option, complex system are hard to comprehend and error-prone; we must accept this and should strive to develop a strategy to deal with this fact in an open manner

Domain Specific Languages and Software Factories

I put some thoughts about Domain Specific Languages (DSL) and UML in a post recently. But there is a broader scope to address DSL - it’s the promising and ongoing approach of Software Factories. The utilization of Software Factories is the wonderful dream to produce software systems like cars or industrial goods (which is a pretty simplified example). Anyway, in order to describe and specify the problem, it needs a language with semantics and syntax. There are examples for existing Domain Specific Languages, needed to solve these problems: SQL, Regular Expressions or HTML for instance. Based on a DSL, a standard development environment (like Eclipse) could be used to build a software factory. Sure, the tool must be customized, and it needs other components to have a Software Factory in place - templates, processes, patterns, frameworks, models (just to list a few of them). From the functional perspective, prototypes or main components of the domain encapsulating key mechanism and basic design elements are other essential parts. I don’t want to list all pro and cons of a Software Factory. Just as a final remark - software development is still a heuristic process containing a few deterministic steps. The utilization of Software Factories would increase the deterministic proportion (but would not make the heuristic part vanish completely which is good news for all folks writing code).

Indirection #2

There is a quotation from a British Computer Scientist, David John Wheeler, I do really like and where I feel a strong need to post this again: “Any problem in computer science can be solved with another layer of indirection. But that usually will create another problem." It is not new and we all know about the meaning behind it but it is still relevant today and tomorrow, and it describes the dilemma of toooo many “genericcontainerwrappingadaptergatewayframeworkolalasolutions” perfectly.

UML and Domain Specific Languages

I joined an interesting conversation on the utilization of UML recently. The talk was about code generation out of UML (tools). Mmmh. Pro and cons will always exist when it comes to UML. I use UML for sketching and for my architecture and design documents. Use case, class, deployment and sequence diagrams are important tools to me. They create a better understanding in the team on the expected outcome based on a unified description - much better than proprietary pictures which often lead to different views in the minds of the team members. UML models do not cover non-functional requirements sufficiently. This is a real drawback when is comes to code generation. My hope is that Domain Specific Languages (DSL) will improve, gain more acceptance and will be extended to areas and domains where they appropriate.

Prioritize your non-functional requirements

Non-functional requirements (like usability, security, performance, fidelity) are important success criterions in software architecture and development. It is a misconception that all requirements can be covered by functional requirements. Many requirements are of twofold nature, say security, that typical consists of functional and non-functional types. If this has been recognized and accepted, prioritization is the next important step. It should be not acceptable to have TWO non-functional requirements (also known as Quality Attributes) with the very same high level of priority. This is a critical contradiction that must be communicated and resolved. If not, you might be end with a beast that is highly secured and comfortable to use (let’s call it digital monkey wrench) – which is basically not feasible. It is always a challenge to make compromises for each Quality Attribute and to communicate and explain this. Utility Trees are a great tool to list, depicts and prioritize non-functional requirements.

Handling of Non-Functional Requirements

Non-Functional Requirements (aka Quality Attributes) are the „bread and butter business" for software architects. Unfortunately, non-functional requirements are often abstract and hard to grasp. A good approach to tackle this are so-called “utility trees”. Utility trees list important non-functional requirements (say security, usability and performance) in a tree structure; beginning with “utility as root”. It’s a good approach to use the list for a first prioritization. Each level in the utility tree represents a refinement of the non-functional requirement and ends with leaves. Leaves are scenarios. And this is a good approach from my point of view. A scenario is easier to comprehend for all stakeholders. It is based on the - Context Stimulus Response Pattern – and should lead to a decent understanding what, for instance, security really means for a software product. I will present an example in one of my upcoming posts.