Complexity
theory in Cyber Security
It is the
mark of an educated mind to be able to entertain a thought without accepting
it.”
Abstract
With computer systems becoming
ubiquitous and the IT lexicon becoming omnipresent in most organisations, IT
security is one of the top priorities for most organisations. As firms start
collecting more data about customers, rise of business analytics and Big data
capabilities, companies now have more to keep safe. The rising popularity of
Cloud computing on the other hand leaves organisations with less control on its
assets. The annual economic impact of Cyber crime is estimated to be higher
than that of the Drug trade [1] and by some
estimates it’s twice as much as the economic impact of the 9/11 attack [2] . While organisations
have had focus on IT security for a long time and have spent vast amounts of
money, cyber attacks and news of hacked systems are far from being history.
About 6.5M new Malware were created across the Internet in the first quarter of
2013 alone [3] .
Our traditional approaches have
brought limited success so far. This paper argues that Cyber systems are
Complex Adaptive systems. Principles from Complexity science – inspired by
system thinking and natural science, something that has been extensively used
social science, finance & economics, and epidemiology – should be explored
for use in Cyber security to complement the more traditional methods. This
paper introduces some of the high level approaches, it however does not get
into the implementation details.
1.
Introduction
Computers and the Internet have
become indispensable for homes and organisations alike. The dependence on them
increases by the day, be it for household users, in mission critical space
control, power grid management, medical applications or for corporate finance
systems. But also in parallel are the challenges related to the continued and
reliable delivery of service which is becoming a bigger concern for
organisations. Cyber security is at the forefront of all threats that the
organizations face, with a majority rating it higher than the threat of
terrorism or a natural disaster.
In spite of all the focus Cyber
security has had, it has been a challenging journey so far. The global spend on
IT Security is expected to hit $120 Billion by 2017 [4] , and that is one
area where the IT budget for most companies either stayed flat or slightly
increased even in the recent financial crises [5] . But that has not
substantially reduced the number of vulnerabilities in software or attacks by
criminal groups.
There is a need to fundamentally
rethink our approach to securing our IT systems. Our approach to security is
siloed and focuses on point solutions
so far for specific threats like anti viruses, spam filters, intrusion
detections and firewalls [6] . But we are at a
stage where Cyber systems are much more than just tin-and-wire and software.
They involve systemic issues with a social, economic and political component.
The interconnectedness of systems, intertwined with a people element makes IT
systems un-isolable from the human element. Complex Cyber systems today almost
have a life of their own; Cyber systems are complex adaptive systems that we have tried to understand and
tackle using more traditional theories.
2.
Complex Systems – an Introduction
Before getting into the
motivations of treating a Cyber system as a Complex system, here is a brief of
what a Complex system is. Note that the term “system” could be any combination of people, process or technology
that fulfils a certain purpose. The wrist watch you are wearing, the
sub-oceanic reefs, or the economy of a country – are all examples of a “system”.
In very simple terms, a Complex system is any system in
which the parts of the system and their interactions together represent a
specific behaviour, such that an analysis of all its constituent parts cannot
explain the behaviour. In such systems the cause and effect can not necessarily
be related and the relationships are non-linear - a small change could have a
disproportionate impact. In other words, as Aristotle said “the whole is
greater than the sum of its parts”. One of the most popular examples used in
this context is of an urban traffic system and emergence of traffic jams;
analysis of individual cars and car drivers cannot help explain the patterns
and emergence of traffic jams.
While a Complex Adaptive system (CAS) also has
characteristics of self-learning, emergence and evolution among the
participants of the complex system. The participants or agents in a CAS show
heterogeneous behaviour. Their behaviour and interactions with other agents
continuously evolving. The key characteristics for a system to be characterised
as Complex Adaptive are:
ü The
behaviour or output cannot be predicted simply by analysing the parts and
inputs of the system
ü The
behaviour of the system is emergent and changes with time. The same input and
environmental conditions do not always guarantee the same output.
ü The
participants or agents of a system (human agents in this case) are
self-learning and change their behaviour based on the outcome of the previous
experience
Complex processes are often
confused with “complicated”
processes. A complex process is something that has an unpredictable output,
however simple the steps might seem. A complicated process is something with
lots of intricate steps and difficult to achieve pre-conditions but with a
predictable outcome. An often used example is: making tea is Complex (at least
for me … I can never get a cup that tastes the same as the previous one),
building a car is Complicated. David Snowden’s Cynefin framework gives a more
formal description of the terms [7] .
Complexity as a field of study isn’t
new, its roots could be traced back to the work on Metaphysics by Aristotle [8] . Complexity theory
is largely inspired by biological systems and has been used in social science,
epidemiology and natural science study for some time now. It has been used in
the study of economic systems and free markets alike and gaining acceptance for
financial risk analysis as well (Refer my paper on Complexity in Financial risk
analysis here).
It is not something that has been very popular in the Cyber security so far,
but there is growing acceptance of complexity thinking in applied sciences and
computing.
3.
Motivation for using Complexity in Cyber
Security
IT systems today are all designed
and built by us (as in the human community of IT workers in an organisation
plus suppliers) and we collectively have all the knowledge there is to have
regarding these systems. Why then do we see new attacks on IT systems every day
that we had never expected, attacking vulnerabilities that we never knew
existed? One of the reasons is the fact that any IT system is designed by
thousands of individuals across the whole technology stack from the business
application down to the underlying network components and hardware it sits on.
That introduces a strong human element in the design of Cyber systems and
opportunities become ubiquitous for the introduction of flaws that could become
vulnerabilities [9] .
Most organisations have multiple
layers of defence for their critical systems (layers of firewalls, IDS,
hardened O/S, strong authentication etc), but attacks still happen. More often
than not, computer break-ins are a collision of circumstances rather than a
standalone vulnerability being exploited for a cyber-attack to succeed. In
other words, it’s the “whole” of the circumstances and actions of the attackers
that cause the damage.
3.1
Reductionism vs Holisim approach
Reductionism and Holism are two
contradictory philosophical approaches for the analysis and design of any
object or system. The Reductionists argue that any system can be reduced to its
parts and analysed by “reducing” it
to the constituent elements; while the Holists argue that the whole is greater
than the sum so a system cannot be analysed merely by understanding its parts [10] .
Reductionists argue that all
systems and machines can be understood by looking at its constituent parts.
Most of the modern sciences and analysis methods are based on the reductionist
approach, and to be fair they have served us quite well so far. By understanding
what each part does you really can analyse what a wrist watch would do, by
designing each part separately you really can make a car behave the way you
want to, or by analysing the position of the celestial objects we can
accurately predict the next Solar eclipse. Reductionism has a strong focus on
causality – there is a cause to an affect.
But that is the extent to which
the reductionist view point can help explain the behaviour of a system. When it
comes to emergent systems like the human behaviour, Socio-economic systems,
Biological systems or Socio-cyber systems, the reductionist approach has its
limitations. Simple examples like the human body, the response of a mob to a
political stimulus, the reaction of the financial market to the news of a
merger, or even a traffic jam – cannot be predicted even when studied in detail
the behaviour of the constituent members of all these ‘systems’.
We have traditionally looked at
Cyber security with a Reductionist lens with specific point solutions for
individual problems and tried to anticipate the attacks a cyber-criminal might
do against known vulnerabilities. It’s time we start looking at Cyber security
with an alternate Holism approach as well.
3.2
Computer Break-ins are like pathogen infections
Computer break-ins are more like viral or
bacterial infections than a home or car break-in [9] . A burglar breaking
into a house can’t really use that as a launch pad to break into the
neighbours. Neither can the vulnerability in one lock system for a car be
exploited for a million others across the globe simultaneously. They are more
akin to microbial infections to the human body, they can propagate the
infection as humans do; they are likely to impact large portions of the
population of a species as long as they are “connected” to each other and in
case of severe infections the systems are generally ‘isolated’; as are people
put in ‘quarantine’ to reduce further spread [9] . Even the lexicon of
Cyber systems uses biological metaphors – Virus, Worms, infections etc. It has
many parallels in epidemiology, but the design principles often employed in
Cyber systems are not aligned to the natural selection principles. Cyber
systems rely a lot on uniformity of processes and technology components as
against diversity of genes in organisms of a species that make the species more
resilient to epidemic attacks [11] . More on this later
in the paper …
Complexity theory has gained
great traction and proven quite useful in epidemiology, understanding the patterns
of spread of infections and ways of controlling them. Researchers are now
turning towards using their learnings from natural sciences to Cyber systems.
4.
Approach to Mitigating security threats
Traditionally there have been two
different and complimentary approaches to mitigate security threats to Cyber
systems that are in use today in most practical systems [11] :
4.1
Formal validation and testing
This approach
primarily relies on the testing team of any IT system to discover any faults in
the system that could expose a vulnerability and can be exploited by attackers.
This could be functional testing to validate the system gives the correct
answer as it is expected, penetration testing to validate its resilience to
specific attacks, and availability/ resilience testing. The scope of this
testing is generally the system itself, not the frontline defences that are
deployed around it.
This
is a useful approach for fairly simple self-contained systems where the
possible user journeys are fairly straightforward. For most other
interconnected systems, formal validation alone is not sufficient as it’s never
possible to ‘test it all’.
Test automation
is a popular approach to reduce the human dependency of the validation
processes, but as Turing’s Halting problem of Undecideability[1]
proves – it’s impossible to build a machine that tests another one in all cases.
Testing is only anecdotal evidence that the system works in the scenarios it
has been tested for, and automation helps get that anecdotal evidence quicker.
4.2
Encapsulation and boundaries of defence
For systems that
cannot be fully validated through formal testing processes, we deploy
additional layers of defences in the form of Firewalls or network segregation
or encapsulate them into virtual machines with limited visibility of the rest
of the network etc. Other common techniques of additional defence mechanism are
Intrusion Prevention systems, Anti-virus etc.
This approach is
ubiquitous in most organisations as a defence from the unknown attacks as it’s
virtually impossible to formally ensure that a piece of software is free from
any vulnerability and will remain so.
Approaches using Complexity sciences
could prove quite useful complementary to the more traditional ways. The
versatility of computer systems make them unpredictable, or capable of emergent
behaviour that cannot be predicted without “running it” [11] . Also running it in
isolation in a test environment is not the same as running a system in the real
environment that it is supposed to be in, as it’s the collision of multiple
events that causes the apparent emergent behaviour (recalling holism!).
4.3
Diversity over Uniformity
Robustness to
disturbances is a key emergent behaviour in biological systems. Imagine a
species with all organisms in it having the exact same genetic structure, same
body configuration, similar antibodies and immune system – the outbreak of a
viral infection would have wiped out complete community. But that does not
happen because we are all formed differently and all of us have different
resistance to infections.
Similarly
some mission critical Cyber systems especially in the Aerospace and Medical
industry implement “diversity
implementations” of the same functionality and centralised ‘voting’
function decides the response to the requester if the results from the diverse
implementations do not match.
It’s fairly
common to have redundant copies of mission critical systems in organisations,
but they are homogenous implementations rather than diverse – making them
equally susceptible to all the faults and vulnerabilities as the primary ones.
If the implementation of the redundant systems is made different from the
primary – a different O/S, different application container or database versions
– the two variants would have different level of resilience to certain attacks.
Even a change in the sequence of memory stack access could vary the response to
a buffer overflow attack on the variants [12] – highlighting the central
‘voting’ system that there is something wrong somewhere. As long as the input
data and the business function of the implementation are the same, any
deviations in the response of the implementations is a sign of potential
attack. If a true service-based architecture is implemented, every ‘service’
could have multiple (but a small number of) heterogeneous implementations and
the overall business function could randomly select which implementation of a
service it uses for every new user request. A fairly large number of different
execution paths could be achieved using this approach, increasing the
resilience of the system [13] .
Multi variant
Execution Environments (MVEE) have been developed, where applications with
slight difference in implementation are executed in lockstep and their response
to a request are monitored [12] . These have proven quite
useful in intrusion detection trying to change the behaviour of the code, or
even identifying existing flaws where the variants respond differently to a
request.
On similar
lines, using the N-version programming concept [14] ; an N-version
antivirus was developed at the University of Michigan that had heterogeneous
implementations looking at any new files for corresponding virus signatures.
The result was a more resilient anti-virus system, less prone to attacks on
itself and 35% better detection coverage across the estate [15] .
4.4
Agent Based Modelling (ABM)
One of the key
areas of study in Complexity science is Agent Based Modelling, a simulation
modelling technique.
Agent Based
Modelling is a simulation modelling technique used to understand and analyse
the behaviour of Complex systems, specifically Complex adaptive systems. The
individuals or groups interacting with each other in the Complex system are
represented by artificial ‘agents’ and act by predefined set of rules. The
Agents could evolve their behaviour and adapt as per the circumstances.
Contrary to Deductive reasoning[2]
that has been most popularly used to explain the behaviour of social and
economic systems, Simulation does not try to generalise the system and agents’
behaviour.
ABMs have been quite popular to study things like crowd management
behaviour in case of a fire evacuation, spread of epidemics, to explain market
behaviour and recently financial risk analysis. It is a bottom-up modelling
technique wherein the behaviour of each agent is programmed separately, and can
be different from all other agents. The evolutionary and self-learning
behaviour of agents could be implemented using various techniques, Genetic
Algorithm implementation being one of the popular ones [16] .
Cyber systems
are interconnections between software modules, wiring of logical circuits,
microchips, the Internet and a number of users (system users or end users).
These interactions and actors can be implemented in a simulation model in order
to do what-if analysis, predict the impact of changing parameters and
interactions between the actors of the model. Simulation models have been used
for analysing the performance characteristics based on application
characteristics and user behaviour for a long time now – some of the popular
Capacity & performance management tools use the technique. Similar
techniques can be applied to analyse the response of Cyber systems to threats,
designing a fault-tolerant architecture and analysing the extent of emergent
robustness due to diversity of implementation.
One of the key
areas of focus in Agent Based modelling is the “self-learning” process of agents.
In the real world, the behaviour of an attacker would evolve with
experience. This aspect of an agent’s behaviour is implemented by a learning
process for agents, Genetic Algorithm’s being one of the most popular technique
for that. Genetic Algorithms have been used for designing automobile and
aeronautics engineering, optimising the performance of Formula one cars [17] and simulating the
investor learning behaviour in simulated stock markets (implemented using Agent
Based models).
An interesting
visualisation of Genetic Algorithm – or a self-learning process in action – is
the demo of a simple 2D car design process that starts from scratch with a set
of simple rules and end up with a workable car from a blob of different parts: http://rednuht.org/genetic_cars_2/
The
self-learning process of agents is based on “Mutations”
and “Crossovers” - two basic
operators in Genetic Algorithm implementation. They emulate the DNA crossover
and mutations in biological evolution of life forms. Through crossovers and
mutations, agents learn from their own experiences and mistakes. These could be
used to simulate the learning behaviour of potential attackers, without the
need to manually imagine all the use cases and user journeys that an attacker
might try to break a Cyber system with.
5.
Conclusion
Complexity in Cyber systems,
especially the use of Agent Based modelling to assess the emergent behaviour of
systems is a relatively new field of study with very little research done on it
yet. There is still some way to go before using Agent Based Modelling becomes a
commercial proposition for organisations. But given the focus on Cyber security
and inadequacies in our current stance, Complexity science is certainly an
avenue that practitioners and academia are increasing their focus on.
Commercially available products
or services using Complexity based techniques will however take a while till
they enter the mainstream commercial organisations.
References
[1]
|
J. A. Lewis and S. Baker, “The
Economic Impact of Cybercrime and Cyber Espionage,” 22 July 2013. [Online].
Available:
http://csis.org/publication/economic-impact-cybercrime-and-cyber-espionage.
[Accessed 17 June 2014].
|
[2]
|
L. Kugel, “Terrorism and the
Global Economy,” E-Internatonal Relations Students, 31 Aug 2011. [Online].
Available:
http://www.e-ir.info/2011/08/31/what-is-the-impact-of-terrorism-on-the-ipe/.
[Accessed 17 June 2014].
|
[3]
|
“Cybersecurity - Facts and
Figures,” International Telecommunications Union, [Online]. Available:
http://www.itu.int/en/ITU-D/Partners/Pages/Call4Partners/CYBLDCStats.aspx.
[Accessed 11 June 2014].
|
[4]
|
“Interesting Facts on
Cybersecurity,” Florida Tech University Online, [Online]. Available:
http://www.floridatechonline.com/online-degree-resources/interesting-facts-on-cyber-security/.
[Accessed 12 June 2014].
|
[5]
|
“Global security spending to hit
$86B in 2016,” 14 Sep 2012. [Online]. Available: http://www.infosecurity-magazine.com/view/28219/global-security-spending-to-hit-86b-in-2016/.
|
[6]
|
S. Forrest, S. Hofmeyr and B.
Edwards, “The Complex Science of Cyber Defense,” 24 June 2013. [Online].
Available: http://blogs.hbr.org/2013/06/embrace-the-complexity-of-cybe/.
|
[7]
|
“Cynefin Framework (David
Snowden),” [Online]. Available: http://en.wikipedia.org/wiki/Cynefin.
|
[8]
|
“Metaphysics (Aristotle),”
[Online]. Available: http://en.wikipedia.org/wiki/Metaphysics_(Aristotle).
|
[9]
|
R. Armstrong, “Motivation for the
Study and Simulation of Cybersecurity as a Complex System,” 2008.
|
[10]
|
S. A. McLeod, Reductionism and
Holism, 2008.
|
[11]
|
R. C. Armstrong, J. R. Mayo and
F. Siebenlist, “Complexity Science Challenges in Cybersecurity,” March
2009.
|
[12]
|
B. Salamat, T. Jackson, A. Gal
and M. Franz, “Orchestra: Intrusion Detection Using Parallel Execution and
Monitoring of Program Variants in User-Space,” Proceedings of the 4th
ACM European conference on Computer systems, pp. 33-46, April 2009.
|
[13]
|
R. C. Armstrong and J. R. Mayo,
“Leveraging Complexity in Software for Cybersecurity (Abstract),” Association
of Computing Machinery, pp. 978-1-60558-518-5, 2009.
|
[14]
|
C. Liming and A. Avizienis,
“N-VERSION PROGRAMMINC: A FAULT-TOLERANCE APPROACH TO RELlABlLlTY OF
SOFTWARE OPERATlON,” Fault-Tolerant Computing, p. 113, Jun1995.
|
[15]
|
J. Oberheide, E. Cooke and F.
Jahanian, “CloudAV: N-Version Antivirus in the Network Cloud,” University
of Michigan, Ann Arbor, MI 48109, 2008.
|
[16]
|
J. H. Holland, Adaptation in
natural and artificial systems: An introductory analysis with applications
to biology, control, and artificial intelligence, Michigan: University
of Michigan Press, 1975.
|
[17]
|
K. &. B. P. J. Wloch,
“Optimising the performance of a formula one car using a genetic
algorithm,” Parallel Problem Solving from Nature-PPSN VIII, pp.
702-711, January 2004.
|
[18]
|
P. E. (. o. D. Leon, “Press
Transcript,” US Department of Defense, 11 Oct 2012. [Online]. Available:
http://www.defense.gov/transcripts/transcript.aspx?transcriptid=5136.
[Accessed 12 June 2014].
|
[1]
Alan Turing – a mathematician who came to fame for his role in breaking the
Enigma machines used to encrypt communication messages during the second world
war – proved that a general algorithm whether or not a program would even
terminate (or keep running forever) for all program-input pairs cannot exist.
[2]
Deductive reasoning is a ‘top-down’ reasoning approach starting with a
hypothesis and data points used to substantiate the claim. Inductive reasoning
on the other hand is a ‘bottom-up’ approach that starts with specific
observations which are then generalised to form a general theory
No comments:
Post a Comment