Models and Tools for the Computational Support of Technology Impact Assessments, Applied in the Context of Mass Transportation
Fig. 4.1
A map of issues and actors in TIA in mass transportation (Figure translated, originally from: Hempel, Leon et al., “SIAM – Security Impact Assessment Measures. Ein System zur Unterstützung von Security Technology Assessments an Flughäfen und im öffentlichen Nahverkehr”, Oranienburger Schriften, (Fachhochschule der Polizei des Landes Brandenburg. 2011))
This chapter describes the features and conceptual underpinnings of a web-based software prototype which was created in the course of the SIAM FP7 project.5 The term assessment support system was coined to characterise this system and distinguish it from common decision support systems6 which use different methods to quantify a limited set of assessment criteria in order to try and determine a particular solution that best fits a purpose in comparison to others.
Such systems would fall short in light of the complexity associated with more comprehensive technology impact assessments, where human reflection and judgment are generally considered to achieve better results. The aims of an assessment support system are to collect and structure the information elicited from different actors, and provide effective representations that help human assessors to navigate and deal with the vast space of assessment issues more easily. The system presented here makes use of an extensive catalogue of assessment questions that take different perspectives on the technological solution at hand, and elicits information about a planned technology acquisition, as well as expertise and opinions from the actors involved in the assessment. This information is then processed and compiled in a structured assessment report to give insight into how different actors have evaluated various aspects of a technology and its impact, to identify areas of agreement or conflict, and give guidance on aspects where the assessment process could be improved.
In essence, an assessment support system is more concerned with the process rather than the result. That means the particular measures employed to provide guidance and decision-support do not directly point towards a particular technological solution that seems to be more favourable than another, based on performance measures, for example. In contrast, an assessment support system uses a range of semantic metadata of the issues addressed, and of the actors who address them, in order to also indicate higher-level properties of the assessment process itself. For example, this is useful to find out whether the assessment of certain topics was balanced, in terms of a sufficient number of different relevant actors having contributed their view; or, to point out whether certain actors are currently missing and should be included in the assessment. In this general way, an assessment support system is meant to enhance, rather than replace, traditional decision support approaches. The focus here is to guide the assessment process and improve the reflexivity of information, opinions, and expertise among the contributors and decision-makers as they engage in technology impact assessment.
In the next section, the application context for which an assessment support system has been developed will be described, followed by details on some related knowledge which has informed the design of the software system. In particular, details will be provided on the kinds of users which were considered for the software, how activities were mapped to specific user interactions, and a general overview of the system’s architecture. In Sect. 4.3, some of the conceptual underpinnings of the system will be discussed. Finally, the main features of the software will be presented in Sect. 4.4, followed by some details on the evaluation procedures in Sect. 4.5, and a closing summary (Sect. 4.6).
4.2 Application Context and Mapping
This part provides an overview of the assessment process supported, details on how this process was mapped to a general use case to be implemented in software tools, and a description of the overall architecture envisioned for the computational assessment support system.
4.2.1 Background
The application context for the software is situated in the field of security technology acquisition. In particular, the related decision-making processes in mass transportation sites have been taken as an exemplar for devising an abstract, idealised assessment process which should be supported by a software system (Fig. 4.2). This development was informed by a series of case studies conducted with industrial partners from the mass transportation domain, such as airport and railway operators.7 The case studies were based on the idea of reconstructing the innovation journeys8 of previous technology acquisitions at the respective sites, and concerned the elicitation of related information such as the relevant actors involved, assessment criteria applied, and the structure of the overall decision-making process.
Fig. 4.2
General phases of the idealised SIAM assessment process (From: Graeme Jones and Ronald Grau, “The SIAM Assessment Support System: Initial Application and Database Specification”, SIAM deliverable D11.1, (European Commission, 2012), 6)
In this context, an assessment is made in the context of evaluating technology options for solving new or existing security problems. Figure 4.2 shows the basic activities underlying this process. At the beginning of the assessment process is a scenario formulation phase, where the security need is explained, and a technological solution suggested. This information is usually available as result of a locally conducted threat assessment and constitutes the beginning of the “Concept/New Option” phase of the technology acquisition process. In this phase, a problem has been identified and technological solutions are being considered to address the problem.
Different actors get involved in a subsequent evaluation process which may have different roles, depending on the stakeholders they represent, and their associated interests, needs, motivations, and responsibilities. The different actors engage in the definition and negotiation of the assessment criteria that are deemed important to consider before a decision is made about whether an investment into some technology option should go forward. After the first phase, part of the process is repeated to address further phases of the technology acquisition process, such as “Testing/Development”, for instance, which may involve other actors and related assessment criteria. This idealised process assumes that after the technological solution has been further assessed in the context of development, implementation, testing, diffusion, sustainment, and wider change, a well-informed decision can be made.9 Since every phase will invokes different sets of actors and assessment issues, the first one (“Concept New Option”) was chosen as an exemplar for demonstrating the capabilities of an assessment support system in the course of the SIAM project.10
4.2.2 General Use Case
The development of a software system usually involves the design of one or more use cases mapping a problem-related workflow to purposeful human interactions with a user interface.
At its core, the assessment process illustrated in Fig. 4.2 has been realised through a mechanism involving three major steps (Fig. 4.3):
Fig. 4.3
General use case mapping of the SIAM assessment support process (From: Ronald Grau and Graeme Jones, “The SIAM Assessment Support Toolkit: A Software System for the Support of Technology Impact Assessments”, SIAM deliverable D11.2, (European Commission, 2014), 10)
1.
In the assessment configuration step, a specific assessment case is created which has all relevant information about a given scenario and the suggested solutions, a technology proposal for addressing the problem, and a first set of actors which are to be involved in the assessment case (which can later be extended).
2.
Assessment questions are posed to the actors about the particular problem scenario which is to be assessed, covering a wide range of topics concerning organisational, economic, technological, legal, societal, as well as trust- and security-related issues. The questions are aimed towards acquiring information from different perspectives, targeted at particular actors, and differentiated by the particular attributes of the technological solution proposed. In other words, the configuration of the assessment case determines the kinds and number of particular questions asked, such that those which are not relevant for the current case can be hidden. Semantic data is used to target questions more effectively to the different actors, and questions can be interconnected and have mutual conditions in place which control their presentation. The answers given by the various actors are eventually stored in a database and associated with the current case.
3.
Assessment reports can be compiled from the information gathered for a case which summarise the contributions made by the actors and show an analysis of the answers given based on the defined assessment perspectives, tasks, and subjects, in order to provide further guidance on how the assessment process can be improved to achieve a more comprehensive and reflective evaluation of the proposed solution.
4.2.3 Actor Roles and User Functions
For the design of a software system, it is important to determine the kinds of users which will interact with it before any useful features can be implemented. Aside from scientific researchers who are free to take any particular stance in using the system, two distinct actor groups and some of their roles, responsibilities and interests were initially identified:
Policy Setters
Policy Implementers
Stakeholders:
Transport and Civil Aviation Authorities, Watchdogs, Non-Government Organisations
Roles:
Establishers and enforcers of security standards and regulations, Supervision of facility managers, Safeguards of security and safety, Creators of subjective security, Safeguards of freedom and fundamental rights.
Responsibilities:
Compliance with other legalistic requirements, realisation of state policies, Increasing homeland security, Maintaining passenger rights, Harmonisation of legal frameworks, Funding of research, Identification of infringements, Demanding or creating counter infringement measures and –technologies.
Interests:
Understanding threats, understanding security measures and -technologies, Advocating privacy and data protection, Understanding infringements, international recognition and acceptance.
Stakeholders:
Facility managers, Facility personnel, Police and contracted security providers.
Roles:
Investors, Implementers, Providers of retail space, Operators, Police forces.
Responsibilities:
Operating a transport facility within a business context subject to the security requirements specified in security policies, Legal compliance.
Interests:
Keeping up a secure state of the facility (prevent damage and disturbance of operations), Retail income maximisation, Security Investment optimisation, Minimising acceptance issues (including Convenience, Health and Infringement implications).
Source: SIAM Initial Application and Database Specification11
In addition to a specification of professional roles for the participants in an assessment case which reflect certain interests and responsibilities of the related stakeholders, it was also recognised that the users of the system must perform different tasks, or functions, which are more closely related to what they do when using the software. Consequently, user functions were introduced as a means for distributing different organisational and contributory tasks in the assessment process (Table 4.1). These user functions go along with certain permissions and access rights to features of the software that are relevant for each user to fulfil their particular set of tasks. The functions are, in principle, independent of an actor’s role.
Table 4.1
User functions in the SIAM AST
Function | Permissions |
|---|---|
Assessment leader (Coordinator) | Can set up new assessment cases |
Can specify and invite other actors | |
Can answer assessment questions | |
Can edit and generate the overall assessment report | |
Assessment participant | Can access assessment cases (once invited) |
Can answer assessment questions according to their specified role | |
Can create custom questions to be considered by other actors | |
Can generate a summary report of their personal contributions | |
Information provider (External consultant) | Can access specific assessment cases (once invited) |
Can answer only those assessment questions which have been delegated by other actors | |
Observer (Auditor) | Can access specific existing assessment cases |
Cannot answer any questions | |
Can inspect the assessment report |
For instance, any user of the software can create assessment cases, which assigns the function “Assessment Leader” to that particular user, limited to the case created. The same user could fulfil different functions in other assessment cases, acting as a participant, observer, or information provider.
4.2.4 System Architecture
In the assessment of security technologies, there is a need to adhere to established standards and utilise up-to-date information that can be retrieved from the public domain or from government authorities (e.g., legal texts, open standards, technical reports, crime statistics, threat assessments, research papers, etc.).12 On the other hand, there is also a desire to keep certain information within the facility or institution where an assessment is made, such as the physical layout of security sensitive areas, the functionality of existing installations, or the contents of location-specific threat scenarios. Based on these considerations, the assessment support system was envisioned to make use of distributed data, with some information retrieved from the external sources, and further, privately held information about facility-specific models of threats, processes and technologies. This is not to say that local information cannot be shared: It is imaginable that an exchange of relevant standards, best practices, assessment criteria or -scenarios between organisations can in fact bring positive synergy effects and improve the speed and quality of technology acquisition processes overall. However, a separation of the data will leave the assessing facility or institution in control of what private data to share and with whom.
At the heart of the computational implementation is a browser-based application that implements the mapped SIAM assessment support process.13 The application runs on a web server, and can be installed at different local sites. Private data is kept in a local database, and public assessment information retrieved over the Internet from other repositories, provided and maintained by public institutions or government authorities. Figure 4.4 shows how this architecture could look like should the system be chosen to be adopted and deployed at a comprehensive, possibly national or transnational scale.
Fig. 4.4
General architecture of the SIAM AST (From: Jones and Grau, “The SIAM Assessment Support System: Initial Application and Database Specification”, 12)
Within the SIAM project, the aim was to create a prototype of such a system which illustrates the capabilities of the assessment support application. As the partnerships with policy setter organisations which are required for implementing the above approach are clearly not existent in this phase of development, the architecture was simulated with a local demo database, which was used for storing all data. The prototype functionality comprises a modular software platform which has a technology impact assessment support application as core unit. A database stores various kinds of assessment information, such as that about the actors involved in the technology acquisition process, their particular roles, as well as the assessment criteria and questions that need to be considered. Within the assessment support application, these questions are then posed to users of the system in a structured fashion; information is collected from the users; and reports are generated based on the questions which were asked and the answers that were received.
4.3 Conceptual Developments
Software development is substantially informed by requirements engineering activities where subject experts provide the necessary information and knowledge required for the design of suitable software components.14 As the SIAM project aimed to include information related to several factual and process-based concepts, such as different kinds of technology, security measures, threats, and various impacts, a conceptual basis needed to be developed for inter-relating the relevant information. This presented a tremendous challenge, as many of those concepts form a complex domain of their own. The task of a software architect is to gain a sufficient conceptual understanding to be able to design the required software functionality. For this purpose, the conceptual models employed do not need to represent all domains in their full complexity and detail; they merely need to be adequate for the purpose such that they facilitate the design of data structures and functions to implement a specified use case in a software program. The conceptual models also need to be sufficiently general in order to enable understanding and communication about the assessment issues between different kinds of users of the software, considering these may have different areas and levels of expertise. As a result, somewhat simplified structures are often adequate to represent and interconnect information about processes and impacts at an abstract conceptual level, whereas the complexity of individual assessment issues can be expressed through carefully designed assessment questions. There have been previous efforts to structure and model the entities and relations that exist in the security domain.15 , 16 These provided useful inputs for modelling but were found to be of limited use as an off-the-shelf conceptualisation, considering the ambitious development goals of SIAM.
This section presents some of the conceptual ideas and models developed as a result of the requirements engineering activities mentioned above. These were conducted as knowledge elicitation and modelling exercises performed in workshops, surveys, and direct requests for information targeted at the content-providers available in the project, which included sociologists, criminologists, lawyers, engineers, and technologists from various international organisations. The conceptual ideas reflect the conceptual mapping of any knowledge retrieved into structured representations that can be implemented into computer programs. These models might not present a universal solution for all impact assessments and be quite specific to the current security technology application context, and the requirements of the software. However, they may be useful for other computer scientists that work in related areas or to inform future extensions of this work. Those readers who are mainly interested in the description of the implemented features can skip this part and go directly to Sect. 4.4.
4.3.1 Security Measures and Technologies
One of the core concepts of the system relates to the so-called Security Measures and Technologies (SMT) which are being assessed. In the most general sense, technology refers to the procedures and technical artefacts which have been designed for a particular purpose. In our model, technology is thus present in both an activity and a tool. SIAM’s conceptualisation of SMT has been based on a process-based view of the security domain which identifies the nature of security policies as the defining element for security measures, which are purposeful activities in the first place. The model then differentiates the actions for achieving the purpose of a policy from their implementation aspects, i.e. the actors and/or technology involved in executing the actions.
In essence, the underlying idea of this model is that every Security Process comprises one or more Security Measures, which are further described by three components: Actors, Activities, and Tools. Based on the specification of some security need or purpose (Security Policy), Security Processes can be represented as Security Policies that are implemented by the components of Security Measures (Fig. 4.5). This approach considers that technology refers to more than just the naïve perception of technical devices. A big advantage of describing security measures and technologies at this level of granularity is that relations to other concepts like threat, infringement, or trust can be established that are sufficiently meaningful to aid decision-makers in their evaluation of an SMT. Extension of this model were created to determine the relations, where Threats can be described in a similar fashion and connected to existing models of SMT. For instance, consider a handgun is an object that could be defined as the tool of a threat, and a statement can then be made whether this tool can be detected by a component of a particular security measure.17
Fig. 4.5
A process-based model of SMT (From: Graeme Jones and Ronald Grau, “A Typology of Security Measure Technologies and Counter Infringement Technologies”, SIAM deliverable D2.3, (European Commission, 2012), 6)
Real-world impact assessments do not usually address single technologies rather than complex technology stacks embedded within solutions, where every individual part may have a distinct impact on various issues, depending on what it is used for. For example, a person- or luggage scanner device actually combines a number of different technologies for imaging, analysis, reporting, etc., in order to offer a technological solution to a problem, and in compliance to the policies outlined in a security plan. With respect to infringement and the development of counter-infringement measures and technologies, the resolution adopted by this knowledge representation allows pinpointing quite clearly what exact part of a security solution presents a problem, acknowledging that it is actually part of a wider security process involving human operators and their specific activities, as well as architectural elements and economic constraints. For example, a body scanner is a complex technological solution, where the radiation emitted by some technical part (tool) may be the origin of health problems and thus, possible infringements of the bodily integrity of the scrutinised. Further, the officer inspecting the images delivered by the solution may be an actor carrying out an activity responsible for privacy issues. This list of examples could be continued easily.18
The range of security measure technologies that need to be grouped within this typology is enormous. To provide a practical limit to this range yet enable future extension, the following scoping has been adopted (Fig. 4.6). The Protection domain can be partitioned into two broad categories: Safety and Security. The Safety category includes accidental or environmental sources of threat such as fire, severe weather, flooding and major accidents.
Fig. 4.6
A protection typology (From: Jones and Grau, “A Typology of Security Measure Technologies and Counter Infringement Technologies”, 7, 5)
The Security category addresses threats that mainly arise from deliberate human action. The Security category addresses threats to both the physical and the informational realm. The SIAM project focused on security measures and technologies that address the People and Physical Infrastructure category which is a sensible constraint as many of the assessed impacts (e.g., infringement of rights, norms and other liberties, or socio-technical normativity, etc.) do not apply to the virtual realm and thus need to be examined at a level where information technology and people intersect.19
Given the conceptualisation and the scoping constraints outlined above, Security Measure Technologies (SMT) can be broadly partitioned into nine subclasses embedded within four groups, as shown in Fig. 4.7, and further detailed in the following sub-sections. The grouping of the SMTs corresponds to main directives of a security regime, some of which were taken from current security plans.20 Further, the typology accommodates measures that are commonly classified as either preventive or corrective measures, as well as measures used for establishing and protecting the security status of objects or people within a facility.
Fig. 4.7
The SIAM SMT typology (Figure and subsequent descriptions: Jones and Grau, “A Typology of Security Measure Technologies and Counter Infringement Technologies”, 8)
In the following, the individual groups, and the types embedded within the groups will be described in more detail.
4.3.1.1 SMT Typology
Threat Detection
Threat Detection SMTs are used in security measures designed to detect potential threats by some signature of the attack. Both Threat and Security Measure can be modelled as a Process carried out by Actors using Tools to carry out an Action or Activity. Threat detection, therefore, can focus on the actors, the actions related to an event or the tools used.