Analytics in a Collaborative Solution-Discovery Project
We use our computers and smart phones in many ways. We play games, communicate via social media, navigate through our world, search for information, keep track of our finances, do tasks in our jobs, and so on. Common to all of this: the computers do the heavy lifting and number crunching. We assess the results and make decisions. That is the role of analytics in a Collaborative Solution-Discovery project.
Analytics are for handling the numbers
When a CSD project needs to do the numbers to produce quantitative results, what we are calling “Analytics” is the method. Typical objectives of analytics in a CSD project might be:
- To assess the extent of a problem
- To select the features and right-size the design parameters of a potential solution
- To evaluate competing prospective solutions to select the best
Analytics refers to all the possible computational procedures – formulae, algorithms, models, symbolic functions, graphical methods, and the like that are commonly employed to produce quantitative answers. The range of analytics is so vast and so specifically related to the disciplines pertinent to a particular project that a general treatment will not be attempted here. Only a summary of the place for analytics in a CSD project with a couple of examples will be provided. The focus is on issues that might arise.
Analysis would naturally be important for a hard system project like improving regional transportation or mitigating the impact of sea level rise, but could often be part of a soft-system project as well. For instance, design of a Regional Integrated Social Services Delivery System would depend on estimating the expected demand for various social services, and when the system was in operation, an important performance metric would be the actual client demand profile as compared with the available capacity for the various services.
Analytics and Complexity
How do analytics help a CSD collaborative team to overcome the obstacle of complexity? Quite simply, analytics enable a large measure of complexity to be encapsulated in a compact symbolic expression. A mathematical function involving several variables can show how variations of one or several variables would affect the others. A model of a dynamic system can be used to simulate the behavior of a system as it unfolds through time under a variety of conditions, supporting exploration of the system’s capability, and exposure of unexpected emergent behavior. This can be used as a tool for understanding the problem system, designing solutions, and then comparing several solutions to select the best.
A particularly important benefit of analytics is their ability to expose otherwise unexpected behaviors of complex systems. Quite often, such systems behave in a counter-intuitive manner that eludes common sense, but is revealed by the analysis. Totally unexpected emergent behaviors occur, and are discovered in time to deal with their consequences.
An example occurs in traffic planning. A wide four-lane street (two lanes each way) is converted to a configuration with a single through lane each direction that allows space for left turn lanes and entry lanes from side streets. This configuration actually has higher throughput in cars per hour because the through lanes are not held up by cars turning left or entering traffic. However, drivers don’t like it because average speed, travelling the street from end to end, is lower. How can that be: higher throughput but lower average speed? Well, the model predicts that and experience bears it out.
Applying Analytics in a CSD project.
Employing analytic methods in a project lends itself to detailed planning and management. Setting up the analytic methods for a particular project is the job of discipline experts on the project core team, in support of the goals of each phase of the project. In Phase 1 analytics are used to assess the extent of the problem. In Phase 2, analytic methods are selected as a part of the solution-discovery plan, and may also be targeted for inclusion as part of the solution if the solution involves making quantified choices. In Phase 3, analytics will be exercised in searching for and evaluating solutions, and perhaps included in the solution options that are developed.
Two consideration apply to implementing analytics in a project:
- Be sure the experts with the needed skills are on the team and understand their role in supporting, not driving, the solution-discovery effort.
- Set up and execute the analytical processes with full stakeholder engagement. Give full consideration to stakeholder interests. Be sure that stakeholders are fully informed and educated about the place of analytics in the project. Take the opportunity to use analytics as a tool for stakeholder engagement, particularly in expanding stakeholder appreciation for the full extent of the issues to be considered.
Issues
There are issues that challenge the effectiveness of analytics in a CSD project.
- Innumeracy
- Skepticism about models, and analytics in general
- Risk assessment
Innumeracy ranges from discomfort with numbers and math to flat-out incapacity to understand and work with numbers and mathematics as tools for understanding. Unfortunately, innumeracy is common in the population, and that will be the case with stakeholders. Those with the problem may have difficulty understanding and accepting the process of analysis and its results, and may even be suspicious or hostile toward its use. Sensitivity in handling this will be an important part of stakeholder engagement. Leadership should respect the quandary of people who have trouble grasping and trusting analytic methods that are necessary for the project.
Even among those comfortable with analysis, skepticism of models may be a problem. The attitude may be “It’s just a model, not reality, and it can’t be trusted.” Such an attitude is worthy of respect if it is a matter of exercising due caution in the interpretation of the results of modeling. After all, it is said, “The map is not the territory.” A map on paper is useful for locating travel objectives and planning routes but tracing the route on the map is definitely not the same a travelling over the landscape, and most of the features experienced on such a trip will not be illustrated on the map.
However, if resistance to models is more extreme, point out that everything we do depends on models. Psychologically, we do not live in external reality. We live in models of external reality that exist in our minds, that are continually updated with sensory input. If we are sane and good observers, those models serve us well. We remain safe while driving our cars because we have reasonably trustworthy internal models of how other drivers will behave in traffic, and of how our car will respond to driving commands. Those models we use when driving are so well en grained that we use them without thinking., as a natural extension of ourselves. Weather forecasts and financial models are others we depend on in our everyday lives.
Uncertainty and risk are common in life and will certainly be a factor in many CSD projects. They will occur in understanding and representing the problem addressed by the project. They will be factors in design and selection of the preferred solution, and will often be issues in the operation of the solution. Unfortunately, the ability to reliably assess risk and uncertainty are talents mostly lacking in the general population. Not only are they lacking, but quite often the logically and mathematically correct results appear counter-intuitive, or at least incomprehensible, and so are rejected by many stakeholders. Again, the analysis of risk and uncertainty in support of a CSD project must be set up by qualified experts, and then introduced with sensitivity to the broad range of stakeholders so the stakeholders are comfortable and trusting of the results.
Stakeholder Engagement
Participation in setting up and running a model is a powerful technique for stakeholder engagement. This forces stakeholders to expand their viewpoint on the reality within the problem and its solution as they struggle to construct a model that actually represents and faithfully imitates that full reality. It builds trust in the outcome through participation in the process that leads to the outcome.
A glimpse into the “How to” of modeling
Modeling is a tool for understanding a problem, supporting the design of solution options, evaluation of options to select the preferred one, and not without importance, as a communication tool for engaging stakeholders.
Modeling begins with an overall view of the system to be modeled. The Form and Function views, and perhaps others, are the usual starting point. Working from those views, identify the state variables of the system. They are the parameters whose values characterize a snapshot of the state or condition of the system at a particular time, and distinguish that state from all other states the system might assume. Then set up mathematical relationships among the state variables, the inputs and outputs of the system, and time. Create mathematical expressions representing the behavior of the elements in the selected view.
If the system has only a few elements, a single “all-in-one” expression that describes the whole system may be possible, with all the pertinent variables in one function. For a system with lots of elements, the “all-in-one” approach may be too difficult, so create separate models for each element, linked by the “output to input” or interface variables shared among the various elements. In that approach, each element has internal functions that receive inputs from other elements or the environment, and process them into outputs to other elements or the environment.
For those not used to working in this way, to get the idea, let’s start with a simple example, a system of just one element and one state variable. The example we choose is a bathtub with open drain, and water running in from the faucet. The system element is the bathtub, and the state variable is the water level (call it L) in the tub. Starting with L = 0, initially the water level will rise pretty fast because, with low water level, the outflow through the drain will be small. As water level rises the outflow also rises, because the pressure at the drain that forces water through the drain increases in proportion to L. Eventually the outflow reaches the same value as the inflow from the faucet. At that point the water level stops rising and the system has reached a steady state. That qualitative description of the bathtub system is adequate to illustrate how to set up a model. The more mathematically inclined would be able to set up the differential equation that represents the bathtub system, and actually calculate the timewise behavior.
To extend that illustration to something more complex, envision a garden water-feature consisting of bowls in a cascade arrangement. The outlets from bowls higher up sometimes divide, sometimes merge, feeding spouts to the bowls in lower levels. Each bowl is an input-reservoir-outlet element like the simple bathtub system. The inter-dependencies among elements are created by the plumbing that routes water from above to below. To make it even more complex, imagine that the inter-bowl plumbing includes valves that can be opened or closed by linkages connected to floats on various of the bowls. This introduces the possibility of feedback as bowls lower down, later in the water flow sequence, can influence bowls higher up and earlier in the sequence. It’s hard to imagine just how that system might behave, starting with all the bowls empty and the faucet to the topmost bowl just turned on. It might turn out be quite dynamic and fun, or it might just settle into a boring steady state. Wouldn’t be better to set up the model and find out before actually building it and being disappointed?
Various platforms can be used to do the math for running a model. A spreadsheet is a simple choice. The values in the cells on any line would be the state variables, and the functions entered behind the cells relate the state variables to one another, to the inputs, and to time. The spreadsheet calculates the values of all the variables in the line below from the values in the line above, after the passage of an increment of time. Making the time increment small, and calculating behavior for a large number of time increments, gives a pretty good approximation of how the real system would behave.
Sometimes systems involve situations where inputs vary in a random manner, and the behavior of the system over a range of input situations is important to study. The behavior of a retirement investment portfolio, under different market conditions and different needs for cash during the retiree’s lifetime, would be a typical example. A technique called a Monte Carlo study would be appropriate for this analysis. Many runs of the model are done, each with a different scenario for market performance and cash needs, chosen at random from the set of all reasonable possibilities. The aggregate of all these runs characterizes the expected range of future outcomes, with most likely outcomes clustered around the center of the output range, flanked by less likely favorable and unfavorable outcomes at the extremes. A good retirement plan would have a high likelihood (lots of of possible outcomes) in the acceptable range, and very few in the unfavorable range.
“System Dynamics,” technically called a stock and flow representation of a dynamic system, is a widely used modeling tool. It is well suited to modeling social systems, and is available in a number of software versions. Typically, model creation is a matter of building a flow chart of the system on the computer screen, then entering variable names and functions as associated with the symbols in the flow chart.