Consensus and Cooperation in Networked Multi-Agent Systems

Consensus and Cooperation in Networked Multi-Agent Systems

Journal: Proceedings of the IEEE

  • Proceedings of the IEEE is the leading journal to provide an in-depth review, survey, and tutorial coverage of the technical developments in electronics, electrical and computer engineering, and computer science. Consistently ranked as one of the top journals by Impact Factor, Article Influence Score and more, the journal serves as a trusted resource for engineers around the world



  • This paper provides a theoretical framework for analysis of consensus algorithms for multi-agent networked systems with an emphasis on the role of directed information flow, robustness to changes in network topology due to link/node failures, time-delays, and performance guarantees. An overview of basic concepts of information consensus in networks and methods of convergence and performance analysis for the algorithms are provided. Our analysis framework is based on tools from matrix theory, algebraic graph theory, and control theory. We discuss the connections between consensus problems in networked dynamic systems and diverse applications including synchronization of coupled oscillators, flocking, formation control, fast consensus in small world networks, Markov processes and gossip-based algorithms, load balancing in networks, rendezvous in space, distributed sensor fusion in sensor networks, and belief propagation. We establish direct connections between spectral and structural properties of complex networks and the speed of information diffusion of consensus algorithms. A brief introduction is provided on networked systems with non-local information flow that are considerably faster than distributed systems with lattice-type nearest neighbor interactions. Simulation results are presented that demonstrate the role of small world effects on the speed of consensus algorithms and cooperative control of multi vehicle formations.


  • In networks of agents (or dynamic systems), “consensus” means to reach an agreement regarding a certain quantity of interest that depends on the state of all agents. A “consensus algorithm” (or protocol) is an interaction rule that specifies the information exchange between an agent and all of its (nearest) neighbors on the network (pp 215)
    • In my work, this is agreement on heading and velocity
  • Graph Laplacians are an important point of focus of this paper. It is worth mentioning that the second smallest eigenvalue of graph Laplacians called algebraic connectivity quantifies the speed of convergence of consensus algorithms. (pp 216)
  • More recently, there has been a tremendous surge of interest among researchers from various disciplines of engineering and science in problems related to multi-agent networked systems with close ties to consensus problems. This includes subjects such as consensus [26]–[32], collective behavior of flocks and swarms [19], [33]–[37], sensor fusion [38]–[40], random networks [41], [42], synchronization of coupled oscillators [42]–[46], algebraic connectivity of complex networks [47]–[49], asynchronous distributed algorithms [30], [50], formation control for multi-robot systems [51]–[59], optimization-based cooperative control [60]–[63], dynamic graphs [64]–[67], complexity of coordinated tasks [68]–[71], and consensus-based belief propagation in Bayesian networks [72], [73]. (pp 216)
    • That is a dense lit review. How did they order it thematically?
  • A byproduct of this framework is to demonstrate that seemingly different consensus algorithms in the literature [10], [12]–[15] are closely related. (pp 216)
  • To understand the role of cooperation in performing coordinated tasks, we need to distinguish between unconstrained and constrained consensus problems. An unconstrained consensus problem is simply the alignment problem in which it suffices that the state of all agents asymptotically be the same. In contrast, in distributed computation of a function f(z), the state of all agents has to asymptotically become equal to f(z), meaning that the consensus problem is constrained. We refer to this constrained consensus problem as the f-consensus problem. (pp 217)
    • Normal exploring/flocking/stampeding is unconstrained. Herding adds constraint, though it’s dynamic. The variables that have to be manipulated in the case of constraint to result in the same amount of consensus are probably what’s interesting here. Examples could be how ‘loud’ does the herder have to be? Also, how ‘primed’ does the population have to be to accept herding?
  • …cooperation can be informally interpreted as “giving consent to providing one’s state and following a common protocol that serves the group objective.” (pp 217)
  • Formal analysis of the behavior of systems that involve more than one type of agent is more complicated, particularly, in presence of adversarial agents in noncooperative games [79], [80]. (pp 217)
  • The reason matrix theory [81] is so widely used in analysis of consensus algorithms [10], [12], [13], [14], [15], [64] is primarily due to the structure of P in (4) and its connection to graphs. (pp 218)
  • The role of consensus algorithms in particle based flocking is for an agent to achieve velocity matching with respect to its neighbors. In [19], it is demonstrated that flocks are networks of dynamic systems with a dynamic topology. This topology is a proximity graph that depends on the state of all agents and is determined locally for each agent, i.e., the topology of flocks is a state dependent graph. The notion of state-dependent graphs was introduced by Mesbahi [64] in a context that is independent of flocking. (pp 218)
    • They leave out heading alignment here. Deliberate? Or is heading alignment just another variant on velocity
  • Consider a network of decision-making agents with dynamics ẋi = ui interested in reaching a consensus via local communication with their neighbors on a graph G = (V, E). By reaching a consensus, we mean asymptotically converging to a one-dimensional agreement space characterized by the following equation: x1 = x2 = … = x (pp 219)
  • A dynamic graph G(t) = (V, E(t)) is a graph in which the set of edges E(t) and the adjacency matrix A(t) are time-varying. Clearly, the set of neighbors Ni(t) of every agent in a dynamic graph is a time-varying set as well. Dynamic graphs are useful for describing the network topology of mobile sensor networks and flocks [19]. (pp 219)
  • GraphLaplacianGradientDescent(pp 220)
  • algebraic connectivity of a graph: The algebraic connectivity (also known as Fiedler value or Fiedler eigenvalue) of a graph G is the second-smallest eigenvalue of the Laplacian matrix of G.[1] This eigenvalue is greater than 0 if and only if G is a connected graph. This is a corollary to the fact that the number of times 0 appears as an eigenvalue in the Laplacian is the number of connected components in the graph. The magnitude of this value reflects how well connected the overall graph is. It has been used in analysing the robustness and synchronizability of networks. (wikipedia) (pp 220)
  • According to Gershgorin theorem [81], all eigenvalues of L in the complex plane are located in a closed disk centered at delta + 0j with a radius of delta, the maximum degree of a graph (pp 220)
    • This is another measure that I can do of the nomad/flock/stampede structures combined with DBSCAN. Each agent knows what agents it is connected with, and we know how many agents there are. Each agent row should just have the number of agents it is connected to.
  • In many scenarios, networked systems can possess a dynamic topology that is time-varying due to node and link failures/creations, packet-loss [40], [98], asynchronous consensus [41], state-dependence [64], formation reconfiguration [53], evolution [96], and flocking [19], [99]. Networked systems with a dynamic topology are commonly known as switching networks. (pp 226)
  • Conclusion: A theoretical framework was provided for analysis of consensus algorithms for networked multi-agent systems with fixed or dynamic topology and directed information flow. The connections between consensus problems and several applications were discussed that include synchronization of coupled oscillators, flocking, formation control, fast consensus in small-world networks, Markov processes and gossip-based algorithms, load balancing in networks, rendezvous in space, distributed sensor fusion in sensor networks, and belief propagation. The role of “cooperation” in distributed coordination of networked autonomous systems was clarified and the effects of lack of cooperation was demonstrated by an example. It was demonstrated that notions such as graph Laplacians, nonnegative stochasticmatrices, and algebraic connectivity of graphs and digraphs play an instrumental role in analysis of consensus algorithms. We proved that algorithms introduced by Jadbabaie et al. and Fax and Murray are identical for graphs with n self-loops and are both special cases of the consensus algorithm of Olfati-Saber and Murray. The notion of Perron matrices was introduced as the discrete-time counterpart of graph Laplacians in consensus protocols. A number of fundamental spectral properties of Perron matrices were proved. This led to a unified framework for expression and analysis of consensus algorithms in both continuous-time and discrete-time. Simulation results for reaching a consensus in small-worlds versus lattice-type nearest-neighbor graphs and cooperative control of multivehicle formations were presented. (pp 231)

Schooling as a strategy for taxis in a noisy environment

Schooling as a strategy for taxis in a noisy environment

Journal: Evolutionary Ecology: Evolutionary Ecology is a conceptually oriented journal of basic biology at the interface of ecology and evolution. The journal publishes original research, reviews and discussion papers dealing with evolutionary ecology, including evolutionary aspects of behavioral and population ecology. The objective is to promote the conceptual, theoretical and empirical development of ecology and evolutionary biology; the scope extends to all organisms and systems. Research papers present the results of empirical and theoretical investigations, testing current theories in evolutionary ecology.

Author: Daniel Grunbaum: My research program seeks to establish quantitative relationships between short-term, small-scale processes, such as individual movement behaviors, and their long-term, large-scale population level effects, such as population fluxes and distributions.


  • A common strategy to overcome this problem is taxis, a behaviour in which an animal performs a biased random walk by changing direction more rapidly when local conditions are getting worse.
    • Consider voters switching from Bush->Obama->Trump
  • Such an animal spends more time moving in right directions than wrong ones, and eventually gets to a favourable area. Taxis is ineffcient, however, when environmental gradients are weak or overlain by `noisy’ small-scale fluctuations. In this paper, I show that schooling behaviour can improve the ability of animals performing taxis to climb gradients, even under conditions when asocial taxis would be ineffective. Schooling is a social behaviour incorporating tendencies to remain close to and align with fellow members of a group. It enhances taxis because the alignment tendency produces tight angular distributions within groups, and dampens the stochastic effects of individual sampling errors. As a result, more school members orient up-gradient than in the comparable asocial case. However, overly strong schooling behaviour makes the school slow in responding to changing gradient directions. This trade-off suggests an optimal level of schooling behaviour for given spatio-temporal scales of environmental variations.
    • This has implications for everything from human social interaction to ANN design.


  • Because limiting resources typically have `patchy’ distributions in which concentrations may vary by orders of magnitude, success or failure in finding favourable areas often has an enormous impact on growth rates and reproductive success. To locate resource concentrations, many aquatic organisms display tactic behaviours, in which they orient with respect to local variations in chemical stimuli or other environmental properties. (pp 503)
  • Here, I propose that schooling behaviours improve the tactic capabilities of school members, and enable them to climb faint and noisy gradients which they would otherwise be unable to follow. (pp 504)
  • Schooling is thought to result from two principal behavioural components: (1) tendencies to move towards neighbours when isolated, and away from them when too close, so that the group retains a characteristic level of compactness; and (2) tendencies to align orientation with those of neighbours, so that nearby animals have similar directions of travel and the group as a whole exhibits a directional polarity. (pp 504)
    • My models indicate that attraction isn’t required, as long as there is a distance-graded awareness. In other words, you align most strongly with those agents that are closest.
  • I focus in this paper on schooling in aquatic animals, and particularly on phytoplankton as a distributed resource. However, although I do not examine them specifically, the modelling approaches and the basic results apply more generally to other environmental properties (such as temperature), to other causes of population movement (such as migration) and to other socially aggregating species which form polarized groups (such as flocks, herds and swarms). (pp 504)
  • Under these circumstances, the search of a nektonic filter-feeder for large-scale concentrations of phytoplankton is analogous to the behaviour of a bacterium performing chemotaxis. The essence of the analogy is that, while higher animals have much more sophisticated sensory and cognitive capacities, the scale at which they sample their environment is too small to identify accurately the true gradient. (pp 505)
    • And, I would contend for determining optimal social interactions in large groups.
  • Bacteria using chemotaxis usually do not directly sense the direction of the gradient. Instead, they perform random walks in which they change direction more often or by a greater amount if conditions are deteriorating than if they are improving (Keller and Segel, 1971; Alt, 1980; Tranquillo, 1990). Thus, on average, individuals spend more time moving in favourable directions than in unfavourable ones. (pp 505)
  • A bacterial analogy has been applied to a variety of behaviours in more complex organisms, such as spatially varying di€usion rates due to foraging behaviours or food-handling in copepods and larval ®sh (Davis et al., 1991), migration patterns in tuna (Mullen, 1989) and restricted area searching in ladybugs (Kareiva and Odell, 1987) and seabirds (Veit et al., 1993, 1995). The analogy provides for these higher animals a quantitative prediction of distribution patterns and abilities to locate resources at large space and time scales, based on measurable characteristics of small-scale movements. (pp 505)
  • I do not consider more sophisticated (and possibly more effective) social tactic algorithms, in which explicit information about the environment at remote points is actively or passively transmitted between individuals, or in which individual algorithms (such as slowing down when in relatively high concentrations) cause the group to function as a single sensing unit (Kils, 1986, described in Pitcher and Parrish, 1993). (pp 506)
    • This is something that could be easily added to the model. There could be a multiplier for each data cell that acts as a velocity scalar of the flock. That should have significant effects! This could also be applied to gradient descent. The flock of Gradient Descent Agents (GDAs) could have a higher speed across the fitness landscape, but slow and change direction when a better value is found by one of the GDAs. It occurs to me that this would work with a step function, as long as the baseline of the flock is sufficiently broad.
  • When the noise predominates (d <= 1), the angular distribution of individuals is nearly uniform, and the up-gradient velocity is near zero. In a range of intermediate values of d(0.3 <= d <= 3), there is measurable but slow movement up-gradient. The question I will address in the next two sections is: Can individuals in this intermediate signal-to-noise range with slow gradient-climbing rates improve their tactic ability by adopting a social behaviour (i.e. schooling)? (pp 508)
  • The key attributes of these models are: (1) a decreasing probability of detection or responsiveness to neighbours at large separation distances; (2) a social response that includes some sort of switch from attractive to repulsive interactions with neighbours, mediated by either separation distance or local density of animals*; and (3) a tendency to align with neighbours (Inagaki et al., 1976; Matuda and Sannomiya, 1980, 1985; Aoki, 1982; Huth and Wissel, 1990, 1992; Warburton and Lazarus, 1991; Grunbaum, 1994). (pp 508)
    • * Though not true of belief behavior (multiple individuals can share the same belief), for a Gradient Descent Agent (GDA), the idea of attraction/repulsion may be important.
  • If the number of neighbours is within an acceptable range, then the individual does not respond to them. On the other hand, if the number is outside that range, the individual turns by a small amount, Δθ3, to the left or right according to whether it has too many or too few of them and which side has more neighbours. In addition, at each time step, each individual randomly chooses one of its visible neighbours and turns by a small amount, Δθ4, towards that neighbour’s heading. (pp 508)
  • The results of simulations based on these rules show that schooling individuals, on average, move more directly in an up-gradient direction than asocial searchers with the same tactic parameters. Figure 4 shows the distribution of individuals in simulations of asocial and social taxis in a periodic domain (i.e. animals crossing the right boundary re-enter the left boundary, etc.). (pp 509)
  • Gradient Schooling
  • As predicted by Equation (5), asocial taxis results in a broad distribution of orientations, with a peak in the up-gradient (positive x-axis) direction but with a large fraction of individuals moving the wrong way at any given time (Fig. 5a,b). By comparison, schooling individuals tend to align with one another, forming a group with a tightened angular distribution. There is stochasticity in the average velocity of both asocial and social searchers (Fig. 5c). On average, however, schooling individuals move up-gradient faster and more directly than asocial ones. These simulation results demonstrate that it is theoretically possible to devise tactic search strategies utilizing social behaviours that are superior to asocial algorithms. That is, one of the advantages of schooling is that, potentially, it allows more successful search strategies under `noisy’ environmental conditions, where variations on the micro-scales at which animals sense their environment obscure the macro-scale gradients between ecologically favourable and unfavourable regions. (pp 510)
  • School-size effects must depend to some extent on the tactic and schooling algorithms, and the choices of parameters. However, underlying social taxis are the statistics of pooling outcomes of independent decisions, so the numerical dependence on school size may operate in a similar manner for many comparable behavioural schemes. For example, it seems reasonable to expect that, in many alternative schooling and tactic algorithms, decisions made collectively by less than 10 individuals would show some improvement over the asocial case but also retain much of the variability. Similarly, in most scenarios, group statistics probably vary only slowly with group size once it reaches sizes of 50-100. (pp 514)
  • when group size becomes large, the behaviour of model schools changes in character. With numerous individuals, stochasticity in the behaviour of each member has a relatively weaker effect on group motion. The behaviour of the group as a whole becomes more consistent and predictable, for longer time periods. (pp 514)
    • I think that this should be true in belief spaces as well. It may be difficult to track one person’s trajectory, but a group in aggregate, particularly a polarized group may be very detectable.
  • An example of group response to changing gradient direction shows that there can be a cost to strong alignment tendency. In this example, the gradient is initially pointed in the negative y-direction (Fig. 9). After an initial period of 5 time units, during which the gradient orients perpendicularly to the x-axis, the gradient reverts to the usual x-direction orientation. The school must then adjust to its new surroundings by shifting to climb the new gradient. This example shows that alignment works against course adjustment: the stronger the tendency to align, the slower is the group’s reorientation to the new gradient direction. This is apparently due to a non-linear interaction between alignment and taxis: asymmetries in the angular distribution during the transition create a net alignment flux away from the gradient direction. Thus, individuals that pay too much attention to neighbours, and allow alignment to overwhelm their tactic tendencies, may travel rapidly and persistently in the wrong direction. (pp 516)
    • So, if alignment (and velocity matching) are strong enough, the conditions for a stampede (group behavior with negative outcomes – in this case, less food) emerge
  • The models also suggest that there is a trade-off in strengthening tendencies to align with neighbours: strong alignment produces tight angular distributions, but increases the time needed to adjust course when the direction of the gradient changes. A reasonable balance seems to be achieved when individuals take roughly the same time to coalesce into a polarized group as they do to orient to the gradient in asocial taxis. (pp 518)
    • There is something about the relationship between explore and exploit in this statement that I really need to think about.
  • Social taxis is potentially effective in animals whose resources vary substantially over large length scales and for whom movements over these scales are possible. (pp 518)
    • Surviving as a social animal requires staying in the group. Since belief can cover wide ranges (e.g. religion), does there need to be a mechanism where individuals can harmonize their beliefs? From Social Norms and Other Minds The Evolutionary Roots of Higher Cognition :  Field research on primate societies in the wild and in captivity clearly shows that the capacity for (at least) implicit appreciation of permission, prohibition, and obligation social norms is directly related to survival rates and reproductive success. Without at least a rudimentary capacity to recognize and respond appropriately to these structures, remaining within a social group characterized by a dominance hierarchy would be all but impossible.
  • Interestingly, krill have been reported to school until a food patch has been discovered, whereupon they disperse to feed, consistent with a searching function for schooling. The apparent effectiveness of schooling as a strategy for taxis suggests that these schooling animals may be better able to climb obscure large-scale gradients than they would were they asocial. Interactive effects of taxis and sociality may affect the evolutionary value of larger groups both directly, by improving foraging ability with group size, and indirectly, by constraining alignment rates. (pp 518)
  • An example where sociality directly affects foraging strategy is forage area copying, in which unsuccessful fish move to the vicinity of neighbours that are observed to be foraging successfully (Pitcher et al., 1982; Ranta and Kaitala, 1991; Pitcher and Parrish, 1993). Pitcher and House (1987) interpreted area copying in goldfish as the result of a two-stage decision process: (1) a decision to stay put or move depending on whether feeding rate is high or low; and (2) a decision to join neighbours or not based upon whether or not further solitary searching is successful. Similar group dynamics have been observed in foraging seabirds (Porter and Seally, 1982; Haney et al., 1992).
  • Synchrokinesis depends upon the school having a relatively large spatial extent: part of a migrating school encounters an especially favourable or unfavourable area. The response of that section of the school is propagated throughout the school by alignment and grouping behaviours, with the result that the school as a whole is more effective at route-finding than isolated individuals. Forage area copying and synchrokinesis are distinct from social taxis in that an individual discovers and reacts to an environmental feature or resource, and fellow group members exploit that discovery. In social taxis, no individual need ever have greater knowledge about the environment than any other — social taxis is essentially bound up in the statistics of pooling the outcomes of many unreliable decisions. Synchrokinesis and social taxis are complementary mechanisms and may be expected to co-occur in migrating and gradient-climbing schools. (pp 519)
  • For example, in the comparisons of taxis among groups of various sizes, the most successful individuals were in the asocial simulation, even though as a fraction of the entire population they were vanishingly small. (pp 519)
    • Explorers have the highest payoff for the highest risks

Speaker–listener neural coupling underlies successful communication

Speaker–listener neural coupling underlies successful communication (2010)

Greg J. Stephens

Lauren J. Silbert

Uri Hasson (HassonLab at Princeton)


  • Verbal communication is a joint activity; however, speech production and comprehension have primarily been analyzed as independent processes within the boundaries of individual brains. Here, we applied fMRI to record brain activity from both speakers and listeners during natural verbal communication. We used the speaker’s spatiotemporal brain activity to model listeners’ brain activity and found that the speaker’s activity is spatially and temporally coupled with the listener’s activity. This coupling vanishes when participants fail to communicate. Moreover, though on average the listener’s brain activity mirrors the speaker’s activity with a delay, we also find areas that exhibit predictive anticipatory responses. We connected the extent of neural coupling to a quantitative measure of story comprehension and find that the greater the anticipatory speaker–listener coupling, the greater the understanding. We argue that the observed alignment of production- and comprehension-based processes serves as a mechanism by which brains convey information.
    • This seems to be the root article for neural coupling. It seems to be an area of vigorous study, with lots of work coming out from the three authors.
    • The study design is also really good.


  • In this study we directly examine the spatial and temporal coupling between production and comprehension across brains during natural verbal communication. (pp 14425)
  • Using fMRI, we recorded the brain activity of a speaker telling an unrehearsed real-life story and the brain activity … (n = 11) of a listener listening to the recorded audio of the spoken story, thereby capturing the time-locked neural dynamics from both sides of the communication. Finally, we used a detailed questionnaire to assess the level of comprehension of each listener. (pp 14425)
  • …because communication unfolds over time, this coupling will exhibit important temporal structure. In particular, because the speaker’s production-based processes mostly precede the listener’s comprehension-based processes, the listener’s neural dynamics will mirror the speaker’s neural dynamics with some delay. Conversely, when listeners use their production system to emulate and predict the speaker’s utterances, we expect the opposite: the listener’s dynamics will precede the speaker’s dynamics. (pp 14425)
  • To analyze the direct interaction of production and comprehension mechanisms, we considered only spatially local models that measure the degree of speaker–listener coupling within the same Talairach location. (pp 14426)
  • we also observed significant speaker–listener coupling in a collection of extralinguistic areas known to be involved in the processing of semantic and social aspects of the story (19), including the precuneus, dorsolateral prefrontal cortex, orbitofrontal cortex, striatum, and medial prefrontal cortex. (pp 14426)
  • In agreement with previous work, the story evoked highly reliable activity inmany brain areas across all listeners (8, 11, 12) (Fig. 2B, yellow). We note that the agreement with previous work is far from assured: the story here was both personal and spontaneous, and was recorded in the noisy environment of the scanner. The similarity in the response patterns across all listeners underscores a strong tendency to process incoming verbal information in similar ways. A comparison between the speaker–listener and the listener–listenermaps reveals an extensive overlap (Fig. 2B, orange). These areas include many of the sensory related, classic linguistic-related and extralinguistic-related brain areas, demonstrating that many of the areas involved in speech comprehension (listener–listener coupling) are also aligned during communication (speaker–listener coupling). (pp 14426)
  • To test whether the extensive speaker–listener coupling emerges only when information is transferred across interlocutors, we blocked the communication between speaker and listener. We repeated the experiment while recording a Russian speaker telling a story in the scanner, and then played the story to non–Russian speaking listeners (n = 11). In this experimental setup, although the Russian speaker is trying to communicate information, the listeners are unable to extract the information from the incoming acoustic sounds. Using identical analysis methods and statistical thresholds, we found no significant coupling between the speaker and the listeners or among the listeners. At significantly lower thresholds we found that the non–Russian-speaking listener–listener coupling was confined to early auditory cortices. This indicates that the reliable activity in most areas, besides early auditory cortex, depends on a successful processing of the incoming information, and is not driven by the low-level acoustic aspects of the stimuli. (pp 14426)
  • Neural Coupling
    • In my model, the anticipation is modeled by the alignment and velocity, but others come to similar conclusions. It may be a way of dealing with noisy environments. Which would be another way of saying group dynamics with incomplete information.
  • Our analysis also identifies a subset of brain regions in which the activity in the listener’s brain precedes the activity in the speaker’s brain. The listener’s anticipatory responses were localized to areas known to be involved in predictions and value representation (pp 14428)
  • Such findings are in agreement with the theory of interactive linguistic alignment (1). According to this theory, production and comprehension become tightly aligned on many different levels during verbal communication, including the phonetic, phonological, lexical, syntactic, and semantic representationsAccordingly, we observed neural coupling during communication at many different processing levels, including low-level auditory areas (induced by the shared input), production-based areas (e.g., Broca’s area), comprehension based areas (e.g., Wernicke’s area and TPJ), and high-order extralinguistic areas (e.g., precuneus and mPFC) that can induce shared contextual model of the situation(34). Interestingly, some of these extralinguistic areas are known to be involved in processing social information crucial for successful communication, including, among others, the capacity to discern the beliefs, desires, and goals of others.(pp 14429)



The Group Polarization Phenomenon

The Group Polarization Phenomenon

David G. Myers

Helmut Lamm

Experiments exploring the effects of group discussion on attitudes, jury decisions, ethical decisions, judgments, person perceptions, negotiations, and risk taking (other than the choice-dilemmas task) are generally consistent with a “group polarization” hypothesis, derived from the risky-shift literature. Recent attempts to explain the phenomenon fall mostly into one of three theoretical approaches: (a) group decision rules, especially majority rule (which is contradicted by available data), (b) interpersonal comparisons (for which there is mixed support), and (c) informational influence (for which there is strong support). A conceptual scheme is presented which integrates the latter two viewpoints and suggests how attitudes develop in a social context.

  • Pictures may be important as part of an argument. Need to be able to support that.
  • This polarization concept should also be distinguished from a related concept, extremization. Whereas polarization refers to shifts toward the already preferred pole, extremization has been used to refer to movement away from neutrality, regardless of direction. Since all instances of group polarization are instances of extremization, but not vice versa, extremization may be easier to demonstrate than polarization. (pp 603)
  • For convenience we have organized these studies into seven categories: attitudes, jury decisions, ethical decisions, judgments, person perceptions, negotiation behavior, and risk measures other than the choice dilemmas. This categorization is admittedly somewhat arbitrary. (pp 604)
  • In other studies, however, it is possible to infer the direction of initial preferences. Robinson (1941) conducted lengthy discussions of two attitudes. On attitude toward war, where students were initially quite pacifistic, there was a nonsignificant shift to even more pacifism following discussion. On attitude toward capital punishment, to which students were initially opposed, there was a significant shift to even stronger opposition. (pp 604)
  • Varying the stimulus materials. Myers and Kaplan (1976) engaged their subjects in discussion of stimulus materials which elicited a dominant predisposition of guilty or not guilty. After discussing traffic cases in which the defendants were made to appear as low in guilt, the Subjects Were even more definite in their judgments of innocence and more lenient in recommended punishment. After discussing “high-guilt” cases, the subjects polarized toward harsher judgments of guilt and punishment. (pp 605)
  • Group composition studies. Vidmar composed groups of jurors high or low in dogmatism. The high-dogmatism juries shifted toward harsher sentences following discussion, and the low-dogmatism groups shifted toward more lenient sentences, despite the fact that the high- and low-dogmatism juries did not differ in their predeliberation judgments. (pp 606)
  • Main and Walker (1973) observed that these constitutionality decisions were also more libertarian in the group condition (65% versus 45%). Although a minority of the single-judge decisions were prolibertarian, Walker and Main surmised that the preexisting private values of the judges were actually prolibertarian and that their decisions made alone were compromised in the face of antilibertarian public pressure. Their private values were then supposedly released and reinforced in the professional group context (pp 606)
  • From what we have been able to perceive thus far, the process is an interesting combination of rational persuasion, sheer social pressure, and the psychological mechanism by which individual perceptions undergo change when exposed to group discussion (pp 606)
  • Myers (1975) also used a faculty evaluation task. The subjects responded to 200 word descriptions of “good” or “bad” faculty with a scale judgment and by distributing a pay increase budget among the hypothetical faculty. As predicted by the group polarization hypothesis, good faculty were rated and paid even more favorably after the group interaction, and contrariwise for the bad faculty. (pp 608)
  • in general, the work on person perception supports the group polarization hypothesis, especially when the stimulus materials are more complex than just a single adjective. (pp 608)
  • Myers and Bach (1976) compared the conflict behavior of individuals and groups, using an expanded prisoner’s dilemma matrix cast in the language of a gas war. There was no difference in their conflict behavior (both individuals and groups were highly noncooperative). But on postexperimental scales assessing the subjects’ evaluations of themselves and their opponents, individuals tended to justify their own behavior, and groups were even more inclined toward self-justification. This demonstration of group polarization supports Janis’s (1972) contention that in situations of intergroup conflict, group members are likely to develop a strengthened belief in the inherent morality of their actions.  (pp 608)
  • Skewness cannot account for group polarization. This is particularly relevant to the majority rule scheme, which depends on a skewed distribution of initial choices. On choice dilemmas, positively skewed distributions (i.e., with a risky majority) should produce risky shift, and negatively skewed distributions should yield a conservative shift. Several findings refute this prediction. (pp 612)
  • Shifts in the group median, although slightly attenuated, are not significantly smaller than shifts in the group mean (pp 612)
  • Group shift has also been shown to occur in dyads (although somewhat reduced), where obviously there can be no skewness in the initial responses (pp 612)
  • while group decision models may be useful in other situations in which discussion is minimal or absent and the task is to reach agreement (e.g., Lambert, 1969), the models (or at least the majority rule model stressed in this analysis) are not a sufficient explanation of the group polarization findings we are seeking to explain. There are still a variety of other decision schemes that can be explored and with other specific tasks. But clearly, group induced shift on choice dilemmas is something more than a statistical artifact. (pp 612)
  • Interpersonal Comparisons theory suggests that a subject changes when he discovers that others share his inclinations more than he would have supposed, either because the group norm is discovered to be more in the preferred direction than previously imagined or because the subject is released to more strongly act out his preference after observing someone else who models it more extremely than himself. This theory, taken by itself, suggests that relevant new information which emerges during the discussion is of no consequence. Group polarization is a source effect, not a message effect. (pp 614)
    • This is very close to the flocking theory where one agent looks at the alignment and velocity of nearby agents.
  • Differences between self, presumed other, and ideal scores. One well-known and widely substantiated assumption of the interpersonal comparisons approach is the observation from choice-dilemmas research that if, after responding, the subjects go back over the items and guess how their average peer would respond and then go back over the items a third time and indicate what response they would actually admire most, they tend to estimate the group norm as more neutral than their own initial response and their ideal as more extreme (pp 613)
  • Lamm et al. (1972) have also shown that not only do subjects indicate their ideal as more extreme than their actual response, but they also suspect that the same is true of their peers. The tendency of people to perceive themselves as more in what they consider to be the socially desirable direction than their average peer extends beyond the choice dilemmas (see Codol, Note 13). For example, most businessmen believe themselves to be more ethical than the average businessman (Baumhart, 1968), and there is evidence that people perceive their own views as less prejudiced than the norm of their community (Lenihan, Note 14). (pp 613)
  • The tendency to perceive others as “behind” oneself exists only when the self response is made prior to estimating the group norm (McCauley, Kogan, & Teger, 1971; Myers, 1974). Evidently it is after one has decided for himself that there is then a tendency to consider one’s action as relatively admirable (by perceiving the average person as less admirable than oneself). (pp 613)
  • it has been reliably demonstrated that subjects perceive other persons who have responded more extremely than themselves (in the direction of their ideal) as more socially desirable than persons who have not (Baron, Monson, & Baron, 1973; Jellison & Davis, 1973; Jellison & Riskind, 1970, 1971; Madaras & Bern, 1968). A parallel finding exists in the attitude literature (Eisinger & Mills, 1968): An extreme communicator on one’s side of an issue tends to be perceived as more sincere and competent than a moderate. (pp 614)
  • Burnstein, Vinokur, and Pichevin (1974) took an informational influence viewpoint and showed that people who adopt extreme choices are presumed to possess cogent arguments and are then presumably admired for their ability. They also demonstrated that subjects have much less confidence in others’ choices than in their own, suggesting that the tendency to perceive others as more neutral than oneself simply reflects ignorance about others’ choices (pp 614)
  • self-ideal difference scores are less affected by order of measurement than self versus perceived other differences (Myers, 1974)—suggest that the self-ideal discrepancy may be the more crucial element of a viable interpersonal comparisons approach. (pp 614)
  • One set of studies has manipulated the information about others’ responses by providing fake norms. More than a dozen separate studies all show that subjects will move toward the manipulated norm (see Myers, 1973) (pp 615)
    • Can’t find this paper, but herding!
  • Consistent with this idea, they observed that exposure to others’ choices produced shift only when subjects then wrote arguments on the item. If knowledge of others’ choices was denied or if an opportunity to rethink the item was denied, no shift occurred. (pp 615)
  • On the other hand, it may be reasoned that in each of the studies producing minimal or nonexistent shift after exposure to others’ attitudes, the subjects were first induced to bind themselves publicly to a pretest choice and then simply exposed to others’ choices. It takes only a quick recall of some classic conformity studies (e.g., Asch, 1956) to realize that this was an excellent procedure for inhibiting response change. (pp 615)
  • Bishop and Myers (1974) have formulated mathematical models of the presumed informational influence mechanisms. These models assume that the amount of group shift will be determined by three factors: the direction of each argument (which alternative it favors), the persuasiveness of each argument, and the originality of each argument (the extent to which it is not already known by the group members before discussion). In discussion, the potency of an argument will be zero if either the rated persuasiveness is zero (it is trivial or irrelevant) or if all group members considered the argument before discussion (pp 616)
  • the simple direction of arguments is such an excellent predictor of shift (without considering persuasiveness and originality), it is not easy to demonstrate the superiority of the models over a simple analysis of argument direction as undertaken by Ebbesen and Bowers (1974). (pp 617)
    • This supports the notion that alignment and heading, as used in the model may really be sufficient to model polarizing behavior
  • A group that is fairly polarized on a particular item before discussion is presumably already in general possession of those arguments which polarize a group. A less extreme group has more to gain from the expression of partially shared persuasive arguments. (pp 617)
  • Passive receipt of arguments outside an interactive discussion context generally produces reduced shift (e.g., Bishop & Myers, 1974; Burnstein & Vinokur, 1973; St. Jean, 1970; St. Jean & Percival, 1974). Likewise, listening to a group discussion generally elicits less shift than actual participation (pp 617)
    • There may be implications here with respect to what’s being seen and read on the news having a lower influence than items that are being discussed on social media. A good questions is at what point does the reception of information feel ‘interactive’? Is clicking ‘like enough? My guess is that it is.
  • Verbal commitment could produce the increased sense of involvement and certainty that Moscovici and Zavolloni (1969) believe to be inherent in group polarization. (pp 618)
    • This reinforces the point above, but we need to know what the minimum threshold of what can be considered ‘verbal commitment’.
  • By offering arguments that tend toward the outer limits of his range of acceptability, the individual tests his ideals and also presents himself favorably to the group since, as we noted earlier, extremity in the direction of the ideal connotes knowledgeability and competence. (pp 618)
  • Diagram (pp 619) PolarazationDiagram
  • Arguments spoken in discussion more decisively favor the dominant alternative than do written arguments. The tendency for discussion arguments to be one-sided is probably not equal for all phases of a given discussion. Studies in speech-communications (see Fisher, 1974) suggest that one-sided discussion is especially likely after a choice direction has implicitly emerged and group members mutually reinforce their shared inclination. (pp 619)
    • This review is pre IRC, and views writing as non-interactive. THis may not be true any more.
  • The strength of the various vectors is expected to vary across situations. In more fact-oriented judgment tasks (group problem solving tasks being the extreme case), the cognitive determinants will likely be paramount, although people will still be motivated to demonstrate their abilities. On matters of social preference, in which the social desirability of actions is more evident, the direct and indirect attitudinal effects of social motivation are likely to appear. The direct impact will occur in situations in which the individual has ideals that may be compromised by presumed norms but in which exposure to others’ positions informs him that his ideals are shared more strongly or widely than he would have supposed. These situations—in which expressed ideals are a step ahead of prior responses—will also tend to elicit discussion content that is biased toward the ideals. (pp 620)
  • What is the extent of small group influence on attitudes? McGuire (1969) noted, “It is clear that any impact that the mass media have on opinion is less than that produced by informal face-to-face communication of the person with his primary groups, his family, friends, co-workers, and neighbors (p. 231,).” (pp 220)



Sick echo chambers

Over the past year, I’ve been building a model that lets me look at how opinions evolve in belief space, much in the manner that flocks, herds and schools emerge in the wild.


Recently, I was Listening to BBC Business Daily this morning on Facebook vs Democracy:

  • Presenter Ed Butler hears a range of voices raising concern about the existential threat that social media could pose to democracy, including Ukrainian government official Dmytro Shymkiv, journalist Berit Anderson, tech investor Roger McNamee and internet pioneer Larry Smarr.

Roger McNamee and Larry Smarr in particular note how social media can be used to increase polarization based on emergent poles. In other words, “normal” opposing views can be amplified by attentive bad actors [page 24] with an eye towards causing generalized societal disruption.

My model explores emergent group interactions and I wondered if this adversarial herding in information space as it might work in my model.

These are the rough rules I started with:

  • Herders can teleport, since they are not emotionally invested in their belief space position and orientation
  • Herders appear like multiple individuals that may seem close and trustworthy, but they are actually a distant monolithic entity that is aware of a much larger belief space.
  • Herders amplify arbitrary pre-existing positions. The insight is that they are not herding in a direction, but to increase polarization
  • To add this to the model, I needed to do the following:
    • Make the size of the agent a function of the weight so we can see what’s going on
    • When in ‘herding mode’ the overall heading of the population is calculated, and the agent that is closest to that heading is selected to be amplified by our trolls/bot army.
    • The weight is increased to X, and the radius is increased to Y.
      • X represents AMPLIFICATION BY trolls, bots, etc.
      • A large Y means that the bots can swamp other, normally closer signals. This models the effect of a monolithic entity controlling thousands of bots across the belief space

Here’s a screenshot of the running simulation. There is an additional set of controls at the upper left that allow herding to be enables, and the weight of the influence to be set. In this case, the herding weight is 10. Though the screenshot shows one large agent shape, the amplified shape flits from agent to agent, always keeping closest to the average heading.


The results are kind of scary. If I set the weight of the herder to 15, I can change the change the flocking behavior of the default to echo chamber.

  • Normal: No Herding
  • Herding weight set to 15, other options the same: HerdingWeight15

I did some additional tweaking to see if having highly-weighted herders ignore each other (they would be coordinated through C&C) would have any effect. It doesn’t. There is enough interaction through the regular populations to keep the alignment space reduced.

It looks like there is a ‘sick echo chamber’ pattern. If the borders are reflective, and the herding weight + influence radius is great enough, then a wall-hugging pattern will emerge.

The influence weight is sort of a credibility score. An agent that has a lot of followers, or says a lot of the things that I agree with has a lot of influence weight The range weight is reach.

Since a troll farm or botnet can be regarded as a single organization,  interacting with any one of the agents is really interacting with the root entity.  So a herding agent has high influence and high reach. The high reach explains the border hugging behavior.

It’s like there’s someone at the back of the stampede yelling YOUR’E GOING THE RIGHT WAY! KEEP AT IT! And they never go off the cliff because they are a swarm Or, it never goes of the cliff, because it manifests as a swarm.

A loud, distributed voice pointing in a bad direction means wall hugging. Note that there is some kind of floating point error that lets wall huggers creep off the edge.Edgecrawling

With a respawn border, we get the situation where the overall heading of the flock doesn’t change even as it gets destroyed as it goes over the border. Again, since the herding algorithm is looking at the overall population, it never crosses the border but influences all the respawned agents to head towards the same edge: DirectionPreserving

Who’d have thought that there could be something worse than runaway polarization?


Suppressing the Search Engine Manipulation Effect (SEME)

Suppressing the Search Engine Manipulation Effect (SEME)

  • Authors
    • Robert Epstein, (American Institute for Behavioral Research and Technology) Epstein and Robertson have found in multiple studies that search rankings that favor a political candidate drive the votes of undecided voters toward that candidate, an effect they call SEME (“seem”), the Search Engine Manipulation Effect.
    • Ronald Robertson (Northeastern University) I design experiments and technologies to explore the ways in which online platforms can influence the attitudes, beliefs, and behavior of individuals and groups. Currently, I am a PhD student in the world’s first Network Science PhD program at Northeastern University and am advised by Christo Wilson and David Lazer.
    • David Lazer (Northeastern University) professor of political science and computer and information science and the co-director of the NULab for Texts, Maps, and Networks
    • Christo Wilson (Northeastern University) Assistant Professor in the College of Computer and Information Science atNortheastern University. I am a member of the Cybersecurity and Privacy Institute and the Director of the BS in Cybersecurity Program in the College.


  • Abstract: A recent series of experiments demonstrated that introducing ranking bias to election-related search engine results can have a strong and undetectable influence on the preferences of undecided voters. This phenomenon, called the Search Engine Manipulation Effect (SEME), exerts influence largely through order effects that are enhanced in a digital context. We present data from three new experiments involving 3,600 subjects in 39 countries in which we replicate SEME and test design interventions for suppressing the effect. In the replication, voting preferences shifted by 39.0%, a number almost identical to the shift found in a previously published experiment (37.1%). Alerting users to the ranking bias reduced the shift to 22.1%, and more detailed alerts reduced it to 13.8%. Users’ browsing behaviors were also significantly altered by the alerts, with more clicks and time going to lower-ranked search results. Although bias alerts were effective in suppressing SEME, we found that SEME could be completely eliminated only by alternating search results – in effect, with an equal-time rule. We propose a browser extension capable of deploying bias alerts in real-time and speculate that SEME might be impacting a wide range of decision-making, not just voting, in which case search engines might need to be strictly regulated.
  • Introduction
    • Recent research has shown that society’s growing dependence on ranking algorithms leaves our psychological heuristics and vulnerabilities susceptible to their influence on an unprecedented scale and in unexpected ways
    • Experiments conducted on Facebook’s Newsfeed have demonstrated that subtle ranking manipulations can influence the emotional language people use
    • Similarly, experiments on web search have shown that manipulating election-related search engine rankings can shift the voting preferences of undecided voters by 20% or more after a single search
    • While “bias” can be ambiguous, our focus is on the ranking bias recently quantified by Kulshrestha et al. with Twitter rankings
    • Our results provide support for the robustness of SEME and create a foundation for future efforts to mitigate ranking bias. More broadly, our work adds to the growing literature that provides an empirical basis to calls for algorithm accountability and transparency [24, 25, 90, 91] and contributes a quantitative approach that complements the qualitative literature on designing interventions for ranking algorithms
    • Our results also suggest that proactive strategies that prevent ranking bias (e.g., alternating rankings) are more effective than reactive strategies that suppress the effect through design interventions like bias alerts. Given the accumulating evidence, we speculate that SEME may be impacting a wide range of decision-making, not just voting
  • Related Work
    • Order effects are among the strongest and most reliable effects ever discovered in the psychological sciences [29, 88]. These effects favorably affect the recall and evaluation of items at the beginning of a list (primacy) and at the end of a list (recency).
      • There does not seem to be an equivalent primacy effect in maps that I can find
    • online systems can: (1) provide a platform for constant, large-scale, rapid experimentation, (2) tailor their persuasive strategies by mining detailed demographic and behavioral profiles of users [1, 6, 9, 18, 121], and (3) provide users with a sense of control over the system that enhances their susceptibility to influence
      • Is this flocking from the flock’s perspective? Sort of an Ur-flock?
      • This is that Trust/Awareness equation again
    • A recent report involving 33,000 people found that search engines were the most trusted source of news, with 64% of people reporting that they trust search engines, compared to 57% for traditional media, 51% for online media, and 41% for social media [10]. Similarly, a 2012 survey by Pew found that 73% of search engine users report that “all or most of the information they find is accurate and trustworthy,” and 66% report that “search engines are a fair and unbiased source of information” [105].
    • Suggestions for fostering resistance can be broken down into two primary strategies: (1) providing forewarnings [43, 49] and (2) training and motivating people to resist [79, 120].
      • Interesting that alternate, non-ordered design approaches aren’t even mentioned
    • Part of the reason that forewarnings work is explained by psychological reactance theory [12], which posits that when people believe their intellectual freedom is threatened – by exposing an attempt to persuade, for example – they react in the direction opposite that of the intended one
    • In the context of online media bias, researchers have primarily explored methods for curbing the effects of algorithmic filtering and selective exposure [87, 96] rather than ranking bias [71]. In this vein, researchers have developed services that encourage users to explore multiple perspectives [97, 98] and browser extensions that gamify and encourage balanced political news consumption [19, 20, 86]. However, these solutions are somewhat impractical because they require users to adopt new services or exert additional effort.
  • Methods – Experiment Design
    • To construct biased search rankings we asked four independent raters to provide bias ratings of the webpages we collected on an 11-point Likert scale ranging from -5 “favors Cameron” to +5 “favors Miliband”. We then selected the 15 webpages that most strongly favored Cameron and the 15 that most strongly favored Miliband to create three bias groups
    • The query in the search engine was fixed as “UK Politics ‘David Cameron’ OR ‘Ed Miliband’”, and subjects could not reformulate it.
    • On top of assignment to a bias group, subjects were randomly assigned to one of three alert experiments.We drew from the literature on decision-making and design intervention to implement so-called debiasing strategies for improving decision-making in the presence of biased information [39, 78, 82]. Specifically, we constructed and placed alerts in the search results produced by our mock search engine that provided forewarnings with salient graphics, autonomony-supportive language, and details on the persuasive threat
  • Methods – Procedure
    • After providing informed consent and answering basic demographic questions
      • Do this and use this phrase!
    • Subjects then rated the two candidates on 10-point Likert scales with respect to their overall impression of each candidate, how much they trusted each candidate, and how much they liked each candidate. Subjects also indicated their likelihood of voting for one candidate or the other on an 11-point Likert scale where the candidates’ names appeared at opposite ends of the scale and 0 indicated no preference, as well as on a binary choice question where subjects indicated who they would vote for if the election were held today.
      • This is a good way to set up the game. People read the dilemma, formulate an initial solution and their level of commitment to it. They can choose to make it “public” as their first statement or to keep it private and display a “no opinion” initial statement
    • We asked: “While you were doing your online research on the candidates, did you notice anything about the search results that bothered you in any way?” and prompted subjects to explain what had bothered them in a free response format: “If you answered “yes,” please tell us what bothered you.” We did not directly ask subjects whether they had “noticed bias” to avoid the inflation of false positive rates that leading questions can cause
  • Methods – Participants
    • We recruited 3,883 subjects between April 28, 2015 and May 6, 2015 on Amazon’s Mechanical Turk (AMT;, a subject pool frequently used by behavioral, economic, and social science researchers [8, 13, 102]. We excluded from our analysis subjects who reported an English fluency level of 5 or less (on a scale of 1 to 10) (n=26)
      • MTurk would be a good source of participants as well
  • Analysys – Search metrics
    • Utilizing Kolmogorov-Smirnov (K-S) tests of differences in distributions, we found significant differences in the patterns of time spent on the 30 webpages between subjects in the no alert experiment (correlation with ranking: Spearman’s ρ = -0.836, P <0.001) and the high alert experiment (ρ = -0.654, P <0.001) (K-S D = 0.467, P <0.01), and between subjects in the low alert experiment (ρ = -0.719, P <0.001) and the high alert experiment (K-S D = 0.400, P <0.01)
      • A way of looking for explore/exploit populations? And how fast can it be determined? Google uses a mechanism to stop an experiment once a confidence level is reached. Also, bootstrap would be good here
    • Similarly, we also found significant differences in the patterns of clicks that subjects made on the 30 webpages between subjects in the no alert experiment (ρ = -0.865, P <0.001) and the high alert experiment (ρ = -0.795, P <0.001) (K-S D = 0.500, P <0.001), and between subjects in the low alert experiment (ρ = -0.876, P <0.001) and the high alert experiment (K-S D = 0.367, P <0.05)
    • Among all conditions,we found that differences in the patterns of time and clicks on the individual rankings primarily emerged on the first SERP, but less so on the second, fourth, and fifth SERPs
  • Analysys – Attitude Shifts
    • we found that the mean shifts in candidate ratings for the bias groups significantly converged on the mean shift found in the neutral group as the level of detail in the alerts increased, with high alerts creating higher convergence than low alerts
      • As more diverse information is injected, populations compromise
  • Analysys – Vote Shifts
    • Vote Manipulation Power (VMP)is the percent change in the number of subjects, in the two bias groups combined, who indicated that they would vote for the candidate who was favored by their search rankings. That is, if x and x ′ subjects in the bias groups said they would vote for the favored candidate before and after conducting the search, respectively, then VMP = (x ′ − x)/x.
      • This could also be applied to the game to watch how votes for an outcome change over time. In the case of the game, new candidates can come into existence, so we need to watch for that.
  • Analysys – Bias Awareness
    • We found 8.1% of subjects that showed awareness of the bias in the no alert experiment, a figure identical to the 8.1% awareness rate found by Eslami et al. in their audit of [37], and similar to the 8.6% of subjects who showed awareness in the original study [30]. The percentage of subjects showing bias awareness increased to 21.5% in the low alert experiment, and 23.4% in the high alert experiment.
  • Discussion
    • However, despite the additional suppression of the high alerts, the lowest VMP was found among the neutral group subjects: rankings alternating between favoring the two candidates prevented SEME.
      • This configuration forces users to “explore” more, within the context of a list affordance.
    • As with previous research on SEME [30], and with research on attitude change and influence more generally [3, 72, 120], we found that subjects vary in their susceptibility to SEME, as well as in their responsiveness to the alerts, based on their personal characteristics (Figure 6 and Figure 7 in the Appendix).
      • Explorer and exploiter populations?
    • As more people turn to the internet for political news [85, 115], designing systems that can monitor and suppress the effects of algorithm biases, like ranking bias, will play an increasingly important role in protecting the public’s psychological vulnerabilities.
      • And one of the big issues is finding bias at scale with domain independence
    • Real-time automated bias detection could potentially be achieved by utilizing a Natural Language Processing (NLP) approach. One could utilize opinions [75], sentiment [99], linguistic patterns [109], word associations [14], or recursive neural networks [59] with human-coded data to classify biased language.
      • Scale and domain problems.
  • Discussion – Awareness of bias
    • Awareness of ranking bias appears to suppress SEME only when it occurs in conjunction with a bias alert, perhaps because an alert is a kind of warning–inherently negative in nature.
      • According to Moscovici, an inherently negative construct should reduce polarization movement.
    • Awareness of ranking bias in the absence of bias alerts might increase VMP because people perceive the bias as a kind of social proof [111, 112], made all the more powerful because of the disproportionate trust people have in search rankings [10, 95, 105]. The user’s interpretation might be, “This candidate MUST be good, because even the search results say so.”

Some thoughts about awareness and trust

I had some more thoughts about how behavior patterns emerge from the interplay between trust and awareness. I think the following may be true:

  1. Awareness refers to how complete the knowledge of an information domain is. Completely aware indicates complete information. Unaware indicates not only absent information but no knowledge of the domain at all.
  2. Trust is a social construct to deal with incomplete information. It’s a shortcut that essentially states “based on some set of past experiences, I will assume that this (now trusted) entity will behave in a predictable, reliable, and beneficial way for me”
  3. Healthy behaviors emerge when trust and awareness are equivalent.
  4. Low trust and low awareness is reasonable. It’s like walking through a dark, unknown space. You go slow, bump into things, and adjust.
  5. Low trust and high awareness is paralytic.
  6. High trust and low awareness is reckless. Runaway conditions like echo chambers. The quandary here is that high trust is efficient. Consider the prisoner’s dilemma:
      1. dilemma
      2. In the normal case, the two criminals have to evaluate what the best action is based on all the actions the other individual could choose, ideally resulting in a Nash Equilibrium. For two players (p), there are 4 choices (c). However, if each player believes that the other player will make the same choice, then only the two diagonal choices remain. For two players, this reduces the complexity by half. But for multiple dissimilar players, the options go up by cp, so that if this were The Usual Suspects, there would be 32 possibilities to be worked out by each player. But for 5 identical prisoners, the number of choices remains 2, which is basically “what should we all do?”. The more we believe that the others in our social group see the world the same way, the less work we all have to do.
  7. Diversity is a mechanism for extending awareness, but it depends on trusting those who are different. That may be the essence of the explore/exploit dilemma.
  8. Attention is a form of focused awareness, can reduce general awareness. This is one to the reasons that Tufekci’s thoughts on the attention economy matter so much. As technology increases attention on proportionally more “marketable” items, the population’s social awareness is distorted.
  9. In a healthy group context, trust falls off as a function of awareness. That’s why we get flocking. That is the pattern that emerges when you trust more those who are close, while they in turn do the same, building a web of interaction. It’s kind of like interacting ripples?
  10. This may work for any collection of entities that have varied states that undergo change in some predictable way. If they were completely random, then awareness of the state is impossible, and trust should be zero.
    1. Human agent trust chains might proceed from self to family to friends to community, etc.
    2. Machine agent trust chains might proceed from self to direct connections (thumb drives, etc) to LAN/WAN to WAN
    3. Genetic agent trust chain is short – self to species. Contact is only for reproduction. Interaction would reflect the very long sampling times.
    4. Note that (1) is evolved and is based on incremental and repeated interactions, while (2) is designed and is based on arbitrary rules that can change rapidly. Genetics are maybe dealing with different incentives? The only issue is persisting and spreading (which helps in the persisting)
  11. Computer-mediated-communication disturbs this process (as does probably every form of mass communication) because the trust in the system is applied to the trust of the content. This can work in both ways. For example, lowering trust in the press allows for claims of Fake News. Raising the trust of social networks that channel anonymous online sources allows for conspiracy thinking.
  12. An emerging risk is how this affects artificial intelligence, given that currently high trust in the algorithms and training sets is assumed by the builders
    1. Low numbers of training sets mean low diversity/awareness,
    2. Low numbers of algorithms (DNNs) also mean low diversity/awareness
    3. Since training/learning is spread by update, the installed base is essentially multiple instances of the same individual. So no diversity and very high trust. That’s a recipe for a stampede of 10,000 self driving cars.

Since I wrote this, I’ve had some additional thoughts. I think that our understanding of Awareness and Trust is getting confused with Faith and Doubt. Much of what we believe to be true is no longer based on direct evidence, or even an understandable chain of reasoning. Particularly as more and more of our understanding comes from statistical analysis of large sets of fuzzy data, the line between Awareness and Faith becomes blurred, I think.

Doubt is an important part of faith, and it has to do with the mind coming up against the unknowable. The question does God exist? contains the basics of the tension between faith and doubt. Proving the existence of God can even be thought of as distraction from the attempt to come to terms with the mysteries of life. Within every one of us is the ability to reject all prior religious thought and start our own journey that aligns with our personal understandings.

Conversely, it is impossible to increase awareness without trusting the prior work. Isaac Newton had to trust in large part, the shoulders of the giants he stood on, even if he was refining notions of what gravity was. So too with Albert Einstein, Rosalind Franklin and others in their fields. The scientific method is a framework for building a large, broad-based, interlocking tapestry awareness.

When science is approached from a perspective of Faith and Doubt, communities like the Flat Earth Society emerge. It’s based on the faith that the since the world appears flat here, it must be flat everywhere, and doubt of a history of esoteric measurements and math that disprove this personally reasonable assumption. From this perspective, the Flat Earthers are a protestant movement, much in the way that the community that emerged around Martin Luther, when he rejected the organized, carefully constructed orthodoxy of the Catholic Church, based on his personally reasonable interpretation of scripture.

Confusing Awareness and Trust with Faith and Doubt is toxic to both. Ongoing, systemic doubt in trustworthy information will destroy progress, ultimately unraveling the tapestry of awareness. Trust that mysteries can be proven is toxic in its own way, since it gives rise to confusion between reality and fantasy like we see in doomsday cults.

My sense is that as our ability to manipulate and present information is handed over to machines, that we will need to educate them in these differences, and make sure that they do not become as confused as we are. Because we are rapidly heading for a time where these machines will be co complex and capable that our trust in them will be based on faith.