Notes on “Sociophysics, an Introduction”

Sociophysics (Parongama Sen, Bikas K. Chakrabarti – 2013)

These are my notes as I was reading the book, which I found to be a very good overview with good detail that didn’t get in the way of the narrative. The references are stellar. When I found an appropriate paper mentioned in the text, I’ve included it as a link, usually with an accompanying abstract.

I read the book to support the model I’m working on for my PhD on trustworthy news. I’ve been doing agent-based simulations since the ’90s when I was working on my Master’s thesis on the The Coevolution of Weapons and Aggression. I certainly feel as though it has helped update my awareness of progress in the field since that effort, back when the term sociophysics didn’t even exist.

  • Chapter 2: Basic features of social systems and modelling
    • Minority Opinion Spreading in Random Geometry
      • Abstract: The dynamics of spreading of the minority opinion in public debates (a reform proposal, a behavior change, a military retaliation) is studied using a diffusion reaction model. People move by discrete step on a landscape of random geometry shaped by social life (offices, houses, bars, and restaurants). A perfect world is considered with no advantage to the minority. A one person-one argument principle is applied to determine locally individual mind changes. In case of equality, a collective doubt is evoked which in turn favors the Status Quo. Starting from a large in favor of the proposal initial majority, repeated random size local discussions are found to drive the majority reversal along the minority hostile view. Total opinion refusal is completed within few days. Recent national collective issues are revisited. The model may apply to rumor and fear propagation.
      • Clustering coefficient (video)
        CC = 0
        numNodes = 0
        for(i = 0 to max)
        	for(j = 0 to max)
        		n = node(i,j)
        		k = n.numNeighbors()
        		a = n.numLinksBetweenNeighbors()
        		CC += n.getNodeCC()
        CC = CC/numNodes
      • Clustering coefficient ordering: random -> small world -> regular
      • To build a scale-free network, AL Barabási, R Albert in Emergence of scaling in random networks start with a small random network and incrementally add nodes where the probability of connecting a new node with existing nodes is proportional to how many connections the current nodes have.
        for(i = 0 to desired)
        	n = createNewNode()
        	totalLinks = countAllLinks()
        	for(j = 0 to network.numNodes)
        		curNode = getNode(j)
        		links = curNode.getLinks
        		probability = links/totalLinks
        		curNode.addNeighbor(n, probability)
      • Does node aging matter in this model?
      • Null Models For Social Networks (for comparison and testing)
      • Downloaded the following from the references section to my Group Polarization folder
      • A bubble could be an example of a strong community [pg 17] would need to figure out a way of establishing in and out links in knowledge space
      • Benchmark networks to test community detection algorithms [pg 17]. Artificially generated and the Zachary Karate club
      • I appear to be working with (maybe?) class ‘C’ social networks, where links connect people indirectly [pg 19].Covered in chapter 7 – Of Flocks, Flows and Transports
      • Page 25 discusses Marian Boguña et al Models of Social Networks based on Social Distance Attachment which uses the concept of social distance. A set of quantities (e.g. profession, religion, location) are used and the social distance between two individuals is the difference in the quantities.
      • More state-space simulation from page 28: Spin-glass-like Dynamics of Social Networks. Digging around uncovered her thesis: Information and Entropy in Neural Networks and Interacting Systems. From the abstract:
        • Like neural networks, large ensembles of similar units that interact also need a generalization of classical information-theoretic concepts. We extend the concept of Shannon entropy in a novel way, which may be relevant when we have such interacting systems, and show how it differs from Shannon entropy and other generalizations, such as Tsallis entropy.
      • Mean Field Approximation – In physics and probability theory, mean field theory (MFT also known as self-consistent field theory) studies the behavior of large and complex stochastic models by studying a simpler model. Such models consider a large number of small individual components which interact with each other. The effect of all the other individuals on any given individual is approximated by a single averaged effect, thus reducing a many-body problem to a one-body problem.
    • Chapter 3: Opinion formation in a society
    • Chapter 4: Social choices and popularity – skimmed, not appropriate
    • Chapter 5: Crowd-avoiding dynamical phenomena – skimmed, not appropriate
    • Chapter 6: Social phenomena on complex networks
      • Claudio Castellano (Google Scholar)
      • Loops of nodes behave differently from trees. what to do about that? I think loops drive the echo chamber process? It is, after all, feedback..
      • There is also a ‘freezing’ issue, where a stable state is reached where two cliques containing different states are lightly connected, but not enough that the neighbors in one clique can be convinced to change their opinion [Fig. 6.2, pg 135]
      • Residual Energy: The difference between the actual energy and the known energy of the perfectly-ordered ground state (full consensus).
      • Dynamical Processes on Complex Networks. Got the Kindle edition so now I can search! Interesting section: 10.6 Coevolution of opinions and network
      • Similar chapter in this book – Social Phenomena on coevolutionary networks [pg 166]. One of the interesting things here is the use of the iterated prisoner’s dilemma. On a network, the agents typically calculate and aggregate payoff and imitate the strategy of the neighbor with the best payoff. In the coevolutionary model, an agent can cut off the link to a defector with a probability. This seems a bit like polarization, where the group severs ties with entities with sufficiently divergent views (and individuals leave when the group becomes too extreme)
      • Coevolution of agents and networks: Opinion spreading and community disconnection Abstract: We study a stochastic model for the coevolution of a process of opinion formation in a population of agents and the network which underlies their interaction. Interaction links can break when agents fail to reach an opinion agreement. The structure of the network and the distribution of opinions over the population evolve towards a state where the population is divided into disconnected communities whose agents share the same opinion. The statistical properties of this final state vary considerably as the model parameters are changed. Community sizes and their internal connectivity are the quantities used to characterize such variations.
      • Opinion and community formation in coevolving networks (Gerardo Iñiguez González)
        • Abstract: In human societies opinion formation is mediated by social interactions, consequently taking place on a network of relationships and at the same time influencing the structure of the network and its evolution. To investigate this coevolution of opinions and social interaction structure we develop a dynamic agent-based network model, by taking into account short range interactions like discussions between individuals, long range interactions like a sense for overall mood modulated by the attitudes of individuals, and external field corresponding to outside influence. Moreover, individual biases can be naturally taken into account. In addition the model includes the opinion dependent link-rewiring scheme to describe network topology coevolution with a slower time scale than that of the opinion formation. With this model comprehensive numerical simulations and mean field calculations have been carried out and they show the importance of the separation between fast and slow time scales resulting in the network to organize as well-connected small communities of agents with the same opinion.
        • Citing paper: Effects of deception in social networks (Gerardo Iñiguez González)<— Important???
          • Abstract: Honesty plays a crucial role in any situation where organisms exchange information or resources. Dishonesty can thus be expected to have damaging effects on social coherence if agents cannot trust the information or goods they receive. However, a distinction is often drawn between prosocial lies (‘white’ lies) and antisocial lying (i.e. deception for personal gain), with the former being considered much less destructive than the latter. We use an agent-based model to show that antisocial lying causes social networks to become increasingly fragmented. Antisocial dishonesty thus places strong constraints on the size and cohesion of social communities, providing a major hurdle that organisms have to overcome (e.g. by evolving counter-deception strategies) in order to evolve large, socially cohesive communities. In contrast, white lies can prove to be beneficial in smoothing the flow of interactions and facilitating a larger, more integrated network. Our results demonstrate that these group-level effects can arise as emergent properties of interactions at the dyadic level. The balance between prosocial and antisocial lies may set constraints on the structure of social networks, and hence the shape of society as a whole.
      • Section 6.5: Is it really a small world? Searching post Milgram
        • In the introduction to this section [page 168], the authors say a very interesting thing: “Although the network may have the small world property, searches are usually done locally: the individual may not know the global structure of the network that would help them find the shortest path to the target node“. I think that they are talking about social networks explicitly here, but the same concept applies to an information network. This is a network description of the information horizon problem. You can’t find what you can’t see, at least in a broad outline.
        • Also this: “Searching can regarded as a learning process; repeating the search several times can avoid infinite loops and lead to better solutions
        • 6.5.8 Funneling properties.
            • The funneling capability of a node can be defined as the fraction of successful dynamic paths through it when the target is fixed and the source is varied. Two thoughts: First, this seems to be a measurement of centrality. Second, Large, vague nodes are needed for ‘laundering’ information into misinformation or conspiracy theory.
            • Consider four agents. Who have characteristics that can vary between (0, 1).
              • Agent 1 has two color intensities: R=0.1, G= 0.7
              • Agent 2 has one color and two note volumes R=0.3, A=0.2, F=0.6
              • Agent 3 also has one color and two note volumes B=0.4, D=1, E=0.2
              • Agent 4 has three notes A=0.3, D=0.4, E=0.5
            • Let’s assume that funneling is not required if agents share a color or note. This means that A4 can get to A1 through A2, but A3 has to get to A1 via A4 and then A2. In a matrix this looks like
          R G B A D E F
          Agent1 0.1 0.7
          Agent2 0.3 0.2 0.6
          Agent3 0.4 1.0 0.2
          Agent4 0.3 0.4 0.5
            • But if we add the hypernyms Color and Notes, we can get funneling. I am summing the color and notes to give a sense of the agent’s ‘projection’ into the larger, more general space. I think the ‘size’ of the funnels are the number of items that go in them times the range of each item. So Color would have a range of (0, 3) and Notes would have a range of (0, 4), since I’m not including B, C, and G here:
          R G B A D E F Color Notes
          Agent1 0.1 0.7 0.8
          Agent2 0.3 0.2 0.6 0.3 0.8
          Agent3 0.4 1.0 0.2 0.4 1.2
          Agent4 0.3 0.4 0.5 1.2
            • Now agents 2 and 3 can get to each other through either Color or note in two hops, and the Agents 1 and 4 can reach each other by going through each of the funnels.
            • There should be a cost in using a funnel though. You loose the information about which color or which note. Intuitively, a series of steps with non-funnel links should be somehow more specific than the same number of steps through a funnel.
            • Practical uses would be a way to detect poorly reasoned conclusions, as long as the beginning and end of the train of thought could be identified.
      • Knowing a network by walking on it: emergence of scaling (Alexei Vázquez) Looks like an interesting guy with a wide range of publications.
    • Chapter 7:  of flocks, flows and transports [page 179]
      • Boids (Flocks, herds and schools: A distributed behavioral modelCraig Reynolds):
        • Try to avoid collisions with other boids (repulsion)
        • Attempt to match velocity with neighboring boids
        • attempt to stay close to nearby boids
      • If the collision avoidance is taken out and the number of dimensions increased, then this could be the model. Rather than the flock converging around a position, look at the distances between the individuals using DBSCAN and cluster.
      • Density and noise need to be independent variables and saved on runs. This would also be true in information space. You can have high organization in high density, low noise states. Thinking about that, this also implies one of the emergent properties of an information bubble is the low noise. Even though the environment may be very noisy, the bubble isn’t.
      • As with the other social models, individuals can have weight. That way the flock can have leaders and followers. (See Misinformed leaders lose influence over pigeon flocks to inform the model)
      • Also, I like the idea of a social network being built from belief proximity, which raises the cost for switching to another flock, even if they are nearby. It could be that once a social network forms that anti-belief repulsion starts to play a role.
      • Another component to include would be a Levy Flight (truncated?). That could account for cases where a leader makes a big jump and then the crowd follows with some ejection for those who can’t/won’t keep up.
      • Power law distribution of weight and max step size in the creation of the population
      • Thomas Schelling (Another Herbert Simon type) Segregation Model
      • Phase diagram of a Schelling segregation model (L Gauvin, J Vannimenus, JP Nadal – The European Physical Journal B, 2009). I’m beginning to think that the model could be a combination of a flocking and segregation model. That could be really interesting. I also seem to get nothing when I do a Scholar search on “flocking and segregation agent simulation
        • Satisfaction criteria – when the number of unlike agents is less than a fixed proportion F. As F gets larger there is an abrupt transition to a segregated state.
        • Definition of segregation coefficient – the weighted average (normalized) of all cluster sizes averaged over all configurations. When only two clusters survive, n(c) = N/2
      • Migration in a small world: A network approach to modeling immigration processes (B Fotouhi, MG Rabbat – Communication, Control, and Computing, 2012 –
    • Chapter 8: Endnote [page 202]
      • Frustration in Complexity (2008 – Philippe Binder)- The common thread between all complex systems may not be cooperation but rather the irresolvable coexistence of opposing tendencies.
      • Definition of consensus in an opinion model – the emergence of long-range order.
      • Looking for phase changes from heterogeneous to homogeneous or clustered states is important. Finding what parameters are causal and the values is considered a publishable result. Canonical types of transitions, such as the percolation threshold are discussed in the appendices.

Trustworthy News Model Assumptions


  • 12.13.16: Initial post
  • 12:16:16: Added reference to proposal and explicitly discussed explorer and exploiter types.

A web version of my Google Docs dissertation proposal is here. Blame them for the formatting issues. The section this is building on is Section 5.3.1. A standalone description of this task is here.

The first part of my dissertation work is to develop an agent-based simulation that exhibits information bubble/antibubble behavior. Using Sen and Chakrabarti’s Sociophysics as my guide, I’m working up the specifics of the model. My framework is an application (JavaFX, because that’s what I’m using at work these days). It’s basically an empty framework with a trivial model that allows clustering based on similar attributes such as color: strawmanapp

Going forward, I need to clarify and defend the model, so I’m going to be listing the components here.

Agent assumptions

  • Agents get their information from global sources (news media). They have equal access, but visibility is restricted
    • Agents are Explorers or Exploiters (Which may be made up of Confirmers and Avoiders)
    • Agents have ‘budgets’ that they can allocate
    • Finding sources has a cost. Sources from the social network has a lower cost to access
    • Keeping a source is cheaper than getting a new one
    • For explorers, the cost of getting a new source is lower than an exploiter.
    • The ‘belief’ as a set of ‘statements’ appears to be valid
    • The collection of statements and the associated values create a position in an n-dimensional hilbert space of information. Position and velocity should be calculable.
    • Start at one dimension to reproduce prior opinion models

Network assumptions

  • There are two items that we are looking for.
    • The first is the network configuration over time. What nodes do agents connect to for their information.
    • The second is the content of that information. For that, we’ll probably need some dimensionality reduction, such as NMF (look for a post on implementing this later). This is where we look for echo chambers of information, as opposed to the agents participating in them
  • Adjustable to include scale-free, small world, and null configurations
  • What about loops? Feedback could be interesting, since a small group that is semi-isolated could form into a very loud bubble that could lower the cost of finding information. So a notion of volume might be needed that emerges from a set of agreeing agents. This could be attraction, though I think I like an economic approach more?
  • There is also a ‘freezing’ issue, where a stable state is reached where two cliques containing different states are lightly connected, but not enough that the neighbors in one clique can be convinced to change their opinion [Fig. 6.2, pg 135]


  • Residual Energy: The difference between the actual energy and the known energy of the perfectly-ordered ground state (full consensus).
  • Deviation from null network.
  • Clustering as per community detection (Girard et. al)

Implementation details

  • Able to be run multiple times with the same configuration but different seed
  • Outputs to… something. MySql or Excel probably
  • Visualization using t-SNE? Description plus Java implementation is here:

More to come as the model fleshes out.


Opinion Dynamics With Decaying Confidence: Application to Community Detection in Graphs

Opinion Dynamics With Decaying Confidence: Application to Community Detection in Graphs

  • Irinel-Constantin Morarescu
  • Antoine Girard (some supporting slides from 2006. Very helpful!)
  • Really important reference: Community detection in graphs.
  • Handy chart of symbols, and a bigger chart
  • Data sources for the paper:
  • Italics indicate direct quotes
  • From the slides, a flock is an entity in a network where the members have agreed upon a direction and a velocity. In the paper, rather than movement vector, the value is an ‘opinion’
  • We consider a network of agents where each agent has an opinion. At each time step, the agents exchange their opinion with their neighbors and update it by taking into account only the opinions that differ from their own less than some confidence bound. This confidence bound is decaying: an agent gives repetitively confidence only to its neighbors that approach sufficiently fast its opinion.
    • This seems like a nice way to form bubbles. Agents only see their neighbors and have to accommodate with their neighbors within a narrowing range of acceptance. This means that other agents elsewhere in the network (and depending on the connectivity) would converge differently, and different opinions would be created.
  • Under that constraint, global consensus may not be achieved and only local agreements may be reached. The agents reaching a local agreement form communities inside the network.
    • If the decay rate is low enough, then global consensus can be reached. Faster, and the network starts to break apart.
  • Our model can be interpreted in terms of opinion dynamics. Each agent has an opinion. At each time step, the agent receives the opinions of its neighbors and then updates its opinion by taking a weighted average of its opinion and the opinions of its neighbors that are within some confidence range of its own. The confidence ranges are getting smaller at each time step: an agent gives repetitively confidence only to the neighbors that approach sufficiently fast its own opinion. This can be seen as a model for a negotiation process where an agent expects that its neighbors move significantly towards its opinion at each negotiation round in order to keep negotiating.
  • We assume that the relation is symmetric and anti-reflexive
    • Undirected graph where no nodes are connected to themselves
  • This model can be related to the one discussed in [17], [18] where agents harden their position by increasing over time the weight assigned to their own opinion. In our model, the agents implicitly increase also the weights assigned to their neighbors whose opinion converges sufficiently fast to their own opinion, by disregarding the opinions of the other agents. As noticed in [18], hardening the agents positions may hamper the agents to reach an asymptotic consensus. This will be observed in our model as well. However, the aim in this paper is not to exogenously increase the self-confidence of the agents, but to meet a prescribed convergence speed towards the final opinion profile.
    • This last line follows my thinking on bubbles somewhat. I think the hardening is a function of the information distance between the two positions. Convergence can only happen at a certain rate, so the farther apart the harder it is to converge. In this model, that’s done by arbitrarily reducing the confidence, but I think the math should be pretty similar. I do wonder if anti-agreement is useful here.
  • our model would coincide with Krause model of opinion dynamics with bounded confidence [9][10][11].
    • It looks like Krause is the fountainhead of this area of research. Lots of really interesting work. Everything seems to be from a perspective that agents will converge on one or more opinions, and then the simulation ends. So I know how to make bubbles (and possibly antibubbles, simply by not having agents ‘harden’). What seems to be missing is the notion in Group Polarization that the opinion becomes more extreme. When searching through the works that cite [9], there does seem to be work in this area, but I wasn’t able to find anything that actually has a model using agent-based simulation.
  • In this section, we explore the relation between communities and asymptotically connected components of the network. Let us remark that the set of edges can be classified into two subsets. Intuitively, an edge E(finite)is in if the agents and stop interacting with each other in finite time. E(infinite)consists of the interactions between agents that are infinitely recurrent.
    • So this works in the context that the final opinion is static. I think opinions need a random walk component. Given that there are multiple opinions, is the difference a hypotenuse or manhattan distance?
    • As discussed in the the end of the simulation, any connected agents must be in agreement. That means that you can just look at the connections and determine the group?
  • Asymptotic Agreement Implies Asymptotic Connectivity
    • They show that this holds for most but not all conditions. That’s an interesting finding, since it implies in almost any sufficiently connected network, a bubble will engulf most individuals that agree…
    • In this section, we showed that asymptotic connectivity of agents implies asymptotic agreement and that under additional reasonable assumptions these are actually equivalent except for a set of vectors of initial opinions of Lebesgue measure 0. In other words, we can consider almost surely that the communities of agents correspond to the connected components of the graph G(infinity). I think this agrees with my above point.
  • Community Detection: In the usual sense, communities in a graph are groups of vertices such that the concentration of edges inside one community is high and the concentration of edges between communities is comparatively low. Because of the increasing need of analysis tools for understanding complex networks in social sciences, biology, engineering or economics, the community detection problem has attracted a lot of attention in the recent years. The problem of community detection is however not rigorously defined mathematically. One reason is that community structures may appear at different scales in the graph: there can be communities inside communities. Another reason is that communities are not necessarily disjoint and can overlap. We refer the reader to the excellent survey [12] and the references therein for more details. Some formalizations of the community detection problem have been proposed in terms of optimization of quality functions such as modularity [13] or partition stability [14].
  • Essentially, the modularity Q(P)of the partition P is the proportion of edges within the classes of the partition minus the expected proportion of such edges, where the expected number of edges between vertex i and j is assumed to be (degree_i*degree_j)/(all edges)
  • The higher the modularity, the better the partition reflects the community structure of the graph. Thus, it is reasonable to formulate the community detection problem as modularity maximization. However, it has been shown that this optimization problem is NP-complete [21]. Therefore, approaches for community detection rely mostly on heuristic methods. In [15], a modularity optimization algorithm is proposed based on spectral relaxations. Using the eigenvectors of the modularity matrix, it is possible to determine a good initial guess of the community structure of the graph. Then, the obtained partition is refined using local combinatorial optimization. In [16], a hierarchical combinatorial approach for modularity optimization is presented. This algorithm which can be used for very large networks, is currently the one that obtains the partitions with highest modularity.
  • Bubbles at scales? “Stability measures the quality of a partition by giving a positive contribution to communities from which a random walker is unlikely to escape within the given time scale. For small values of t, this gives more weights to small communities whereas for larger values of t , larger communities are favored. Thus, by searching the partitions maximizing the stability for several values of , one can detect communities at several scales.
  • The algebraic connectivity of a graph G is the second-smallest eigenvalue of the Laplacian matrix of G
  • we want to find groups of vertices that are more densely connected than the global graph. This coincides with the notion of community. The larger δ, the more densely connected the communities. This makes it possible to search for communities at different scales of the graph.
  • For each combination of parameter value, the model was simulated for 1000 different vectors of initial opinions chosen randomly in [0,1]34. Simulations were performed as long as enabled by floating point arithmetics.
    • I think that this means that each agent was given a distinct random opinion for each of 1,000 runs. Then they looked for the most common clusterings
  • It is interesting to remark that for δ = 2 we almost obtained the communities that were reported in the original study [23]. Only one agent has been classified differently.
  • When δ increases, the communities become smaller but more densely connected.
    • It should be very interesting to look at belief velocity at different scales.
  • …for the same value of parameter δ, the modularity is very similar for all partitions. Actually, all the partitions obtained for the same value of δ are almost the same. As in the previous example, we can see that the choice of parameters R and α affects the probability of obtaining a given partition. The partition with maximal modularity is obtained for δ = 0.2, it is a partition in 4 communities with modularity 0.523
  • Let us remark that even though the information on the political alignment of the books is not used by the algorithm, our approach allows to uncover this information. Indeed, for δ = 0.1, we obtain 2 communities that are essentially liberal and conservative. For δ = 0.2, we then obtain 4 communities: liberal, conservative, centrist-liberal, centrist-conservative.
    • Note that this is information appears to be latent
  • The last example we consider consists of a significantly larger network of 1222 political blogs [24]. In this network, an edge between two vertices means that one of the corresponding blogs contained a hyperlink to the other on its front page. We also have the information about the political alignment of each blog based on content: 636 are conservative, 586 are liberal.
  • There are 2 main communities: one with 653 blogs, from which 94% are conservative, and one with 541 blogs, from which 98% are liberal. The 28 remaining blogs are distributed in 10 tiny communities. When we progressively increase δ, we can see that the size of the two large communities reduces moderately but progressively until δ = 0.65 where the conservative community splits into several smaller communities, the largest one containing 40 blogs. The liberal community remains until δ = 0.725 where it splits into smaller communities, the largest one containing 54 blogs.

JavaScript’s Gulf of Evaluation and Gulf of Execution


It came to me yesterday why JavaScript development is such slow going for me. I think it’s because it feels like an OO language, but it’s really not. Particularly when using TypeScript, there are classes, ‘this’, inheritance, etc. The thing is that all these ‘artifacts’, for lack of a better term have been added to the language and aren’t there natively. That means that often, things that would behave as I expect them in a more ‘native’ OO language like Java (or Actionscript for that matter) don’t behave intuitively in JS. This means more time in the debugger saying things like “why isn’t that working?”.

Don Norman describes this as the Gulf of Evaluation and the Gulf of Execution. His canonical example is the settings for his freezer. Unless you know what the controls are affecting behind the panel, the odds of getting the temperature correct in the setup he describes are essentially random.

Now, consider probably the most canonical of the JS quirks, the behavior of this. It has scope only in the current function. Nest a function within a function and the scope of the parent ‘object’ doesn’t exist in the parent’s function. This is because in reality, there is no parent object, just a hierarchy of functions. But we have closure, so by setting ‘self = this’ in the parent, we get an approximation of the desired behavior. And now with fat arrow notation, this can indeed be nested (but Typescript won’t let you inherit a fat arrow method). But you have to know that, just like the mechanism behind the panel of Don Norman’s freezer.

The most recent thing that I had to deal with was the assembly of a hierarchical data provider object from a set of database calls. The structure is pretty straightforward:

dataProvider = {

I really thought I should be able declare these items on the fly, something like:

dataProvider = {};
dataProvider['items']['type']['item1'] = {...};
dataProvider['items']['type']['item2'] = {...};
dataProvider['associations'] = [];

Instead, the ‘item1’ object gets created, but the [‘items’] object does not. I expected construction of the object to chain from tail to head like it does with execution (e.g. Instead I get a null object error, and a “why isn’t that working?” moment.

In the end, I had to write some code that created the original object and then before each item was added, to make sure that the appropriate property existed, otherwise create it and add it. So, instead of an hour or so of casual coding, this simple task mushroomed into an afternoon’s worth of careful development and testing.

Interestingly, I had to pick up PHP after a long absence, and for me at least, it behaves the way I expect OO languages to behave, with very consistent quirks that you have to learn once – I’m looking at you __construct()

So that’s why JS development takes too freakin’ long for my calcified OOP brain. But it’s also about why we should also be very careful when we try to make something look like something it’s not.


Gotchas or Special Cases?

I’ve now been working in AngularTypeScript for a while, a few things have cropped up that are probably worth mentioning. Some things are just for clarity, others are because they had me confused for a while.

So without further adieu, my laundry list:

Structuring the main angular app class

I’ve decided that I like using the constructor, rather than instantiating the object and then calling a method. Mostly this is because in TypeScript, the arguments to the constructor aren’t defined in interfaces so it can vary naturally, and because it’s a bit less typing for the same result. Here’s the app I’m currently working on:

module AngularApp {
   // define how this application assembles.

   class QueryMain {


            this.serviceModule = angular.module('phpConnection', []);
            this.serviceModule.service('QueryService', ['$http', queryServicePtr]);

            this.appModule = angular.module('rssApp', ['phpConnection', 'ngSanitize']);
            this.appModule.controller('MainCtrl', ['QueryService', '$timeout','$rootScope', queryControllerPtr]);
            this.appModule.directive('ngFeedPanel', ['$timeout','$rootScope', queryDirectivePtr]);
            this.appModule.directive('ngInfoDialog', ['$timeout','$rootScope', infoDialogDirectivePtr]);
            this.appModule.directive('ngNetworkWebgl', ['$timeout','$rootScope', glNetDirectivePtr]);

   // instantiate Angular with the components defined in the other files. Note
    // that weird things may happen with transclusion?

    new QueryMain(
        new RssAppDirectives.ngFeedPanel().ctor,
        new WGLA2_dirtv.ngNetworkWebgl().ctor,
        new WGLA2_dirtv.ngInfoDialog().ctor,

There are really three patterns to note here. The first is that everything is that everything that the class QueryMain uses is passed in. No globals. And going along with that, please note that directives (and factories) have to be instantiated, so we pass in a pointer to a ctor() function, rather than the class as a whole. The last thing to mention is that there is no chaining. The modules are declared and then the components are added on individually rather than module().directive.controller() etc. The reason for this is that if you happen to declare, for example, a service that is needed by some later item. That service will not be visible if it’s chained with the declaration of the item. Line by line declarations appear to have more predictable results.

The wonderfulness of interfaces and a revisit of fatArrow = ():notation => {}

There seem to be times when it’s nicer to use fat arrow notation. In my experience, this pops up the most often in directives, though it can show up in promises as well. I’ve had some odd experiences where it seemed that ‘this’ doesn’t survive scope/closure changes into a called method. I can’t find any examples where I wasn’t able to make it work with conventional notation, but it’s worth covering anyway.

Since I’ve been working more with directives recently, we’ll use one of the directives used in the above code as an example. This one has had parts stripped out for clarity. By the way, as with every other class that is an Angular/Typescript component this extends my ATSBase class, which is covered here. :

export class ngFeedPanel extends NovettaUtils.ATSBase {
    private myDirective:ng.IDirective;
    private cobj:RssControllersModule.ICallbackPointers;

    constructor() {
    private linkFn (scope:any, element:any, attrs:any) {
        this.cobj = scope.callbackObj;

        scope.nlpQuery = ():void => {
            var mobj:RssControllersModule.IDataResponse = scope.messageObj;

    public ctor(timeout:ng.ITimeoutService, rootscope:ng.IScope):ng.IDirective {

        if (!this.myDirective) {
            this.myDirective = {
                templateUrl: 'directives/rssFeed.html',
                restrict: 'AE',
                scope: {
                    messageObj: '=',
                    callbackObj: '='
                link: this.linkFn
        return this.myDirective;

If you’ve read any earlier posts, you’ll know that I like pointers. The ctor() method is used as a function pointer in the main app, and in turn the link: element of the ng.IDirective object points to the linkFn() method in this class.

Unlike interfaces for other languages, I’ve found interfaces most useful for handling the pattern of:

myObj = {
    thing1 : "thing1",
    thing2 : "thing2",
    thing3 : "thing3",
    thing4 : "thing4"

I hate those things. I’ll make some typo and not notice until the browser crashes in a test. And because there is often no context, I’ll look at the broken code and not see the problem (thingOne, not thing1, or something like that)

Typescript makes sure that doesn’t happen. This interface:

export interface ICallbackPointers{
    setQueryString : Function;
    addToQueryString : Function;
    rssQuery : Function;
    nlpQuery : Function;

Gets declared as a typed object:

export class RssController extends WGLA2_ctrl.Network3DCtrl{
   public callbacks:ICallbackPointers;

Instanced in the constructor (function pointers!):

this.callbacks = {
    setQueryString : this.setQueryString,
    addToQueryString : this.addToQueryString,
    rssQuery : this.googleNewsSubmit,
    nlpQuery : this.nlpQuerySubmit

Passed through the html:

<div class="resultsWrapper">
    <div ng-repeat="item in mc.itemDataArray track by $index">
        <ng-feed-panel message-obj="item" callback-obj="mc.callbacks"></ng-feed-panel>

And used in the directive (also shown above as part of the linkFn()):

scope.nlpQuery = ():void => {
    var mobj:RssControllersModule.IDataResponse = scope.messageObj;

Never a chance for a mistake, but with all the power of ad-hoc object creation. Very cool.

Fat Arrow has been more mysterious. In the following I show two sets of code that use a service to get data from a source. In the fist example all the code is contained within a single class that extends ATSBase, which creates a fat arrow alias for each method. Nonetheless, without fat arrow, ‘this’ does not track back to the class:

public promiseCaller():void{
   this.promise = this.service.getQueries();
   this.promise.then(this.processData, this.errorData);

public processData = (data:any):void => {
   // 'this' is out of scope without fat arrow
   console.log("got data");

public errorData = (data:any):void => {
   // 'this' is out of scope without fat arrow
   alert("error getting data");

However, in the version shown below, ‘this’ is preservedwithin goodUserQuery() and errorResponse.

private goodUserQuery (response:any) {console.log("got data");}
private errorResponse (response:any) {alert("error getting data");}
public promiseCaller():void{
    this.queryService.submit(qstr, this.goodUserQuery, this.errorResponse);

It’s even preserved in the call to the queryService.submit call that takes place in a different service class, part of which is shown below:

public submit(query:string, goodResponse:any, errorResponse:any):any {
   return this.httpService(query).then(goodResponse, errorResponse);

So that’s odd.

The safe pattern appears to be that for small methods that I’m unlikely to extend, I’ll use fat arrow. Wrapping methods will extend just fine and will use the fat arrow functions inside. But I wouldn’t be able to extend the inner methods. Mostly, I think that’s fine and more likely to keep me out of trouble then accidentally typing ‘this’ when I should’ve typed ‘self’. So you basically have the choice:

scope.handleButtonPressFat = (strVal:string):void => {
   this.handleButtonPress(scope, strVal);
scope.handleButtonPress = function(strVal:string):void {
   self.handleButtonPress(scope, strVal);

But if you find yourself in a function that has been called out of a promise and closure isn’t working the way you think it should, try seeing if it can be fixed by using fat arrow.

TypeScript and AngularJS Unification?

Because of certain obscure reasons, AngularJS needs to do very particular things with “this”. Shortly after taking up TypeScript and trying to feed in Angular ‘objects’, I learned about how “Fat Arrow Notation” (FAN) allows for a clean interface with Angular. It’s covered in earlier posts, but the essence is

// Module that uses the angular controller, directive, factory and service defined above.
module AngularApp {
   // define how this application assembles.
   class AngularMain {

      public doCreate(angular:ng.IAngularStatic, tcontroller:Function, tservice:Function, tfactory:Function, tdirective:Function) {
         this.serviceModule = angular.module('globalsApp', [])
            .factory('GlobalsFactory', [tfactory])
            .service('GlobalsService', [tservice]);

         this.appModule = angular.module('simpleApp', ['globalsApp'])
            .controller('MainCtrl', ['GlobalsFactory', 'GlobalsService', '$timeout', tcontroller])
            .directive('testWidget', ['GlobalsService', tdirective]);
   // instantiate Angular with the components defined in the 'InheritApp' module above. Note that the directive and the factory
   // have to be instantiated before use.
   new AngularMain().doCreate(angular,
      new InheritApp.TestFactory().ctor,
      new InheritApp.TestDirective().ctor);

Basically, the Angular parts that need to new() components (Controllers and Services) get the function pointer to the class, while components that depend on the object already being created (Directives and Factories) have a function pointer passed in that returns an object that in turn points to the innards of the class.

So I refactored all my webGL code to FAN and lo, all was good. I made good progress on building my shiny 3D charts.

Well, charts are pretty similar, so I wanted to take advantage of TypeScript’s inheritance, make a BaseChart class, which I would then extend to Area, Bar, Column, Scatter, etc. What I expected to be able to do was take a base method:

public fatArrowFunction = (arg:string):void => {
   alert("It's Parent fatArrowFunction("+arg+")");

And extend it:

public fatArrowFunction = (arg:string):void => {
   alert("It's Child fatArrowFunction("+arg+")");

“Um, no.”, said the compiler. “super() cannot be used with FAN”.

“WTF?” Said I.

It turns out that this is a known not-really-a-bug, that people who are combining Angular and TypeScript run into. After casting around for a bit, I found the fix as well:

class Base {
   constructor() {
      for (var p in this) {

         if (!, p) && typeof this[p] == 'function') {
            var method = this[p];
            this[p] = () => {
               method.apply(this, arguments);
            // (make a prototype method bound to the instance)

Basically what this does is scan through the prototype list and set a bunch of fat arrow function pointers that point back to the prototype function. It seems that there are some people that complain, but as a developer who cut his teeth on C programming, I find function pointers kind of comforting. They become a kind of abbreviation of some big complex thing.

The problem is that the example doesn’t’ quite work, at least in the browsers I’m currently using (Chrome 41, IE 11, FF 36). Instead of pointing at their respective prototypes, all the pointers appear to reference the last item of the loop. And the behavior doesn’t show up well in debuggers. I had to print the contents of the function to see that the pointer named one thing was pointing at another. And this happened in a number of contexts. For example, this[fnName] = () => {this[‘__proto__’][fnName].apply(this, arguments);} gives the same problem.

After a few days of flailing and learning a lot, I went back to basics and tried setting the function pointers explicitly in the constructor of each class. It worked, and it wasn’t horrible. Then, and pretty much just for kicks, I added the base class back in with this method:

public setFunctionPointer(self:any, fnName:string):void{
   this[fnName] = function () {
      //console.log("calling ["+fnName+"]")
      return self['__proto__'][fnName].apply(self, arguments);

And gave it a shot. And it worked! I was pleasantly surprised. And because I’m an eternal optimist, I added the loop back, but this time using the function call:

constructor() {
   var proto:Object = this["__proto__"];
   var methodName:string;

   for (var p in proto){
         methodName = p;
         if(methodName !== 'constructor'){
            this.setFunctionPointer(this, methodName);
         //console.log("\t"+methodName+" ("+typeof proto[p]+")");

And that, my droogs, worked.

I think it’s a prototype chaining issue, but I’m not sure how. In the non-working code, we’re basically setting this[fnName] = function () { this[fnName].apply(self, arguments)}. That should chain up to the prototype and work, but I don’t think it is. Rather, all the functions wind up chaining to the same place.

function Base() {
    var _this = this;
    for (var p in this) {
        if (!, p) && typeof this[p] == 'function') {
            var method = this[p];
            this[p] = function () {
                method.apply(_this, arguments);

On the other hand, look at the code generated when we use the function we get the following:

var ATSBase = (function () {
    function ATSBase() {
        var proto = this["__proto__"];
        var methodName;
        for (var p in proto) {
            methodName = p;
            if (methodName !== 'constructor') {
                this.setFunctionPointer(this, methodName);
    ATSBase.prototype.setFunctionPointer = function (self, fnName) {
        this[fnName] = function () {
            //console.log("calling ["+fnName+"]")
            return self['__proto__'][fnName].apply(self, arguments);
    return ATSBase;

Now, rather than starting at the root, the actual call is done in the prototype. I think this may cause the chain to start in the prototype object, but then again, looking at the code, I don’t see why that should be the case. One clear difference is the fact that in the first version, “this” can be in two closure states (this[p] = function (){method.apply(_this, arguments);};). So it could be closure is behaving in less than obvious ways.

Unfortunately, we are at the point in development where something works, so it’s time to move on. Maybe later after the codebase is more mature, I’ll come back and examine this further. You can explore a running version here.

Web Dev Jenga

Back in the distant past, some very smart people wrote a paper about the past, present and future of user interface software tools. In it they discuss the idea of a tool having a low threshold to learning and a high ceiling of capability. Inevitably, they say, we build tools that start with low threshold and slowly add capability until the high ceiling of capability is reached. Unfortunately, this (almost?) always means that the low threshold to learn is lost amid all the added complexity.

I have seen this happen with FORTRAN, C/C++, Java, JavaScript, and HTML. It’s a pain, but I think it’s inevitable. Interestingly, I think that if you keep things hard, they paradoxically stay simple. The difference between GL, OpenGL and WebGL is really not all that different. There was a big change with the introduction of shaders, but that’s one major shift in something like 20 years.

Now it’s happening to tools. It’s easy to write a quick tool that handles some aspect of development. If it has low threshold for learning and good utility, then it gets picked up and suddenly we have a new way of doing the same old thing. Maybe it’s better, but often it’s just different. The unfortunate result is now we have stacks of frameworks, languages and tools that we don’t understand well. The normal scenario is:

  • Have a confounding problem.
  • Ask Google/StackOverflow about it.
  • Try the responses that seem best until something works
  • Move on to the next confounding problem

As a professional developer, I only have so much time to drill down into things to obtain deep understanding. Many times, you have to trust. It’s faith-based coding, and it really reminds me of building a tower from Jenga blocks. We add and subtract things all the time. It’s a miracle that the thing stays up as often as they do.

My adventures with ‘thrangularJS’ has settled down to the point where I’m building reasonably complex pieces that need to be assembled in a particular order. The watcher in IntelliJ will compile TypeScript to JavaScript, but just in the context of that one file. If a change has been made in a TypeScript file, chances are that it will have to ramify through the project. This is one of those things that has to happen using compilers that doesn’t happen with interpreters.

So, I start to look at what the web development community is doing with dependency management these days. The answer seemed to be Grunt with a typescript task. This worked, but it compiled all the files even if only one needed to be changed. What I really wanted was a makefile. But the makefile needed to be triggered by a watcher. It turns out that there has been some thought on this, and it works in a clean way.

Since I’m on a windows box, I’m using GnuMake. It’s extremely stable, last updated in 2006 (when iPhones were introduced, I think). I’m also using Grunt, installed by npm. Not as stable as make, but it’s never done me wrong. Following the install of make, and Grunt the components have to be set up in the project directory:

npm init
npm install grunt --save-dev
npm install grunt-exec --save-dev
npm install grunt-contrib-watch --save-dev

Then we need a makefile. This is mine, and there are probably better ways to do it. But it’s clear (you can learn everything you ever wanted to know about make here):

CC = tsc 
CFLAGS=  --declaration --noImplicitAny --target ES5 --sourcemap

build: modules/AppMain.js

modules/AppMain.js : directives/WGLA2_directives.js controllers/WGLA1_controller.js \
   $(CC) $(CFLAGS) modules/AppMain.ts

classes/WebGlInterfaces.js : classes/WebGlInterfaces.ts
   $(CC) $(CFLAGS) classes/WebGlInterfaces.ts

classes/WebGlCanvasClasses.js : classes/WebGlInterfaces.js \
   $(CC) $(CFLAGS) classes/WebGlCanvasClasses.ts

classes/WebGlComponentClasses.js : classes/WebGlInterfaces.js  \
   $(CC) $(CFLAGS) classes/WebGlComponentClasses.ts

controllers/WGLA1_controller.js : classes/WebGlInterfaces.js classes/WebGlComponentClasses.js classes/WebGlCanvasClasses.js \
   $(CC) $(CFLAGS) controllers/WGLA1_controller.ts

directives/WGLA2_directives.js : classes/WebGlInterfaces.js classes/WebGlComponentClasses.js classes/WebGlCanvasClasses.js \
   $(CC) $(CFLAGS) directives/WGLA2_directives.ts

Last, we need a GruntFile.js to knit it all together:

module.exports = function (grunt) {

        pkg: grunt.file.readJSON('package.json'),

        exec: {
            make: {
                command: 'make build'
        watch: {
            files: ['**/*.ts', '!**/*.d.ts'],
            tasks:['exec:make'] //tasks: ['ts']


    grunt.registerTask('default', ['watch']);


And that’s it. Make is a little tricky to learn (high threshold), but has tremendous power and flexibility (high ceiling). Grunt is kept simple and obvious and can be swapped out easily if something better comes along. Other tasks can be added to make such as uglify, test, deploy, pretty much whatever you want. And it’s guaranteed to go in the right order.

It’s a good block to keep in your Jenga tower.