Starting in the January 2012, Scott E. Page (one of my PhD thesis advisors) will teach Model Thinking (a free online course offered via the consortium that brought you AI Class, Machine Learning, etc.)

Scott and I have previously teamed up to teach Complex Systems @ the ICPSR Summer Methods Program (where I teach the model implementation lab).  Over 7,000 people and counting have are already signed up …

I am going to bump this post to front of the blog one last time as there has been some interest in this material. It has now been several weeks since we completed the full four week class here at the ICPSR Program in Quantitative Methods. In this course, I (together with my colleagues) highlight the methods of complex systems as well as several environments designed to explore the field. These include Netlogo (agent based models and network models), Vensim (system dynamics / ecological modeling) and Pajek (empirical network analysis).  In the final week, we cover a variety of advanced topics:

• (a) Community Detection in Networks
• (b) Computational Linguistics / Natural Language Processing
• (c) Diffusion Models and Mathematical Modeling with Data
• (d) Exponential Random Graph (p*) Models
• (e) Information Retrieval / Webscraping

Although, we do not work with more advanced languages within the course, those who need to conduct complex analysis are directed to alternatives such as R, Python, Java, etc.

Anyway, the slides are designed to be fully self-contained and thus allow for individually paced study of the relevant material. If you work through the slides carefully you should be able to learn the software as well as many of the core principles associated with the science of complex systems. The material should be available online indefinitely. If you have questions, feel free to email me.

This week’s issue of the Economist has an interesting article entitled Riders on a Swarm. Among other things, the article discusses how attempts to computationally model ant, bee and bird behavior have offered insight into major problems in artificial intelligence.

For those not familiar, the examples discussed within the article are classic models in the science of complex systems. For example, here is the Netlogo implementation of bird flocking. It will run in your browser but requires Java 4.1 or higher. If you decide to take a look — please click setup – then go to make the model run. Once inside the Netlogo GUI, you can explore how various parameter configurations impact the model’s outcomes.

One of the major insights of the bird flocking model is how random starting conditions and local behavioral rules can lead to the emergence of observed behavioral patterns that appear (at least on first glance) to be orchestrated by some sort of top down command structure.

This is, of course, not the case. The model is bottom up and not top down. Both the simplicity and the bottom up flavor of the model are apparent when you explore the model’s code. For those interested, I will take a second and plug the slides from my ICPSR class. In the class, I dedicated about an hour of class time to bird flocking model. Click here for the slides. In the slides, I walk through some of the important features of the code (discussion starts on slide 16).

]]> http://computationallegalstudies.com/2010/08/17/riders-on-a-swarm-might-mimicking-the-behavior-of-ants-bees-birds-be-the-key-to-artificial-intelligence/feed/ 0 http://computationallegalstudies.com/2010/07/27/agents-of-change-agent-based-models-and-methods-the-economist/ http://computationallegalstudies.com/2010/07/27/agents-of-change-agent-based-models-and-methods-the-economist/#comments Tue, 27 Jul 2010 06:41:30 +0000 Daniel Martin Katz http://computationallegalstudies.com/?p=4693 Continue reading →]]>

This week’s “economic focus” in the Economist highlights Agent Based Modeling as an alternative to traditional economic models and methods. As I am currently teaching Agent Based approaches to modeling as part of the ICPSR Introduction to Computing for Complex Systems, I am quite pleased to see this coverage.  Indeed, the timing could not be better and I plan to highlight this article in the course!

Here are some highlights from the article: “… Agent-based modelling does not assume that the economy can achieve a settled equilibrium. No order or design is imposed on the economy from the top down. Unlike many models, ABMs are not populated with “representative agents”: identical traders, firms or households whose individual behaviour mirrors the economy as a whole. Rather, an ABM uses a bottom-up approach which assigns particular behavioural rules to each agent. For example, some may believe that prices reflect fundamentals whereas others may rely on empirical observations of past price trends. Crucially, agents’ behaviour may be determined (and altered) by direct interactions between them, whereas in conventional models interaction happens only indirectly through pricing. This feature of ABMs enables, for example, the copycat behaviour that leads to “herding” among investors. The agents may learn from experience or switch their strategies according to majority opinion. They can aggregate into institutional structures such as banks and firms …” For those who are interested, I have made similar points in the post “Complex Models for Dynamic Time Evolving Landscapes –or– Herb Gintis Offers a Strong Rebuke of “Meltdown.“

The State of the Union often provides for dramatic political theatre. While watching President Obama’s first State of the Union Address last night, I could not help but think about a particular subplot associated with the speech–the Republican caucus and the “standing ovation problem.” With respect to being the party not currently occupying the White House–from the individual member all the way up to the full caucus–it is difficult for the individual member to determine (1) whether to applaud (2) if a given statement by the President is worthy of a standing ovation. From my passive consumption of the television coverage, there was clearly significant variation in the number of Republican caucus members standing at any given applause moment.

For those not familiar, here is a State of the Union based description of the standing ovation problem. “The standing ovation model illustrates a familiar decision-making problem: after hearing a given statement by the President a subset of the audience begins to applaud. The applause builds and a few members of the respective caucus may decide to stand up in enthusiastic recognition. In this situation every other member of the respective caucus must decide whether to join the standing individuals in their ovation, or else remain seated. It is not a trivial decision; imagine, for example, that you initially decide to stay down quietly but then find yourself surrounded by people standing and clapping vigorously. It seems plausible that you may feel awkward, change your mind and end up standing up, saving yourself a significant dose of potential embarrassment. Analogously, you probably wouldn’t enjoy being the only person standing and clapping alone in the middle of a crowded chamber of seated people.”

While often considered along with other related information cascade problems, generating agent based models for the so called “standing ovation problem” has been the focus of a number of scholars.  For example, along with John Miller (Carnegie Mellon), Michigan CSCS Director Scott E. Page has authored a leading article on the “standing ovation problem.”  Using an agent based modeling approach, Miller & Page analyze a variety dynamics associated with this rich problem. For those interested, here is a link to a standing ovation ABM in Netlogo (requires Java).

In this tutorial, we will build a model that can simulate outbreaks of disease on a small-world network (although the code can support arbitrary networks).  This tutorial represents a shift away from both:

a) the mass-action mixing of the first two and and

b) the assumption of social homogeneity across individuals that allowed us to take some shortcuts to simplify model code and speed execution. Put another way, we’re moving more in the direction of individual-based modeling.

When we’re done, your model should be producing plots that look like this:

Outbreak on a small-world network

Red nodes are individuals who have been infected before the end of the run, blue nodes are never-infected individuals and green ones are the index cases who are infectious at the beginning of the run.

And your model will be putting out interesting and unpredictable results such as these:

Time vs. # of cases

In order to do this one, though, you’re going to need to download and install have igraph for Python on your system.

Individual-Based Networks

It is important to make the subtle distinction between individual and agent based models very clear here. Although the terms  are often used interchangeably, referring to our nodes, who have no agency, per se, but are instead fairly static receivers and diffusers of infection, as agents, seems like overreaching. Were they to exhibit some kind of adaptive behavior, i.e., avoiding infectious agents or removing themselves from the population during the infective period, they then become more agent-like.

This is not to under- or over-emphasize the importance or utility of either approach, but just to keep the distinction in mind to avoid the “when all you have is a hammer, everything looks like a nail” problem.

In short, adaptive agents are great, but they’re overkill if you don’t need them for your specific problem.

Small World Networks

The guiding idea behind small-world networks is that they capture some of the structure seen in more realistic contact networks: most contacts are regular in the sense that they are fairly predicable, but there are some contacts that span tightly clustered social groups and bring them together.

In the basic small-world model, an individual is connected to some (small, typically <=8) number of his or her immediate neighbors. Some fraction of these network connections are then randomly re-wired, so that some individuals who were previously distant in network terms – i.e., connected by a large number of jumps – are now adjacent to each other. This also has the effect of shortening the distance between their neighbors and individuals on the other side of the graph. Another way of putting this is that we have shortened the average path length and increased the average reachability of all nodes.

These random connections are sometimes referred to as “weak ties”, as there are fewer of these ties that bridge clusters than there are within clusters. When these networks are considered from a sociological perspective, we often expect to find that the relationship represented by a weak tie is one in which the actors on either end have less in common with each other than they do with their ‘closer’ network neighbors.

Random networks also have the property of having short average path lengths, but they lack the clustering that gives the small-world model that pleasant smell of quasi-realism that makes them an interesting but largely tractable, testing ground for theories about the impact of social structure on dynamic processes.

Installation and Implementation Issues

If you have all the pre-requisites installed on your system, you should be able to just copy and paste this code into a new file and run it with your friendly, local Python interpreter. When you run the model, you should first see a plot of the network, and when you close this, you should see a plot of the number of infections as a function of time shortly thereafter.

Aside from the addition of the network, the major conceptual difference is that the model operates on discrete individuals instead of a homogeneous population of agents. In this case, the only heterogeneity is in the number and identity of each individual’s contacts, but there’s no reason we can’t (and many do) incorporate more heterogeneity (biological, etc.) into a very similar model framework.

With Python, this change in orientation to homogeneous nodes to discrete individuals seems almost trivial, but in other languages it can be somewhat painful. For instance, in C/++, a similar implementation would involve defining a struct with fields for recovery time and individual ID, and defining a custom comparison operator for these structs. Although this is admittedly not a super-high bar to pass, it adds enough complexity that it can scare off novices and frustrate more experienced modelers.

Perhaps more importantly, it often has the effect of convincing programmers that a more heavily object-oriented approach is the way to go, so that each individual is a discrete object. When our individuals are as inert as they are in this model, this ends up being a waste of resources and makes for significantly more cluttered code. The end result can often be a model written in a language that is ostensibly faster than Python, such as C++ or Java, that runs slower than a saner (and more readable) Python implementation.

For those of you who are playing along at home, here are some things to think about and try with this model:

1. Change the kind of network topology the model uses (you can find all of the different networks available in igraph here).
2. Incorporate another level of agent heterogeneity: Allow agents to have differing levels of infectivity (Easier); Give agents different recovery time distributions (Harder, but not super difficult).
3. Make two network models – you can think of them as separate towns – and allow them to weakly influence each other’s outbreaks. (Try to use the object-oriented framework here with minimal changes to the basic model.)

That’s it for tutorial #3, (other than reviewing the comment code which is below) but definitely check back for more on network models!

In future posts, we’ll be thinking about more dynamic networks (i.e., ones where the links can change over time), agents with a little more agency, and tools for generating dynamic visualizations (i.e., movies!) of stochastic processes on networks.

That really covers the bulk of the major conceptual issues. Now let’s work through the implementation.

Click Below to Review the Implementation and Commented Code!

Okay, thanks for clicking through… now follow along with the comments in the code below:

]]> http://computationallegalstudies.com/2009/11/15/programming-dynamic-models-in-python-3-outbreak-on-a-network/feed/ 3 http://computationallegalstudies.com/2009/08/26/model-of-intellectual-diffusion-on-the-american-legal-academy-repost-from-422/ http://computationallegalstudies.com/2009/08/26/model-of-intellectual-diffusion-on-the-american-legal-academy-repost-from-422/#comments Wed, 26 Aug 2009 09:34:24 +0000 Daniel Martin Katz http://computationallegalstudies.com/?p=1923 Continue reading →]]> For the third installment of posts related to Reproduction of Hierarchy? A Social Network Analysis of the American Law Professoriate, we offer a Netlogo simulation of intellectual diffusion on the network we previously visualized.  As noted in prior posts, we are interested legal socialization and its role in considering the spread of particular intellectual or doctrinal paradigms. This model captures a discrete run of the social epidemiological model we offer in the paper.   As we noted within the paper, this represents a first cut on the question—where we favor parsimony over complexity.  In reality, there obviously exist far more dynamics than we engage herein.  The purpose of this exercise is simply to begin to engage the question. In our estimation, a positive theory of law should engage the sociology of the academy — a group who collectively socialize nearly every lawyer and judge in the United States. In the paper and in the model documentation, we offer some possible model extensions which could be considered in future scholarship.

Once you click through to the model, here is how it works:

(1) Click the Setup Button in the Upper Left Corner.  This will Display the Network in the Circular Layout.

(2) Click the Layout Button.  Depending upon the speed of your machine this may take up to 30 seconds.  Stop the Layout Button by Re-Clicking the Button.

(3) Click the Size Nodes by Degree Button. You Will Notice the Fairly Central Node Colored in Red.  This is School #12 Northwestern University Law School.  Observe how we have set the default infected school as #12 Northwestern (Hat Tip to Uri Wilensky).  A Full List of School Number is available at the bottom of the page when you click through.

(4) Now, we are ready to begin.  Click the Spread Once Button.  The idea then reaches its neighbors with probability p (set as a default at .05).  You can click the Toggle Infection Tree button (at any point) to observe the discrete paths traversed by the idea.

(5) Click the Spread Once Button, again and again.  Notice the plot tracking the time on the x axisand the number of institution infected on the y axis.  This is an estimate of the diffusion curve for the institution.

(6) To restart the simulation, click the Reinfect One button.  Prior to hitting this button, slide theInfected Slider to any Law School you would like to observe.  Also, feel free to adjust the p slider to increase or decrease the infectiousness of the idea.

Please comment if you have any difficulty or questions.  Note you must have Java 1.4.1 + installed on your computer.  The Information Technology professionals at many institutions will have already installed this on your machine but if not you will need to download it.   We hope you enjoy!

The Forest Fire Model is a commonly invoked example of non-linear system–where a very small perturbation can generate significant differences in observed outcomes. Consider the above Netlogo–to Run the Model: (1) Adjust the Density Slider to set the concentration within the Forest.  (2) Hit the Setup Button (3) Hit the Go Button  …. Rinse and Repeat at different levels of Density.

Above is the output for a run of the model at several levels of Density {48%, 56%, 62%}.  Notice the differences in the Percent Burned {1.6%, 5.2%, 86.5%}.

This is obviously a theoretical model but it has potential application to a wide class of substantive questions including regulatory failure.  In addition, the Forest Fire Model is important because it has been invoked in the critique of the popular book The Tipping Point. Specifically, in discussing the book network scientist Duncan Watts notes ”It sort of sounds cool … But it’s wonderfully persuasive only for as long as you don’t think about it.” Watts notes “…trends are more like forest fires: There are thousands a year, but only a few become roaring monsters. That’s because in those rare situations, the landscape was ripe: sparse rain, dry woods, badly equipped fire departments. If these conditions exist, any old match will do…. and nobody… will go around talking about the exceptional properties of the spark that started the fire.” (Quotes from Jan 2008 Is the Tipping Point Toast? Fast Company Magazine).

The visual above is drawn from the Netlogo Simulation of preferential attachment. ”In the model, a given node prefers to connect to other nodes that already display high indegree.  As the number of connections a given agent displays is a function of the number the agent possessed in earlier time periods, the distribution of connections is highly susceptible to the initial starting conditions. For instance, consider a network that has four nodes A, B, C and D where A is connected to B and C is connected to D. If node E enters the network, assume the initial probability of attachment to the AB community is equal to that of the CD community. Once E connects to either the AB or CD community, subsequent entrants such as node F, G and beyond are more likely to connect with the community selected by E.” The model offers one of the generative processes responsible for creating a network with a power law distribution.  

There are important differences between the abstract model as initially described in Albert-László Barabási & Reka Albert, Emergence of Scaling in Random Networks, 286 Science 509 (1999) and the dynamics of broader social world.  While a number of extensions of the model have been authored in the period following the original article, what is striking is how much leverage on basic dynamics can be gleaned from the graph analog of a Yule process. 

For purposes of positive legal theory consider the following passage … “In order to contextualize what a particular observed network structure implies, it is critical to remember that the social landscape need not take any particular form. Scaffolding could indeed assume a variety of flavors and there are causal mechanisms that act at the micro-level to produce the observed macro-architecture.”  While such distributions have been documented in a variety of context relevant for positive legal theory, it is important to note this distribution of social authority is by no means a given.  Specifically, as we described in Social Architecture, Judicial Peer Effects and the ‘Evolution’ of the Law: Toward a Positive Theory of Judicial Social Structure social systems can embrace a wide variety of architectures.  Thus, we believe the documented tendency of common law and its constitutive systems to generate such highly skewed distributions is highly relevant. 

In Part II of this post, we will highlight the current state of the relevant applied legal literature. This includes not only our work but also important studies by a wide number of other legal scholars. To preview, check out this post from a few days ago…    

Several months ago, I put together this syllabus for use in a future seminar course Law as a Complex System.   A number of my friends and colleagues noted that if were to actually use this syllabus in a course, it would be necessary to reduce the total reading in contained herein. While I completely agree, I still thought I would post it to the blog in its current form. I am proud to say that I am an award winning instructor.  Notwithstanding, I am always interested in improving my pedagogical skills. Thus, if you see any law related scholarship you believe should be included please feel free to email me.

This is the final installment of posts related to Reproduction of Hierarchy? A Social Network Analysis of the American Law Professoriate. Thanks for your emails.

Here is the plot we provide within the paper.  As a general proposition, we believe this represents an upper bound measure for the intellectual reach of an agenda offered by a given institution.  With respect to our version of the Reed Frost Epidemiological Model, we use the p parameter to model “idea infectiousness.”  When p = 1 every institution “contacted” by the idea is infected with the idea. When p = 0 no institution “contacted” by the idea is infected.  In this version, we use the programming language python to run the model 500 times per institution. The above plot represents an estimate of the “diffusion curve” for each of the 184 institutions in our model. Building off central limit type properties, this leaves a far better estimate of reach than is offered in the single model run from the previous Netlogo GUI.

A cursory review of the above plot demonstrates, we are far from the land of linearity.  Namely, a large number of institutions are able to reach much of the graph with very small changes in the value of p.

In the Structure of Scientific Revolutions, Kuhn quotes from Max Planck:  ”a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.” Following Planck, we believe retirement is indeed be an important mechanism.  However, we also argue the nature of the p parameter is a relevant consideration.  In fact, unpacking various dimensions of p is the key to the broader model. Specifically, what are the properties of an idea that generate its infectiousness? Of course, we might like to believe infectiousness is related to a class of normatively attractive properties such as promoting efficiency or justice.  However, it is not clear that this follows.

We took no pass on the question of whether some institutions would be better or worse at producing ideas with greater or lesser values of p. The motivated question for this post considers whether, in general, the institutions which are top producers of law professors are (1) leaders in innovation, (2) subsequent ratifiers of a newly established paradigm or (3) defenders of the status quo. In a deep sense, we are asking how to reasonably model decision making by the heterogeneous agents located at such institutions.  Do institutions reward or punish intellectual risk-taking, search, etc.?

While this is an empirical question beyond the scope of this post, it worth asking because it partially informs the micro-dynamics plausibly responsible for generating the spread of new intellectual paradigms.

I recently attended a conference at the Santa Fe Institute.  During the trip, I made a point of eating at the El Farol Bar & Restaurant. This restaurant holds a special place in the lore of complex systems.  Thus, I thought I would take the opportunity to highlight the model on the CLS blog.  

Here is a subset of the model description…. “The bar is popular — especially on Thursday nights when they offer Irish music — but sometimes becomes overcrowded and unpleasant. In fact, if the patrons of the bar think it will be overcrowded they stay home; otherwise they go enjoy themselves at El Farol. This model explores what happens to the overall attendance at the bar on these popular Thursday evenings, as the patrons use different strategies for determining how crowded they think the bar will be.”   

The original paper written by Brian Arthur is located here. An interesting follow up paper employing reinforcement learning is located here.    This above is a screen print from the Netlogo model.  Netlogo offers an easy interface useful for exploring a variety of agent based models.  

The model will run in your browser provided you have Java 1.4.1+.  

To run the El Farol model, please go here.   

A Netlogo 3D screenprint of one of the classic agent based models—the Shelling Segregation Model is above. We offer it as a holdover until CLS Blog Returns Sunday Night with more exciting content…..

NEXT WEEK:
(1) Discussion of a New Paper: Computer Programming and the Law
(2) Visualizing the 110th Congress — The House of Representatives
(3) For Law Students and Law Professors — Data on the Law Clerk Tournament
(4) And More …..