While Google Ngrams is a fun exploratory tool, it is merely a glimpse at the real possibilities in the era of Big Data. Two major conferences this year — Princeton CITP: Big Data Conference and ECCS 2010: High Throughput Humanitiesoffered a preview of the world that is coming. In my presentation at these conferences, I tried to underscore the ways in which these developments are meaningful for social scientists, legal scholars and practicing lawyers. In short, the prospects for arbitrage here are significant. It will be exciting to watch creative folks try to put things together …
While some of these cost and revenue projections could be debated, the New York Times has developed a nifty interface that allows end users to consider how they would attack the problem of deficit reduction. Using the New York Times Budget puzzle, here is my approach. In developing my approach, I tried to consider a configuration that I thought could actually attract majority support in a time of divided government. Thus, I leaned pretty hard on spending cuts (70%) with relatively fewer tax increases (30%). Obviously, your are free to disagree but I would suggest that try it for yourself and see where you come out.
While I am hardly here to shill for Bloomberg, the introduction of Bloomberg Government into the market for government information does represent an important development worthy of highlighting. Coverage from a few weeks back is located here and here.
While I hope to explore the actual product in the coming months, the front page highlights both its coverage and its informational interface. Whether aimed at sophisticated and non sophisticated actors, the selection of this sort of dashboard style interface is important as it is precisely the sort of HCI that has been shown to help end users navigate complex information environments.
The ever increasing access to digitized governmental information provides a real arbitrage opportunity for a specific firm to serve as the default third party provider of that information. Whether Bloomberg Government will fill this void is likely a function of (1) the novelity of its informational inputs and (2) the quality of the HCI experienced by target end users. Only time will tell…
From this week’s issue of Science comes Filling the Light Pipe by David J. Richardson. This is an important article highlighting a serious challenge facing the both the scientific and policy community.
From the abstract: “It has been a landmark year for the field of optical telecommunications, with the award of the 2009 Nobel Prize to Charles Kao for his insight in the mid-1960s that the future of communications lay in single-mode silica-based optical fibers (1) as well as the 50th anniversary of the first demonstration of the laser—both key technologies responsible for the development of the global-scale communications networks of today (2). Recently, however, a growing realization has emerged within the telecommunications industry that the end of the phenomenal growth in optical fiber communication capacity is within sight. At this year’s Optical Fiber Communication Conference (OFC 2010), several groups reported results within a factor of 2 of the ultimate capacity limits of existing optical fiber technology. Without radical innovation in our physical network infrastructure—that is, improvements in the key physical properties of transmission fibers and the optical amplifiers that we rely on to transmit data over long distances—we face what has been widely referred to as a “capacity crunch” that could severely constrain future Internet growth, as well as having social and political ramifications.”
Sebastian Seung is mapping a massively ambitious new model of the brain that focuses on the connections between each neuron. He calls it our “connectome,” and it’s as individual as our genome — and understanding it could open a new way to understand our brains and our minds.
Congratulations to Carl Malamud and publicresource.org for being selected as a Google 10^100 project winner! Among other things, the $2 million grant “will support the Law.Gov initiative, which aims to make all primary legal materials in the United States available to all.” I had the pleasure of presenting at the UC-Boulder Law.gov meeting and Texas Law.gov meeting earlier this year. Thus, I am very excited about Google’s decision to fund this very worthy project. For those who are interested in reading more, O’Reilly Radar has additional information on the law.gov / google 10^100 announcement here.
As we speak, I am currently in route to Portugal for the very exciting High Throughput Humanities meeting on Wednesday. As we believe it fits well within the goals of the meeting, I will briefly present our work on the United States Code (including a brief preview of our still unreleased new paper). Anyway, for those not familiar with the meeting, if you click on the image above you will be taken to the main page. Also, here is the announcement for the meeting:
“The High Throughput Humanities satellite event at ECCS’10 establishes a forum for high throughput approaches in the humanities and social sciences, within the framework of complex systems science. The symposium aims to go beyond massive data aquisition and to present results beyond what can be manually achieved by a single person or a small group. Bringing together scientists, researchers, and practitioners from relevant fields, the event will stimulate and facilitate discussion, spark collaboration, as well as connect approaches, methods, and ideas.
The main goal of the event is to present novel results based on analyses of Big Data (see NATURE special issue 2009), focusing on emergent complex properties and dynamics, which allow for new insights, applications, and services.
Utilizing a complex systems approach to harness these data, the contributors of this event aim to make headway into the territory of traditional humanities and social sciences, understanding history, arts, literature, and society on a global-, meso- and granular level, using computational methods to go beyond the limitations of the traditional researcher.”
1. Scour the web: We continually scan thousands of news publications, blogs, niche sources, trade publications, government web sites, financial databases and more.
2. Extract, rank and organize: We extract information from text including entities, events, and the time that these events occur. We also measure momentum for each item in our index, as well as sentiment.
3. Make it accessible and useful: You can explore the past, present and predicted future of almost anything. Powerful visualization tools allow you to quickly see temporal patterns, or link networks of related information.
Diversity is one the major topics of interest here at Michigan Center for the Study of Complex Systems. This includes diversity as experienced in physical as well as in social systems. On this topic, here is a lecture that one of my dissertation advisors, Scott Page gave earlier this year at the Darden School of Business at Virginia. This lecture is related to his book about the power of diversity entitled: The Difference: How The Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Although I realize it is roughly a 90 minute talk, I believe that you will find that it is well worth the time!
“At TEDGlobal 2010, author Matt Ridley shows how, throughout history, the engine of human progress has been the meeting and mating of ideas to make new ideas. It’s not important how clever individuals are, he says; what really matters is how smart the collective brain is.”
2 of the ultimate capacity limits of existing optical fiber technology. Without 
