ABSTRACT: Scholars have increasingly investigated “crowdsourcing” as an alternative to expert-based judgment or purely data-driven approaches to predicting the future. Under certain conditions, scholars have found that crowd-sourcing can outperform these other approaches. However, despite interest in the topic and a series of successful use cases, relatively few studies have applied empirical model thinking to evaluate the accuracy and robustness of crowdsourcing in real-world contexts. In this paper, we offer three novel contributions. First, we explore a dataset of over 600,000 predictions from over 7,000 participants in a multi-year tournament to predict the decisions of the Supreme Court of the United States. Second, we develop a comprehensive crowd construction framework that allows for the formal description and application of crowdsourcing to real-world data. Third, we apply this framework to our data to construct more than 275,000 crowd models. We find that in out-of-sample historical simulations, crowdsourcing robustly outperforms the commonly-accepted null model, yielding the highest-known performance for this context at 80.8% case level accuracy. To our knowledge, this dataset and analysis represent one of the largest explorations of recurring human prediction to date, and our results provide additional empirical support for the use of crowdsourcing as a prediction method. (via SSRN)
I was revisiting some of our old stuff for this Oslo event -early on for us on our #LegalPhysics #LegalAnalytics path – published in Physica A – “By applying our sink clustering method, we obtain a dendrogram of the network’s largest weakly connected component shown in Fig. 4. However, despite their general topical relatedness, these two clusters of cases engage substantively different sub-questions, and are thus appropriately divided into separate clusters. While not a major focus of the docket of the modern court, the early court elaborated a number of important legal concepts through the lens of these admiralty decisions. For example, the red group of cases engages questions of presidential power and the laws of war, as well as general interpretations of the Prize Acts of 1812. Meanwhile, the blue cluster engages questions surrounding tort liability, jurisdiction, and the burden of proof.”
Yesterday was the 2nd Edition of The Forum on Legal Evolution – Hosted at Northwestern University Pritzker School of Law — The Forum is comprised of legal innovators and early adopters, organized around a shared interest in the changing legal market. Paul Lippe + Mark Chandler received lifetime achievement awards. Thanks to William Henderson and his team for organizing the event!
Tomorrow I will be speaking at The Future of Law Schools Conference – Organized by the Thomson Reuters Legal Executive Institute.
Here is our panel which kicks off the afternoon –
David Curle, Director, Market Intelligence, Thomson Reuters Legal
Daniel B. Rodriguez, Dean @ Northwestern Pritzker School of Law
Joseph Harroz, Dean @ University of Oklahoma
Daniel Martin Katz, Assoc. Prof @ Illinois Tech – Chicago Kent Law
Gabriel H. Teninbaum, Professor @ Suffolk University Law School
Tanina Rostain, Professor @ Georgetown Law
See article here
Last night – I was inducted into the College of Law Practice Management – it is a real honor to join an amazing class of inductees and current fellows ! #COLPM
Very excited to be here for the Futures Conference of the College of Law Practice Management. Tonight I will be induced as as Fellow of the College (along with a range of other inductees).
See coverage of our conference here
This is my Presentation for the NALP Conference on Emerging Legal Careers.
Tomorrow I will deliver the Thursday Keynote at the Legal Week Strategic Technology Forum in San Diego. I look forward to connecting with old friends and making new friends at the event!