Tag: prediction models

-

Quantitative Legal Prediction { Draft of Paper Available on SSRN }

clsadmin


“Do I have a case? What is our likely exposure? How much is this going to cost? What will happen if we leave this particular provision out of this contract? How can we best staff this particular legal matter? These are core questions asked by sophisticated clients such as general counsels as well as consumers at the retail level. Whether generated by a mental model or a sophisticated algorithm, prediction is a core component of the guidance that lawyers offer. Indeed, it is by generating informed answers to these types of questions that many lawyers earn their respective wage.

Every single day lawyers and law firms are providing predictions to their clients regarding their prospects in litigation and the cost associated with its pursuit (defense). How are these predictions being generated? Precisely what data or model is being leveraged? Could a subset of these predictions be improved by access to outcome data in a large number of ‘similar’ cases. Simply put, the answer is yes. Quantitative legal prediction already plays a significant role in certain practice areas and this role is likely increase as greater access to appropriate legal data becomes available.

This article is dedicated to highlighting the coming age of Quantitative Legal Prediction with hopes that practicing lawyers, law students and law schools will take heed and prepare to survive (thrive) in this new ordering. Simply put, most lawyers, law schools and law students are going to have to do more to prepare for the data driven future of this industry. In other words, welcome to Law’s Information Revolution and yeah – there is going to be math on the exam.”

Access the paper HERE.

-

A Rational But Ultimately Unsuccessful Critique of Nate Silver

clsadmin

This article is reasonable in so much as it is a rational argument against Nate Silver’s work at 538 (rather than the ridiculous nonsense he had to endure from folks who are totally clueless – UnSkewedPolls.com, etc.).  However, it is ultimately unsuccessful.

“Nate Silver didn’t nail it; the pollsters did.”  Not true.  They both got it correct (or as accurate as can be when there is only 1 event that is being modeled).

“To be fair, the art of averaging isn’t simple.”  Well it is not just averaging.  Pure averaging is totally stupid.  This is weighting and it is non-trivial because you need to build a notion of how much signal vs. noise to assign to each {pollster, time point combo}.  Some of these polling outfits are totally disreputable and some have historic “house effects” (see e.g. Rasmussen).  With respect to time – the question is how much of the past is useful for predicting the future – so you need some sort of decay function to phase out the impact of prior data points (prior polls) on your current prediction.

It is correct to say that Nate Silver’s model cannot be validated in a traditional sense – he uses simulation – because on every day other than election day – there is no way to execute a direct test of the accuracy of the model.  Simulation is basically as good as we can do in an environment where there is only one event and it is perfectly valid as a scientific endeavor.  If folks want to complain and actually be taken seriously – they can come up with their own positive approach.  The scientific community can engage the competing claims.  For example, the Princeton Election Consortium is a good example of a challenge to the 538 methodology.

No matter what 538 is a hell of a lot better than the status quo practices previously existed to its founding in early 2008. The level of jealously directed toward Nate Silver is completely transparent.  If you want to get all Popperian – go right ahead but then you have little or nothing to say about this or most other prediction problems.  This is what happened in quantitative finance / algo trading and the arbitrage went to those who were not worried about whether what they were doing was science or just engineering [insert Sheldon Cooper quote here] .

One thing we can hope comes out of all this is that all of the data free speculation that was undertaken prior to the election can be put to bed.  I talking about you Dick Morris, Karl Rove, etc. – perhaps you guys should consider retirement and leave the arguments to the serious quants.