{"id":406,"date":"2011-05-01T07:36:29","date_gmt":"2011-05-01T11:36:29","guid":{"rendered":"http:\/\/scripts.mit.edu\/~zong\/wpress\/?p=406"},"modified":"2011-07-05T23:42:26","modified_gmt":"2011-07-06T03:42:26","slug":"how-to-summarize-a-long-tail","status":"publish","type":"post","link":"https:\/\/blog.yhuang.org\/?p=406","title":{"rendered":"how to summarize a long tail?"},"content":{"rendered":"<blockquote><p>Where to Live to Avoid a Natural Disaster<br \/>\nWeather disasters and quakes: who\u2019s most at risk? The analysis below, by Sperling\u2019s Best Places, a publisher of city rankings, is an attempt to assess a combination of those risks in 379 American metro areas &#8230; and take(s) into account the relative infrequency of quakes, compared with weather events and floods.<\/p><\/blockquote>\n<p><img decoding=\"async\" src=\"wp-content\/uploads\/images\/01safe-custom1.gif\" alt=\"http:\/\/graphics8.nytimes.com\/images\/2011\/05\/01\/weekinreview\/01safe\/01safe-custom1.gif\" width=\"600\" \/><\/p>\n<p>I don&#8217;t know what exact metric they used here but it seems to be more or less expected value of &#8220;disaster points&#8221; accumulated during a unit period, for the lack of a better description. Here is the problem with these expected value based metrics, trying to summarize very different distributions: the variance matters! One catastrophic disaster isn&#8217;t quite the same as several smaller disasters costing the same number of disaster points. Yet &#8220;maximizing&#8221; survival using this map, humanity moves to the West Coast then becomes extinct in the next big earthquake. Even imposing a convex utility function on the distribution isn&#8217;t entirely satisfying. When it comes to decision making, tail risk is in a different bucket than central risk. Somehow an important aspect of the decision making process isn&#8217;t captured by &#8220;soft&#8221; metrics.<br \/>\n<!--more--><br \/>\nBut let&#8217;s go back a bit. The reason why distributions are summarized by a single number is often based on the notion of repeated experiments such that the law of large numbers kicks in. For a single sample path, repetition is not relevant for truly rare events, so expected value and other metrics that depend on repetition should be discarded immediately. It isn&#8217;t clear what a reasonable replacement is though. If something like a simple survival threshold were more appropriate (let&#8217;s say you could survive a two sigma lifetime event and no larger), people certainly don&#8217;t behave as such, since there would be no solution: there is always a catastrophic risk, say, of the universe ending. It may be necessary to treat three different severity regimes separately: soft metrics for the &#8220;mundane&#8221; and multiply occurring central risks, thresholding for the &#8220;rare&#8221; but &#8220;statistically identifiable&#8221; tail risks, and disregard for the &#8220;never occurred and unknown&#8221; acts of god.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Where to Live to Avoid a Natural Disaster Weather disasters and quakes: who\u2019s most at risk? The analysis below, by Sperling\u2019s Best Places, a publisher of city rankings, is an attempt to assess a combination of those risks in 379 American metro areas &#8230; and take(s) into account the relative infrequency of quakes, compared with [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[1985,1988,2019,1874,1977,1989,1984,12,1986,1978],"class_list":["post-406","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-avoid","tag-big-earthquake","tag-catastrophic-disaster","tag-disaster","tag-disaster-weather","tag-law-of-large-numbers","tag-natural","tag-number","tag-weather","tag-weather-disasters"],"_links":{"self":[{"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=\/wp\/v2\/posts\/406","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=406"}],"version-history":[{"count":0,"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=\/wp\/v2\/posts\/406\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=406"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=406"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.yhuang.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}