Closedness of a set in a metric space (“includes all limit points”), by the sound of it, really wants to be something akin to “has solid boundaries.” But it isn’t. The problem is that the existence of limit points depends on the embedding space. If the embedding space lacks those limit points, then a set in it can be technically closed even though it isn’t really “like” other closed sets. For example, the set \(\mathbb R\) in space \((\mathbb R, d_{\text{Eucl.}})\) is closed, because the space has no point called \(\infty\).

The first stab at “has solid boundaries” was to get rid of infinities by adding boundedness (“can be covered by some ball”) to the condition. Now, the set \(\mathbb R\) in space \((\mathbb R, d_{\text{Eucl.}})\) is closed yet not bounded, so it doesn’t have solid boundaries. Excellent. Upon further inspection, however, it doesn’t really address the real issue. For instance, set \(\mathbb Q \cap [0,1]\) in space \((\mathbb Q, d_{\text{Eucl.}})\) is closed and bounded, but it’s still porous at all the irrational numbers. It doesn’t have solid boundaries.

The second stab then was to replace closedness with completeness (“contains all Cauchy limits”). Completeness gets rid of the dependency on the embedding space and uses points in the set itself to define limits. This along with boundedness takes care of the above two examples. But there is still a deeply unsatisfying outcome: completeness and boundedness both depend on the metric and therefore are affected by rescaling. First let’s rewrite the boundedness condition as equivalently “can be covered by a finite number of \(r\)-balls for a fixed \(r\)“. The set \(\mathbb Z\) in space \((\mathbb Z, d_{\text{Disc.}}^1)\) where \(d_{\text{Disc.}}^k(x,y)=k\) if \(x\neq y\) is complete and bounded, if we choose \(r>1\), yet under a rescaling, say by replacing \(d_{\text{Disc.}}^1\) with \(d_{\text{Disc.}}^{r+\epsilon}\), the set is no longer bounded.

So finally, a third stab was to require sets to be bounded at once for all rescaling, by replacing boundedness with total boundedness (“can be covered by a finite collection of \(r\)-balls for all \(r\)“).

This last definition, complete and totally bounded, turns out to be equivalent to compactness (“every open cover has a finite subcover”) in metric spaces. And it shows. We’ve managed to drop all the stuff dependent on the embedding space and the metric scaling. All that is left are the topological properties of the set under open balls generated by the metric.

In some sense, what we get out of compactness (and really wanted from the outset) is the notion that a set is well contained regardless of scaling. This is what it means to have solid boundaries. It has to be a topological property of the set itself. More interestingly, because we can scale the set however we want (provided the scaling is non-singular), we can turn infinitesimals into infinities and vice versa. Small-scale infinitesimals (limiting sequence) and large-scale infinities (unbounded sequence) are really the same thing. For instance, the set \(\{1/n\}_{n\in \mathbb Z}\) under the metric \(d_{\text{Eucl.}}\) has an unincluded Cauchy limit at 0 and is thus not complete, but if we rescale the underlying space by \(1/x\) at \(x\), then it is basically \(\mathbb Z\) under the metric \(d_{\text{Disc.}}^1\), with no Cauchy limits to include and is complete. Similarly the first set is totally bounded while the second is not. This scale-dependence is awkward. Apparently, completeness and total boundedness take care of the small-scale and the large-scale separately, but in the end, they are all infinities and should be treated the same. The true distinction of consequence is between finitude (solid boundaries, well contained) and infinitude (no solid boundaries, ill contained), a distinction which compactness identifies.

One of the immediately intuitive notions coming from this understanding of compactness is the theorem that states “the image of a compact set under continuous mapping is also compact” (just a rescaling after all). We can regurgitate this as, a set with solid boundaries under continuous rescaling of the space still has solid boundaries. Other nice properties about limits being attained on compact sets, and about functions on compact sets, follow from the same intuition of their having solid boundaries. It also makes sense why the intersection of a compact set with a closed set is compact (well containment only needs to be ensured once), and therefore the intersection of any number of compact sets is compact. In some sense, compact sets are a much more useful dual to open sets, than closed sets — even under their topological definition (“complement is open”) — ever were.

So being at the event captured in the image, I got to ask a question toward the end. Actually I asked two questions. The first was whether Watson would ring in and use the remaining 3 seconds or whatever to continue to compute. Gondek said it would if it helped. In actual competition it doesn’t appear to be the case, as the buzz-in thresholding condition ensured that further computation would not have been helpful. The second question was a follow-up on the identified weakness of Watson — learning on “common sense” knowledge. I asked what path AI research would take to tackle such knowledge, which are by its very definition, “not in the books.” Gondek said that IBM is building up semantic information (e.g. a “report” is something that can be “turned in” and “assessed,” etc.) from corpus. That wasn’t exactly what I was asking, however.

My point was whether all “knowledge” is written down. There is such a thing as experiential “knowledge,” and humans take years to learn it/be trained in it through parenting (i.e., to “mature”). If only there were a handbook on life, or that life could be learned through reading a series of textbooks, then perhaps I’d believe that the kind of general-purpose AI that most people are probably imagining (rather than an expert/Q&A system) can be achieved along the lines of current methods.

In any case, earthquake prediction may be a misnomer. One can never predict the precise time of an earthquake. But with more data and detection of ever smaller features, one can give more granular probabilistic predictions. So instead of saying there is a probability \(p\) of earthquake in the next 30 years, we may be able to either say (at any given moment) there is a probability \(p_1 \ll p\), or probability \(p_2 \gg p\) of one within the next year.