The idea of plenoptic cameras has been thrown around for quite a while. Instead of a conventional camera with a single lens focusing a single object plane onto the in-camera image plane (i.e. the sensor), a plenoptic camera attempts to capture enough additional information so as to be able to reconstruct “all possible” images that can be obtained from the light entering the same aperture. The most talked-about application is subsequent refocusing; if it were just this, then multi-capture with mechanical focal-length sweeps using a conventional camera would suffice. Another is stereograms, but again, two spaced shots would suffice for that. A plenoptic camera does more in one shot and makes these merely special post-processing cases. The simplest conception of a plenoptic camera is essentially an array of mini-cameras (microlens + micropixel array for each logical pixel) that separately captures light from all directions at each point in the image. In between conventional cameras and plenoptic cameras are perhaps smarter, sparser non-regular arrays like these coded aperture systems that hark back to old radio-astronomy literature. These have good signal processing properties from a deconvolution perspective, but the full-array plenoptic camera like the R11 seems fully general, and with some future industrial scaling, the saved expenses of a compromise may be inconsequential.

Fine, so a plenoptic camera may make clever use of its given aperture size, but do we really get something for nothing? To answer that, first a digression.

Why does a conventional camera lose information? According to the cartoon version of Fourier optics, a lens is a spatially-variant phase transformer and space itself is an array of lightfield integrators. One can sort of see that light from the object arriving at farther off-axis locations on the lens will have greater (i.e. higher-frequency) phase variations corresponding to various points of the object, the result being that the pattern at the front end of the lens is the Fourier Transform of the object. The lens multiplies its aperture shape onto this, and by its phase transforming capability, sends the product out to the image plane, where the analogous thing happens. The pattern at the image plane is thus the Inverse Fourier Transform of the pattern coming out of the lens. The net result is the image is the object convolved with the transform pair of the lens aperture shape, known as the Point Spread Function (PSF).

So right off the bat, the object has been low-passed through the finite aperture due to diffraction. Nothing can be done about finite bandwidth, but at least the focused distance to the image plane is also where usually the PSF main lobe is narrowest for a particular object plane distance (coincides with geometric optics results). Away from the designated object plane distance, we no longer get good approximations to Fourier Transforms so the image is further distorted by non-invertible transformations that can never be uniquely decoded.

A plenoptic camera is supposed to get around this with its microlens array structure because (1) each small aperture microlens is a large depth-of-field subsystem operating in parallel with others, (2) total bandwidth is not sacrificed since the synthetic aperture is still large, (3) better than a pinhole array as microlenses still do your Inverse Fourier Transforms for you, no deconvolution of potentially non-invertible transformations involved. But for this, there is now a design tradeoff between higher spatial resolution per depth vs. depth resolution. So it’s not a free gain.

In particular, once the camera is built (at least the current type), we don’t get the option of choosing non-uniform tradeoffs across the image. We get some spatial resolution and we get some depth resolution that is the characteristic of the camera, then all we can do is to degrade it computationally for effect, always throwing away the bulk of the data in each computation, but having the flexibility to choose what to throw away. In a conventional camera, the optical resources are deployed differently. We could obtain high spatial and depth resolution around one particular depth and decaying spatial and depth resolution at depths away from that. Or we could obtain high spatial resolution at all depths, but no depth resolution. Or something in between. The choices are limited but given it is what we actually want, the data is combined fully in the desired way. It is only when we think we are mistaken do we regret that information has been “lost” (but really just combined in the “wrong” way.) Because a conventional camera has evolved for 100+ years, in capable hands it is kind of matched to exactly what you want to to do in photography. Of course you can’t do weird things like having two depths at which things are focused, but that is also weird in a way that our (conventional) eyes are not used.

Then this brings us to the question of what is photography. Is it a reproduction of the physics of radiation? Then a plenoptic camera isn’t enough, one would need a hyperspectral holographic recorder of some sort. But it isn’t that. Is it to reproduce what our eyes see? It isn’t that, either, because photographs are somewhat realistic but surely not real. With more and more in-camera and post-processing gimmicks, real may even be unacceptable. Is it just free painting with a really complicated and constrained brush — in other words, masochism? No, isn’t that, because then no interaction with the real world input would be required. The meaning of photography, so far as I can see, is an art which not unlike other arts, is the communcation of an emotional state. But this emotional state is defined by that which the photographer experiences at the acute moment when a particular real world phenomenon (the scene) is observed. It’s what the photographer imagines is seen that he tries his best to manipulate his instrument to reproduce, but this imagination is seeded irrevocably by the transient real phenomenon. Anything else, he could just patiently paint from scratch; but for this, the imagination seeded by the transient real phenomenon, he somehow needs photography.

But does he need post-processing? Does he need a fully capable plenoptic camera so he can do post-processing? Is that still true to the moment or is it okay to have the emotional state last a while (and possibly change) all the way back to the photoshop computer? Heck, why not have photoshop onboard? That would free the imagination. I lean towards yes, if only for the reason that more options can’t hurt. But there is unresolved tension on this question. A camera with low capabilities can have its quirks and constraints become part of the real phenomenon, if the photographer is presumed to feel the world through its viewfinder. Is this a bit recursive? Once the instrument has too much software, then what is to separate photography from virtual brush painting in its onboard photoshop?

So being at the event captured in the image, I got to ask a question toward the end. Actually I asked two questions. The first was whether Watson would ring in and use the remaining 3 seconds or whatever to continue to compute. Gondek said it would if it helped. In actual competition it doesn’t appear to be the case, as the buzz-in thresholding condition ensured that further computation would not have been helpful. The second question was a follow-up on the identified weakness of Watson — learning on “common sense” knowledge. I asked what path AI research would take to tackle such knowledge, which are by its very definition, “not in the books.” Gondek said that IBM is building up semantic information (e.g. a “report” is something that can be “turned in” and “assessed,” etc.) from corpus. That wasn’t exactly what I was asking, however.

My point was whether all “knowledge” is written down. There is such a thing as experiential “knowledge,” and humans take years to learn it/be trained in it through parenting (i.e., to “mature”). If only there were a handbook on life, or that life could be learned through reading a series of textbooks, then perhaps I’d believe that the kind of general-purpose AI that most people are probably imagining (rather than an expert/Q&A system) can be achieved along the lines of current methods.

You may keep getting

Enter your WordPress.com API key to link this blog to your WordPress.com account. Be sure to use your own API key! Using any other key will lock you out of your stats.

even if the API key is correct. or if you hard code the API key

An API Key is present in the source code but it did not work.

The WordPress.com Stats Plugin is not working because it needs to be linked to a
WordPress.com account.

I didn’t find a solution in any of the forums, so I looked at the stupid script some more. Basically it uses the API key to get a “blog_id” (database index, most likely) from WordPress.com and can’t find one. So I made up a blog_id in the code. That shut up the plug-in, but of course stats aren’t tracked.

Finally, I went to http://dashboard.wordpress.com, logged into the account, made a new garbage *.Wordpress.com blog, then a bit later took out the made-up blog_id from the code, de-activated and re-activated, and … everything works. The external blog shows in the “Global dashboard.” Also the real blog_id is returned from code. But, if I take out the hard coded API key, it stops working again.

This is definitely a WordPress.com problem with registering externally hosted blogs, so to make it work, hard code the API key, make sure there is at least one *.Wordpress.com blog, wait a little bit, then re-activate the stats plug in.

and statistics

Hmm…

The answer of course is in the form of an axis of rotation preference. Rotation about the bodyline axis (left right flip) and rotation about waist axis (top bottom flip) are not equivalent. But let’s first be clear that the mirror does treat them equivalently in terms of optics, so that’s not where the non-equivalence comes from. In fact, it is futile to think too much about the mirror.

Instead, let’s think about the person. When you physically turn around to face the other way, you make a choice to rotate around one particular axis, and not the other. That’s a very obvious breaking of equivalence. When you look at somebody else or at your image in the mirror, you are likewise choosing a preferred axis — except in your mind. You make a mental rotation and that’s where the equivalence is broken. That’s why left and right flip but top and bottom do not, because your mind chooses to do the left-right flip and not the top-bottom flip.

But why does the mind choose one rotation over the other? Because one can be done, and the other can’t. Left and right in the body plan are symmetric, so the image seen in a mirror makes “sense” as a rotated image along the bodyline axis. It would not make “sense” as a rotated image along the waist axis, because head and feet are not symmetric.

On a related but different matter, if you look into a lake, you see a top-bottom flip. That’s perfectly normal because the plane of mirroring in this case is itself preferential. In this case, it is a purely optical equivalence breaking. So far so good. But you are somehow aware that the image in the lake is abnormal (i.e. upside down). Now if you face a wall mirror sideways, you have likewise a left-right optically flipped image. But nothing seems abnormal there. What is it about upside down that is abnormal? Well, it has to be due to the external reference of updown-ness known as gravity. In fact, if you stayed in the space shuttle long enough, I bet seeing an upside down person or image would no longer seem at all weird.

I got a chance to use Ghost again. Ghost 10.0 that is. Unbelievable! What a piece of crap! I just wanted to image a disk, but now you have to run the ugly yellow UI in Windows — wait, you have to first install it in Windows so it can “help” you “automatically” “define” “restore points” so you can “backup your computer.” What does that user-fuddy gibberish mean?! Oh look here, I can be an “advanced” user and make a straight disk-to-disk copy (no disk to image?) but every time I click the button it wants to install .NET Runtime 1.1 first, what the …? And it keeps wanting me to activate the product and get “LiveUpdates.” Umrghh! Booting the CD up by itself gives me a patchy “recovery console.” No option to image disks in sight. Needless to say I junked the CD.

Fortunately the package tucks in another CD called “Ghost 2003″ for “older” computers. So it turns out Ghost 2003 is the Ghost that I remember. Man, thank goodness for older computers… Snorton has totally killed Ghost. Caveat emptor.

Hills, yes.

Yesterday I drove up a fairly steep hill called Phinney Ridge (really quite steep, but not super steep by Seattle standards). I was also going west, so suddenly I was reminded of this geographic and geologic fact of Seattle and thought … “Yeah, I must be hitting a large gradient against one of these hill-spines. Gee, it’s even called a ‘ridge.’ Wonder if I can skirt around it,” and so on.

In fact there are lots of descriptive place names that didn’t register with me — like, what is “Capitol Hill” anyway? — until I saw this image. I cut it out of the USGS data site, where you can play around with such things to your heart’s content. (I also took two looks at topozone.com but decided they sucked.)

Seattle is supposed to be built originally on “seven” “hills,” (c.f. Rome) and I labeled them here in red numbers 1-7: First Hill, Capitol Hill, Queen Anne, Magnolia, Beacon Hill, Denny Hill (razed early on), and Crown Hill. I call BS on the seven hills theory. Not only do some of these “hills” not look like distinct hills, some of them aren’t even that impressive. There are lots more hills around … and Crown Hill looks sorely out of place, like an after-thought to make the number 7.

I haven’t really been everywhere in Seattle, so I can’t say where the hills are the steepest — they probably keep records of this. Where I have been though … some parts of the eastern ridge of Capitol Hill (green 1), the Downtown (green 2), and Magnolia have made for hair-raising experiences in a manual transmission car. By comparison, Phinney Ridge (green 3) really isn’t so bad.

The only really flat part of Seattle is the industrial/stadium/international district along the shores of the Duwamish River, which can be seen in the image, flanked by West Seattle and Beacon Hill. Seattle could have developed along the only river in the city: Seattle was almost called Duwamps, after all … but no, people had to go live on hills instead of flat land (yeah ok fine, there was too little of it and it was an Asian ghetto from the days of Chin Chun Hock).

Part 6.

Today I can go no farther (so I stopped)

The last days of this project are spent on two tiring tasks that do not gain me very much, but must be done to carry this project to its logical conclusion. One of these is to decrypt a few very small, but important NTFS-encrypted files. The other is to wring the last readable bits out of the broken Seagate drive by splitting all the error regions to isolate the unreadable regions as much as possible. These can proceed in parallel.

I set Knoppix to slave away on the broken Seagate drive using ddrescue again. This ended up taking more than one day constantly running and made the disk really hot. I had to manually intervene from time to time just so this will get somewhere. This is where I wished ddrescue was smarter about skipping around unreadable regions.

That is the end of it. The story has a happy ending, but what a giant waste of time. Back up your data.

Lessons today:

• AEFSDR is nice. But I wouldn’t buy it, either.
• Remove your private keys from the computer.
• Near total data recovery is very possible from an apparently dead drive, but a delicate procedure needs to be followed.
• The right tools are very important, and a complete disk recovery toolchain simply does not exist — you are left to find your own pieces.

On to the Appendix.

Part 5.

The tide turns (rather quickly)

After the exceedingly annoying but ultimately inconsequential ext2 interlude, I’m back on track with the original problem of recovering data from the broken Seagate drive.

After the disaster with file-copying using the Windows ext2ifs driver last time, I made sure to make a copy of the disk image while the external USB drive holding it was mounted under Knoppix Linux. The destination was another external drive formatted with ext2.

There is a lot more to do and this project is far from over, but I can see the end-game now, and so the recovery effort can be declared a success.

Lessons today:

• MIP is pretty useful, but I still won’t buy it.
• NTFS is truly robust, and the NTFS version of chkdsk is remarkable.

On to Part 6.

* Edit: I’ve since found a free tool called vdk that makes Mount Image Pro obsolete. Yes, it mounts ddrescue images. Yay!