While I've been too busy to post, hopefully you've taken a gander at the various DNI RSS feeds. Right? Well, if you haven't been keeping up, the HCI/interface world has been a non-stop flurry of exciting and interesting innovations.
Why all the excitement? Three things.
One, battery technology and device efficiency have gotten to the point where rich media on portable devices doesn't drain the power before you're done thinking, "Hey, I'd like to experience some rich media content right now?"
Two: The lines between every traditional computer category are collapsing all around us. Your phone is a slim client, your ebook reader is a netbook, your htpc is a nettop, your desktop is always connected and serving up content, up is down, down is left, cats and dogs are living together! And all this requires that we think a little differently about how we handle, view, sort, and access our delicious data.
And Three: Ohhhh how much data we have! And it's cheap to store. And for the price of a 14" CRT in 1993, you can grab a high def 60" mega-monitor (okay, you'd have to get it second hand and last year's model, but still...) In other words, we have more sand and bigger sandbox to play in.
So, I underestimated the amount of work leading up to my prelim defense, and going to start my little break now-ish. It isn't the prelim work as much as the other stuff, like my little Thunderbird on a USB stick took forever to configure perfectly, and was then corrupted by an old version of TrueCrypt hiding on one of my computers. *poof* All that work disappeared. Ugh. Not sure how to handle that now, but might have to think up a little hack to make it work properly.
In the meantime, here are a few stories...
Serious: 60 Minutes covered DEKAon Sunday. Thank you to the 20-ish people that let me know about it, including my parents. They're silly. They still live in a world where missing a TV show means you have to wait for the rerun. Video clip is up on the 60 Minutes site.
Pravda has a very vague and almost not worth reading article on a Russian BCI group.
The Boston Globe discusses some of the hurdles of setting up an implantable BCI project, and chronicles the situation over at the now defunct Cyberkinetics. I don't want to say anything about the project that could anger one of the 14,000 government institutions regulating aspects of it (kidding on the number btw), so I will point to here (yes, the page is in a very very early alpha state).
And some sort of licensing setup for ECoG algorithms has been setup with a company called Neurolutions. (Really? Neurolutions? Have we gotten to the point where all the good brain related names are taken? Even DNI, which was a 3am, caffeine-induced hyper-mania induced naming beats the snot out of that.)
Seriously?: I could bring myself to watch this clip, but here's the title: "Robotic baby seal has healing powers" - CNN (also known as, "We fired out entire science department, but what could go wrong?" News) Video clip. Be sure to get the tshirt, too.
Of course almost all TED talks are worth noting, but three are particularly rocking. The first two are by Aimee Mullins. The first is more of a show and tell Q&A session, discussing the various legs she has and the stories behind them. The second is a formal talk with an interesting story about her interactions with children.
The third video, featuring Pattie Maes, discusses what her lab calls a "sixth sense" - a combination camera and projector system and is location and context aware. If you've seen some of the MIT 'cyborg' types before, think of this as the iPod-ization of that concept. Though I don't really consider this a new 'sensory' experience, the demo is pretty amazing.
What a sweet looking little interface on the upcoming BenQ MID.
No, I am not a Linux user, and especially not a fanboy/fanboi, but from an interface design perspective, Linux is poised to become the clear winner of the alternative interface wars. The reasons are simple.
A) No corporate identity. If a Linus distro wants a certain branding-type identity, someone just branches the distribution and codes away. The roots of this 'feature' are in the design of X Windows and the various window managers, like KDE and Gnome, and the philosophy that the interface is an extension of the underlying commands in the system. Don't like that Apple menu bar at the top? Sick of a certain Start button? Too bad. Millions have been spent on figuring out how big to make the menu fonts, the order of each item, etc., and with widespread adoption of Windows and MacOS in their iconic interface form, probably 100x's as much money is spent on making sure version X.5 looks and operates similar enough to version X that people won't be calling tech support and businesses don't need to retrain employees - a huge time/money sink, and one that is almost never budgeted.
B) The unstoppable march of the digital device swarm. While the popular WMs are still 'meh' in many respects, they have also been competing with highly organized, profit driven companies with millions of dollars spent on consumer research. People generally don't go to Linux because it is particularly easy to use of good looking, but now that companies are starting to put dollars behind Linux interface design (due to it widely being recognized as the Achilles' heel of Linux) the money factor is about to change. Netbook, MID, smartphone, and specialty computer makers are realizing that they don't need all the library overhead and background processes used in a modern desktop OS on devices meant for stereotyped tasks. The result? Better battery life (fewer computations, simpler computations incorporated into low level, modern instruction sets, less disk access), faster performance, and fewer errors/crashes/conflicts.
C) Multimedia barrage. I remember downloading MP3s back in 1997, while at college. You better believe that once I found a patch cable long enough to reach my stereo, my room was THE party room in the dorm. Pissed off that smug bastard with the 100-disc CD changer, too! But, seriously, if there is one positive outcome from the RIAA and MPAA* suing instead of innovating, it is that media formats have remained mostly the same. Good thing they spent years and tens to hundreds of millions of dollars to find out people want to play the music they buy on any device they choose (give Apple time - they'll pack it up soon enough), because the encoding and decoding to the few popular codecs used now can now be done in hardware. Before this stagnation, it made less sense for, say, NVidia to put mpeg decoding in hardware, or at least to repeatedly refine the ability in order to consume less power and operate with better stability. Same goes for software. Coders have been given the time to optimize, understand, and tweak media decoding. Sure there are a couple dozen different codecs floating around, but that could have been much worse**. Let's not forget that the whole open source software movement was also figuring out how to work out collaborations among many programmers in sparse locations at the same time. The result is that we now have cell phones running full motion video, etc. On the Linux side, this has translated into valuable time to support modern media formats (attract media type programmers to a decidedly computer-y OS, work out revisioning logistics, create a legal base/consciousness within the community). Without the benefit of corporate talks/alliances/contracts, OSS needed to implement media-centric features in a stable and reliable fashion. That has been largely accomplished.
* - To their credit, the MPAA has been MUCH more reasonable on this front, which is why I still go to movies and pay for rentals, but have not purchased a new CD or a paid music download in at least the last 6 years. I've contributed to artists through secondary means, like concerts, buying games they have tracks on, and snatching up any DVD-Audio they might possibly release. ** - Innovation on the media front has been largely squashed by acceptance from IP holders, once they figured out people were copying content for free. Any new attempt to make a better codec would run the risk of be legally hammered into oblivion if it gained a sizable user base, mired by copy protection at the insistence of IP holders, or developed far enough for the owners to sell it off, get their money, and run. The legacy codecs we use are there only because corporations acted so remarkably slowly and then started a legal shit-storm for anyone associated. When was the last time you saw someone distributing a product for free to millions of people, customized to their exact tastes and with immediate access, and it took the IP holders years to even recognize its existence?
D) Dr. Halo or How I stopped worrying about my child's social life and learned to love the video game. And, thank video games for unstoppable march of realism and complexity in 3D environments. Unlike video, which has several targets (movie, transmission, streaming, local playback, etc.), video games have had relatively few, making hardware the limiting factor. We can argue about changing 3D standards helping this, but the mantra of "if you're not maxing out the hardware, you're not taking full advantage of it" has egged developers on. Video games also pioneered the HUD, menu system, navigation nuances, and many foggy cognitive features of human-computer interaction that we probably won't even scratch the surface of over the next 50 years. While 3D support for Linux is still largely dependant on how nice the hardware folks was to be about releasing source code (which varies greatly), basic 3D operations have been removed from proprietary APIs and placed in lower level, quasi-auto-detected interactions of hardware and software (instruction sets), optimized at the library level, and generally more 'universalized'. So, while specular mapping from 8x anisotropic filtered internal reflections on bump mapped surface of a volumetricly rendered light source might not be in the cards right now, fade rectangle and rotate isn't a big deal, and that's all you need to a neat-o menu effect.
E) A new breed of 'digital ready' people. 97% of school children play video games. 1% of incoming freshmen at colleges have land lines, while 99% have cell phones. 17% of US homes overall (including mine) have only a cell phone. How many high school students do you think have ever seen a hand-cranked window in a car? And for the parents, remember when YOU discovered Playboy/Playgirl? Yeah, imagine now. The biggest change in culture between the PC of 1995 and today is the user, and this is arguably true for any type of significant tool at the timescale of generations. We evolve with the tools we produce because of the tools we produce. This rule applies to culture as much as it applies to the mess of neural networks in our heads.
Younger folks, and I like to include myself as one, just 'get' electronics. We can fuck with a car stereo for 30 seconds and, through some ingrained, trial and error process, figure out how to set the clock, add favorite radio stations, and contact the International Space Station. Whether it is Windows, Mac OS, Linux, BeOS, AmigaOS, or whatever, as long as some thought was put into the interface, we're (mostly) fearless. Yes, grandpa, we know all about how efficient the command line was for punch card computing. Unless you are actually interested in computing, no one cares. This is where the Mac has generally shined, but mostly because it was only compared to Windows, and generally from the perspective of someone with enough familiarity with both to know that both tools would accomplish the tasks they needed a computer for.
Once this seemingly innate understanding of computing interfaces is understood, it will be implemented in niche devices. Right now it is mostly trial, focus group testing, and error, which has taken us far enough that device makers for low adoption devices to get some pretty slick interfaces. Once we have better rules and theories about the Generation X (or Y or whatever this generation is called) specific differences in HCI, it will be easier for those closest to the goal and with smaller user bases to adapt. Those devices are largely Linux based.
Anyhow, don't wipe your PC yet. I'm not talking about desktop computers, and those are likely to run a completely different course. But, don't shy away from other devices because they happen to have an unfamiliar interface. You might just find it more usable than your *ack!* iMac!
Mark my words: Video games will be the biggest driving factor outside of traditional research for the BCI field. You have at the same time fine and robust input control, reward, motivation, community, and accomplishment all wrapped up in a neat little package. Add to that art, music, plot, and history, and you have a task that not only seems rewarding, but is also life enriching (things that might appear to be lacking in Pong or Super Mario Bros.).
So, I will be adding bits and pieces of video game news, as they pertain to interface and assistive technologies. If you don't like it, too bad. You'll thank me later. (Ask the people that didn't believe me in 1995 when I said the most important advancements in the internet will not be in 'finding' information, but 'filtering' it.)
First, we have One Thumb to Rule Them All, linked from Gizmodo. An inspiring story about a writer and gamer who plays mainstream games with only his thumb (due to spinal muscle atrophy).
Next, we have TecEBlog, with a mini-survey of various alternative input devices, like EOG, EMG, BCI, tongue, etc. Nice little gallery of videos.
And finally, the latest in non-neural, neural control of games, the OCZ Actuator. This has been making the rounds since the press release, and there are plenty of videos on YouTubeif you're interested. If you watch carefully, you'll note that the player's muscle movements are driving the game.
Still purging my old news items worth noting. So here we go...
Human 2.0, H2.0, H2O, whatever you want to call it, it is undeniably cool at all levels. Human 2.0 is a yearly event at the MIT Media Lab which highlights the latest in Human-Machine Interface in the broadest sense. Unlike other such events, H2.0 is actually based on current research and not far flung exaggerations of where things could lead 50 years out. Somewhat 'looser' than other scientific conferences, the event serves as an opportunity to let the imagination run a little further than usual. Wired has a mini-interview with Frank Moss, as well as its own coverage.
Doug Smith at U Penn is working on a way to coax neural processes over long distances - up to 1 cm per day. This is only mentioned in New Scientist, and after looking at the lab site and pub med, I can't find any other info on it. If you have more info, please post it in comments.
SCLin's blog has a nice thread on keeping up with the literature. He does the same thing I do, for the most part, but one of the comments to the article is a nice little gem. This link to an example of how to use Yahoo Pipes to filter out good articles is really sweet. I'll have to work up to something like that when I revise my current method.
The goddess of neuro blogs at The Retrospectacle pointed out that Joseph LeDoux has a band that sings "nerdtastic" songs about brains. The Amygdaloids. Enjoy! Worth noting from the comments: This and this.
Gizmodo had a post on a 3D mouse the size of a ring which could be used to control a computer cursor using standard mouse protocols. $155 worth of parts. The device uses an ultrasonic sound transmitter which is detected by five ultrasonic microphones to interpolate the ring's position. Not too shabby for a bunch of students, and I bet the power requirements are reasonable!
There are plenty of theories for how information is storied in neocortex. Neurobot discusses Hierarchical Temporal Memory (HTM) work being done as part of the Numenta project. Groovy.
The Neurocritic has a post on DBS in the Nucleus Accumbens for the treatment of depression. I heard a speaker discuss this at the NIH BCI workshop, and the prospects sound amazing.
And finally, there is a monster-long PsyBlog series on non-verbal communication and gestures. This is one of those 'bookmark it now, read it in pieces later' type pages.
And one bonus HCI item. Fresh from YouTube, and complete with description, here's a little machine vision type ditty where hand configurations seen by a webcam become cursor control.
I'm being super productive today, so I'm allowing myself a few extra entries.
Earlier last year, during a meeting with one of my advisors, I mentioned that the push to use cortical implants to control a cursor worried me a little. Sure the brain is plastic, and that plasticity combined with the inherent coding properties of motor cortex should result in decent control of a 2D cursor, but it was still 'unnatural'. The people involved in many tried of BCI devices are adults - well beyond the so called 'critical period' (a controversial term, I know). My guiding philosophy is that the interfaces of tomorrow will appeal more and more to the distinct advantages of our biology. That is, future human-computer interaction will become more and more organic and complementary to uniquely human traits. This is why the need to create more able-bodied types of interfaces, like prosthetic limbs, is much more important. technology sure isn't going to be a sitting target, and we need to anticipate the direction of its enhancement (no, I don't think a simple velocity filter will work here :) ).
So, what am I getting at. Last night, Microsoft announced "Surface". It is arguably the coolest thing I have seen in computer interfaces to this day (and remember I'm in the tech biz as well, so that should carry some weight!) One problem: Current cursor based navigation is nearly worthless. Sure, it won't replace the desktop PC anytime soon, but it is a sign of what is to come. Cursor navigation is extremely important, and a much easier target to hit than full limb control, don't get me wrong, but sometimes advances like Surface make me take a step back until the big picture is back in focus.
Enough pondering... Back to reading Fetz papers...
