A week away

So, I underestimated the amount of work leading up to my prelim defense, and going to start my little break now-ish. It isn't the prelim work as much as the other stuff, like my little Thunderbird on a USB stick took forever to configure perfectly, and was then corrupted by an old version of TrueCrypt hiding on one of my computers. *poof* All that work disappeared. Ugh. Not sure how to handle that now, but might have to think up a little hack to make it work properly.

In the meantime, here are a few stories...

Serious:
60 Minutes covered DEKA on Sunday. Thank you to the 20-ish people that let me know about it, including my parents. They're silly. They still live in a world where missing a TV show means you have to wait for the rerun. Video clip is up on the 60 Minutes site.

This best computer interfaces story is a trip down memory lane.

Pravda has a very vague and almost not worth reading article on a Russian BCI group.

The Boston Globe discusses some of the hurdles of setting up an implantable BCI project, and chronicles the situation over at the now defunct Cyberkinetics. I don't want to say anything about the project that could anger one of the 14,000 government institutions regulating aspects of it (kidding on the number btw), so I will point to here (yes, the page is in a very very early alpha state).

And some sort of licensing setup for ECoG algorithms has been setup with a company called Neurolutions. (Really? Neurolutions? Have we gotten to the point where all the good brain related names are taken? Even DNI, which was a 3am, caffeine-induced hyper-mania induced naming beats the snot out of that.)

Seriously?:
I could bring myself to watch this clip, but here's the title:
"Robotic baby seal has healing powers" - CNN (also known as, "We fired out entire science department, but what could go wrong?" News)
Video clip. Be sure to get the tshirt, too.

TED talks worth noting

Of course almost all TED talks are worth noting, but three are particularly rocking. The first two are by Aimee Mullins. The first is more of a show and tell Q&A session, discussing the various legs she has and the stories behind them. The second is a formal talk with an interesting story about her interactions with children.





The third video, featuring Pattie Maes, discusses what her lab calls a "sixth sense" - a combination camera and projector system and is location and context aware. If you've seen some of the MIT 'cyborg' types before, think of this as the iPod-ization of that concept. Though I don't really consider this a new 'sensory' experience, the demo is pretty amazing.

The upcoming rise of Linux

What a sweet looking little interface on the upcoming BenQ MID.



No, I am not a Linux user, and especially not a fanboy/fanboi, but from an interface design perspective, Linux is poised to become the clear winner of the alternative interface wars. The reasons are simple.

A) No corporate identity. If a Linus distro wants a certain branding-type identity, someone just branches the distribution and codes away. The roots of this 'feature' are in the design of X Windows and the various window managers, like KDE and Gnome, and the philosophy that the interface is an extension of the underlying commands in the system. Don't like that Apple menu bar at the top? Sick of a certain Start button? Too bad. Millions have been spent on figuring out how big to make the menu fonts, the order of each item, etc., and with widespread adoption of Windows and MacOS in their iconic interface form, probably 100x's as much money is spent on making sure version X.5 looks and operates similar enough to version X that people won't be calling tech support and businesses don't need to retrain employees - a huge time/money sink, and one that is almost never budgeted.

B) The unstoppable march of the digital device swarm. While the popular WMs are still 'meh' in many respects, they have also been competing with highly organized, profit driven companies with millions of dollars spent on consumer research. People generally don't go to Linux because it is particularly easy to use of good looking, but now that companies are starting to put dollars behind Linux interface design (due to it widely being recognized as the Achilles' heel of Linux) the money factor is about to change. Netbook, MID, smartphone, and specialty computer makers are realizing that they don't need all the library overhead and background processes used in a modern desktop OS on devices meant for stereotyped tasks. The result? Better battery life (fewer computations, simpler computations incorporated into low level, modern instruction sets, less disk access), faster performance, and fewer errors/crashes/conflicts.

C) Multimedia barrage. I remember downloading MP3s back in 1997, while at college. You better believe that once I found a patch cable long enough to reach my stereo, my room was THE party room in the dorm. Pissed off that smug bastard with the 100-disc CD changer, too! But, seriously, if there is one positive outcome from the RIAA and MPAA* suing instead of innovating, it is that media formats have remained mostly the same. Good thing they spent years and tens to hundreds of millions of dollars to find out people want to play the music they buy on any device they choose (give Apple time - they'll pack it up soon enough), because the encoding and decoding to the few popular codecs used now can now be done in hardware. Before this stagnation, it made less sense for, say, NVidia to put mpeg decoding in hardware, or at least to repeatedly refine the ability in order to consume less power and operate with better stability. Same goes for software. Coders have been given the time to optimize, understand, and tweak media decoding. Sure there are a couple dozen different codecs floating around, but that could have been much worse**. Let's not forget that the whole open source software movement was also figuring out how to work out collaborations among many programmers in sparse locations at the same time. The result is that we now have cell phones running full motion video, etc. On the Linux side, this has translated into valuable time to support modern media formats (attract media type programmers to a decidedly computer-y OS, work out revisioning logistics, create a legal base/consciousness within the community). Without the benefit of corporate talks/alliances/contracts, OSS needed to implement media-centric features in a stable and reliable fashion. That has been largely accomplished.

* - To their credit, the MPAA has been MUCH more reasonable on this front, which is why I still go to movies and pay for rentals, but have not purchased a new CD or a paid music download in at least the last 6 years. I've contributed to artists through secondary means, like concerts, buying games they have tracks on, and snatching up any DVD-Audio they might possibly release.
** - Innovation on the media front has been largely squashed by acceptance from IP holders, once they figured out people were copying content for free. Any new attempt to make a better codec would run the risk of be legally hammered into oblivion if it gained a sizable user base, mired by copy protection at the insistence of IP holders, or developed far enough for the owners to sell it off, get their money, and run. The legacy codecs we use are there only because corporations acted so remarkably slowly and then started a legal shit-storm for anyone associated. When was the last time you saw someone distributing a product for free to millions of people, customized to their exact tastes and with immediate access, and it took the IP holders years to even recognize its existence?

D) Dr. Halo or How I stopped worrying about my child's social life and learned to love the video game. And, thank video games for unstoppable march of realism and complexity in 3D environments. Unlike video, which has several targets (movie, transmission, streaming, local playback, etc.), video games have had relatively few, making hardware the limiting factor. We can argue about changing 3D standards helping this, but the mantra of "if you're not maxing out the hardware, you're not taking full advantage of it" has egged developers on. Video games also pioneered the HUD, menu system, navigation nuances, and many foggy cognitive features of human-computer interaction that we probably won't even scratch the surface of over the next 50 years. While 3D support for Linux is still largely dependant on how nice the hardware folks was to be about releasing source code (which varies greatly), basic 3D operations have been removed from proprietary APIs and placed in lower level, quasi-auto-detected interactions of hardware and software (instruction sets), optimized at the library level, and generally more 'universalized'. So, while specular mapping from 8x anisotropic filtered internal reflections on bump mapped surface of a volumetricly rendered light source might not be in the cards right now, fade rectangle and rotate isn't a big deal, and that's all you need to a neat-o menu effect.

E) A new breed of 'digital ready' people. 97% of school children play video games. 1% of incoming freshmen at colleges have land lines, while 99% have cell phones. 17% of US homes overall (including mine) have only a cell phone. How many high school students do you think have ever seen a hand-cranked window in a car? And for the parents, remember when YOU discovered Playboy/Playgirl? Yeah, imagine now. The biggest change in culture between the PC of 1995 and today is the user, and this is arguably true for any type of significant tool at the timescale of generations. We evolve with the tools we produce because of the tools we produce. This rule applies to culture as much as it applies to the mess of neural networks in our heads.

Younger folks, and I like to include myself as one, just 'get' electronics. We can fuck with a car stereo for 30 seconds and, through some ingrained, trial and error process, figure out how to set the clock, add favorite radio stations, and contact the International Space Station. Whether it is Windows, Mac OS, Linux, BeOS, AmigaOS, or whatever, as long as some thought was put into the interface, we're (mostly) fearless. Yes, grandpa, we know all about how efficient the command line was for punch card computing. Unless you are actually interested in computing, no one cares. This is where the Mac has generally shined, but mostly because it was only compared to Windows, and generally from the perspective of someone with enough familiarity with both to know that both tools would accomplish the tasks they needed a computer for.

Once this seemingly innate understanding of computing interfaces is understood, it will be implemented in niche devices. Right now it is mostly trial, focus group testing, and error, which has taken us far enough that device makers for low adoption devices to get some pretty slick interfaces. Once we have better rules and theories about the Generation X (or Y or whatever this generation is called) specific differences in HCI, it will be easier for those closest to the goal and with smaller user bases to adapt. Those devices are largely Linux based.

Anyhow, don't wipe your PC yet. I'm not talking about desktop computers, and those are likely to run a completely different course. But, don't shy away from other devices because they happen to have an unfamiliar interface. You might just find it more usable than your *ack!* iMac!

Interface design

Oh, sweet, sweet procrastination. I have defragged my hard drive, run a virus scan, reindexed all my files, but there is always you, Internet. You are my favorite source of procrastination. Uh, (cough) sorry.

(Edit: While typing this, you might think I just had an idea shot down or a frustrating meeting. that's not the case. The articles linked toward the end were the thing that inspired me to write this.)

When designing tasks, especially for human types, it makes sense to take a step back and make sure you covered the basics. Too many times I see task design go to committee when all that does is soak up time and energy. There are two parts to any interface design: Style and Technical. Style is that inexplicable, non-critical bias we all have. Technical ensures that the task answers the intended question. This all goes awry when we start nitpicking. By nitpicking, I mean minor changes based on loose evidence that happens to be significant in some cases, but not in the one in question. I strongly, strongly, strongly urge all labs involved in interface design for behavioral tasks to do two very simple things:
1) Limited the number of people designing the task to 3 at the maximum.
2) Make a checklist that every task must pass.

Three people? Yes, well no. Two people! The process goes like this:
START with style. The 'creative' person (who also implements the task) should design the interface. The 'technical' person should ensure that it answers the question. I'm assuming a lab situation where no one is an idiot, so the technical person is really scrutinizing the actual science and comparing it to other similar tasks with a more critical eye wile the creative person is more concerned with fitting it into their overall scheme/study. The creative person should be the less experienced, the technical a post-doc or PI. Any concerns pointed out by the technical should be handled by the creative in whatever way they see fit. The result should be discussed between the two and then presented to the third, who has had no influence on the design. This meeting should be no longer than 30 minutes.

So, it goes: Creative -> Technical -> Creative -> Technical/Creative -> Outside Confirmation (-> back to Creative/Technical if significant issues identified)

You may say, "That's what we do!" I doubt it. Having worked in five labs and three companies, no one has an actual process. It generally begins with a discussion in a group where personal bias is introduced and emphasized before the actual task or interface creation. Advice is sought by multiple people, sometimes because of expert opinions or a wish to collaborate with a specific person, which means the nice linear design above branches by N! times as each change and objection is propagated throughout all involved members.

It is still important to get everyone's feedback and make sure other lab mates know about the tasks, so presenting it to others is still a good idea (isn't that partially what lab meetings are for?), but when presenting, do the following:
- Any idea that doesn't relate or has previously been discussed between the group of three should receive a, "that's a good idea, I'll look into it" response. Do not explain, do not argue, and at most ask why that's significant. You do not need to rehash the last 4 months of interface design to someone that wasn't involved. They were interested, they had a good point, thank them, maybe rehash in your head what your previous conversations were about concerning it, and move on.
- Any undiscussed ideas or unknown sources ("The X lab did something like that, but did X(i)") should be written down and assigned a value 1 to 3. 1-not important, 2-medium or UNKNOWN importance, and 3-Crap, should have thought of that. Address those issues in that order. Address the #3 issues immediately after the meeting and look up the info for unknown sources. make sure all three people have assigned the same values to the identified issues and leave the creative/subordinate to iron them out. Repeat the linear approval process above once these new issues have been resolved.

You are now done. Sounds long, but it really is about 1 week. Each step should take around half a day so you have time to mull over the info. Don't throw things up the line that aren't done, don't meet in between to discuss anything other than the vital issues and any possible implementation problems. The creative person is the one most vested in seeing the task done right, and will make sure the design goes through.

This is how businesses work when they hire designers. The designer(s) (usually 1-3 people), meet with the client, find out who they are trying to attract, get enough technical details to know what the business is and is not, leave and take time without any company involvement until the next formal meeting, meet to present several sketch ups of the final product, return to the studio to alter the chosen mock up to the specifications, call/email/fax the final design, get the approval, and make the widget. For bigger things, like whole websites with many categories, each meeting will discuss a specific subcategory after an initial unifying design meeting (similar to a student meeting with the PI to discuss his/her thesis work).

In order for this linear design to work, each hand off of the task should be preceded by a checklist.
A) Who else has done this task? What were the major problems and are they more or less a concern for you?
B) It is technically and financially feasible?
C) What about the currently available setup needs to be changed?
D) Does it conform to the following (what inspired me to write this whole babbling article):
http://www.componenthouse.com/article-21
http://www.componenthouse.com/article-22

LAST) Does it still answer the questions you are looking to ask?

Alright. Back to work.

Webcam multitouch

I can't believe I didn't post this already. Based on Remy's comment on the Wiimote interface, here's a similar thing done with a standard webcam by a Microsoft researcher.



And while I'm at it, might as well post this Minority Report type interface using the Wiimote as well. I have to say that the Wii has inspired some crazy-awesome interface hacks that get me excited about what will soon be available. Hey, the hardware and drivers are there, the manufacturing is in place (and cheap!), so all we need is someone to tie it all together. And give me 20%.

Two interface briefs

First, the more 'boring' one. BBC reports on auditory interfaces, and using objects to emit sound with function. Cool? Sure. But, I'm not sure what it would be used for.

Second, and totally awesome, an independent programmer has turned the Nintendo Wii remote into a head tracking device. The result is absolutely incredible in both its simplicity and resulting quality. Watch the video. You can skip to 2:05 if you don't want the background info. Stuff like this makes me giddy about the innovative uses for simple technology.

Sight and sound

Alright, Super Paper Friday will be tomorrow. I don't have the attention span to compile the whole list right now (about 1/3 done). Instead, here are a few stories...

An interesting idea here. Glasses that have embedded microphones and with attached hearing aids which amplify sounds that the wearer is looking toward and reduce background noise. Not BCI, but neat-o concept. (@ Medgadget)

Again, not BCI, but interface-y, the new JSF (Joint Strike Fighter) helmet prototype "augments skills and the senses". That's all anyone is saying right now. That and that it makes everyone pee themselves in fear. (@ Gizmodo)

And lastly, what robot programmers do when they get bored...

Head typing

I don't have all the details, but here's a link to a link regarding a keyboard/mouse hybrid combo that can be controlled by head position. I thought it was worth mentioning (kinda means this is becoming a BCI/neuroscience/assistive devices blog, which I have to qualms about).

It's an MFin' multi-touch extravaganza!

Yet another reason that full movement restoration is necessary for the paralyzed. As technology advances, the interfaces will become more and more geared toward uniquely human forms of interaction. Controlling a cursor doesn't mean much if it goes the way of trackballs. (Okay, I still know a few people that use trackballs, but you get the idea.)

Multitouch is nothing too new, but has really gained fame for being used in Apple's iPhone. The idea is that a touch sensitive surface should be able to detect when a user is fiddling with it with more than one finger. On the iPhone, this gives the user the ability to zoom in and out from pictures by 'pinching' (or I guess anti-'pinching) their fingers across the screen.

Got $1600 lying around and an itching to show off your latest animation and movie laden Powerpoint presentation. Well Hitachi just rolled out the 77 inch Starboard FX-77 whiteboard with multitouch. I like that you can link up to 50 of them together. Not sure why, but I think that would make for some zany mega-multitouch action.

Another groovy multitouch related product from Sharp is being shown off at CEATEC right now (yes now!). Basically it's an LCD screen that also acts as a scanner. How does multitouch fit in? It can scan the screen for fingers planted against it, rather than using the various technologies listed in the article. How would you feel to have your screen looking back at you!?!?! Creepy. This comes to mind...

And lastly, Microsoft and Mitsubishi has been working on what they call LucidTouch, a fully touch sensitive, two-sided interface idea. I'm not sure what the real utility is besides having some ability to influence the front surface by the grip on the device, but I'm sure some of you creative folks could turn this into an interpretive art thingy. Video below.



Man, Engadget loves the multitouch.

Interface interface interface

My pal Giovanni send me a link to this video on seeing with the tongue. The idea is simple - use an array of electrodes tied to a grayscale camera to perform position reflecting stimulation of the tongue with intensities that vary with the pixel darkness. For instance, if the top left pixel is black, stimulate the front left electrode minimally. If the pixel tot he right of that is 50% gray, stimulate the next lead on the electrode with 50% intensity. The big issue right now is resolution of the device, but I would think that there is a limitation in general on the resolution a person can discern. I don't know much about density of receptors on the tongue, but I think the number is around 50-100/cm^2. Say the pad can be about 4cm wide when tensed and 5 1/3cm (to maintain 4:3 aspect ratio), that's between 1066.6 and 2133.3 pixels total. To put that in perspective, those junky 14" monitors from over a decade ago, what went up to VGA resolution (640x480) 307,200 pixels. That that's not even taking into account any local crosstalk, which could work for or, more likely, against better resolution. Maybe some sort of fovea limitation, with a dispersion from the center, though a the fovea, the human eye's resolution is about 500 MEGA pixels. Still an interesting idea and worth a watch.

Giovanni and Maria Laura pointed me to this blog on general tech stuff, which has a thing for alternative interfaces. Nothing new to me, but I don't think I've posted all the videos they have up there, so worth a look. Not maintained by a scientist, and most of the videos are posted as new, but really old. A few nice demos of the wonky "EEG" systems (I use quotes when I mention EEG and those consumer devices with a single dry electrode). The below video is a must-see, and was caught by this site.