Wednesday, May 28, 2008

Monkey self feeding BCI control

Alright, I get it. Several people have sent along links to Andy Schwartz's Nature publication, so here's the info. Thanks to Remy Wahnoun and Natalia "Get Moose and Squirrel" Bilenko for doing my work for me and compiling the links, and being the first to send info along.

The work reported involves real time decoding to drive a robot arm, which fed the monkey. I haven't had time to run through the whole thing, so here's the "shoot from the hip" version of my thoughts. I'll update these

Interesting points:

- The monkey was trained to use the arm with a joystick, and followed by, the part I thought was very interesting, various levels of pre-programmed control assistance while using brain signals. I will expand on this later, but it is an interesting task design for several reasons.

- Having seen some of the raw video at the Neurobotics 2007 workshop, there is much more obviously correlated arm motion. They have an entire, substantial paragraph devoted to this, since they probably knew it would be a criticism.

The three points they make are:
1) Movements were with the arm ipsilateral to the electrodes.
2) The movements were delayed by up to a second.
3) Moving is required, since BCI control has been demonstrated before without it.
They cite some supplementary info and write, "... monkey’s hand movement was only loosely coupled to prosthetic control." The video I saw before tells a slightly different story. There was a pulling movement 100% of the time (every trial), and when the arm didn't reach the monkey's mouth it would 'paw' or 'dig' (repeated movements similar to pulling an invisible object toward the body) repeatedly until the arm reached the monkey's mouth. You can see this pretty clearly in the video linked above (watch the hand in the plexiglass tube). Yes, there is a delay, but the whole situation would be interesting to examine. It may not require the 'pulling' at onset, but monkeys very often use both hands to eat. Those of you with monkeys, go give them a treat. They will grab it with they right hand (usually), and then pull it towards their body and engage their left hand as they do so. Feeding it almost always a bimanual task for macaques, explaining the ipsilateral activation and the delay. Reason #3 above is kind of a brush off reason - hey, movement is needed to use a BCI, so no big deal. The three cited papers are all in humans, all were tetraplegic, all lacked some degree of propriocentive feedback, two didn't use PVA or multielectrode arrays, none used an arm, none had confirmed the lack of covert movements with EMG, and the list goes on.

- The trial was very organic and continuous, which is great. From my brief skimming, it looks like simultaneous control of the hand aperture needed to be maintained throughout the task, hinted at when they observe the the hand slowly opening along the path to the food.

In general this is a nice, brief summary with some very interesting points (perfect for a Nature Letter). Obviously they are preparing some studies, and this was released to wet out appetites. The 'organic' nature of the task will open it up to many criticisms, but more studies like this are needed in order to understand the dynamics in play when dealing with a experiences that lack built-in boundaries and environmental awareness. Definitely good thought fodder.


Natalia said...

Thanks for the mention!
That's really interesting about bimanual eating in macaques. I've noticed that before but have never given it much thought. I'm going to go feed my monkey =)

Brendan said...

Nice to see publicity for BCI research. Andy Schwartz and his team have made great progress.

But, blog readers and the general public should know that the approach described by Schwartz and colleagues is not the most common, practical, or effective BCI. Similarly, Nicolelis and colleagues (who Schwartz, Taylor and colleagues hate, and with good reason) got a lot of buzz in 2003 with their invasive monkey BCI and claims that this is 'way beyond anything that was done before.'

People might assume that whatever technology makes it to the media must be the best. This is not so. On the contrary, the most highly regarded BCI labs eschew publicity and only present their work through conferences and proper channels.

To clarify how the real world of BCI research differs from media portrayals:

1) The substantial majority of BCI work involves noninvasive techniques such as EEG. No drilling holes in the head, no surgery, no pain, no need for any doctor.

2) The substantial majority of BCI research uses human subjects.

3) The clunky 2 dimensional control seen in the Nicolelis and the much better Schwartz videos is remarkable, but below noninvasive systems. The Wolpaw lab has both better 2D control and 3D control.

4) The substantial majority of BCI research does not involve robot arm control. This is a very flashy application, but not the most helpful to patients, and it is not a breakthrough to take any signal and use it to control a device. The hard part is getting a reliable signal from the brain in the first place.

Readers who are interested in this technology are encouraged to explore beyond headlines. Friendly articles about BCIs, including reviews, are easy to find online.

Brendan Allison, PhD
Coordinator, BRAINROBOT project
Program Manager, BRAIN consortium
Institute for Automation
University of Bremen

Anonymous said...

Its nice how they hide the freak show behind the aluminum structure. If you do research on an animal but find yourself uncomfortable with showing the public full documentation on what you are doing then one should question the ethics.

Brandon King said...

(This was one of those posts I missed having the reply notification on for, so sorry!)

As Brenden mentioned, do explore beyond the media articles. Part of the trouble, I know, is getting access to primary research articles, and then understanding the background material. But, there is also the issue that none of these systems operates the same - there are no direct comparators.

So, when you imply that invasive systems are more "clunky", that doesn't square with what I've seen. Having seen a few P300 systems operate, the BrainGate control is much 'better'. Many of the EEG systems use targets the size of an entire screen edge, while we use circles placed around the screen (Fitts's Task). One of our goal is to restore the ability to use a computer as is. No fancy front end with giant icons, etc.

But, yeah, there is a boatload of ill-will issues between all the labs (actually have a post on that coming up real soon), which is ridiculous.

I disagree with the statement that "the most highly regarded BCI labs eschew publicity and only present their work through conferences and proper channels". Completely. Yes, they might pick and choose the best clips, but in the end, what works gets noticed. It gets noticed by colleagues, the funding agencies, the users, so shunning publicity simply _can't_ happen in this field. If your device works the best, thousands or tens of thousands of people should be using it. The other aspect of this field that isn't really stated explicitly is that many researchers are actually interested in understanding the brain, not taking a bunch of random looking waveforms and strapping a robotic hand onto a computer. This type of research is far more 'boring' from a media perspective, but really lays the foundation for the eventual implementations.

I might bump this up to a post, since I let it go so long without responding (whoopsy!)

Brandon King said...

And to the anonymous coward, oh man, you are SOOOOOO right! I mean, we shouldn't just hear about Ted Kennedy's brain tumor, we should have every station broadcasting the surgery!

The decision to obstruct view of a headstage is all about presentation to the public, not shame of the scientist. "Joe six-pack" doesn't know what they're looking at, and apparently neither do you. Now go away troll. I will be deleting all future animal rights posts because they are a waste of time, space and energy.

Post a Comment