Neural Code for Faces

tags: [ CV , psychology , src:paper ]

src: (⊕Chang and Tsao 2017Chang, Le, and Doris Y Tsao. 2017. “The Code for Facial Identity in the Primate Brain.” Cell 169 (6): 1013–1028.e14.), Nature news feature.

• primates have special(ized) face cells for disambiguation
• these cells are found in the inferotemporal (IT) cortex
  • two patches are considered: middle lateral/middle fundus (ML/MF), anterior medial (AM)
  • previous work showed a hierarchical relationship (AM being downstream, possibly the “final output stage of IT face processing”)
    • and \(\exists\) sparse set in AM which seemed to fire for specific individuals (independent of head orientation), suggesting it was capturing something very high-level
• question: what are the functions of these cells? one idea is that each cell encodes a particular individual11 “Jennifer Aniston” cells, after a study in epileptic patients found a cell that would only fire when presented with JA, regardless of face or just her name.
  • that does not scale, obviously22 I suspect that they just didn’t have a large enough sample. actually, that’s the whole point of this article! that just because it’s only firing for one thing in your set doesn’t necessarily mean it’s unique. one caveat might be that the things that would trigger activation might not be natural (so practically it’s unique). also, it’s important to distinguish visual/face processing, which is much more low-level, than, say, high-level object recognition, maybe.
• answer: construct the following face space:
  • decompose a face into two broad category of features
    • shape (S): i.e. the geometry of the facial landmarks
    • appearance (A): the rest (i.e. independent of shape)
      • this is accomplished by morphing all faces to the same average face-shape
  • from this decomposition, get 200 S and 200 A features, and perform PCA to get top 25 for each; giving, finally, a 50-d face space!
• result: single neurons are axis-coding (i.e. are projections in the face space)
  • to test this, simply determine the vector for a cell, and see faces that lie orthogonal to that vector have the same firing:
    • and crucially, they are able to determine the null space for those cells that were previously thought to be individual
  • 
  • similarly, they can predict response to new faces, which they also do.
• result: ML/MF cells fired more for shape, while AM fired more for appearance
• result: ~200 face cells needed to decode human faces (and encode)
• implications:
  • axis-coding, i.e. linear projections, are efficient, robust and flexible (see remark below).
• caveats:
  • all faces are neutral (i.e. don’t include expressions/emotions): but that seems fair, since we’re doing facial recognition, not emotion detection
  • potentially missing axes, given the training data

Thoughts:

• a point made in the paper is that this face space is rather constrained, and every point in this space is a valid/realistic face. however, that also suggests that it might be a little bit too restrictive
  • they’ve appreciated that other axes could be missing
  • but my worry is that, with results like “they can encode/decode faces,” the strong caveat there is: for faces falling into this particular space
    • granted, it is clearly a large space, but I suspect there are probably more axes available, and by projecting, you’re missing out on other spaces perhaps.
    • i.e. this is not necessarily the full picture
• a similar point is, it does feel like the features that they’ve come up with are closely related to the function of the cells, but I’m curious if a completely different encoding/feature-set would also produce the same empirical results
  • for instance, as they point out in the section of reproducing these results with a CNN, one doesn’t necessarily have to morph the face to get the appearance features (as it seems biologically implausible for our brains to be morphing faces)
  • and so they say you can probably get the same features by extracting information around the eyes
  • but then why not just do what we think the biology is doing
• a natural follow-up question is, what is the subspace spanned by these vectors: do they complete the space?
  • one would hope that the vectors themselves are orthogonal (or perhaps nearly orthogonal), though perhaps there’s redundancies inbuilt into the system (and perhaps the location of the cells might show that)