↩︎ idea

Backlinks

• [[matrix-completion-experiments]]
  • if we relax the loss function (have it within \(\delta\)), we should be able to get competitive rates in the noisy case? [[idea]]
• [[concurrent-face-and-word-embeddings]]
  • crazy [[idea]]:
    • we have face embeddings, which on their own are just a way to represent faces efficiently in a low-dimension
    • would it be possible to associate faces with adjectives?
    • the goal, unsurprisingly, is to be able to put to text the biases inherent in these models
    • the problem is that it’s currently not biased in the more insidious way
    • but what if it’s the case that, if you were to do some type of alignment, then by doing so, you’re actually inheriting all the biases that the word embeddings have?
    • in other words, whenever you have some sort of process that does some concurrent learning, then the final model will be biased. but that doesn’t feel particularly interesting.
• [[signed-word-embeddings]]
  • [[idea]]
    • I’ve thought about this before, but for some reason I can’t quite find where I wrote about this. basically it follows from two pieces of intuition:
    • that word embeddings cannot distinguish between synonyms and antonyms, because all they care about is co-occurrence
      • as a result, oftentimes you’ll see that antonyms will appear close to each other in the embedding space, which doesn’t really make much sense if we’re think of the embedding space as a proxy for “meaning”
    • that I’ve worked on signed graphs, and so have some intuition about dealing with positive/negative “ties”
    • I think that an interesting place to start will be thinking in terms of the SVD decomposition of the PMI matrix (even though that’s not exactly the same thing as SGNS/Word2Vec)
    • How does this relate to Signed Graphs?
    • The one time I was thinking of embedding positive and negative edges, it was when considering the spectral decomposition (?) of the graph
    • I vaguely remember a paper that tried to spectral methods to do some sort of visualization (though it was very unwieldy)
    • the problem is that when you do things geometrically, then you’re basically going to be conforming to transitivity indirectly. just like when you’re trying to model signed graphs. thus, I think that even though it seems innocuous to have some sort of geometric interpretation, it has this unconscious bias
• [[judicial-demand-for-xai]]
  • Here’s an [[idea]]: what if you can come up with something like the tangent plane, but the explainable plane, in that for every point estimate provided by the machine learning model, you can just find a super simple, interpretable model that does a good, local job of explaining stuff for that particular defendent (has to exactly predict what the ML model predicted, hence the parallels to the tangent plane). Actually, this is very similar to LIME (Ribeiro, Singh, and Guestrin ⊕2016Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. “"Why Should I Trust You?".” In KDD ’16: The 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, 1135–44. New York, NY, USA: ACM.).