#dark_patterns

Project Fairness

References

  • (D’Amour et al., n.d.D’Amour, A, H Srinivasan, J Atwood, P Baljekar on Fairness, and 2020. n.d. “Fairness is not static: deeper understanding of long term fairness via simulation studies.” Dl.acm.org.)

Relevant Literature

Algorithm Helps New York Decide Who Goes Free Before Trial : WSJ

This is a fairly innocuous article on the topic of recidivism, and for some reason makes a very strong case for this particular use of the algorithm.

  1. This is in part because the existing judges were way too bad in terms of letting people off (see Figure 1). However, that’s a problem with the existing judges (and also, since the algorithm utilised the data during this particular period, shouldn’t that mean that the data is somehow problematic, though it’s unclear exactly why it is).

Out of Jail Figure 1: Out of Jail

  1. If you correlate the scores assigned by the algorithm against the number of people who actually returned to the court, then you see that they’re doing good (see Figure 2). In other words, this algorithm is actually predicting well.

Back to Court Figure 2: Back to Court

There are, of course, a few caveats here, that aren’t really mentioned in the article, the most obvious one being that we know nothing of the counterfactual of those that were not released.

Also, from the article, here’s the main factors that are used:

Years since last warrant; more than one warrant in last five years; years since last misdemeanor or felony conviction; misdemeanor convictions in the last three years; felony convictions in the last 10 years; pending cases; length of time at last two addresses; reachable by phone

The last factor, whether or not someone put a phone number down, is something that feels incredibly easy to game. And even in the last paragraph, a staffer encourages a felon to put down a phone number. This definitely rings of [[the-perils-of-explainability]]: if you give people the factors, then basically it’s no longer going to be meaningful.

Thoughts

  • ideas
    • training data
      • something that I hadn’t actually thought about was the specification of the training data, and how the feedback plays in that. which, at a higher level, is a question of: what is your training data? for it to be training data, it must include some notion of “success.”
      • in my first iteration, where I was thinking about estimating \(\theta\), there was no notion of a training data, just the current iteration, and predicting the signal. this doesn’t seem right?
  • Categorization/Flowchart
    • supervised
      • selection (0/1): asymmetry of selection means that one option leads to no data
        • individual (independent choices, maybe slight interaction, but very small)
          • recidivism (jail means can’t tell if right choice)
          • lending (no loan means you can’t default)
        • potential group effect (performance a function of chosen group)
          • hiring (no hire means don’t know if would have been good)
      • top-k: I guess you have the same thing here, the counterfactual of other choices
        • college admissions
        • women in data science
      • ranked
        • search
      • allocation/assignment
        • police (allocate (fixed) balls to bins)
        • advertising (allocate ads to people, no constraints)
      • classification
        • facial recognition (this can be more problematic downstream, but this is just about detecting faces, and the skewed data)
      • estimation (holy grail 🏆)
    • unsupervised
      • word embedding bias
  • Examples
    • Feedback mechanisms in advertising
      • old
        • this actually works out pretty nicely, and it’s very different to the other examples, as it’s not just a straightforward selection problem
        • you’re essentially predicting a person’s interest (actually you can even do away with this part, and just assume that actually we have a good read on people’s interests)
        • call the person’s view \(x_i\)
        • you, as the company, are doing utility maximization, which boils down to click-rates
          • that is, you’re going to learn a model that provides an article \(y_i\) such that it maximizes the probability the individual will click it
        • unbeknownst to you, there’s a feedback mechanism going on behind the scenes
          • a simple example would be the \(x_i\) get updated in the direction of \(y_i - x_i\)
        • while trying to code this up, I came to realize that the important thing here is to specify what exactly the function we’re trying to learn is:
          • we’re sort of learning a function of two parameters, but then the eventual goal is to pick the second parameter to maximize the final output
      • intuition:
        • companies are maximizing click-through rates, without concern for how this might be sharing people’s preferences
        • on the other hand, this assumes that people are very malleable (which I guess from the stuff on #dark_patterns, this is a reasonable assumption)
        • I guess we don’t need to posit one way or another. we simply consider the possibility of such a feedback loop, and see how that plays out. and that’s the whole point – we should be doing “scenario” runs (like how the military do war simulations or contingency scenario experiments, whatever they’re called)
      • simple model:
        • it’s on a per individual level basis (since they’re basically independent, there’s no scarcity)
        • then, the feedback mechanism is simply the fact that you’re “extremizing” people as that leads to better clicks/attention
      • relation to fairness:
        • the example where advertising becomes “sexist” when it starts to give nurse degree ads to women and doctor ads to men.
          • it’s unclear what’s the mechanism at play here, but I imagine this is more about the algorithm learning biased things about the way people respond to ads, and not so much about the feedback mechanism. is there one here?
  • Outline suggestions
    • merging of economics and statistics
      • economics has become more ML-heavy (Athey), and more data-driven, statistical (Chatty)
      • statistics/ML has become more economics, causal-inference-y, more “social” (social networks)
    • we focus on fairness, but this extends to all systems that seek to predict/influence human behavior (hence encroaching on the turf of economics)
      • the reality is that humans respond to signals, and will adapt accordingly: hence forming feedback loops
    • if we don’t seek to understand/consider these mechanisms then we risk inadvertently shaping the world around us in some direction that we might not wish for
    • this paper is a “call-to-arms,” for everyone involved to think about the long-term repercussions of the algorithms that they’re deploying
      • not just the short-term
      • case in point: word embeddings, you going to try to make them “unbiased?” this is the way the world works. the best we can hope for is to not perpetuate these biases, and try to move them away
      • clearly this is an ambitious goal, and usually the feedback is long-term, but that doesn’t mean that we shouldn’t be thinking about it, speculating on what happens over the long-run
      • and then there are examples where the feedback mechanism is very short, producing changes in the underlying system very quickly (the political advertising, news article suggestion space in social networks)
    • what we find is that, in these admittedly artificial models of society (in the particular context that we’re interested in, say women in science), we see interesting tradeoffs
      • there’s the short-term-ism that, even if one tries to be pro-fairness and anti-bias, can lead to unintended consequences
      • since everything is artificial, and these feedback mechanisms are hidden (and glacial movements of a societal nature), at this point we can only hope to draw attention to this problem
      • we speculate that it might be possible to estimate the feedback mechanisms, given enough data, so it is no longer a speculative endeavor
  • Reminder of desirables:
    • it would be very interesting to be able to actually try this model on some kind of dataset (it might have to be synthetic, unfortunately)
      • other papers can do it because they’re just looking at the current state (or a one-time-step)
      • we would have to use something like the opengym perhaps
    • it is very easy to set up the scenario in such a way to show that, viola, if my feedback supports diversity/equity, then actually the best thing to do is to promote exactly that
      • and that’s basically what many papers find (is that true?)
      • well, actually when you deal with some type of binary “response,” and have to look at FDR-type considerations, then actually definitions of fairness basically break down, so there it’s not clear what the best thing to do would be
      • I wonder if looking at things in the long-run can resolve what is best
        • so, it’s no longer saying, okay let’s define fairness by some equality of some rate
        • let’s consider these possible “equalizing” policies, and look at how they play out in the long-run. and once we take a step back and look at the big picture, then it’s a little easier to decide what is a “fairer” world (perhaps)
      • back to the topic: we want to find situations where it’s not so “explicit,” that the fairness happens organically somehow