Address AI Bias with Fairness Criteria & Tools

  • anti-discrimination doctrines,
  • using performance metrics to analyze fairness,
  • fairness criteria,
  • generalize fairness criteria, and
  • tools to identify bias.

Understand US Anti-Discrimination Doctrines

Model performance metrics

  • Subgroup AUC: Subgroup positives v.s. subgroup negatives
  • “BPSN” AUC: Background positives v.s. subgroup negatives
  • “BNSP” AUC: Background negatives v.s. subgroup Positives

Fairness Criteria

Modified from source
Source
Source
Source Gray: service area, White: Non-service area, Blue: Black population, Green population
Modified from source
Source
Source
Source
Source
Source
Source

Generalize Fairness Criteria

Demographic parity
Equal opportunity
Predictive parity
Source

Tools

slices = [tfma.slicer.SingleSliceSpec(columns=['sex'])]
Slice by height
A classifier on the UCI census problem in predicting whether a person earns more than $50K.

Computer Vision Datasets for Fairness

Source

Next

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store