how to solve problems w/o relying on math – this is about social justice
intro’d to Cathy via pdf 15
notes/quotes from livestream that day:
data from policing/evals/et al .. cloaked behind mathematical obscurity.. unjust/control
on the authority of the inscrutable.. ie: algorithms as weapons
notes from video:
if don’t know notation.. feel like some kind of authority there.. some objectivity.. some scientific truth.. that they’re not allowed to question because they’re not experts..
that .. authority of the inscrutable.. is translated .. to algorithms..
theory of algo’s.. used as weapons.. through authority of the inscrutable.. weapons of math destruction
algos that are used in all sorts of places.. as a form of social control.. not actually helping people
1\ secret/opaque.. people who are targeted by them don’t understand them
2\ affected by them
3\ widespread .. a lot of people involved
4\ questionable defn of success.. people targeted don’t agree with .. ie: saving money
5\ create pernicious feedback loops.. make problems worse..
ie: value added model for teachers.. algo.. that can fire teachers.. chicago teachers strike about this.. idea.. this algo is supposed to hold teachers accountable for good teaching.. contract.. that no one can see inside.. accountability for teachers.. but no accountability for model..
5 min – bottom layer from data – from policing events.. 2nd layer – predictive policing.. next layer – evidence-based sentencing.. all biased…
6 min – unconstitutional because cloked behind math obscurity.. lacks accountability
models are embedded opinions.. so unless specifically make sure not unfair to some people.. they will be
8 min – micro targeting.. goal: to understand you as consumer/voter and to show you what the campaign wants.. giving you what they think will make you believe the candidate.. ie: political cookies.. show her the thing that she likes about me.. so.. it’s efficient for campaigns.. but at end of day.. what is efficient for campaigns is inefficient for democracy
9 min – part of living in democracy is understanding the rules.. and these algo’s are a set of secret rules..
10 min – we have to add effects up and we see unequal effects..
11 min – these are really hard problems: get ed better; decrease mass incarceration; … lots of good intentions that aren’t succeeding.. we have to think about how to solve problems.. rather than how to rely on obscure math
this is not a question of privacy.. it’s about social justice..
I’m interested in algorithmic accountability, civil disagreement, and the social mechanism of shame.
from her about page:
Cathy O’Neil lives in New York City. She hopes to someday have a better answer to the question, “what can a non-academic mathematician do that makes the world a better place?”
go deeper and help make this happen Cathy.. a nother way
Cathy (Catherine) Helen O’Neil is the author of the blog mathbabe.org and several books, including Weapons of Math Destruction. She was the former Director of the Lede Program in Data Practices at Columbia University Graduate School of Journalism, Tow Center and was employed as Data Science Consultant at Johnson Research Labs.
weapons of math destruction
She lives in New York City and is active in the Occupy movement.
O’Neil attended UC Berkeley as an undergraduate, received a Ph.D. in mathematics from Harvard University in 1999, and afterward held positions in the mathematics departments of MIT and Barnard College, doing research in arithmetic algebraic geometry. She left academia in 2007, and worked for four years in the finance industry, including two years at the hedge fund D. E. Shaw. After becoming disillusioned with the world of finance, O’Neil became involved with the Occupy Wall Street movement, participating in its Alternative Banking Group.
She is a co-author (with Rachel Schutt) of Doing Data Science: Straight Talk from the Frontline (O’Reilly 2013, ISBN 1449358659). She also wrote an e-book On Being a Data Skeptic (O’Reilly Media 2013, ASIN: B00G3M9JY2). Her book Weapons of Math Destruction was published in 2016 (Crown, ISBN 0553418815) and has been nominated for the 2016 National Book Award for Nonfiction
O’Neil lives in New York City with her husband Aise Johan de Jong and their three sons
Aise Johan de Jong (born 30 January 1966) is a Dutch mathematician born in Belgium. He currently is a professor of mathematics at Columbia University. His research interest includes algebraic geometry.
De Jong attended high school in The Hague, obtained his master’s degree at Leiden University and earned hisdoctorate at the Radboud University Nijmegen in 1992, under supervision of Frans Oort and Joseph H. M. Steenbrink.
He won a Cole Prize in 2000 for his work on singularity. In the same year De Jong became a correspondent of the Royal Netherlands Academy of Arts and Sciences.
Professor de Jong has also spent the past few years working on the Stacks Project, “an open source textbook and reference work on algebraic stacks and the algebraic geometry needed to define them.” The book the project has generated currently runs to more than 5,300 pages as of 6 August 2016.
rapid prototyping .. to slow/human
oct 26 2016
the good thing about the financial crisis was that everyone saw it
i’m defining good food.. i’m defining success.. my objecting my agenda on that model.. the person in defining algo are one’s defining success.. and usually means – success for them.. this is huge.. this is how we choose what to do next…
people often don’t even understand that they are being scored.. for this algo.. and no way to make it accountable.. if don’t know it exists.. un appealable..
algos often create larger feedback loop of destruction..
broken feedback loop ness
the broken feedback loop.. has made it so that all our lecture/presentation/talks.. focus on validation/measuring of people/transactions.. (ie: her next 10+ minutes was on job apps and teacher verification et al)
actually.. that went on for rest of talk.. analysing the bad ie: prison ness.. rather than spending time working out/trying a nother way to live..
i’m looking for evidence… if this is making things worse or not..
why wanting to focus on that.. rather than.. a nother way
we need to develop tools to understand if these algos are discriminatory..
why not just tools to model a nother way that makes algos (you’re talking about) irrelevant
if we decided.. something was unfair.. i don’t know how to go back and make it fair
again – why not just tools to model a nother way that makes algos (you’re talking about) irrelevant
via Michel fb share
this is a very good presentation:
the era of blind faith in data must end – ted2017
what if the algos are wrong.. to build algo need 1\ data – what happened in past 2\ a defn of success – the thing you’re looking/hoping for
first rule of algos.. you get to choose your success.. opinions embedded in code
we trust in math
a lot can go wrong if we put our faith in big data
algos don’t make things fair.. they repeat our past actions.. the status quo
free market won’t solve this problem.. there’s a lot of money to be made in unfairness
we have to check the algos for fairness.. t
data scientists – we should not be the arbiters of truth.. *we should be the translators of ethical discussions that happen in larger society..t
tech as it could be
Friends, I’m writing a book about shame as a social mechanism. To that end, can you help me out with book or article recommendations? I’m thinking broadly: why do people shame other people for their race, their poverty, their sexuality? When does shame work and when does it fail?
Original Tweet: https://twitter.com/mathbabedotorg/status/1026560511431913472
Thank you in advance and feel free to write to me personally instead of tweeting with your ideas. cathy.oneil at gmail dot com.
Original Tweet: https://twitter.com/mathbabedotorg/status/1026560744052191235
shame.. never works
the social dilemma (doc)