NNSquad - Network Neutrality Squad
[ NNSquad ] Re: Tim Wu and search engines
David Reed brings up the question:
> If the algorithm discriminates against the conversations of (say)
> black people, is it protected speech, or a violation of civil rights?
There have been reports (I've never seen a specific citation, I'm afraid) that a bank used machine learning to build a decision engine for personal loans. They had to scrap it because it turned out that it gave a large negative weight to black applicants. I've heard stories about similar systems for ranking resumes that were found to give a large positive weight to being unmarried.
There are some very difficult issues coming down the road. Opinions in and of themselves may be protected, but when those opinions are embedded in a larger system and have real world consequences, what deserves protection has to be specified very carefully. While truth is an absolute defense against libel in the US, a proof that a rating system that discriminated against black loan applicants or married job applicants actually achieved some acceptable goal (decreasing loss rates, longer work hours with more output) would not render the system legally acceptable in the US. Personally, I wouldn't find it ethically acceptable either.
So ... when do we accept an argument that "it's just what the algorithms came up with"? We have many centuries of experience with bureaucracies - which provide exactly this kind of insulation for each individual. "I'm just following the procedures." There are no absolute rules. If we demand that every member of an organization be personally responsible for every decision or action that organization undertakes, businesses and governments become impossible. But if we go 100% the other way, "It wasn't my job" becomes an absolute defense. Computers are a much better embodiment of the basic drive of bureaucracies: The conversion of individual initiative to uniform policies, uniformly applied. "Government of laws, not men."
-- Jerry
_______________________________________________
nnsquad mailing list
http://lists.nnsquad.org/mailman/listinfo/nnsquad