Abstract
From an analysis of actual cases, three categories of bias in computer systems have been developed: preexisting, technical, and emergent. Preexisting bias has its roots in social institutions, practices, and attitudes. Technical bias arises from technical constraints of considerations. Emergent bias arises in a context of use. Although others have pointed to bias inparticular computer systems and have noted the general problem, we know of no comparable work that examines this phenomenon comprehensively and which offers a framework for understanding and remedying it. We conclude by suggesting that freedom from bias should by counted amoung the select set of criteria—including reliability, accuracy, and efficiency—according to which the quality of systems in use in society should be judged.
- BERLINS,M.AND HODGES, L. 1981. Nationality Bill sets out three new citizenship catego-ries. The London Times (Jan. 15), 1, 15.Google Scholar
- CORBAT~ O, F. J., MERWIN-DAGGETT, M., AND DALEY, R. C. 1962. An experimental time-sharing system. In Proceedings of the Spring Joint Computer Conference. Spartan Books, 335-344.Google Scholar
- FISHLOCK, T. 1981. Delhi press detect racism in Nationality Bill. The London Times (Jan. 20).Google Scholar
- FOTOS, C. P. 1988. British Airways assails U.S. decision to void CRS agreement with American. Aviat. Week Space Tech. (Oct. 24), 78.Google Scholar
- GAO. 1992. Patriot Missile defense: Software problem led to system failure at Dhahran, Saudi Arabia. GAO/IMTEC-92-26, U.S. General Accounting Office, Washington, D.C.Google Scholar
- GRAETTINGER,J.S.AND PERANSON, E. 1981a. The matching program. New Engl. J. Med. 304, 1163-1165.Google Scholar
- GRAETTINGER,J.S.AND PERANSON, E. 1981b. National resident matching program. New Engl. J. Med. 305, 526.Google Scholar
- HUFF,C.AND COOPER, J. 1987. Sex bias in educational software: The effect of designers' stereotypes on the software they design. J. Appl. Soc. Psychol. 17, 519-532.Google Scholar
- JOHNSON,D.G.AND MULVEY, J. M. 1993. Computer decisions: Ethical issues of responsibil-ity and bias. Statistics and Operations Res. Series SOR-93-11, Dept. of Civil Engineering and Operations Research, Princeton Univ., Princeton, N.J.Google Scholar
- LEITH, P. 1986. Fundamental errors in legal logic programming. Comput. J. 29, 225-232. Google Scholar
- MOOR, J. 1985. What is computer ethics? Metaphilosophy 16, 266-275.Google Scholar
- OTT, J. 1988. American Airlines settles CRS dispute with British Airways. Aviat. Week Space Tech. (July 18).Google Scholar
- ROTH, A. E. 1984. The evolution of the labor market for medical interns and residents: A case study in game theory. J. Pol. Econ. 92, 6, 991-1016.Google Scholar
- ROTH, A. E. 1990. New physicians: A natural experiment in market organization. Science 250, (Dec. 14), 1524-1528.Google Scholar
- SERGOT, M. J., SADRI, F., KOWALSKI, R. A., KRIWACZEK, F., HAMMOND, P., AND CORY,H.T. 1986. The British Nationality Act as a logic program. Commun. ACM 29, 370-386. Google Scholar
- SHIFRIN, C. A. 1985. Justice will weigh suit challenging airlines' computer reservations. Aviat. Week Space Tech. (Mar. 25), 105-111.Google Scholar
- SUDARSHAN,A.AND ZISOOK, S. 1981. National resident matching program. New Engl. J. Med. 305, 525-526.Google Scholar
- TAIB, I. M. 1990. Loophole allows bias in displays on computer reservations systems. Aviat. Week Space Tech. (Feb.), 137.Google Scholar
- WILLIAMS, K. J., WERTH, V. P., AND WOLFF, J. A. 1981. An analysis of the resident match. New Engl. J. Med. 304, 19, 1165-1166.Google Scholar
Index Terms
- Bias in computer systems
Recommendations
Moral luck and computer ethics: Gauguin in cyberspace
I argue that the problem of `moral luck' is an unjustly neglected topic within Computer Ethics. This is unfortunate given that the very nature of computer technology, its `logical malleability', leads to ever greater levels of complexity, unreliability ...
Cookies and Web browser design: toward realizing informed consent online
CHI '01: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe first provide criteria for assessing informed consent online. Then we examine how cookie technology and Web browser designs have responded to concerns about informed consent. Specifically, we document relevant design changes in Netscape Navigator and ...
Is ignorance bliss?: informed consent online
CHI EA '01: CHI '01 Extended Abstracts on Human Factors in Computing SystemsIn this panel, we explore: (1) the impacts of electronic media on informed consent; (2) how electronic media can be designed and used to preserve informed consent; (3) the relationship between protecting privacy and informed consent; (4) the ...
Comments