| Images on the Web present a major accessibility | | | | assign proper descriptions to arbitrary images. A |
| issue for the visually impaired, mainly because the | | | | "proper" description is correct if it makes sense |
| majority of them do not have proper captions. | | | | with respect to the image, and sufficient if it gives |
| This paper addresses the problem of attaching | | | | enough information about its contents. Rather |
| proper explanatory text descriptions to arbitrary | | | | than designing a computer vision algorithm that |
| images on the Web. To this end, we introduce | | | | generates natural languagedescriptions for |
| Phetch, an enjoyable computer game that | | | | arbitrary images (a feat still far from attainable), |
| collectsexplanatory descriptions of images. People | | | | we opt for harnessing humans. It is common |
| play the game because it is fun, and as a side | | | | knowledge that humans have little difficulty in |
| effect of game play wecollect valuable | | | | describing the contents of images, although |
| information. Given any image from the World | | | | typically they do not find this task particularly |
| Wide Web, Phetch can output a correct | | | | engaging. On the other hand, many people would |
| annotationfor it. The collected data can be applied | | | | spend a considerable amount of time involved in |
| towards significantly improving Web accessibility. | | | | an activity they consider "fun." Thus, like the ESP |
| In addition toimproving accessibility, Phetch is an | | | | Game, we achieve our goal by working around |
| example of a new class of games that provide | | | | the problem, and creating a fun game that |
| entertainment in exchange forhuman processing | | | | produces the data we aim to collect. |
| power. In essence, we solve a typical computer | | | | We therefore introduce Phetch, a game which, as |
| vision problem with HCI tools alone. | | | | a side effect, generates explanatory sentences |
| The Web is not built for the blind. Only a small | | | | for randomly chosen images. As with the ESP |
| fraction of major corporate websites are fully | | | | Game, we show that if our game is played as |
| accessible to the disabled, let alone those of | | | | much as other popular online games, we can |
| smaller organizations or individuals . However, | | | | assign captions to all images on the Web in a |
| millions of blind people surf the Web every day, | | | | matter of months. Using the output of the game, |
| and Internet use by those with disabilities grows | | | | we mention how to build a system to improve |
| at twice the rate of the non-disabled . | | | | the accessibility of the Web. |
| One of the major accessibility problems is the lack | | | | Design of a Useful Game |
| of descriptive captions for images. Visually | | | | A traditional algorithm is a series of steps that |
| impaired individuals commonly surf the Web using | | | | may be taken to solve a problem. We consider |
| "screen readers," programs that convert the text | | | | Phetch as a kind of algorithm. Analogous to one, |
| of a webpage into synthesized speech. Although | | | | Phetch has well-defined input and output: an |
| screen readers are helpful, they cannot determine | | | | arbitrary image from the Web and its proper |
| the contents of images on the Web that do not | | | | description, respectively. Because it is designed as |
| have descriptive captions. Unfortunately the vast | | | | a game, Phetch needs to be proven enjoyable. |
| majority of images are not accompanied by | | | | We do so by showing usage statistics of a |
| proper captions and therefore are inaccessible to | | | | oneweek trial period. Because it is designed to |
| the blind (as we show below, less than 25% of | | | | collect a specific kind of data, Phetch's output |
| the images on the Web have an HTML ALT | | | | needs to be proven both correct and sufficient. |
| caption). Today, it is the responsibility of Web | | | | We prove this through a specifically designed |
| designers to caption images. We want to take this | | | | experiment. |
| responsibility off their hands. We set our goal to | | | | |