Citizen science

From NoskeWiki
Jump to navigation Jump to search


"Citizen science" is a term used for scientific projects where volunteers, many without scientific training, perform or manage research-related tasks such as observation, measurement or computation.

I became interested in this topic and made this page after a presentation by David Lee at NCMIR in March 2010, where he gave a brief summary of many of the projects listed below. David also referred me to this brilliant article about "games with a purpose" by Luis von Ahn (big name in the field) here.


The big buzzwords in this field are:

  • "Games with a purpose (GWAP)" - describes a computer game aimed at harnessing a human abilities to solve problems which computers are poor at (eg: labeling images), but in an entertaining environment (ie. free labor!). Two classic example are the ESP game and Foldit.
  • Crowdsourcing / clickworking - taking a task traditionally performed by an employee and outsourcing it to a group of people or community by making an open call for help/contributions.
  • Wisdom of the crowd / collective intelligence - the process of taking into account the collective opinion of a group of individuals rather than a single expert to answer a question. The idea is that a large enough group of non-experts is as good, if not better than a single expert. The classic example being Wikipedia.

Interesting Citizen Science Projects

NOTE: All these games work in your browser (unless otherwise stated).

Project Name Description Impression Incentive Estimated Users

Games with a purpose


Citizen science - foldit screenshot.png
Users download an executable program where they can use various tools to fold 3D proteins into optimal energy shapes. Humans (or at least the skilled ones) are good at figuring out the major twists needed to solve these 3D puzzles, while the computer is good at jiggling side chains - hence the game includes many semi-automated tools like "rubber bands" and "wiggle". (read more). This game has become immensely successful and enjoyed MUCH media attention - despite the fact that if you read between the lines it's probably only a handful of the VERY best registered users actually contribute! ("foldit for fun" article) Awesome tutorial, great graphics and sound. Basically it FEELS like a game (a challenging 3D puzzle game) and not a chore! The tutorial itself is 29 puzzels, with LOTS of content and takes ~2hrs to finish - these puzzles helps you get the hang of clearing amino acid clashes and aligning protein backbones to create hydrogen bonding (even for those with NO knowledge of protein structure). The points system is brilliant. The other thing which makes it fun is the number of helpful/smart tools meaning there is multiple ways to solve each puzzle. Fun and challenging game + competing against other users to get best scores for any given protein. 57,000 after only 2 years (updated Sep 2010)!

Citizen science - eterna screenshot.jpg
Very similar to Foldit, except two dimensional and focusing on RNA only. EteRNA released early 2011 and is designed by some of the same people who created, but different in that it allows you to design new RNA in an easy 2D interface and each well they announce a winner and synthesize that protein (with I believe about a 10% success rate). There's a nice article about it in the New York Times (here). Unlike its "predecessor", EteRNA is 2D and much easier to use. It's all Flash-based, but features some nice background bubble effects and a tonne of flashing lights whenever you succeed with a good score. Fun and challenging game + chance of winning each week and getting your RNA synthesized. TOO EARLY TO TELL (updated Feb 2011)!
The ESP Game, Squgil
& other games at

Citizen science - gwap - esp game screenshot.png
Several interactive and cooperative games where you are matched with another player:
  • ESP Game - popular game where you both see the same image, type multiple keywords & get points for each match *
  • Squgil - both trace an outline in an image matching a single keyword (eg: dog) & get points according to how similar your shapes are **
  • Tag a Tune - both type words and guess if you are listening to the same song
  • Verbosity - one user describes a word & the other guesses the word
  • Matchin - both users see two images & click the one they prefer
  • PopVideo - five users type tagwords as you watch the same video
  • Flipit - "memory" game where you match "similar images" (*played individually)
Easy to sign up and fun to play, although buggy in some browsers. Nice point system and thermometer bar as you and your anonymous partner race against the clock. I warn you, some of the games can be a bit addictive, so expect to be there a few hours trying each game. Interestingly, Google has made their own version of the "ESP Game" called Google Image Labeler, which is very similar.

(read more).

Fun, help improve internet tagwords & chance to win $20 amazon gift each week. >20,000 member back in 2008

Clicking/classification based

Galaxy Zoo

Citizen science - galaxy zoo.jpg
In the original galaxy zoo, users were asked to classify the shapes of galaxies (eg: spiral, elliptical, edge on) in telescope images (read more on Wikipedia). The project went online Feb 2009 but is no longer live as they reached their target of 60 million classifications of a million galaxy images in April 2010 (see blog)... so now they diverted attention to other "zooniverse projects". Nice and easy to use with a very simple interface and big icons to click and match the shape. To take part you first needed an account, and this "Zooniverse account" now lets you log into "Moon Zoo", "Solar Stormwatch" and more. Contributing to large project and prizes were given every so often for people who reach next milestone. >200,000 users and 6 million classification * (updated Oct 2010)
The Valley of the Khans Project

Citizen science - valley of the khans screenshot.png
Allows volunteers to help search for the Tomb of Genghis Khan, by looking at satellite images and tagging tomb-like structures. Supported by professionals from National Geographic, the web interface was setup in just 3 months in early 2010 so that results could compiled just in time for an expedition to Mongolia ("Field Expedition: Mongolia"). This project is lead by Dr Albert Lim, an avid archeologist and adventurer from the UCSD who has won National Geographic's explorer of the year in 2010 plus many other accolades (here). In Sep 2010 I was lucky enough to meet Albert and one of his team members, Luke Barrington, an expert in machine learning and developer of Herd It, to hear them talk about their citizen science project and how National Geographic helped it become greatly successful in just a short period of time! Although the tomb has not been found (such a feat is incredibly difficult given the secrecy regarding Genghis Khan's burial) the first expedition (mid 2010) helped find, map and protect multiple ancient structures of high cultural significance. The interface is beautiful simple - showing a single tile at a time with five icons (ancient structures, modern structures, tag, roads and rivers) which can be dragged on. In the tutorial phase, all images are pre-tagged by experts and so after you've dragged on up to five icons and click "done" you get immediate feedback to say "you've tagged 3 of 4 features", to show you where the experts placed the tags and useful little notes eg: "did you notice the ...". The first tutorial is 5 pre-tagged images and ~2 mins. After that you register and level up gradually from Novice 1-3 then Intermediate etc, with 10 images per level. Very minimalistic, very effective. 55 burial mounds found. :) Chance to help find the location of Genghis Khan's tomb by guiding Albert's team. 10,000's (I believe) + 450,000 tiles processed + 1M tags(updated Oct 2010).

Citizen science - eyewire.png
Asks the public to help map out the "wiring diagram" of neurons in the retina. To do this it loads a cube of data at a time from a huge region imaged by serial block face scanning electron microscope and asks you to take a starting seed point and you can click to extend (left click) or shrink (right click) the region as you scroll up and down the 2D monochrome image slices with the mouse wheel. This citizen science project started Jan 2012, and is part of the "connectome" effort by Prof. Sebastian Seung (see TED talk) at MIT (Massachusetts). Seung hopes mapping the neurons will help understand how the eye works (biotechniques article).

SIDE NOTE: On a personal note I could early be bitter that this is *pretty similar* to a project I'm working on and they beat me! My own project I tired to keep hidden, but it was accidentally leaked and can be found if you search deep enough. Makes me wish I had the type of resources they have (big development team, good media and millions in funding!) - with just myself programming part time I don't know when I'll be ready for stable release. Since, however, they did such a great job and made such clever use of clustering pixels, it's hard to be bitter and I hope this project does well. :)

Great interface - very clean and great use of transparent color! In terms of usability I found it very fast and easy to use - although my background is 3D segmentation, so that might be a bit biased. It looks like the main window is javascript, loading what looks like a 4 tiles (probably 128x128 each) over 400 slices, with some HTML controls on the left and if you have a newer browser there is a brilliant little 3D HTML5 / WebGL window which shows what you have mapped in 3D. More brilliant again is the way they use (I believe) regions of super-voxels. An advanced algorithm has already been run and found the pathway for most of the cell's axon/dendrites, but your job is to "connect the blobs" that belong together so you can refine the algorithm's results. The only fault with this approach is that if the blobs are incorrect or outside-the-line you currently can't fix them up - but in this specific challenge it's a great speed trade-off. In this game (actually it's not very-game like, but been advertised as such) they aren't interested in precise lines - it's the overarching network diagram they really want to get right. Early days yet, but it's a very polished looking site already and they say their next game will help people locate synapses, although in this particular dataset I really can't see any, so I presume they'll use a different dataset. Don't get me wrong - their image dataset is great though - it comes from Winfried Denk at el from the Max Planck Institute for Medical Research in Heidelberg. Help learn more about how the eye is networked - especially with regard to Retinal Ganglion cells. Since this was only opened in 2012 it's hard to say and they have limited the number of users for their beta testing phase.

Citizen science - stardust at home screenshot.jpg
Encourages volunteers to search for tiny interstellar dust particles (~1 micron diameter) by scrolling through "focus movies" and clicking any likely interstellar dust tracks. There are ~1 million focus movies, each 40 images of 480x360 microns, collectively representing a set of aerogel "Stardust Interstellar Dust Collector" blocks which were exposed to open space on the Stardust spacecraft in 2000. NOTE: They estimate only 45 actual particles are in entire sample though and may take years to confirm each one for sure. (read more) Great concept, but my big complaint is lack of feedback/interactivity. The only feedback was returning to the main menu and seeing an arbitrary score. I lasted ~1 hr scrolling through poor contrast focus movies and clicking "No Track", or occasionally guessing there *might* be a track - though looking at high scores it's obvious many user persist for LONG periods. Each focus movies appears to be shown/reviewed by multiple people, so in my opinion they should show "good, 4/5 other people saw no track" after each movie and *perhaps* contrast options or automatic segmentation tools to help highlight candidate regions. The first user to discover a new particle can name it and appear co-author on any paper announcing discovery. Users with high scores are listed. 1000's (I believe)

Citizen science - whale fm screenshot.jpg
Scientists have made sound recordings of both Killer Whales and Pilot Whales (technically both species of dolphin) and allow members of the public to categorized them. For each recording, it shows the location in Google maps, and also presents a visual representation. The user sees several different sounds + representation and you click the best match. BLURB: You can help marine researchers understand what whales are saying. Listen to the large sound and find the small one that matches it best. Click 'Help' below for an interactive guide. (read more) Very easy to use and great interface built on Google Maps. By adding Google Maps, the user can see all the sounds he's classified and has a better feeling of accomplishment. Is hosted on zooniverse. Contributing to science / whale research. unsure
Be a Martian

Citizen science - how to be a martian - tag mars.pngCitizen science - how to be a martian - map mars.png
A NASA site designed mostly for kids, allows users to login and help "map mars" (align high-res satellite images over lower res images) and "count craters" (drawing circles over craters). After creating a login you earn "experience points" and get trophies by visiting the "Map Room". There is also a large educational element, and from the main menu, users can watch videos or read about mars, view a full map of Mars and send virtual postcards to the Spirit Rover on Mars. The site was launched in 2009, a collaborative effort between Microsoft and NASA. It includes many hundreds of thousands satellite images and cost many hundreds of thousands to build! While the graphical artwork is brilliant (each room has a beautiful 3D artwork) they are perhaps too much (to the point of distraction) and I found it was not particularly obvious where to go or what to do! Even when you work out the interface, and how to navigate to the screens that matter, there is no feedback system to tell you when you've done bad, nor sound effects. The only feedback is getting experience points after going through each - absolutely nothing (and I tried this) is stopping you from quickly/blindly clicking done on each images (without aligning or drawing circles) to get level up faster. Help map mars. 50,000 registered users (updated Oct 2010).
Clickworkers (NOTE: site down) Represents a one-year pioneer pilot study by NASA from Nov 2000 - Sep 2001 where volunteer users clicked on images to identify craters on Mars (similar to "Be a Martian" directly above). An analysis showed: "the automatically computed consensus of a large number of clickworkers is virtually indistinguishable from the inputs of a geologist with years of experience in identifying Mars craters."... and that while some clickworkers worked on the project for weeks, 37% of the work was done by one-time contributors. (read more) This project has concluded so sadly you can't test the interface - articles like this one and this one are about all that remains. However this project is still noteworthy as it was one of the first to demonstrate the public's willingness to donate time to help science, and coined the phrase "click-worker" for these users. Contributing to science online. 85,000 visitors over one year!
Gene Games

Dizeez & GenESP
A relatively new project by the Department of Molecular and Experimental Medicine at the Scripps Research Institute and lead by Andrew Su. More games might be added, but in Dizeez people are paired together and have to click what gene they believe is related to a particular disease. In GenESP, you and a partner are shown a keyword and must type genes you think of until you match - very much like the ESP game. I think the theory is to work out what people associate with different words, but I'm far from convinced yet. (read more) The interface is very well designed, but sadly I don't know any gene names, so I was just guessing. Their threshold for participation is very high and probably only aimed at professional chemists / biologists who know gene names and unpublished information. With this limited pool I'm a bit unsure what new information can be gleamed that isn't already commonly known and/or published. Unsure. Unsure.

Reporting (real world) observations


Citizen science - ebird.jpg
User report bird sightings and information is collected into their eBird database. (read more) Haven't tested, but I like the way you can view data on a map. Allow bird-watchers to maintain personal records and interact with fellow twitchers. ~35,000

Citizen science - cocorahs.png
Volunteers use "low cost weather measurement tools" - i.e. they enter rain gauge results into database. The Community Collaborative Rain, Hail & Snow Network (CoCoRaHS) strives for 200,000 members by the end of 2010. N/A Good feeling of contributing 1000's


Wikipedia The famous online encyclopedia where any user can change entries. Despite the occasional people who spam and misuse the system, the greater good prevails and most entires very accurate. No online encyclopedia comes close to Wikipedia in terms of size! Takes a little while to learn syntax, but after that, it's a very nice interface. Contributing to the greatest encyclopedia on Earth! 6+ million
Amazon Mechanical Turk (MTurk) MTurk is a crowdsourcing marketplace. Requesters write these programs and post tasks known as HITs (Human Intelligence Tasks), such as choosing the best among several photographs of a store-front, writing product descriptions, or identifying performers on music CDs. Workers browse HITs and complete them for a monetary payment set by the Requester. (read more) Haven't done, but looks like most HITs are only a few cents for minutes worth of work and interface more complex than should be (article). Making money (although is in VERY small amounts) 100,000+



  • Wikipedia - Citizen Science
  • - great site listing almost a hundred different citizen science projects - most of which concern observation and reporting of animals such as squirrels, fish ("REEF"), birds etc. Others are reporting observation of earthquakes, the weather, stars, some the sending in of DNA (swab kits) ("The Genographic Project"), blood sugar level of diabetics and one of the more unusual one is searching literature to report length of dinosaur bones ("The Open Dinosaur Project").
  • Project Noah - interesting in that it is designed to work with smart phones (iPhone & Android), but is relatively new (youtube).
  • Games for Change - a large collection of games "selected as games that engage contemporary social issues in meaningful ways to foster a more just, equitable and/or tolerant society". 3rd World Famer is a good example.

Acknowledgements: David Lee from UCSD for teaching me about and getting me interested in Citizen Science. Thanks also to Albert Lin and Luke Barrington for taking the time to explain some of the lessons they had learnt from his Valley of the Khans project