[iDC] Can DIY education be crowdsourced?

[John Bell]

Yes, I think identifying and distinguishing types of peers is an important aspect of the kind of system I'm talking about.  The part that's problematic is--without falling back on external validation like degrees and academic positions--figuring out which people are which type, and what the scope of the types are.  For example, I just did something similar for a proposal as part of the Mozilla+Journalism project where I was trying to identify commenters with expertise in different fields so they could add annotation to mass media articles.  In that system a commenter could claim a level of expertise when they made a comment and a trust metric would adjust their long-term credibility based on how other users rate that comment.  It's a refinement of the old Slashdot karma model, but one that seems useful in this situation.

(http://www.nmdjohn.com/2011/08/05/moznewslab-week-4-pitching-reposte/ if anybody is curious.)

But I think there are limits to how much participation can be incentivized without ending up back at cash, which I suspect introduces its own problems.  Look at the situation with Wikipedia where they rewarded participation by turning users into bureaucrats, creating a system that's often accused of being petty and detrimental to the health of the project.  Amazon's biggest reviewer is widely regarded as untrustworthy by people who know who she is, writing reviews of books that she clearly hasn't read (those who don't recognize her of course don't know this, and Amazon doesn't expose enough information for casual users to reach that conclusion on their own).

So the question I'm left with is how to create incentives that go beyond status in the internal community.  Can external incentives be used without creating the equivalent of Warcraft gold farmers?  What would they be?

- John

On Sep 5, 2011, at 6:02 PM, Anya Kamenetz wrote:

> Really interesting stuff, John! Definitely agree with you on the "necessary but not sufficient" formulation. 
> 
> >>But the issue we’d like to discuss with the list is what a system with the same goals--ongoing, deep evaluation of complex learning--would look like if it were designed to work on the same scale as, say, the Khan Academy.  Is peer feedback sufficient to meet those goals?  If so, quality would somehow need to be controlled so that it doesn’t turn into a stream of YouTube comments, and if not some other method would have to be used to deal with large volumes of students.  
> 
> What strikes me is that there are different types of peers--some peers perhaps more equal than others. In a community of practice model there are fellow beginners, who have one type of feedback to offer, then there are people just ahead of you--like the sophomore, junior, senior to your freshman, who have a different type of feedback (less grounded in immediate understanding of what you're going through and more grounded in knowledge and experience), and then graduate student/TA/professor with a more sophisticated offering still. 
> 
> One can imagine a scalable system that incentivizes feedback according to the experience and sophistication of the person offering it, and thus its likely value to the user. Maybe it's a "freemium" model where learners give and receive feedback freely as a condition of participation up to a certain level of experience, and the most experienced participants receive other kinds of incentives (even money?) in exchange for offering the most detailed, sophisticated, time-consuming forms of feedback. 
> I often think back to my summer studying capoeira where the most experienced students took on more and more responsibilities instructing the beginners, as an honor--but only the mestre gets paid. 
> 
> Of course there are other technological ways of encouraging quality control on a large system that depends for its value on freely offered feedback. These are all over the net. TripAdvisor, Amazon, eBay, Quora, Yelp are all good examples--Yelp in particular, again for the way it incentivizes its best providers of feedback, making them a recognized part of a community, allowing the raters to earn ratings. LinkedIn with its endorsement structure another one to look at.
> Maybe you need a system of badges, tags or profile keywords so you can ask a native Brazilian to read your Portuguese paper or a nationally ranked chess player to check out your game or someone with a stellar Github rating to look at your code. 
> a