Knight News Challenge application to improve Memetracker and Content Recommendation Engine modules

I just submitted my application to the Knight News Challenge. My application is for money to bring the Memetracker and Content Recommendation Engine modules to production ready status. Memetracker, is of course, the module I wrote this past summer as part of Google Summer of Code.

Title:

Improve modules for semi-automated news aggregation and content recommendation in Drupal

Describe your project:

All organizations, large and small, have a vital need to deliver relevant and timely information to its members. My project will be to make it possible for organizations to easily meet this need. I will work on two Drupal modules, write documentation, and build a Drupal install profile so that organizations can easily add sophisticated news aggregation and recommendation tools into their Drupal website.

The two Drupal modules I will improve are Memetracker and Content Recommendation Engine.

I wrote Memetracker as part of the 2008 Google Summer of Code. The Memetracker module uses machine learning algorithms to intelligently filter and group all types of content. The module’s purpose is to find and display to a community in real time the most interesting conversations and memes on relevant topics as they emerge.

My goal for the memetracker module is for it emulate functionality of successful commercial memetrackers such as Techmeme, Google News, Tailrank, and Megite. I want it to be a robust open-source implementation of memetracking technology that can be easily plugged into Drupal-based community sites.

The Content Recommendation Engine module is designed to provide personalized content recommendation. It learns what types of content individuals are interested in and recommends new content as it comes in.

Both modules are powerful ideas but need quite a bit of work to be usable in real-life situations. I would use the Knight Foundation money to do fix bugs, add requested functionality, and to create an install profile which makes it very easy for non-technical end-users to install a sophisticated Drupal-based news aggregation and recommendation site.

How will your project improve the way news and information are delivered to geographic communities?

There are many tools available to communities to aggregate and distribute information. What’s missing are open source tools which leverage not just human intelligence to filter content but also artificial intelligence.

There is far too much information for any person or organization to sort through manually. These automated tools can be thought of as pre-processors that improve the signal-to-noise ratio reducing the stress people endure trying to follow news. By filtering out the noise, important news is much more likely to be identified and acted upon.

How is your idea innovative? (new or different from what already exists)

There are not any open source tools that do the automated content filtering and recommendation these two tools will do. In addition, the few tools that are similar to these are standalone applications where as my work would be built on Drupal, the most widely used open-source social publishing platform. This means two things. First, my tools would get wider adoption as they would fit into many organization’s existing technology stack. Second, they are more useful as they can take advantage of many other powerful modules available for Drupal. The tools will be the basic building blocks of a rich flowering of content aggregation / filtering web applications based on these modules and Drupal.

What experience do you or your organization have to successfully develop this project?

I was a Google Summer of Code student this past summer where I wrote the Memetracker module.

I traveled to Drupalcon in Hungary and presented there on the Memetracker module and on content filtering in general. The session and Q&A can be viewed here:

http://www.archive.org/details/memtracker

Currently I’m building and maintaining a large Drupal-based social learning platform at Brigham Young University. As I develop tools for the site and work with and observe the 100s of student users (soon to be thousands), I’m developing a deep understanding of how information spreads though a community and how to develop technology to facilitate that process. I hope to use both modules extensively in this website. The social learning website can be viewed here: https://island.byu.edu

I have been heavily involved in the Drupal community for the past 1.5 years. From this experience, I have obtained a good understanding about how open source development works and am confident I will be able to build these tools such that they are easily modified and extensible to meet the varying needs of different organizations.

I am passionate about building tools that help organizations digest, interact around, and act upon information. I would very much appreciate support from the Knight Foundation to continue to improve the Memetracker and the Content Recommendation Engine modules and make them widely available. Thank you.

Subscribe to get updated on new posts!