Blog

I gave a talk at BarCampNYC4 on Saturday on common data problems and a very light overview of algorithms that address them.

My talk at barcampnyc4 – photo courtesy of dynamist on flickr.

I delivered the majority of the content verbally, by talking through examples of problems and how to solve them, so there’s no guarantee that these slides will make sense, but they might be funny!

Sanford took some excellent notes during the presentation.

There were some very nice comments on twitter.

The discussion was so lively and engaging that I’m planning to expand on this content — I really welcome your suggestions and comments!

Jon Udell hosted Charlie and I on his Interviews with Innovators podcast.

We discussed Path101’s approach to career advice through data, and how the high availability of data is changing the way we make decisions.

Listen here.

The April edition of Communications of the ACM has an interesting article on recent advances in active learning by Graeme Stemp-Morlock.

In passive learning (a more traditional approach), you build a large training set of classified data by (often) manually assigning labels. This data is used as the basis of your analysis.

In the real world, we find that generating these large sets of labeled data is often expensive and time consuming. With active learning, you identify the most ambiguous data to label, resulting in a much higher payoff for each label defined (and fewer headaches for your labelers).

The article goes on to mention that active learning is being used in practice with excellent results (for example in music identification, text classification, and even bioinformative), but that the theory lags. This is another example of a gap between the world of the practitioner and the academic work behind it.

Second Life news stand

One of the more popular objects that I created in Second Life is the News Stand. This charming device takes any query term and displays the two latest headlines from Yahoo! News. Clicking the news stand will load the top news story for the topic in a web browser.

I’ve gotten more requests to customize the script than I can possibly keep up with, so I’ve decided to release the code under a creative commons license. I hope you find it useful! Please let me know if you make improvements, so that I can link to them from here.

Unfortunately, parsing XML at all and RSS feeds in particular is extremely messy in LSL. This script doesn’t contain a general RSS feed parser — the parsing code was written specifically for the Yahoo! news feed.

// News Stand script
// by Ann Enigma
 
string url="http://news.search.yahoo.com/news/rss?p=";
key requestid;
string current_url = "http://news.yahoo.com";
integer gLine = 0;
key data_card; // for settings notecard
string query;
 
integer mins = 10; // number of minutes between refreshes
 
get_news() {
    llWhisper(0, "Refreshing the news...");
    requestid = llHTTPRequest(url,[HTTP_METHOD,"GET"],"");       
}
 
default
{
    state_entry()
    {
        data_card = llGetNotecardLine("Settings",gLine); // load settings from notecard
    }
 
    // read data from the notecard
    dataserver(key query_id, string data) {
        if (query_id == data_card) {
            if (data != EOF) {    // not at the end of the notecard
                if (gLine == 0) {
                    query = data;
                } 
            }
        }
        state setup;
    }
    changed(integer change) {
        llResetScript();
    }
}
 
state setup {
    state_entry()
    {
        url += llEscapeURL(query); // set the url
        get_news(); // get the news!
        llSetTimerEvent(60*mins); // every mins # of minutes
    }
 
    touch_start(integer total_number)
    {
       llLoadURL(llDetectedKey(0), "Current Top Story", current_url);
    }
    http_response(key req_id, integer status, list metadata, string body) {
 
        integer count = 0;
        string display;
 
        if (req_id == requestid) { // parse the results (unfortunately messy)
            list result = llParseString2List(body,["<",">","\n","[","]"],[]);
            integer length = llGetListLength(result);
            integer i = 0;
            for (i = 0; i < length; i++) {
                if (llList2String(result,i) == "title" && llList2String(result,i+4) == "/title") {
                    display += llList2String(result,i+3) + "\n";
                    if (count == 1) { 
                        current_url = llList2String(result,i+6);
                    }
                    count++;
                }
            }
            llSetText(display, <1,1,1>, 1.0);
        }
    }
    timer() {
        get_news();
    }
    changed(integer change) {
        llResetScript();
    }
 
}

This script is released under . Have fun!

Create a notecard called “Settings”, with a single line containing your search term. For example:

Educational Technology

The original news stand comes with these instructions, which may come in handy:

To set up the News Stand, simply open the "Settings" notecard, delete the default search term, and type in your own!

This gadget will refresh every 10 minutes, and will display the top two headlines on Yahoo! News for any search term. If you click on the news stand, it will load the top story in a web browser.

If you would like to build your own news stand, you can simply drag the script and "Settings" notecard into any object.

If you want the no-assembly-required version, the News Stand is available for L$350 at Xstreet SL Marketplace or from the ICT Library on Info Island in-world.

The original news stand object was created by (now former) student Kyle Pouliot.

WHEREAMI is a mobile application that accepts a username as input, searches public profiles on various location-aware services, and returns the user’s last known location via text message.

Just text 41411 with whereami <username>, where <username> is a username that you or someone you know is likely to use.

For example, if you text whereami hmason to 41411, you’ll see a response much like the image to the left.

This app works on the principle that people tend to use the same username for many applications. The WHEREAMI script will search through a variety of web services for a result for that username. All of the information is public and available without logging in.

Right now, the script will search Brightkite, then Dopplr, and finally Twitter. If you know of another site with public user location information, please comment below and I’ll add it!