Author: hmason

I read Out of the Loop in Silicon Valley in the NYTimes today, which explores how and why women are under-repesented in tech startups. From the number of retweets I saw and the clicks through bit.ly links (12,579 at the time of this posting), it’s been getting a lot of attention.

There are some very strong, compelling themes in this article. Computer science and engineering to have an “image problem”; the way we teach math to elementary school students is horrible and turns way too many away.

I don’t want to nitpick the article, but there are a few statements that reinforce the very damaging stereotypes that the article sets out to dispel.

“When women take on the challenges of an engineering or computer science education in college, some studies suggest that they struggle against a distinct set of personal, psycho-social issues… Even women who soldier though [sic] demanding computer science and engineering programs in college…”

I’ve been both a computer science student and a computer science professor. I have not seen any evidence that the average undergraduate computer science education is harder than physics, math, chemistry, biology, or many other ‘hard’ disciplines with a much stronger gender balance. Implying that women are unwilling to meet the intellectual challenges of the discipline is bullshit.

“Girls have certain family goals they want to accomplish,” she says. “Working 60 hours a week is difficult because it requires a life sacrifice.”

The men that I know and work with also have wonderful personal lives. Working 60 hour weeks is a sacrifice for them, too.

Please read the whole article. Let me know what you think when you see the material in context.

I’m going to make the assumption that we all believe that having more women in technology is a Good Thing^tm.

Many groups have popped up that support women in technology, like Girls in Tech, She’s Geeky, and many others (enumerated in Digiphile’s thoughtful post Why Including women matters for the future of technology and society). More often than not, these groups are the canned food drives of the women in technology movement. They make you feel better, they might do a little good, but they offer no fundamental change to the system that created the problem in the first place.

The Grace Hopper Celebration of Women in Computing does this well. GHC invites women to come to one place, be together, and do science together.

We don’t need affirmative action for women in tech. We need to create experiences that nurture women and men so that more people are inspired to can create beautiful, technical things together.

I recently attended the Third Annual Workshop on Search and Social Media, an academic workshop with very strong industry participation. The workshop was packed, and had some of the most informative and interesting panel discussions I’ve seen (not counting the one I spoke on!).

Daniel Tunkelang did a great job of writing up the specific presentations on his site and on the ACM blog, so I won’t attempt to re-create the presentations line by line at this late date. Rather, I’d like to highlight a few open problems and research questions that came out of the discussions that I hope to see developed in the next year.

Social search consists of a set of problems including (but hardly limited to) search of social content like status updates, real-time search, generating, labeling, and finding user-generated content, ‘long-tail’ events and interests, finding vs re-finding, and trend identification.

What data is available to social search? There are many kinds of social data, from e-mail (private) to blogs (public) and tweets (mostly public) — what is and should be searchable? How do we handle issues of privacy and identity management?

How do we compute relevance, taking into account freshness, accuracy, and degrees of social separation?

Will the architecture of these search engines look like the search engines we’re currently familiar with?

How do we evaluate accuracy and truthiness of social data?

How do we characterize social connections, around concepts like strong vs weak ties, and friend-of-a-friend vs friend-of-a-friend’s-friend? Can we converge on a single social graph representation?

How do we best filter social data to lead to accurate recommendations for content discovery? How do we accommodate the fact that as we move beyond static factual data, two people using the same query may be looking for very different results?

Finally, how do we deal with the chasm between the industry participants (who have LOTS of data) and the academic participants, who suffer from a lack of public (and publishable) data?

Thanks again to the organizers – Eugene Agichtein, Marti Hearst, Ian Soboroff, and Daniel Tunkelang – who put together a fantastic event.

For more on this and a cool demo, check out Gene Golovchinsky’s look at the SSM2010 twitter coverage.

Over at NYC Resistor, it was getting cold, and we needed a doorbell so visitors wouldn’t be stranded outside when the building was locked. A standard wireless model didn’t work reliably (the space is on the fifth floor, just out of range), so various members generally resorted to writing their phone numbers on a sign on the front door when they were expecting guests.

Since almost everyone has a mobile phone already, and SMS-based solution seemed appropriate. In order to implement this we need two things:

1. An SMS shortcode
2. A system to notify when the shortcode is triggered

It’s irritating and expensive to acquire your own shortcode, but there are several services that will allow you to use one in exchange for a small fee or advertisements in your messages. TextMarks is my favorite (I used TextMarks for my WhereAmI project). While TextMarks markets their service as a system for mobile mailing lists, they allow you to reserve a keyword and define a behavior (that can include pulling data from a URL!) to occur when that keyword is triggered.

Configuring TextMarks

Sign up for TextMarks and choose a keyword. Configure the keyword to respond with the “First 120 characters on web page”, and point it at the future home of your script (you can always come back and modify this later).

Note the \0 as the value of the msg parameter — this instructs TextMarks to send along any additional message contents as the value of that parameter. That means if someone were to text 41411 “doorbell hi this is hilary”, TextMarks would call the script with the param msg=hi this is hilary. This can be quite useful.

The Script

This script is written in Python, but you can use any scripting language you like. This particular script just sends an e-mail to an account when the ‘doorbell’ is rung, but you could have it do pretty much anything up to and including ringing a real bell (which may be coming soon!).

#!/usr/bin/env python
# encoding: utf-8
"""
doorbell.py
Created by Hilary Mason, feel free to use this code in your own projects.
"""
 
import sys, os
import smtplib
import cgi
import cgitb; cgitb.enable()
 
class Doorbell(object):
	GMAIL_USERNAME = 'YOURGMAILACCOUNT@gmail.com'
	GMAIL_PASSWORD = 'YOURPASSWORD'
 
	def __init__(self, msg):
		message = """\
From: YOURGMAILACCOUNT@gmail.com
To: YOURGMAILACCOUNT@gmail.com
Subject: KNOCK KNOCK, someone is at the door!
 
%s
		""" % msg
 
		server = smtplib.SMTP('smtp.gmail.com:587')
		server.ehlo()
		server.starttls()
		server.ehlo()
		server.login(self.GMAIL_USERNAME, self.GMAIL_PASSWORD)
		server.sendmail('YOURGMAILACCOUNT@gmail.com', ['YOURGMAILACCOUNT@gmail.com'], message)
		server.quit()
 
		print "You knocked! You can also call us at 347-586-9270. <3, NYC Resistor"
 
 
if __name__ == '__main__':
	print "Content-Type: text/plain\n\n"
 
	form = cgi.FieldStorage()
	if 'msg' in form:
		w = Doorbell(form['msg'].value)
	else:
		w = Doorbell('There is an anonymous monkey at the door.')

And that’s it! Provided you have your keyword configured to point at your script, and the script living at an accessible address, you’ll get an e-mail whenever your SMS doorbell is rung and the person who sent the message will get back a cute response confirming their action.

Finally…

This setup can be easily extended such that a message containing ‘doorbell hilary’ could e-mail only me, or be forwarded to my phone.

I’m curious to see if having a remotely accessible ‘doorbell’ will encourage pranksters — we might need to add a password.

Last weekend Yahoo came to New York for an Open Hack Day, and it was great!

I was invited to speak on a panel on semantic metadata, moderated by Paul Ford (harpers.org) along with Marco Neumann (KONA) and Paul Tarjan (Yahoo/Search Monkey). The panel was a lively discussion, and we got some great questions from the audience.

After the panel, I stayed around to participate in the hack competition. Yahoo! provided a fantastic space, with free-flowing coffee, snacks, comfy chairs and plenty of Yahoo folks and other hackers around to give advice and play foosball with. I teamed up with Diana Eng, Alicia Gibb, and Bill Ward to create the Del.icio.us Cake!

The cake is attached to a laptop via USB. A program running on the laptop accepts a delicious tag and retrieves a list of recent popular sites for that tag from the delicious API. Finally, it iterates through each URL, downloads the page, and computes the sentiment of that page relative to the tag — basically, is the content of the page positive, neutral or negative?

The signal is output to an ardiuno (hidden in the middle of the cake) which turns on the appropriate set of LEDs. There are four sets of LEDs on the cake, one in each quadrant of the delicious logo, one each for positive sentiment, neutral or inconclusive sentiment, and negative sentiment, and, of course, one to let us know that the cake is turned on.

I wrote the sentiment classifiers between around 3am and 6am Saturday morning, so they really were a hack! I trained them on movie reviews data, working with the assumption that 5-star reviews contain positive terms and 1-star reviews contain negative terms. I wouldn’t recommend this approach for a serious attempt at sentiment analysis, but it worked well enough.

We won the food/hardware hack prize, shared with the awesome MakerBot team!

We had a great time creating and presenting the hack. Thanks, Yahoo, and most of all, thanks to Alicia, Bill, and Diana for a really fantastic, silly weekend.

Further coverage:

• Yahoo’s summary of the Open Hack NYC event
• Diana’s writeup for Eyebeam
• CNN.com: Hackers Take Over Times Square

I’ve found myself in need of a name distribution for a few projects recently, so I thought I would post it here so I won’t have to go looking for it again.

The data is available from the US Census Bureau (from 1990 census) here, and I have it here in a friendly MySQL *.sql format (it will create the tables and insert the data). There are three tables: male first names, female first names, and surnames.

I’ve noted several issues in the data that are likely the result of typos, so make sure to do your own validation if your application requires it.

The format is simple:

1. the name
2. frequency (percentage of people in the sampled population with that name)
3. cumulative frequency (as you read down the list, the percentage of total population covered)
4. rank

If you want to use this to generate a random name, you can do so very easily with a query like this:

SELECT name FROM ref_census_surnames n ORDER BY (RAND() * (n.freq + .01)) LIMIT 0,1;

Download it here: census_names.tar.gz