Category: Uncategorized

Posted: August 12, 2013 | Author: Hilary Mason | Filed under: blog | Tags: datagotham, nyc | Leave a comment »

Registration is open for DataGotham 2013, our second annual New York data community conference, September 12th and 13th. The core of the conference is a series of brilliant data practitioners telling the stories about what they work on. The content is technically-oriented but not all deeply technical, and we really welcome anyone curious about how New York companies and institutions are pushing the boundaries on data to attend.

We have two goals for the conference. The primary goal is to connect people in the greater New York data community who are working on interesting things. If our community is strong and supportive, we will all do better work.

Our second goal is to highlight the amazing working happening here, so that people near and far will realize that New York is the best place in the world to do data science.

Come join us to hear these stories firsthand and meet fellow data-minded practitioners! Register now:

(Readers of this blog can use discount code “IheartNYC” for 10% off, and I hope to see you there!)

Posted: April 14, 2013 | Author: Hilary Mason | Filed under: blog | Tags: google, search | 4 Comments »

In 2008, cuil, a search engine startup, displayed my bio alongside a photo of deceased actress Hilary Mason. In January 2013, Bing confused us, this time putting my photo next to her bio (they fixed it after a suitable amount of mocking on Twitter).

Today, Google did the same thing. (live search link)

Today I win the internet?

If you zoom in on the bio section, you can clearly see that it’s her bio with a photo of me (originally from Crain’s New York 40 under Forty). Further, if you go into her filmography, you continue to see my photo.

I’m most proud of my starring role in the amazing film Robot Jox. (bottom right of the image below)

I know that entity disambiguation is a hard problem. I’ve worked on it, though never with the kind of resources that I imagine Google can bring to it. And yet, this is absurd!

Posted: April 1, 2013 | Author: Hilary Mason | Filed under: blog | Tags: bitly, data, engineering, infrastructure | 3 Comments »

Data engineering is when the architecture of your system is dependent on characteristics of the data flowing through that system.

It requires a different kind of engineering process than typical systems engineering, because you have to do some work upfront to understand the nature of the data before you can effectively begin to design the infrastructure. Most data engineering systems also transform the data as they process it.

Developing these types of systems requires an initial research phase, where you do the necessary work to understand the characteristics of the data, before you design the system (and perhaps even requiring an active experimental process where you try multiple infrastructure options in the wild before making a final decision). I’ve seen numerous people run straight into walls when they ignore this research requirement.

Forget Table is one example of a data engineering project from our work at bitly. It’s a database for storing non-stationary categorical distributions. We often see streams of data and want to understand what the distributions in that data look like, knowing that they drift over time. Forget Table is  designed precisely for this use, allowing you to configure the rate of change in your particular dataset (check it out on github).

Posted: March 23, 2013 | Author: Hilary Mason | Filed under: blog | Tags: google | 8 Comments »

Google Now is an extension to Google’s Android search app that uses all of the data that Google has about you along with what it can guess about your current context to present the information it thinks you need when it thinks you need it.

It’ll tell you to leave a bit early to make your next calendar event because of heavy traffic, or that it’s a friend’s birthday, or that there’s a cool cafe nearby where you are.

I think it’s amazing.

It’s amazing because this is the first Google product that takes ALL OF THE DATA that they have about us and actually makes it useful for us. Not for advertisers.

Finally.

Posted: February 25, 2013 | Author: Hilary Mason | Filed under: blog | Tags: blogs, data science | 4 Comments »

I’m gathering a bundle of data science blogs to share. I’m looking to include blogs that update regularly and aren’t either personal opinion and project blogs (like this one) or primarily about marketing any particular company. Let me know if you have a favorite that I’ve forgotten.

If you’re just looking for one place to start, hop on over to Simply Statistics.