Friday, December 2, 2011

The Search For Analysts To Make Sense Of 'Big Data'

DJ Patil explains a visualization of a person's LinkedIn network, where different groups of contacts are illustrated by different colors. Patil developed big data projects for LinkedIn — like algorithms to determine whom you might know. He now recruits mathematicians for a venture capital firm. Enlarge Joi Ito/Flickr

DJ Patil explains a visualization of a person's LinkedIn network, where different groups of contacts are illustrated by different colors. Patil developed big data projects for LinkedIn ? like algorithms to determine whom you might know. He now recruits mathematicians for a venture capital firm.

Joi Ito/Flickr

DJ Patil explains a visualization of a person's LinkedIn network, where different groups of contacts are illustrated by different colors. Patil developed big data projects for LinkedIn ? like algorithms to determine whom you might know. He now recruits mathematicians for a venture capital firm.

Second in a two-part series

Businesses keep vast troves of data about things like online shopping behavior, or millions of changes in weather patterns, or trillions of financial transactions ? information that goes by the generic name of big data.

Now, more companies are trying to make sense of what the data can tell them about how to do business better. That, in turn, is fueling demand for people who can make sense of the information ? mathematicians ? and creating something of a recruiting war.

DJ Patil, with venture capital firm Greylock Partners, is on a perpetual manhunt, looking for a rare breed: someone with a brain for math, finesse with computers, the eyes of an artist and more.

"There's one common element across all these people that stands out above everything, and that's curiosity," Patil says. "It's an intense curiosity to understand what's behind the data."

He compares raw data to clay: shapeless until molded by a gifted mathematician. A good mathematician can write algorithms that can churn through billions or trillions of data points and show where patterns emerge.

For example, patterns indicated early on that mothers were heavy users of social networks, which in turn led to the creation of social circles. Patil says a good mathematician can figure out what matters and what doesn't in a huge trove of data.

Courses at Brown University visualized.

Courtesy Coursekick

Dylan Field, a junior at Brown University who is being recruited by big data companies, created this visual analysis of course data at Brown with fellow students. Each dot represents a course, and its size indicates how interdisciplinary it is. Similar courses are grouped together, showing how different academic fields relate. The information is then used for CourseKick, which helps students discover new courses to take.

The Climate Corp's visualization of soil type.

Courtesy The Climate Corp

The Climate Corp. uses 50 terabytes of live data ? including weather measurements from 2.5 million locations and 150 billion soil observations ? to create weather insurance for farmers. Its founders learned big data principles from their time at Google and now predict risk for weather events like droughts or flooding. This graphic is the first of its kind to visualize soil type; darker blue coloring shows a higher water holding capacity, better for growing crops, while lighter colors have less capacity.

"Everybody's looking for these people, because they know these individuals can move the needle by themselves. They're that impactful," he says.

Firms are going beyond offering piles of cash and equity to get these people. The chief executive officer at TellApart, a company that helps online marketers profile its customers, invited one math recruit to dinner with other tech luminaries to sell him on the company. Another California firm, Cataphora, which helps firms monitor employee behavior, opened a satellite office in Ann Arbor, Mich., so it could recruit from the University of Michigan's math department.

Often, executives say it's not just about money; they have to appeal to an ideal.

David Friedberg says his company does that by touting the power of risk management to change agriculture. Friedberg, chief executive officer of The Climate Corp., says crop insurance is basically nonexistent in large parts of Africa and Southeast Asia. He says the ability to better model changing climate patterns means his company can provide insurance to small farmers who otherwise might not take on the risk of farming more land.

"We can actually encourage agricultural development and provide a sustainable living for them," Friedberg says. "And that's one of the long-term missions of our organization."

The type of people drawn to this work aren't necessarily what you might expect. Greylock Partners' Patil says his successful recruits have included an oceanographer and a neurosurgeon, as well as people who barely graduated from high school but were brilliant at math. He approaches math majors the way baseball scouts look for young stars.

"I have a list that I track. There's one student who I think is phenomenal; I've been tracking him since he was 16," Patil says.

That student is Dylan Field, now 19 and a junior at Brown University. He remembers that the summer of first grade, he found an algebra book and dug into it. Now, he says math and statistics are the sexiest skills around, and his early aptitude and interest in math have become, literally, his meal ticket.

Dylan Field is a junior at Brown University who has been on Greylock Partners recruiter DJ Patil's watch list since he was 16. Enlarge ?

Dylan Field is a junior at Brown University who has been on Greylock Partners recruiter DJ Patil's watch list since he was 16.

?

Dylan Field is a junior at Brown University who has been on Greylock Partners recruiter DJ Patil's watch list since he was 16.

"The joke is that so many companies come to recruit these days, that you don't need to be on the meal plan, you can just go to their recruiting events and get food there," he says.

Field says it's a happy coincidence the market wants what he loves to do.

"You can understand something that is much bigger than yourself, and I think that is the most interesting property of creating a big data application," he says.

Field plans to spend much of next year working at a big data startup called Flipboard. His ultimate goal, he says, is to start his own big data company.

Source: http://www.npr.org/2011/11/30/142893065/the-search-for-analysts-to-make-sense-of-big-data?ft=1&f=1007

crystal cathedral sarah vowell fire in reno plane crash plane crash kelly ripa reno wildfire

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.