AI News, The Real Lesson for Data Science That is Demonstrated by Palantir's Struggles

The Real Lesson for Data Science That is Demonstrated by Palantir's Struggles

Buzzfeed recently published a long article on the struggles of the secretive data science company, Palantir.

Over the last 13 months, at least three top-tier corporate clients have walked away, including Coca-Cola, American Express, and Nasdaq, according to internal documents.

Palantir mines data to help companies make more money, but clients have balked at its high prices that can exceed $1 million per month, expressed doubts that its software can produce valuable insights over time, and even experienced difficult working relationships with Palantir’s young engineers.

It looks like they are going through pretty standard growing pains, trying to scale the business and diversify the customer base.

Data science activities live on a spectrum with software on one end and highly customized consulting on another end (I lump a lot of things into consulting, including methods development, modeling, etc.).

The idea being that if someone comes to you with a data problem, there are two extremes that you might offer to them: The first option is cheap, simple, and if you had a good enough web site, the person probably wouldn’t even have to talk with you at all!

For example, I use this web site for sample size calculations and I’ve never spoken with the author of the web site.

The second option is labor intensive, time-consuming, ongoing in nature, and is only scalable to the extent that you are willing to forgo sleep and maybe bend the space-time continuum.

In April 2015, employees were informed that American Express (codename: Charlie’s Angels) had dumped Palantir after 18 months of cybersecurity work, including a six-month pilot, an email shows.

“We struggled from day 1 to make Palantir a sticky product for users and generate wins,” Sid Rajgarhia, a Palantir business development employee, said in the email.

He added that Coca-Cola’s “working relationship” with the youthful Palantir employees was “difficult.” The Coke executive acknowledged that the beverage giant “needs to get better at working with millennials,” according to Kelt.

For example, Palantir’s long experience working with the US military and intelligence agencies gave them deep expertise in those areas, but how does that help them with a consumer products company?

In fact, running the models probably is the easy part, but getting to the point where you can actually fit models can be incredibly hard.

I think the key to answering this question lies in the description of the company itself: The company, based in Palo Alto, California, is essentially a hybrid software and consulting firm, placing what it calls “forward deployed engineers” on-site at client offices.

Given the tremendous difficulty of turning data analysis into a software problem, my guess is that they are more akin to a consulting company and are overvalued.

Palantir Technologies

It has since expanded its customer base to serve state and local governments, as well as private companies in the financial and healthcare industries.[5] Karp, Palantir's chief executive officer, announced in 2013 that the company would not pursue an IPO, as going public would make 'running a company like ours very difficult'.[6] The company was valued at US$9 billion in early 2014, with Forbes stating that the valuation made Palantir 'among Silicon Valley's most valuable private technology companies'.[6] As of December 2014, Peter Thiel was Palantir's largest shareholder.[6] In January 2015, the company was valued at US$15 billion after an undisclosed round of funding with US$50 million in November 2014.[7] This valuation rose to US$20.33 billion in late 2015 as the company closed an $880 million round of funding.[2] Palantir has never reported a profit, and in 2018 Morgan Stanley valued the company at $6 billion.

Byers executive lectured the founders about the inevitable failure of their company.[12] The only early investments were $2 million from the U.S. Central Intelligence Agency's venture capital arm In-Q-Tel, and $30 million from Thiel himself and his venture capital firm, Founders Fund.[3][4][13][14][15] Palantir developed its technology by computer scientists and analysts from intelligence agencies over three years, through pilots facilitated by In-Q-Tel.[16][3] The company stated computers alone using artificial intelligence could not defeat an adaptive adversary.

However, at the time the United States Army continued to use its own data analysis tool.[28] Also, according to TechCrunch, the U.S. spy agencies such as the CIA and FBI were linked for the first time with Palantir software, as their databases had previously been 'siloed.'[28] In September 2013, Palantir disclosed over $196 million in funding according to a U.S. Securities and Exchange Commission filing.[29][30] It was estimated that the company would likely close almost $1 billion in contracts in 2014.[31] CEO Alex Karp announced in 2013 that the company would not be pursuing an IPO, as going public would make “running a company like ours very difficult.”[6] In December 2013, the company began a round of financing, raising around $450 million from private funders.

As of December 2014, Thiel was Palantir's largest shareholder.[6] The company was valued at $15 billion in November 2014.[33] In June 2015, Buzzfeed reported the company was raising up to $500 million in new capital at a valuation of $20 billion.[34] By December 2015, it had raised a further $880 million, while the company was still valued at $20 billion.[35] In February 2016, Palantir bought Kimono Labs, a startup which makes it easy to collect information from public facing websites.

The software connects to commercial, proprietary and public data sets and discovers trends, relationships and anomalies, including predictive analytics.[39][40] Aided by 120 “forward-deployed engineers” of Palantir during 2009, Peter Cavicchia III of JPMorgan used Metropolis to monitor employee communications and alert the insider threat team when an employee showed any signs of potential disgruntlement: the insider alert team would further scrutinize the employee and possibly conduct physical surveillance after hours with bank security personnel.[39][40] The Metropolis team used emails, download activity, browser histories, and GPS locations from JPMorgan owned smartphones and their transcripts of digitally recorded phone conversations to search, aggregate, sort, and analyze this information for any specific keywords, phrases, and patterns of behavior.[39][40] In 2013, Cavicchia may have shared this information with Frank Bisignano who had become the CEO of First Data Corporation.[39] The company has been involved in a number of business and consumer products, designing in part or in whole.

The employee was later reinstated.[51] On September 26, 2016, the Office of Federal Contract Compliance Programs of the U.S. Department of Labor filed a lawsuit against Palantir alleging that the company discriminated against Asian job applicants on the basis of their race.[53] According to the lawsuit, the company 'routinely eliminated' Asian applicants during the hiring process, even when they were 'as qualified as white applicants' for the same jobs.[54] Palantir settled the suit in April 2017 for $1.7 million while not admitting wrongdoing.[55] During questioning in front of the digital, culture, media and sport select committee, Christopher Wylie, the former research director of Cambridge Analytica, said that several meetings had taken place between Palantir and Cambridge Analytica, and that Alexander Nix, the chief executive of SCL, had facilitated their use of Aleksandr Kogan's data which had been obtained from his app 'thisismydigitallife' by mining personal surveys.

Palantir Connects the Dots With Big Data

When you live and work in Silicon Valley, you grow accustomed to a kind of semantic saturation from overused buzzwords.

Terms such as “disruptive,” “innovative,” and “mission driven” come to mind—all favorites among Valley startups, whether they’re building operating systems for robotic arms or phone-based bowling games.

Palantir, currently valued at about $20 billion, is one of tech’s biggest “unicorns.” Its two main products, Gotham and Metropolis, serve the same basic purpose—bringing together massive, disparate data sources and scouring them for connections and patterns that aren’t obvious to the human eye.

There’s nothing ironic about the superhero references: Having deployed their products to address crises like cybercrime, natural disasters, and the ugly byproducts of civil war, Palantir’s employees firmly believe that they’re members of the Justice League.

“If we could help them make sense of data, they could end indiscriminate surveillance.” “This is protecting civil liberties through the use of software,” says Karp, referring to the company’s partnerships with governments, “and therefore protecting societies from hitting the button against all civil liberties.” For more on Apple’s privacy case, watch this Fortune video: It’s a tricky balancing act, but it’s at the core of Palantir’s operations—and one that will guide the company as it takes on serious business challenges.

Launched in 2004 by Karp and a group of techies led by PayPal PYPL co-founder Thiel, Palantir has raised about $2 billion in funding over the years from investors like hedge fund Tiger Global, Yelp YELP co-founder Jeremy Stoppelman, and Thiel’s venture firm, Founders Fund.

It doesn’t report current revenue, but the company told Fortune that it brought in more than $1 billion in 2014 and that its 2015 commercial bookings nearly doubled from the previous year;

They also took Fortune inside some of their humanitarian projects—most notably, an ambitious refugee-relief effort in Syria—which aim to show a skeptical public that Palantir’s data mining can be an unequivocal force for good.

The Real Lesson for Data Science That is Demonstrated by Palantir's Struggles

Buzzfeed recently published a long article on the struggles of the secretive data science company, Palantir.

Over the last 13 months, at least three top-tier corporate clients have walked away, including Coca-Cola, American Express, and Nasdaq, according to internal documents.

Palantir mines data to help companies make more money, but clients have balked at its high prices that can exceed $1 million per month, expressed doubts that its software can produce valuable insights over time, and even experienced difficult working relationships with Palantir’s young engineers.

It looks like they are going through pretty standard growing pains, trying to scale the business and diversify the customer base.

Data science activities live on a spectrum with software on one end and highly customized consulting on another end (I lump a lot of things into consulting, including methods development, modeling, etc.).

The idea being that if someone comes to you with a data problem, there are two extremes that you might offer to them: The first option is cheap, simple, and if you had a good enough web site, the person probably wouldn’t even have to talk with you at all!

For example, I use this web site for sample size calculations and I’ve never spoken with the author of the web site.

The second option is labor intensive, time-consuming, ongoing in nature, and is only scalable to the extent that you are willing to forgo sleep and maybe bend the space-time continuum.

In April 2015, employees were informed that American Express (codename: Charlie’s Angels) had dumped Palantir after 18 months of cybersecurity work, including a six-month pilot, an email shows.

“We struggled from day 1 to make Palantir a sticky product for users and generate wins,” Sid Rajgarhia, a Palantir business development employee, said in the email.

He added that Coca-Cola’s “working relationship” with the youthful Palantir employees was “difficult.” The Coke executive acknowledged that the beverage giant “needs to get better at working with millennials,” according to Kelt.

For example, Palantir’s long experience working with the US military and intelligence agencies gave them deep expertise in those areas, but how does that help them with a consumer products company?

In fact, running the models probably is the easy part, but getting to the point where you can actually fit models can be incredibly hard.

I think the key to answering this question lies in the description of the company itself: The company, based in Palo Alto, California, is essentially a hybrid software and consulting firm, placing what it calls “forward deployed engineers” on-site at client offices.

Given the tremendous difficulty of turning data analysis into a software problem, my guess is that they are more akin to a consulting company and are overvalued.

Inside Palantir, Silicon Valley's Most Secretive Company

Among the projects Palantir runs at BP is an oil well management system that creates data-rich views of wells being drilled, to help prevent a catastrophic collapse, and a system that generates insights for energy futures trading, according to emails and a person familiar with the deal.

In April 2015, employees were informed that American Express (codename: Charlie’s Angels) had dumped Palantir after 18 months of cybersecurity work, including a six-month pilot, an email shows.

In April, with the pilot done, Palantir engaged in a “somewhat wacky brainstorm” with Nasdaq about potential projects beyond security, according to an email from Melody Hildebrandt, a top executive in Palantir’s commercial division.

News Corp was “pleased” with Palantir’s work, though it “evolved to an internal team, as planned,” meaning News Corp now has its own employees doing something similar, News Corp spokesperson James Kennedy told BuzzFeed News in an email.

Palantir CEO says investors will be 'positively surprised' at the company's margins

Some 1,500 employees at the carmaker, including assembly-line managers, use the software to identify production problems and potential safety issues, he said.

Global revenue for big data and business analytics will grow to $210 billion by 2020 from $151 billion in 2017, according to research firm IDC.

The tech giants are all playing in the space, with Microsoft, IBM, Oracle and SAP selling big data and analytics tools.

Palantir Technologies

Palantir Technologies, Inc. is an American computer software and services company, specializing in data analysis. Primary clients are the US government ...

Facebook CEO Mark Zuckerberg testifies before Congress on data scandal

Facebook CEO Mark Zuckerberg will testify today before a U.S. congressional hearing about the use of Facebook data to target voters in the 2016 election.

Rose Psalmond - Field Development Planning

Field development modeling, rig scheduling, and portfolio optimization are made easy with Palantir's Field Development Planning™ software suite. Our software ...

Lecture 12 - Building for the Enterprise (Aaron Levie)

Lecture Transcript: Aaron Levie - founder of Box, enterprise master, Twitter ..

Dataminr in Action

At a rate of 500 million tweets per day, finding actionable information when it first emerges on Twitter is a tremendous challenge. Dataminr algorithmically ...

Power of Information Conference: Analyzing Disparate Data

Moderator: Paul Light, Professor of Public Service, New York University Panel: Diego May, CEO and Founder, Junar; Raphael Majma, Senior Advisor, White ...

Data Driven Models for Change #SkollWF 2017

Corporations increasingly leverage data to dissect potential markets and target relevant consumers. How can the social sector apply this methodology for ...

Rescale : Joris Poort インタビュー|Digital Research

【日本語字幕あり】Rescale - Joris Poort Founder & CEO インタビュー 「膨大なリソースから解決策を探るクラウドベースのシミュレーションプラットフォ...

Live: Mark Zuckerberg Testimony to Senate Judiciary and Commerce Committees

Mark Zuckerberg is scheduled to appear before a joint hearing of the Senate Judiciary and Commerce Committees today.The hearing will focus on the use of ...

Gurjeet Singh: Shaping the Future of Data [Entire Talk]

Ayasdi Co-Founders Gunnar Carlsson and Gurjeet Singh discuss building a company based on technology developed inside a university, in conversation with ...