Report from the 2013 O’Reilly’s Strata Conference on Big Data
Quo Vadis Big Data?
The Strata Conference has become the watershed event for Big Data in the US, and through its ecosystem’s capillarity, beyond. This year there were two key differences from previous Strata conferences. First, there was a significant presence of the larger players with Cisco, Intel, VMWare, Microsoft, and EMC putting forward solutions and proselytizing their technologies. Second, the subjects covered expanded beyond purely technical and functional subjects to include sessions on Law, Ethics, and Open Data as well as Design and Next Gen Data Scientists. Of course core subjects such as Tools for Hadoop Developers and an Introduction to Apache Drill were also very well represented.
Large enterprise vendors are finding that they can’t escape the decidedly non proprietary nature of Big Data software and hardware. To address this, they are trying various ways to make their offers palatable to enterprises seeking footholds in Big Data. This includes addressing the shortages of resources like Data Scientists, co-existence and even integration of seemingly fractious technologies like SQL and NoSQL, and abstraction of the technical features of the systems through new and innovatively designed user-friendly interfaces.
What does 2013 have in store for Big Data? Certainly we will see consolidation around Hadoop of this currently highly fragmented industry which has almost as many packaged versions as there are vendors. Arguably we will see an increased tempo of acquisitions of pure-play Hadoop start-ups such as Cloudera and MapR. Lastly, we see the maturing of the core Hadoop engine and a shift of its creators to adjacent open source projects such as optimizing real-time performance, scalability, and cost, all factors not addressed in the original core Hadoop specification.
The comprehensive conference report was assembled by the staff of Orange Silicon Valley, combining efforts from the Platform & Middleware and Enterprise groups. The conference organizer, O’Reilly Media, and its founder Tim O’Reilly, is widely regarded as the original evangelist of the Big Data trend in IT, and this conference as its annual gathering place of record.
HADOOP IS 'BIG DATA PLATFORM OF CHOICE'
During Strata Santa Clara 2013, conference chair Alistair Croll asked the rhetorical question “who knew that the future of NoSQL was SQL?”, underscoring that Big Data technologies could only go mainstream by bridging the gap with legacy systems, most importantly SQL. At this year’s conference, Hadoop was confirmed as the Big Data platform of choice, and gained the support of legacy vendors, opening the doors to a much broader adoption.
"WHO KNEW THE FUTURE OF NOSQL WAS SQL?"
Alistair Croll co-founded web performance startup Coradiant, and since that time has also launched Rednod, CloudOps, Bitcurrent, Year One Labs, the Bitnorth conference, the International Startup Festival and several other early-stage companies.
Director, Consumer Group
& Middleware Group
IT & Middleware Group
IT & Middleware Group
VP, Technology Development
This yellow elephant is the logo of Hadoop, the industry standard open-source platform for dealing with Big Data.
1. HortonWorks brings Hadoop to Windows
Microsoft Windows servers still represent 73 percent of the market in 2012, according to IDC. HortonWorks’ move to get Hadoop to run on Windows should help its distribution stand out from competition, and will let enterprises use their favorite analytics tool, Microsoft Excel, to manipulate their data, both structured and unstructured. It also helps Microsoft muscle up its hybrid cloud strategy.
HortonWorks’s distribution will run on Windows Servers, on premise, and on Windows Azure, in the cloud, and will therefore allow Microsoft to better compete with Amazon Web Services. HortonWorks
2. EMC/VMware/Greenplum melts analytics database with Hadoop
Greenplum announced Pivotal HD, a completely re-architected Hadoop distribution that has been natively fused with Greenplum’s analytics database. Pivotal HD promises impressive performance vis-à-vis Hive (an open-source project that allows to run SQL queries on Hadoop), and Cloudera Impala, and perhaps most importantly, brings a trusted name familiar to enterprises worldwide.
Greenplum doesn’t plan on making any significant contribution to Hadoop, as its analytics database is its core product. It has been confirmed to us informally, that Pivotal HD will be offered as a service via Pivotal’s upcoming public cloud offer.
3. Intel leads legacy enterprise vendors into the Big Data world
In a surprising move, Intel has decided to create its own Hadoop distribution. Intel is committed to contributing developments back to the Hadoop community.
The chip company is threatened on multiple fronts by ARM, and expanding its activities into software is a defensive move to protect its x86 architecture.
A lot of the buzz around big data so far has been about harnessing the potential of "unstructured data", but getting there is easier said than done.
On one side, the bulk of organizations have invested in SQL to query their (structured) data, and on the other side, an enormous untapped opportunity lies in unstructured data, which is handled with unfamiliar technologies.
While a whole ecosystem of vendors has emerged to lower the barriers to "Big Data", mastering it has been a tall order for many user organizations.