2012 was a fun year for big data. So much going on, so much excitement and progress. For those of you at Strata in October, you probably saw how all the sql vendors are now big data vendors. But you also saw some like Cloudera coming out with Impala. We also saw companies like Microsoft making a good go at it with Office 2013 integrating big data. So with everything we saw in 2012, what will 2013 bring?
I don’t know, but if I were to speculate, and I am, here is my list of what we may see.
1. The end of Big Data.
Yes, the end of big data as a term, not as a concept. I think everyone can agree it is not the best descriptive term and someone is going to make a new term to try and trump it. And it will probably end up being something like Massive Data. I know, that’s not much of an improvement but I didn’t say it would be!
2. The “Suits” get Serious.
My IT friends call me a suit, that is, someone who is business focused. For big data to really take off, the business people will need to get serious. Most companies I talk to, big data is still an IT driven program. IT is not the driver of the customer experience for the vast majority of businesses out there so business people, get in there and get involved or hire someone who can.
3. Big Math gets Bigger
I have a number of math majors on my team. Why? Quite simple, you can’t have big data without big math. This is something that gets lost in the conversations about data and what your stack looks like. Data by itself is not that useful, you need algorithms to make that data useful, algorithms that often require a good math background. Big math is going to help drive new innovations in big data especially the analytics side of big data.
4. More companies jump in.
This is kind of an easy one. But more companies will jump into the big data space. The goal, capitalize on big data and use it to make money. But they will make the same mistakes over and over again because most will not pay for an experienced big data professional. Those of us in the space know, this is not easy stuff! Yet I keep getting companies offering me less than what I currently make thinking this is just another BI role. No it’s not. And when companies realize, you are going to pay one way or another, they will get the talent they need. You can either pay me a $200-$300k or, you the pay some vendor $8-10 million for the first phase and take on another 20 or 30 million before it’s all said and done. Look, this isn’t hard, if you know what you to and who to hire, that’s why you hire me or people like me! Sure, that salary isn’t small change, but I can save you millions in the long run. Something that has cost other companies $20 million, I’ve built at $2-3 million. Now that’s Fortune 500 enterprise level, hook into every system under the sun cost. Small, means less integration costs so less money. But too many companies don’t understand what they don’t understand and will waste a lot of cash before they get smart.
5. Real Use Case Standards start to Emerge.
One of the big problems with Big Data is the lack of solid use cases for most companies to use. Most of us don’t run search engines, which is where all this started. We sell physical products. So we have physical product issues. If you ask most companies, what do they do with big data, most couldn’t tell you, maybe 2 guys off in the corner can. But most can’t because most have no clue what big data means. If big data is going to survive as a discipline, it needs to have some standard use cases, i.e, if you have problem A, you use big data to help you do X, Y and Z. That’s what is needed. I do think we can reach this because it’s painfully obvious to so many.
If you want to learn more about big data and what is in store for the year. Don’t forget to attend the big data conference in Minneapolis, January 21st. I will be there and so will many others.