Big data initiatives have gone mainstream for CIOs, but collecting petabytes of user data is meaningless if the insights you derive aren’t useful. Data lakes are giving businesses a way to flip the way they derive insight from big data initiatives.
Many of the new product announcements at AWS re:Invent 2016 showed that data lakes are becoming commercialized, especially as enterprises realize they have to adopt an agile mindset to succeed in digital transformation.
Changing Databases from Entry Points into Exit Points
In the past, enterprises relied on a lengthy planning process before executing on any data. You would start with a question about the business and start finding ways to collect structured data to answer that specific question.
This may have worked 20 years ago, but you have to make decisions faster now. You can’t spend months or even years collecting data only to find that your question was irrelevant to begin with.
This is why data lakes are so important now. By setting up a cheap AWS data lake, you can collect massive amounts of unstructured data on an ongoing basis and have data scientists create neural networks that help you derive insights in weeks rather than months or years.
Being able to proactively collect data and answer questions quickly is essential for making agile business decisions and staying ahead of disruption.
However, the more data we collect the harder it will become to move data fluidly within an enterprise. In fact, the new AWS Snowmobile is essentially a moving database that will become the fastest way to move petabytes of data into the cloud—your basic internet connection just won’t be enough at a certain point.
The AWS Snowmobile is just one new product that Amazon is working on to support big data initiatives and the push for mainstream data lakes. At AWS re:Invent, we saw many other important commercial innovations for data lakes.
Data Lake Innovation at AWS re:Invent 2016
One of the best parts about attending AWS re:Invent is listening to some of the biggest AWS customers talk about how they overcame challenges with the latest cloud features and products.
Customers like Netflix and Mapbox were on hand to discuss the different ways they use AWS and data lakes to manage massive amounts of data to improve their products and user experiences. Two new product/feature offerings from AWS that came up during many customer spotlight sessions include:
- AWS QuickSight: According to AWS, Amazon QuickSight is a business analytics service that makes it easy to build visualizations, perform ad-hoc analysis, and quickly get business insights from your data. When a company collects massive amounts of customer demographic data, they can quickly pivot their products and services by using QuickSight to derive meaningful, actionable insights.
- Amazon Athena: Athena is a serverless query service that lets you analyze your data lake data directly in Amazon S3 using standard SQL. This new service lets you derive insights from unstructured data for greater agility.
Even for enterprises that have spent years building structured systems, Amazon’s support for unstructured data insight is essential to achieve the startup mindset that will lead to successful digital transformations. You just have to make sure you have the right strategy in place to execute correctly.
Is Amazon the New Pinnacle of Tech Innovation for the Next Decade?
There’s no doubt that the tech industry has been chasing Apple, Google, and Microsoft for years. But with Alexa, Amazon Go, the continued development and offerings available in AWS, it seems that Amazon is coming to the forefront of the industry.
At the very least, Amazon has become the clear leader in cloud services. But how has Amazon become a major competitor in a space that Apple, Microsoft, and Google have controlled for years?
One key is the tight feedback loop that Amazon keeps with its major customers. These companies spend millions of dollars on AWS annually and Amazon makes a point of integrating their feedback into each new product, service and feature of the platform. If you want to get ahead of your competition through digital transformation, you need to achieve this same level of agility and flexibility.
However, you can’t start leveraging data lakes and valuable insights without a strategy that sets you up for digital transformation success. If you want to learn more about how you can create a strategy that will keep you ahead of disruptors, download our free ebook, Enterprise Technology for Business Outcomes.