Thus far we’ve discussed how Cassandra, Spark, Kafka, Docker, and Kubernetes can be useful to build a global data platform. These components are powerful in their own right and managing them is a little simpler if we decide to use commercial components from DataStax and Confluent.

There are other tools and services we can use to further accelerate our timeline to deliver a world-class global data and analytics platform. Although bringing up a distributed data (Cassandra), distributed computing (Spark), and distributed communication (Kafka) is a great start for a framework, it still needs a few more components to make it a “Platform” which allows quick creation and delivery of services that an enterprise can use.

It’s much easier to iterate your platform on containers before deciding to use more “traditional” computing systems in Staging or Production. It’s not a requirement to use Docker or Kubernetes, but even if the system is using containers all the way up to Production, many of the DevOps cycles can be done more quickly because of how quickly environments can be refreshed.

Scraping website data is like a magic trick that lets you extract web data without having to copy and paste. It can all be done through some lines of code if you know basic Python syntax.

Large companies are using this technology to grow their business. Even Google scraps website data to analyze content and and rank them based on the relevance to your Google search. There are many use cases of web scraping in research, e-commerce, price comparison, market analysis, and lead collection. Regardless of the problem you’re trying to solve, these 5 open source libraries will help you scrape website data.

The Anant STACK process is fairly generic in that it can be applied to any technology whether it’s SaaS, PaaS, IaaS, Open Source or Commercial. You can always elect to implement a full-blown ITIL service strategy if you already have it at your company. Our process is meant to be an easy to understand enterprise architecture process.

This series covers different aspects of architecting and managing a global data & analytics platform. This is not as simple as choosing some technology and installing it. This work involves proper coordination of people, processes, information, and systems to ensure that the business needs are met at all times. We will cover the components of the “SMACK” stack although many people may not necessarily use Akka or Mesos, they will find much value in our coverage of Cassandra, Spark, and Kafka. We will also cover the Anant “STACK” set of procedures which we use at our company to manage data & analytics platforms for our clients.