Business Platform Team

Once a new distributed platform is implemented, any growing organization needs to be able to properly scale the system. Scaling a system for the sake of scaling it doesn’t make sense because cloud resources cost money. The best way to see when and how to scale is to properly monitor the software.

Thus far we’ve discussed how Cassandra, Spark, Kafka, Docker, and Kubernetes can be useful to build a global data platform. These components are powerful in their own right and managing them is a little simpler if we decide to use commercial components from DataStax and Confluent.

There are other tools and services we can use to further accelerate our timeline to deliver a world-class global data and analytics platform. Although bringing up a distributed data (Cassandra), distributed computing (Spark), and distributed communication (Kafka) is a great start for a framework, it still needs a few more components to make it a “Platform” which allows quick creation and delivery of services that an enterprise can use.

It’s much easier to iterate your platform on containers before deciding to use more “traditional” computing systems in Staging or Production. It’s not a requirement to use Docker or Kubernetes, but even if the system is using containers all the way up to Production, many of the DevOps cycles can be done more quickly because of how quickly environments can be refreshed.

The Anant STACK process is fairly generic in that it can be applied to any technology whether it’s SaaS, PaaS, IaaS, Open Source or Commercial. You can always elect to implement a full-blown ITIL service strategy if you already have it at your company. Our process is meant to be an easy to understand enterprise architecture process.

This series covers different aspects of architecting and managing a global data & analytics platform. This is not as simple as choosing some technology and installing it. This work involves proper coordination of people, processes, information, and systems to ensure that the business needs are met at all times. We will cover the components of the “SMACK” stack although many people may not necessarily use Akka or Mesos, they will find much value in our coverage of Cassandra, Spark, and Kafka. We will also cover the Anant “STACK” set of procedures which we use at our company to manage data & analytics platforms for our clients.