Anant Corporation Blog

Our research, knowledge, thoughts, and recommendations about building and leading businesses on the Internet.

Category Archives: Business


What Makes a Good ETL Project?

Bad

  1. Bad ETL (extract, transform, load) projects are ones that don’t have a strategy for different types of information or lack of knowledge management on how to add/remove different data sources, add/remove processors & translators, and add/remove different sinks of information.
  2. It doesn’t necessarily have to be on any particular platform, just that it has structure.. just as any software should have.. an architecture.

 

Good

  1. Simple systems that separate E / T / L into composable blocks that are scriptable or configurable.
  2. Compiled systems are good too if the volume is extreme.
  3. A good bash pipeline is just as good as any as long as its well documented.

 

Ugly

  1. Using ESB (Enterprise Service Bus) for ETL.
  2. Using Spark for ETL.
  3. Basically using things that have advanced features for business logic for doing simple transformations that really don’t need to belong in these computing environments. Conjoining simple message delivery (ETL) to an advanced message delivery (ESB) or advanced computing (Spark).

 

Why should an organization undertake such a project?

To meet a business goal(s). Sometimes it’s to gain intelligence, sometimes it’s to create data products to create value, sometimes it’s to show predictions.

Comes down to how does it affect the organizations’ perpetuity. Some of the questions a business should be able to answer are:

 

What other solutions provide the same end user results?

Tools like Domo, or Tableau, or recently something like Periscope (in the SaaS world) can be useful to gain basic insights without having to do ETL if the data is ready. Other open source tools can be used as well such as Kibana, Metabase, and Redash as long as the data is available.

 

What are the trade-offs between the various solutions?

Ultimately if the data isn’t ready, ETL may be required to get it clean enough for those tools to allow users to visualize/explore it properly.

Data Wrangling & Visualization of Public / Government Data

Rahul Singh, CEO of Anant, had the opportunity to co-organize and MC the May Meetup of Data Wranglers DC where the speakers, John Clune and Timothy Hathaway, covered two topics related to open government and public data for data processing and visualization. We had a great turnout at the event and had the chance to do some networking after.

Our next two meetups will focus on Search & Knowledge Management (June Meetup) and Machine Learning for Data Processing (July Meetup), check out the Meetup page for more details when they become available.

Big thank you to John Clune and Timothy Hathaway for taking the time to present to the group if you have any interest in speaking please don’t hesitate to reach out to Rahul at rahul@anant.us.

 

Below you can find a recording of both presentations.

Software Algebra – Building Applications Without Reinventing the Wheel

As technology has continued to mature in the last two decades there have been many challenges overcome, obstacles faced, and solutions crafted. A recurring theme, in the area of obstacles (more specifically, self-imposed obstacles), has been the propensity for software companies to more often than not 1) turn to developing applications from the ground up for a particular problem or 2) take existing pieces of software that are perfectly fine for their specific use case and then tailor them to a different (but sometimes slightly similar) use case.

 

Software Algebra is essentially a best practice in software development to make sure that we are using 1) the tools best suited to a particular problem 2) while also dodging the trap of re-inventing the wheel by starting from scratch or trying to fit a tool into solving a problem it was never intended to address.

 

There are multiple cases in which this best practice is entirely ignored, most commonly so by inexperienced software architects who have the “my hammer can solve all problems” mindset. Often times, one of the best ways to avoid falling into this trap is to relentlessly focus on getting a Minimal Viable Product (MVP) out of the door in a time-boxed span of time and iterating multiple times on that MVP to steadily bring it up to support all use cases.

 

We recently spoke about this topic at the WebTech Conference in Washington, DC and will be doing so again on Tuesday, April 11th at 6PM at the University of Maryland Baltimore County, you can find additional details as well as register for the event here.

WebTech Conference – Software Algebra

Rahul Singh will be presenting on Thursday, March 30th at WebTech on the topic of Software Algebra. He’ll be speaking about the ways online software (SaaS) and open source applications can work in tandem with web and mobile applications to deliver powerful business results.

 

From this presentation, you will learn how to plan and build web apps that support business process using existing software. This approach takes a very practical approach to taking inventory of business teams, processes, information, and systems and creating future-proof web systems without reinventing the wheel.

 

Thank you, Iron Yard DC for hosting us! Register and view the schedule of speakers here, looking forward to seeing you.

Modern Enterprise: Why Airtable? The Top 20 Links Worth Your Time

Airtable is a relatively new Software-as-a-Service (online software, more commonly known as SaaS Software) to enter the arena for business users. I first saw it when I was looking for process management tools and a search result showed it in comparison to Trello. I was intrigued because I love Trello for what it does for me personally and what it has done for our team. This post is first in a series which will focus on Airtable and how to use it in a modern Enterprise. The Modern Enterprise is an organization or team that uses the Internet and online business software to organize people, processes, information and systems to achieve their goals. We chose to start with Airtable because it hits home for a basic and fundamental need in business which Excel, Google Spreadsheets, and others have thus far met fairly well. If you are no novice to organizing information, you know what I’m talking about. I once knew a professor who said his $500 million / year professional service business was run by his COO on Excel.

 

Business users have been using spreadsheets to organize information for decades. Excel is probably one of the most used business tools in the world because of its versatility by way of simple columns and rows and power by way of formulas and macros. Today, there are many alternatives to Excel that users can use online. Here are a few which I’ve used and think are good for the most part.

 

Google Spreadsheets – We use these extensively.

SmartSheets – I have used it in the past, and think it does some things well.

Microsoft Excel 365  – In addition to it _being_ Excel, syncing with iOS apps, it can also work with Microsoft’s Big Data product HDInsight.

 

These tools are all great and I don’t want to take away from them, but Airtable is another beast altogether. What drew me to Airtable was its simplicity. It looks simple. It feels simple. It _is_ simple. Having had, ahem, some, ahem, experience in databases and online software, I knew how to start using it pretty quickly. I knew I could use it, but I wanted to see if someone else could use it. I asked one of our Project Managers, Danielle, to make an Airtable to track the status of our clients, which ones were on subscription with us vs. working with us on an ad-hoc basis. Danielle is an extremely intelligent and organized team member but she’s not a technologist per se. She’s the model for what I call a technology empowered team member. Danielle had never used Airtable before but was and is very adept at using spreadsheets to track projects and project finances. She was able to whip something up in no-time.

 

Here are my initial thoughts which will guide my evaluation of Airtable for The Modern Enterprise in four upcoming articles (see below).

  • Airtable can be used across the different responsibility areas of The Modern Enterprise (Sales & Marketing, Products & Services, Research & Development, Operations & Infrastructure, Finance & Accounting, and Leadership & Management)
  • Airtable can be used by people internal and external to a company because of its ease in creating “teams” which share a certain set of tables.
  • Airtable can be used for tracking Information or Data, but I think there are better uses for it, especially tracking the Information for Processes.
  • Airtable is a powerful system and can be connected to other systems easily, non-technically, using Zapier or through custom development that interacts with Airtable APIs.

 

Here are some great links to get you started until the next few posts on Airtable.

 

 

This article is part of a larger series:

  1. Modern Enterprise: Why Airtable? The top 20 you need to see.
  2. Modern Enterprise: How Airtable can work for different departments and responsibilities.
  3. Modern Enterprise: How to organize & collaborate with People around the world with Airtable.
  4. Modern Enterprise: How to manage & track Processes with Airtable
  5. Modern Enterprise: How to connect Airtable with anything and create magical automations.

 

Check Airtable out here!

Disclaimer: We aren’t affiliated with Airtable and nor are we getting anything in return for this, we just love it when there’s awesome new technology out there that solves a number of pain points elegantly.