Why automation?
While data science is an important part of any big project in companies these days, there are always some tasks that require frequent repetition, while testing or according to the business needs. We have to write the same code over and over again for different instances of the same test case/dataset or different test cases/datasets. This reduces the efficiency of a data scientist and adds extra unnecessary workload for him.
These repeated tasks include but are not limited to the data collection module, data-preprocessing module, and interpretation of a module in certain data science or data engineering project. Now, to solve this issue, certain companies (like the one I work in) have already designed their own generalized code-base for automating these tasks, while some of them pay a large amount for acquiring services of such an automation tool. Now, when I say it is an automation tool, it does not mean that this tool can work on its own as a smart personal assistant. Whether it has a user interface or not, it is essentially the same thing — a generalized code. So someone has to be occupied with the work to run different modules as per the requirement of the project to get desired outputs.
Here are some of the reasons you might think of willing to pay for these instead of building your own customized automation tool (good news, not all of them are paid):
- You have too much workload on the team and hardware to be able to allocate resources for the task.
- You have too less resources for adding this one to your to-do list.
- You have a timed project and cannot afford to lose time on this task.
- Many more…
Components of a generalized ML automation tool
As I have mentioned earlier, this automation tool isn’t like a smart assistant that would run on its own. It is rather a skeleton of a workflow which most commonly followed while handling any data science-related project. You can also imagine it as a combination of different modules of code that perform specific tasks in all possible approaches in which they are generally executed in a machine learning workflow and that the data scientist has full control over which module to choose and which approach of executing the task in the module to choose. You will get clear with this as you read further.
Any general machine learning workflow consists of the following:
1. Data collection module
This part of the project takes care of the data imports from different formats like an excel spreadsheet, a CSV file, an SQL query, a JSON file, etc. You must have seen a code statement similar to this:
import pandas as pd
pd.read_csv("path/to/csv/file")
#OR
pd.read_excel("path/to/excel/spreadsheet")An automation tool supports imports from the most available file formats.
2. Data exploration or Exploratory Data Analysis
As a data scientist, our work would always involve exploring data or often called Exploratory Data Analysis (EDA). The purposes of exploring data are to know our data better and grasp what we are dealing with.
This is usually done manually using pandas or using a library like pandas-profiling for an extensive description of what kind of data we are going to deal with and what actions we might need to take in the next step — Data preprocessing. An automation tool supports EDA from at least one library to be executed.
3. Data preprocessing and transformation
Considered the most important step in a data science/machine learning project. It is essential to ensure that the data we are dealing with is properly structured and in the correct format, free of any kind of noise. For more information on what Data preprocessing is, you can refer to this article: https://towardsdatascience.com/data-preprocessing-concepts-fa946d11c825
An ideal ML automation tool can support all possible preprocessing and transformation tasks without a hitch as it is an important phase in the process.
4. Training and Fitting of Data:
Irrespective of the type of ML (supervised/unsupervised) and the model being used to train, this step cannot be avoided as this is the crux of the whole machine learning process — a step where the machine learns from the apparent patterns in the data. Hence, an automation tool should support this step irrespective of the problem type — classification, clustering, or regression — in order to minimize the load on a data scientist.
5. Interpretability:
A model must be able to explain why it predicts a certain result for a certain set of input features. There are numerous libraries that help in this and an automation tool with this feature is preferred over one without it. You can learn about Machine Learning interpretability in detail in an article I wrote here: https://athex25.medium.com/a-comprehensive-guide-to-machine-learning-interpretability-b1b49d2bdb31
6. Testing model over unseen results and prediction of accuracy:
This step, like training, is also a common step, irrespective of the model you are using to train and the problem type, you are always required to test your model and deduce its accuracy in the prediction of a result over an unseen set of input features. An automation tool must be able to execute such tasks with support for all possible accuracy metrics (accuracy, precision, recall, etc.).
Additional features of an automation tool
Along with the features of an automation tool mentioned above, there are a few other features that may or may not be present. This might be useful in the context of a large-scale project where the size of data and the code-base itself is huge and that the machine learning process being carried out is just a subset of a larger set of sequential/parallel tasks. You can research these features on your own.
1. CI/CD Support:
CI/CD bridges the gaps between development and operation activities and teams by enforcing automation in the building, testing, and deployment of applications. Modern-day DevOps practices involve continuous development, continuous testing, continuous integration, continuous deployment, and continuous monitoring of software applications throughout their development life cycle. The CI/CD practice or CI/CD pipeline forms the backbone of modern-day DevOps operations. An automation tool with support for this can be of great help for companies that follow the CI/CD development protocol,
2. Version Control Support.
Version Control is an unavoidable feature in not just data science but all domains. It is definitely a better tool if it supports VC.
3. Data visualization.
It is a particularly efficient way of communicating when the data is numerous as for example a Time Series. From an academic point of view, this representation can be considered as a mapping between the original data (usually numerical) and graphic elements (for example, lines or points in a chart).
4. Graphical User Interface.
A tool with an easily navigable user interface instead of a terminal can be preferred in cases where the output from the tool is the end step or the only of the process. In most other cases, generally, a tool with a terminal-based code base is preferred so that the output from the tool can be quickly passed to the further processes in the task pipeline.
5. Leak Detection.
Data leakage is a serious condition and it affects a training of a model and hence its prediction capability. It is like an event when you trying to take an exam without learning what you should have learned to write the most appropriate answer to a question.
The comparison chart
While we were building a generalized machine-learning tool of our own, we made good research about these features in existing tools and came up with a comparison for those to have an idea of what features we can have in our tool. I have also tried to score them based on the features they have but you need not prefer only the one with the highest score, you must choose the best fit for your needs and show that you are an ideal data scientist for the data science community.
Please only consider this as a part of my research and not as an official review of those services. We haven’t actually used them all and the data being shown is based on the documentation and information of features they have on their websites and other articles.
Hope you found this article useful, and also you received some valuable insights if you were willing to build an automation tool of your own! 😄
Feel free to leave a comment if you have some suggestions you would like me to add here! Find me on LinkedIn at: https://www.linkedin.com/in/attharvaj3147/
