What are AutoKeras?

The Neural Architecture Search (NAS), the future of deep learning, is one of the most powerful concepts in artificial intelligence. The NAS framework’s main objective is to take the flaws of human designs out of Neural Network Architecture. To achieve this, many architectures are trained and evaluated. To follow this, each architecture is adjusted on basis of their selected algorithm to try another architecture. This results in a model loss in each model attempt instead of an individual step. Overall, this was a very complicated process!

Earlier, to win over a successful NAS, very intricate implementations viz. Tensorflow, PyTorch, or Keras script were required. Now that’s what exactly AutoKeras did. All thanks to Texas A&M Lab, which built an open-source framework that was made for performing automated machine learning based on the popular Keras package. In recent times, Automated Machine Learning (AutoML) has gained notoriety because of its excellent ML techniques and usage for non-data science folks.

In January 2019, its first version 1.0 was released. The current version of AutoKeras aims to provide a programmed search for the hyperparameters of deep learning models. They are quite easy to understand and use.

How to implement AutoKeras in AutoML?

The overall objective of both AutoML and AutoKera is to simplify the deep machine learning process with the use of automated Neural Architecture Search (NAS) algorithms. A programmer with minimal machine learning expertise can make use of these algorithms to achieve its goal as there are numerous AutoKera tutorials available online.

Sounds good? Let's see how to implement AutoML in Python.

AutoKera is compatible with Python 3.6 version. If you have the same version of Python, you can install AutoKera using a pip. For a simple use case, the code is rather minimal. Using the built-in most dataset, you could load with:

x_train, y_train = most.load_data()

Now for entering the data in between we can create the ImageClassifer object:

import autokeras as ak
model = ak.ImageClassifier(max_trial = 100)

powerful models viz. ResNet, Xception and separable CNNs. Following this, we need to fit the model.

model.fit(x_train, y_train)

On completion of the model with the best score and with the maximum number of trials will be chosen. Now you can save the model as:

x = load_some_image()
model.predict(x) #Returns a softmax array
model.export_model()

Why is Auto-Keras best for Automated Machine Learning?

The Automated Machine Learning ultimate goal is to build a program that has its neural network optimised for specific tasks without any human supervising it. After publishing the first paper Self Organizing Neural Paper in 1988, Automatic Machine Learning became into the limelight as it was a tremendous step regarding both the performance and the computational power as required.

AutoKeras is simple, easy to implement, and gives high performance. It is similar to a mathematical tool, Bayesian Optimization which assists in finding the extremum without calculating derivatives. It mainly comprises of three stages:

  • With the use of the decision function, generate the next architecture
  • Training and observing the current architecture
  • Update the probability distribution

AutoKeras is great for simple tasks, proof-of-concept results, and data investigation. Here are few advantages of using AutoKeras for autoML:

  • AutoKeras provides an easy-to-use interface for different tasks, such as image classification and more. It wins heavily in resizing the image. It assures that no information is lost while resizing your image.
  • AutoKeras can be used to find a top-performing model for a binary classification dataset and regression dataset.
  • Users can achieve the best performance on the dataset by specifying the location of the data and the number of models to try.
  • AutoKeras provides a deep learning experience in AutoML that uses neural architecture search.

Now we shall discuss PyCaret, a new machine library for Python.

What are PyCaret and why use them in AutoML?

PyCaret was developed by a popular data scientist, Moez Ali. It was aimed to assists the data scientist for their efficient and quick end-to-end experiments. They can be proclaimed as a low-code resource in a machine learning experiment that aims to reduce the hypothesis to insight the cycle time. In simple words, it is a library that can simplify complex machine learning tasks with a few coding lines. Due to its low-coding nature, it is simple and easy to use with the help of PyCaret tutorials. It can also be well-utilised by professional data scientists for building rapid prototypes.

PyCaret is a valuable library source that can simplify machine learning (ML) tasks for data scientists. It can also be proved as a cost-effective solution for startup companies. As fewer data scientists can use PyCaret to compete with a larger team using traditional tools. Additionally, startup companies with little knowledge can also explore the field of data science applications.

The Machine learning (ML) libraries and frameworks viz., scikit-learn, XGBoost, Microsoft LightGBM, spaCy, etc have efficiently used AutoML PyCaret as a Python wrapper.

What is PyCaret?

To begin with the machine learning process, the first step is to install PyCaret. It is a simple two-step process.

  • Importing a Module: To begin with it, first, you need to import the module. In the first version of it, there is a total of six modules are available – regression, clustering, classification, anomaly detection, natural language processing (NLP), and associate mining rule.
  • Initialization: In this step, PyCaret is involved in performing some basic tasks like imputing the missing values, encoding the variables, and splitting the dataset, etc.

1 # import the classification module
2 from pyCaret import classification
3 # setup the environment
4 classification_setup =classification.setup(data=data_classification, target='Personal Loan')

Machine Learning using PyCaret is quite simple. It begins with the use of the create_model function. Here, we shall abbreviate decision tree mode as "dt", which will appear as a table with k-fold cross-validated scores.

For supervised learning following are the evaluation metrics used:

  • Classification: Accuracy, AUC, Recall, Precision, F1, Kappa
  • Regression: MAE, MSE, RMSE, R2, RMSLE, MAPE

    1 # build the decision tree model
    2 classification_dt = classification.create_model('dt')

Just like above, pass the string “xgboost” for undergoing XGBoost model.

1 # build the xgboost model
2 classification_xgb = classification.create_model('xgboost')

How to do Hyperparameter Tuning?

Here we are using the tune_model function as the model abbreviation string as one parameter. PyCarets here brings in great flexibility. For example, one can define the number of folds using the fold parameter or one can change the number of iterations using the n_iter parameter. The time spent on n_iter will surely increase the training time but will assure quality performance.

Let’s train a tuned CatBoost model:

How to Build Ensemble Model using PyCarets?
Ensemble model in machine learning brings together the decision

Let's see the ensemble training model here.

1 # ensemble boosting
2 boosting = classification.ensemble_model(classification_dt, method= 'Boosting')

Ensembling technique can also be done by blending. It can be easily done by passing the model created in a list of the blend_models function. That's all! See below.

1 # Ensemble: blending
2 blender =classification.blend_models(estimator_list=[classification_dt,classification_xgb])

Another very significant function of PyCaret library is to compare common evaluation metrics for all the available models in the library of the module.

This function is only available in the pycaret.classification and pycaret.regression modules.

1 # compare performance of different classification models
2 classification.compare_models()

Now let's see the results out of it.

The results of PyCaret are very simple to analyze. Just a small line of code can help you in finding how effective are PyCarets in machine learning. You can get results by providing the model object as the parameter and the type of plot you want.

Here are the results of plot_model:

1 # AUC-ROC plot
2 classification.plot_model(classification_dt, plot = 'auc')
3
4 # Decision Boundary
5 classification.plot_model(classification_dt, plot = 'boundary')

Here are the results of the trained model:

1 # Precision Recall Curve
2 classification.plot_model(classification_dt, plot = 'pr')
3
4 # Validation Curve
5 classification.plot_model(classification_dt, plot = 'vc')

PyCaret also has another amazing function i.e. evaluate_model. For this pass the model object and PyCaret will create an interactive window for you to see and analyze the model in all the possible ways:

1 # evaluate model
2 classification.evaluate_model(classification_dt)

It is important to note that interpreting complex models has great significance in machine learning projects. It assists in debugging the important model. In PyCaret, this step is as simple as writing interpret_model to get the Shapley values. PyCarets easily compile all the steps and passes the unseen data into the pipeline and give us the results. It can be useful for generating quick results under tight timelines.

Endnote

AutoKera and PyCaret are definitely worth using. It may prove to be the right direction in terms of automated machine learning (AutoML) and deep learning procedures. Learning and applying AutoKeras and PyCaret in Python may prove to be a gateway to endless opportunities.

Lets bring your idea to life