LightGBM Algorithm Overview
explaining LightGBM briefly
LightGBM (Light Gradient Boosting Machine) is an open source machine learning algorithm. It is a decision tree based algorithm that uses gradient boosting to ensemble trees. You can find the code repository of the algorithm here on GitHub. LightGBM can be used for classification (binary and multi class), and also regression outcomes.
Why should I use LightGBM?
You can LightGBM for your machine learning tasks and systems. It offers high accuracy and speed. It is complicated model so depending on your use case you can choose to use it. Read below to understand it more.
How is the accuracy performance of LightGBM?
One of the key distinctions of LightGBM, what makes it different from XGBoost, is that it grows leaf wise — by choosing the leaf with the largest decrease in log loss, and continue build tree from there. This offers great accuracy performance. However, due to its growth leaf wise growth, it can converge faster, which makes it prone to overfitting. Thus, tuning the hyperparameters is very important here. Tuning the max_depth hyperparamter helps with making sure that does not happen. A few other important hyperparameters, include learning_rate, num_leaves, min_sample_split, max_bin, num_iterations, application. Look up here for all the other parameters here.
How is the speed performance of LightGBM?
LightGBM also supports parallel processing optimizations that make it very fast. In addition, it also buckets numerical variables into categorical variable behind the scenes to make it faster in speed, and lowers memory usage. It uses Exclusive Feature Bundling (EFB) technique for reducing features. LightGBM also uses Gradient-Based One-Side Sampling for downsampling samples for faster run and higher accuracy.’
What are some use cases for using LightGBM?
If you have a use case where you want to build a model that needs to predict continuous, probabilities, or label outcome, you can use LightGBM for it. Example: use a purchase based model to rank customer’s buying probability and outcome. In this learning problem, the target label is whether the customer bought a product category or not. LightGBM can take features like historical purchase patterns and demographics of customers. Then it can train based on the algorithm, and create predicted labels.
Quick installation command for MacOS
brew install lightgbmReference: