Engineering Best Practices
5 Must-Know Ways to Test ML Models in Production (Implementation Included)
A beginner-friendly guide to model testing.
Avi Chawla
410 posts published
LLMs
NumPy Cheat Sheet for Data Scientists
40 most used methods.
Classical ML and Deep Learning
Model Interpretability (Part 2)
A deep dive into interpretability methods, why they matter, along with their intuition, considerations, how to avoid being misled, and code.
Machine Learning
Implementing a Siamese Network
A beginner-friendly implementation guide.
Machine Learning
Contrastive Learning Using Siamese Networks
Building a face unlock system.
Classical ML and Deep Learning
Model Interpretability (Part 1)
A deep dive into PDPs and ICE plots, along with their intuition, considerations, how to avoid being misled, and code.
37 min read
A Crash Course on Model Calibration
Modern neural networks being trained today are highly misleading. They appear to be heavily overconfident in their predictions. For instance, if a model predicts an event with a 70% probability, then ideally, out of 100 such predictions, approximately 70 should result in the event occurring. However, many experiments have revealed
56 min read
A Crash Course on Graph Neural Networks (With Implementation)
* Google Maps uses graph ML for ETA prediction. * Pinterest uses graph ML (PingSage) for recommendations. * Netflix uses graph ML (SemanticGNN) for recommendations. * Spotify uses graph ML (HGNNs) for audiobook recommendations. * Uber Eats uses graph ML (a GraphSAGE variant) to suggest dishes, restaurants, etc. The list could go on since almost
23 min read
Optimize ML Models to Run Them on Tiny Hardware using Quantization
Typically, the parameters of a neural network (layer weights) are represented using 32-bit floating-point numbers. The rationale is that since the parameters of an ML model are not constrained to any specific range of values, assigning a data type to parameters that cover a wide range of values
Engineering Best Practices
Spark DataFrames and Big Data ML using PySpark on Databricks
A completely hands-on and beginner-friendly deep dive on PySpark using Databricks.
Pandas
Accelerate Pandas 20x using FireDucks
...by changing just one line of code.
Classical ML and Deep Learning
Graph Neural Networks Part 3 (Implementation Included)
A practical and beginner-friendly guide to building neural networks on graph data.
Machine Learning
Knowledge Distillation with Teacher Assistant for Model Compression
A better and intuitive technique to model compression.
Classical ML and Deep Learning
Graph Neural Networks Part 2 (Implementation Included)
A practical and beginner-friendly guide to building neural networks on graph data.
Classical ML and Deep Learning
Graph Neural Networks Part 1 (Implementation Included)
A practical and beginner-friendly guide to building neural networks on graph data.
Classical ML and Deep Learning
Model Calibration (Part 2)
How to make ML models reflect true probabilities in their predictions?
Classical ML and Deep Learning
Federated Learning: A Critical Step Towards Privacy-Preserving ML
Learn real-world ML model development with a primary focus on data privacy – A practical guide.
11 Types of Variables in a Dataset
Beyond features, targets, etc.
Classical ML and Deep Learning
Model Calibration (Part 1)
How to make ML models reflect true probabilities in their predictions?
Engineering Best Practices
A Practical Guide to Scaling ML Model Training
GPUs - GPU Clusters - Distributed Training.
Deep Learning
All-Reduce and Ring-Reduce for Model Synchronization in Multi-GPU Training
Two synchronization algorithms for intermediate-ML models.
Classical ML and Deep Learning
Conformal Predictions: Build Confidence in Your ML Model's Predictions
A critical step towards building and using ML models reliably.
Engineering Best Practices
Quantization: Optimize ML Models to Run Them on Tiny Hardware
A must-know skill for ML engineers to reduce model footprint and inference time.
Engineering Best Practices
How to Structure Your Code for Machine Learning Development
A highly overlooked yet critical skill for data scientists.
Engineering Best Practices
Develop an Elegant Testing Framework For Data Science Projects Using Pytest
A Comprehensive Guide to Pytest for data scientists.
Engineering Best Practices
A Beginner-friendly Guide to Multi-GPU Model Training
Models are becoming bigger and bigger. Learn how to scale models using distributed training.
Engineering Best Practices
Object-Oriented Programming with Python for Data Scientists
A beginner to advanced guide for Python OOP.
Engineering Best Practices
11 Powerful Techniques To Supercharge Your ML Models
Take your ML models to the next level with 11 lesser-known techniques.
The Mathematics Behind RBF Kernel
...and why it so powerful
Machine Learning
Shuffle Feature Importance
An intuitive and reliable technique to measure feature importance.
