Definition
You have probably played a game of 20 Questions, narrowing down choices step by step to reach the right answer. A decision tree is a machine learning algorithm—a computer's method for discovering rules—that classifies data or predicts outcomes by asking a sequence of criteria questions with yes-or-no answers. It begins at a single root question at the top, branches out into different paths based on conditions, and reaches a final decision at the leaf nodes. It gets its name because this upside-down branching structure looks just like a tree.
Narrowing Down Answers Like 20 Questions
Think of an online quiz that recommends a movie. If you answer 'yes' to 'Are you in an energetic mood?', it follows up with 'Do you like action movies?' Before you know it, the perfect recommendation appears on your screen.
A decision tree works the exact same way. When faced with a huge batch of data, the computer first picks the question that splits the data most cleanly into two groups. For instance, when filtering spam emails, it might start by asking, 'Does the subject line contain the word promo?'
Then, it asks another specific question to each separated group. With every question answered, messy data gets sorted into neat, similar piles. Finally, at the very last leaf, it reaches a verdict: 'This email is spam' or 'This email is safe.'
The Secret to Ordering the Questions
How does the computer decide which question comes first and which comes later? The secret lies in calculating how much each question reduces 'impurity'—a measure of how mixed up the data is.
Imagine a bag filled half with red marbles and half with blue marbles. That is high impurity, or total chaos. But if one clever question neatly sorts them into a bag of only red marbles and a bag of only blue marbles, that chaos vanishes instantly. The computer looks for the question that cuts down this confusion the most and puts it at the very top.
In technical terms, this is called maximizing 'information gain.' Using mathematical formulas, the computer measures how orderly the data becomes after each split. Step by step, it builds the tree by picking the question that organizes the data best at that moment. Because it only looks one step ahead, the overall tree is not always guaranteed to be the most optimal possible structure.
Clear Visuals and the Overfitting Trap
The greatest strength of a decision tree is its crystal-clear explainability. Complex neural networks are often called black boxes because it is hard to peek inside and see why they made a decision. A decision tree, however, lays out every question and step like a clear flowchart, so humans can easily understand its reasoning at a glance.
However, making the tree too deep and complex creates a serious trap. If it keeps asking questions to account for every tiny quirk in the training data, it runs into overfitting—scoring 100% on practice data but failing miserably on brand-new data.
It is like a student who memorizes every number and typo in the textbook's practice questions, only to bomb the actual exam when the numbers change slightly. To prevent this, developers prune unnecessary branches or use techniques like Random Forest, which combines predictions from dozens of different decision trees.
Common misconceptions
A decision tree gets better at predicting the longer and more complex its branches become.
When a tree has too many branches, it suffers from overfitting. It memorizes the training data too closely, making it far more likely to get wrong answers on real-world data.
Where you meet it
Email services use it to detect spam by checking the sender address, spam keywords, and attachments in sequence.
E-commerce platforms analyze a customer's visit time, cart items, and recent purchase history to predict whether they will buy a product.
In one sentence
A decision tree is an AI algorithm that reaches an answer by asking a series of questions that split data as cleanly as possible.