Decision Tree
It's like playing a game of 20 Questions, following a chain of Yes/No questions down a roadmap to find the final answer.
Definition You have probably played 20 Questions, narrowing down the possibilities step by step to find the right answer. A Decision Tree is a machine learning algorithmโa computational method computers use to discover rulesโthat classifies data or predicts outcomes using that exact same step-by-step questioning. It starts with a single root question at the top, branches out along different conditions, and arrives at a final decision at the leaves at the bottom. It gets its name because the visual diagram looks just like an upside-down tree.
Narrowing Down Answers Like 20 Questions
Think about taking a movie recommendation quiz on your phone. Answering "Yes" to "Are you in the mood for excitement?" leads to "Do you like action movies?", which finally reveals the perfect film recommendation on your screen.
A decision tree works in the exact same way. When a computer processes a large dataset, it first selects the question that splits the data most cleanly into two groups. For instance, to filter spam emails, it might start by asking, "Does the subject line contain the word 'Sale'?" to divide the inbox into two piles.
Next, it asks another specific question to each split group. With every question answered, mixed data gets sorted neatly by shared characteristics. Finally, at the leaf stage, the computer arrives at a firm verdict: "This email is spam" or "This email is safe."
The Secret to Ordering the Questions
How does the computer decide which question belongs at the very top and which ones should follow? The secret lies in calculating how much a question reduces the data's "impurity"โhow mixed up it is.
A bag with an equal mix of red and blue marbles has high disorder. If a single question neatly divides them into a bag of red marbles and a bag of blue marbles, that disorder vanishes immediately. The computer always chooses the question that reduces disorder the most as its starting question.
In technical terms, this process is called maximizing "information gain." Using mathematical formulas, the computer measures how organized the data becomes with each question and builds the most efficient tree all on its own.
Crystal-Clear Logic vs. The Overfitting Trap
The greatest strength of a decision tree is its explainability. Complex deep neural networks are often called "black boxes" because it is hard to see why they reached a certain conclusion. In contrast, a decision tree lays out every step like a clear map, so humans can understand the reasoning at a single glance.
However, making the tree too deep and complex creates a serious problem. If a tree branches out endlessly to fit every trivial detail, it might score 100% on its practice data but fail miserably on fresh inputs. This is known as overfitting.
It is just like a student who memorizes every number and typo in a practice workbook: change one number on the real exam, and they cannot solve it. To prevent this, engineers prune away unnecessary branches or use Random Forests, which combine the decisions of many trees together.
๐ค Common misconceptions
Making a decision tree as deep and complex as possible always improves its prediction accuracy.
Too many branches cause overfitting, where the model memorizes training data too closely and makes more mistakes when encountering new, real-world data.
๐งบ Where you meet it
A decision tree is an AI algorithm that finds answers by following a chain of smart questions that best split data into pure groups.