Understanding Multinomial Naive Bayes classifier.ipynb

Kernel: Python 3 (ipykernel)

• In text classification, the Multinomial Naive Bayes (represented as MultinomialNB() in Python's sklearn library) is a simple but effective classifier used for categorizing text based on its features. It is a variation of the Naive Bayes classifier that is particularly suited for problems where the features represent counts or frequencies, such as word counts in text classification tasks.

Naive Bayes Assumptions:

• It is based on Bayes' Theorem, which calculates the probability of a class given the features. The Naive Bayes classifier assumes that features are independent of each other given the class. While this assumption is often unrealistic in real-world problems, Naive Bayes classifiers still perform surprisingly well in many cases.

Multinomial Distribution:

• The Multinomial Naive Bayes classifier specifically assumes that the features (e.g., words in a document) follow a multinomial distribution. This means that the occurrence of each word is treated as a count, and the classifier tries to estimate the probability of a document belonging to a particular class based on these counts.

• It works well when the data is represented by discrete counts, such as word counts in text classification.

Text Classification:

• In text classification, the goal is to categorize a given document into one of several predefined classes (e.g., spam or not spam, sentiment analysis as positive or negative).

• The features used by the classifier are typically the words or tokens in the document, and the frequency of occurrence of these words helps the classifier to determine the most likely class.

How Multinomial Naive Bayes Works in Text Classification?

• Training Phase: During training, the model learns the probability distribution of words for each class based on the training data. It counts how many times each word appears in each class.

• Prediction Phase: When making a prediction, the classifier calculates the likelihood of the document belonging to each class based on the observed word frequencies in the document. It then chooses the class with the highest likelihood.

The model uses Bayes' Theorem to update probabilities:

Why Use MultinomialNB for Text Classification?

• Text data is inherently sparse, meaning most words in a document may not appear frequently, and MultinomialNB handles this well with its probabilistic framework.

• The algorithm is efficient and fast, making it suitable for large-scale text classification tasks, such as spam detection, sentiment analysis, or topic categorization.

• It works well when there is a strong association between the presence of specific words and certain classes, such as certain words being more likely in spam emails.

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn import metrics

# Sample text data and labels
documents = ["I love programming", "Python is amazing", "I hate bugs", "Programming is fun"]
labels = ["positive", "positive", "negative", "positive"]

# Convert text data into a matrix of token counts
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(documents)

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, labels, test_size=0.25, random_state=42)

# Create and train the Multinomial Naive Bayes classifier
classifier = MultinomialNB()
classifier.fit(X_train, y_train)

# Make predictions on the test set
y_pred = classifier.predict(X_test)

# Evaluate the classifier's performance
print("Accuracy:", metrics.accuracy_score(y_test, y_pred))
print("Classification Report:\n", metrics.classification_report(y_test, y_pred))

Accuracy: 1.0
Classification Report:
               precision    recall  f1-score   support

    positive       1.00      1.00      1.00         1

    accuracy                           1.00         1
   macro avg       1.00      1.00      1.00         1
weighted avg       1.00      1.00      1.00         1