top of page
Search

Disaster Tweet Classification Using BERT

  • Jan 28
  • 2 min read

Project Description

This project focuses on classifying tweets as disaster-related or not by leveraging a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model. The task involved fine-tuning BERT for text classification, preprocessing tweet data, and evaluating the performance of the model using standard metrics.


Key Highlights

  1. Objective: Predict whether a tweet refers to a disaster event.

  2. Dataset: Tweets labeled as disaster-related (1) or not disaster-related (0).

  3. Model: Fine-tuned BERT model for binary text classification.

  4. Tools and Libraries:

    • Python

    • Libraries: transformers, torch, pandas, and scikit-learn

    • Jupyter Notebook for implementation.


Workflow

  1. Data Preprocessing:

    • Cleaned and tokenized tweets using BERT tokenizer.

    • Managed missing values and ensured proper encoding of input text.

    • Split the dataset into training, validation, and testing sets.

  2. Model Setup:

    • Used a pre-trained BERT model (bert-base-uncased) for transfer learning.

    • Added a classification head to output binary predictions.

  3. Training:

    • Fine-tuned BERT on the disaster tweet dataset with cross-entropy loss.

    • Utilized Adam optimizer and learning rate scheduler for efficient training.

  4. Evaluation:

    • Assessed model performance on the test set using metrics like accuracy, precision, recall, and F1-score.

    • Visualized training and validation loss over epochs.

  5. Prediction:

    • Generated predictions for unseen tweets and saved them as a CSV file for submission.


Results

  • Model Performance:

    • Accuracy: 87%

    • Precision: 86%

    • Recall: 84%

    • F1-Score: 85%


Learning Outcomes

  1. Gained hands-on experience with transfer learning using BERT.

  2. Learned to fine-tune pre-trained transformer models for classification tasks.

  3. Enhanced skills in data preprocessing for natural language processing (NLP) tasks.

  4. Evaluated model performance using appropriate metrics and visualized training progress.


Next Steps

  • Experiment with other transformer models like RoBERTa or DistilBERT for comparison.

  • Perform hyperparameter tuning to optimize performance further.

  • Explore techniques to handle class imbalance, such as oversampling or focal loss.

  • Apply the model to other text classification tasks, such as sentiment analysis.

 
 
 

Комментарии

bottom of page