Large Language Model-Based Online Review Classification for Subfeature-Level Customer Opinion Analysis

Abstract

Abstract. In recent years, many studies have analyzed online reviews to understand customer preferences and requirements for product features. However, most of them have focused on feature categories, whereas companies need to analyze customer preferences regarding subfeatures to gain practical insights for product development. To bridge the gap, this study proposes a new method for subfeature-level review analysis. First, text review sentences are embedded into vectors using a large language model. A sentence bidirectional encoder representation from transformer (SBERT) model is employed. Next, the method trains a neural network model that classifies reviews into subfeatures. The input data are sentence vectors and the outputs are class labels indicating product subfeatures. To address the problem of highly imbalanced labels in review data, a new loss function is proposed based on evaluation metrics. The proposed method was tested using smartphone and headphone reviews collected online. The results showed that the new method achieved higher performance, i.e., F1 scores over 0.80, than a previous BERT-based classifier (F1 scores between 0.39 and 0.69). In addition, the new loss function provides a more balanced precision/recall for all the classes. The developed approach will help companies extract customer opinions at the product subfeature level and has practical implications for early-stage product design.