Detecting and Measuring Product Innovation in News Articles Using Google’s BERT
Date: Tuesday, August 10, 2021 | 10:00 AM to 11:50 AM
Authors: Neil Kattampallil, Gizem Korkmaz, and Gary Anderson
Sponsor: Business and Economic Statistics Section
Abstract: Innovation, the availability and usage of novel ideas, products and business practices is central to the improvement of living standards. Policy makers in part rely on survey-based measures of innovation to design, develop, and implement policies to promote innovation. In the U.S., innovation is measured through nationally representative surveys of businesses such as the Annual Business Survey. To reduce respondent fatigue and to provide more timely information, statistical organizations are interested in exploring non-traditional methods for measuring innovation.
Our goal is to show how non-survey data, in particular news articles, and advanced natural language processing methods can be used to identify and to measure innovation in various sectors(food and beverage, pharmaceutical, and computer software). We present a novel approach utilizing Bidirectional EncoderRepresentation from Transformers (BERT) developed by Google. Our methods include (i) text classification to identify news articles mentioning innovation, (ii) named-entity recognition(NER), and (iii) question answering (QA) to extract company names and (iv) developing innovation indicators for companies.