TwiSeg_ Evaluation of Tweet Segmentation Using Named Entity Recognition

Abstract

Twitter has involved lots of users to share and distribute most recent information, resulting in a large sizes of data produced every day. However, a variety of application in Natural Language Processing and Information Retrieval (IR) suffer harshly from the noisy and short character of tweets. Here, we suggest a framework for tweet segmentation in a batch mode, called HybridSeg. By dividing tweets into meaningful segments, the semantic or background information is well preserved and without difficulty retrieve by the downstream application.