P19-3010@ACL

Total: 1

#1 AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging [PDF] [Copy] [Kimi]

Authors: Bill Yuchen Lin ; Dong-Ho Lee ; Frank F. Xu ; Ouyu Lan ; Xiang Ren

We introduce an open-source web-based data annotation framework (AlpacaTag) for sequence tagging tasks such as named-entity recognition (NER). The distinctive advantages of AlpacaTag are three-fold. 1) Active intelligent recommendation: dynamically suggesting annotations and sampling the most informative unlabeled instances with a back-end active learned model; 2) Automatic crowd consolidation: enhancing real-time inter-annotator agreement by merging inconsistent labels from multiple annotators; 3) Real-time model deployment: users can deploy their models in downstream systems while new annotations are being made. AlpacaTag is a comprehensive solution for sequence labeling tasks, ranging from rapid tagging with recommendations powered by active learning and auto-consolidation of crowd annotations to real-time model deployment.