Amit Beka: Annotating data the right way | PyData Amsterdam 2019

[@nikitajz6560]

0:06 Intro
0:34 Agenda
1:04 Problem statement
2:27 Do we need to annotate?
4:50 Exploratory annotation
5:45 Define the labels
9:28 Starting annotation project
10:31 Tools and mindset
13:00 Test your guidelines / Check scores
15:34 Scale -> internal annotators
17:30 Scaling internally (part 2)
18:05 Explore -> Unbalanced data problem
19:09 Active learning (Define → Unbalanced)
21:46 Balanced accuracy score (Test)
23:25 Active learning: Summary
23:45 Scale -> External annotators
25:05 Simplify your task (Define -> External)
26:34 Denoising the signal (Test -> External)
28:03 External annotators: summary
28:23 Takeaways