Project Creation:

Import Data:

Token Based NER import formats: .txt, .tsv, .json (click on your preferred import format to find the required import format)

Token-based

Token-based projects allow you to label tokens in a text document. This type is well-suited for projects such as NER and POS.

Before submitting your project, you can also adjust additional settings as described below.

  • Every token must have at least one label: useful when you want to enforce that every token in the document must be labeled.
  • Tokens and token spans should have at most one label : does not allow you to add multiple labels to a single token or token span.
  • Allow arrows to be drawn between labels: allows you to draw arrows from one label to another to annotate relationships between words. This is useful for showing an adjective is related to a noun, or a pronoun is referring to a person.
  • Allow character-based labeling: allows you to select and apply labels on a character-level basis, so you don’t have to select the entire token.

Note: If you have already created a project, you can change the configurations through Settings.​

Label Sets

Our project templates require you to upload both a document as well as a label set.

Token-based labeling

For token-based labeling (NER, POS, DEP, COR, and OCR templates), a label set is a single-column .csv or .tsv following the structure below:

Column 1
Label 1
Label 2
Label 3
etc…

Progress Review:

Export Methods

When exporting a file, there are four methods that you can choose.

Download

  • Download is the simplest method. If you choose this method, your project will be downloaded automatically to your computer.

Webhook

  • If you choose this method, you can export the file directly to a URL of your choice.
  • For full explanation about this method, you can view this page.
  • Note: the link will expire in 6 hours.

Email

  • If you choose this method, you will receive an email containing the download link. This can be most helpful for large projects that need to export in the background while you continue on to your next labeling project.
  • Note: the link will expire in 6 hours.

QA/Review (Video Tutorial):

After clicking the Launch Project button, you will be taken to the newly created project.

⚠ Note that as the project creator you start off in Reviewer Mode. If you would like to start labe

ling, you will need to assign yourself, then click on the Reviewer Mode button to switch to Labeler Mode. Any other assigned labelers will automatically start in Labeler Mode.

Reviewer Mode has a Review extension on the right side. This contains a Conflicts section and a Labelers section.

  • The Conflicts section shows any parts of the document where assigned labelers disagreed on the correct label. (The ratio of disagreement is set by the Peer Review Consensus setting above). For example:
    • One labeler has labeled “Cover A”.
    • The second labeler has labeled “Cover B”.
    • Since the two labelers disagreed, this will be shown as a conflict.
    • Click one to resolve the conflict and set the final label that will be exported.
  • The Labelers section shows who has been assigned to this project, along with their current status.

Note: if you are interested in knowing more about QA/Review, please take a look at this video tutorial. After switching the page to Reviewer Mode, don’t forget to refresh the page.