Skip to content

huggingface_ner ML Backend: fit() fails with KeyError 'annotation' for certain webhook payloads #874

@yuka-with-data

Description

@yuka-with-data

Description

When using the huggingface_ner ML Backend in model.py, the fit() method can fail during webhook-triggered training due to an assumption about the webhook payload structure.

Problem

The current implementation assumes that the webhook payload always contains:

data['annotation']['project']

However, in some cases (e.g., after submitting token-level annotations), the payload does not include the annotation key, causing the backend to crash.

Also, according to the webhook payload details in the documentation, project is a top-level field and not inside annotation, so assuming data['annotation']['project'] can lead to a KeyError.

When submitting annotations, the following error occurs:

Traceback (most recent call last):
  File ".../label_studio_ml/api.py", line 126, in webhook
    result = model.fit(event, data)
  File ".../huggingface_ner/model.py", line 146, in fit
    project_id = data['annotation']['project']
KeyError: 'annotation'

Environment

  • Setup: Local (non-Docker) ML Backend
  • Model: huggingface_ner example
  • Trigger: Submitting token-level annotations

Suggested Fix

Handle webhook payloads more defensively by avoiding assumptions about the presence of the annotation key when extracting project_id.

I'd be happy to open a PR implementing this change and linking it to this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions