Issues with Epiphan capture agents

[mpmuc84]

(This is a summary and slight expansion of exchanges in the Opencast community Matrix channel, with Epiphan support, and the Opencast community sysadmin meeting, mainly for documentation purposes.)

TLDR

• Epiphan devices (confirmed at least on Pearl Nexus) acting as capture agents may have issues connecting to Opencast 17.x or higher
• Issues include: connection to Opencast fails completely, scheduled events are not fetched, recorded events are not uploaded
• Affected Epiphan firmware version: 4.24.2
• The root cause appears to be that Epiphan's firmware relies on Opencast endpoints reserved for admin users.
• Epiphan has a fix ready, but needs to do more testing and QA before a public. It should be released with the next firmware update (4.24.3, expected in Q4/2025).
• A possible workaround might be to assign ROLE_ADMIN to the capture agent user (definitely not recommended!). However, this does not appear to fix the issues in all cases.
• In the meantime: We hope Epiphan will implement their CA according to the official guidelines, improve their documentation, and release a new firmware version as soon as possible.

More detailed version

The question of connectivity issues with Epiphan Nexus CAs and Opencast 17 was first raised on Thu 11/09/2025 by Kristina from the University of Konstanz. Other affected institutions include the University of Osnabrück and LMU Munich.

Epiphan Pearl Nexus devices running firmware version 4.24.2 appear to be unable to connect to Opencast 17.x, even when following the integration guide from Epiphan in some cases. Apparently, the CA tries to fetch the service endpoints from /services/services.json, which is reserved for users with ROLE_ADMIN (and is the incorrect endpoint for this purpose in general). Since CA users should usually only have ROLE_CAPTURE_AGENT, the request will either be denied by Opencast, or the response will not be understood by the Nexus device (e.g. in case of a redirect to an external login service).

Diagnosing was difficult, because:

• The Nexus logs only contain limited information on the problem, e.g.

  2025-09-16T18:54:07.532+0200 XXXXX user.err afu[16990]: Opencast REST API: Unsupported response from services/services.json
  

• Opencast responds with a HTTP 200 status code, even when the request is ultimately denied.
  -> If SSO via AAI is enabled, the request is redirected to the discovery service or the external login page.
  -> If SSO is not active, Opencast redirects to its native login form

We tried connecting the Nexus to several Opencast installations, always with the same result:

• our production system with Opencast 17.7 (with SSO via AAI)
• our testing system with Opencast 17.7 (without SSO, only local and LDAP login)
• the public testing instance at https://legacy.opencast.org/ with the current Opencast 17.x

After reporting the issue to Epiphan support, the first response was to follow the integration guide linked above and to assign various roles (i.e. ROLE_ADMIN and several API roles) to the capture agent user. In some cases, this appeared to work, but at least at LMU Munich, it did not. We asked Epiphan to follow the official guidelines for capture agents, with special focus on the endpoints to query for services and the fact that CAs should not need to have admin level privileges.

After this, Epiphan provided us with a slightly modified version of the 4.24.2 firmware, which appears to fix the immediate issues. With the modification, the CA appears to query the correct service endpoint (/services/available.*) to get to the other endpoints needed for registration and upload. It also removes the requirement for ROLE_ADMIN, which was very much appreciated from a security perspective.

While this fix works, it comes with a few caveats:

• Epiphan still needs to conduct further tests and QA before releasing it to the public. They have said the fix is scheduled to be released with the next firmware update (4.24.3, late Q4/2025).
• Since logging and documentation on the CA's behavior remains sparse, we don't know if Epiphan is actually following the documented guidelines for capture agents. The core functionalities of a CA (registration with Opencast, fetching schedule, uploading recordings) appear to be working with the modified firmware, but we have not checked yet whether all functions of the Nexus work with a CA user that only has ROLE_CAPTURE_AGENT assigned.
• It is currently unknown why the CA user is supposed to have several API roles. This might be connected to additional functionality of Epiphan's devices, but Opencast developers have noted several times that hardware vendors should not rely on the external API, because it can (and likely will) be deprecated and/or changed in future Opencast releases.

Conclusion

We will have to wait for the next release of Epiphan's firmware to see how far the changes are going. If you are using Epiphan devices connected to Opencast as CAs, you might want to check (and maybe add to the discussion here):

• Do you have issues with your setup? Which Epiphan firmware and Opencast version are you running?
• In case of issues: Contact Epiphan support asking for assistance.

In general, we hope that Epiphan will follow the official guide for capture agent development and maybe expand their documentation to lay out in more detail why and for what specific roles are required.

[mtneug]

Another thing to test would be if the response from /services/available.(xml|json) is actually decoded correctly. This endpoint is very tricky in the JSON variant in that it can switch the datatype depending on the number of services returned:

If service type does not exist:

{
  "services": ""
}

If service type exists once:

{
  "services": {
    "service": {
      "type": "org.opencastproject.ingest",
      "host": "https://stable.opencast.org",
      "path": "/ingest",
      "active": true,
      "online": true,
      "maintenance": false,
      "jobproducer": true,
      "onlinefrom": "2025-09-24T02:02:15.571+02:00",
      "service_state": "NORMAL",
      "state_changed": "2025-09-24T02:02:15.571+02:00",
      "error_state_trigger": 0,
      "warning_state_trigger": 0
    }
  }
}

If service type exists more than once:

{
  "services": {
    "service": [
      {
        "type": "org.opencastproject.ingest",
        "host": "https://stable.opencast.org",
        "path": "/ingest",
        "active": true,
        "online": true,
        "maintenance": false,
        "jobproducer": true,
        "onlinefrom": "2025-09-24T02:02:15.571+02:00",
        "service_state": "NORMAL",
        "state_changed": "2025-09-24T02:02:15.571+02:00",
        "error_state_trigger": 0,
        "warning_state_trigger": 0
      },
      {
        "type": "org.opencastproject.ingest",
        "host": "https://my-other-stable.opencast.org",
        "path": "/ingest",
        "active": true,
        "online": true,
        "maintenance": false,
        "jobproducer": true,
        "onlinefrom": "2025-09-24T02:02:15.571+02:00",
        "service_state": "NORMAL",
        "state_changed": "2025-09-24T02:02:15.571+02:00",
        "error_state_trigger": 0,
        "warning_state_trigger": 0
      }
    ]
  }
}

[ytourayEpiphan]

TLDR: It’s fixed in Pearl firmware. Beta available support@epiphan.com. Release 2025Q4.

@mpmuc84 thank you for your thoughtful feedback — we truly appreciate the time you took to share your insights. Your comments were instrumental in helping us pinpoint the problem that one of our customers was experiencing with the Epiphan Capture Agent (CA).

As you mentioned, we’ve developed a beta firmware that we believe addresses the issues you raised:

• The Epiphan Capture Agent not being able to connect to Opencast version 17.x or newer.
• The Capture Agent previously relying on endpoints reserved for the ROLE_ADMIN user. We can confirm that the latest beta firmware no longer requires this role, and the CA no longer attempts to fetch data from /services/services.json.

We’ve verified these improvements against stable.opencast.org, and the fixes are scheduled to be included in our upcoming firmware release planned for Q4 2025.

In the meantime, if anyone is interested in evaluating the beta firmware, please reach out to us at support@epiphan.com
— we’ll be happy to share it with you.

That said, one area we recognize still needs improvement is logging. We know we can (and should) do better here. Enhancing our logging framework is an ongoing project, and we’re committed to continuous improvement. Until then, we ask for your patience and invite you to contact us if you come across any log entries that seem unclear or unexpected.

Thank you again for your support and for helping us make our products better for the community.

[ytourayEpiphan]

Hello,

We’re following up to let you know that Epiphan has officially released new firmware for all supported devices, addressing the issues previously raised.

Please download and upgrade your devices to the latest official version. If you continue to experience any issues after updating, or if you have general feedback or suggestions on how we can improve our products, we’d love to hear from you.

You can reach our support team at support@epiphan.com. We greatly appreciate your feedback and continued support.

Best regards,
Yusupha