Skip to content

feature: Add --workers flag to nemoguardrails server for multi-worker uvicorn support #1746

@m-misiura

Description

@m-misiura

Did you check the docs?

  • I have read all the NeMo-Guardrails docs

Is your feature request related to a problem? Please describe.

The server seemingly runs uvicorn with a single worker

Describe the solution you'd like

Add a --workers CLI option (defaulting to 1 for backward compatibility):

@app.command()
def server(..., workers: int = 1):
    ...
    uvicorn.run(server_app, port=port, log_level="info", host="0.0.0.0", workers=workers)

Presumably, we'd need to ensure that make api.py spawn-safe

Describe alternatives you've considered

  • Gunicorn wrapper: can run gunicorn -w 4 -k uvicorn.workers.UvicornWorker nemoguardrails.server.api:app but this bypasses the CLI's config loading IIUC

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions