Get ROCm flash attention

GET

/api/v1/settings/rocm-flash-attention

Return whether AOTriton flash attention is enabled, and whether it applies here.

Response Body

Successful Response

application/json

enabled*Enabled

Whether AOTriton flash attention is enabled

supported*Supported

Whether this machine runs the ROCm backend, where the setting applies

Example Request

curl -X GET 'http://127.0.0.1:8100/api/v1/settings/rocm-flash-attention'

Example Request

curl -X GET 'http://127.0.0.1:8100/api/v1/settings/rocm-flash-attention'

Example ResponseSchema200

{
  "enabled": true,
  "supported": true
}

{  "enabled": true,  "supported": true}

Set low VRAM mode PUT

Toggle low VRAM mode. Unloads the current TTS model if one is loaded.

Set ROCm flash attention PUT

Toggle AOTriton flash attention. Takes effect on the next launch.