Set ROCm flash attention

PUT

/api/v1/settings/rocm-flash-attention

Toggle AOTriton flash attention. Takes effect on the next launch.

Query Parameters

enabled*Enabled

Enable or disable AOTriton flash attention

Successful Response

application/json

success*Success

Whether the operation succeeded

changed*Changed

Whether the setting was actually changed

needs_restart*Needs Restart

Whether the app must restart for the change to take effect

Example Request

curl -X PUT 'http://127.0.0.1:8100/api/v1/settings/rocm-flash-attention?enabled=true'

Example Request

curl -X PUT 'http://127.0.0.1:8100/api/v1/settings/rocm-flash-attention?enabled=true'

Example ResponseSchema

{
  "success": true,
  "changed": true,
  "needs_restart": true
}

{  "success": true,  "changed": true,  "needs_restart": true}

Get ROCm flash attention GET

Return whether AOTriton flash attention is enabled, and whether it applies here.

Get HuggingFace endpoint mode GET

Return the user's HuggingFace endpoint preference ('auto', 'direct', or 'mirror').