The Blog of Jorge de la Cruz

Everything about VMware, Veeam, InfluxData, Grafana, Zimbra, etc.

Looking for the Perfect Dashboard: InfluxDB, Telegraf, and Grafana – Part XLVII (Monitoring Open WebUI)

- Written in: vmware

Greetings friends, I have been talking about Open WebUI in previous articles, this is one of the best open-source software I have ever user used, so intuitive, works extremely well, and it comes with everything we need to run our private LLM models.

It also has super interesting feature where you can see the consumption, and stats per every single answer, giving the user full control, and of course allowing administrators of the infrastructure evaluate different models, performance, etc.

I found that there is a native pipeline for langfuse, in case you are using that already. But as we are using Grafana, InfluxDB, etc. I thought that it would be cool to take the stats as per usual into InfluxDB and do our magic.

Grafana Dashboard for Open WebUI

When you finish the entry you will have something similar to this. It might look a bit different depending of how many models you have, and frequency which you use Open WebUI, but should work out of the box:

Dashboard – Summary

  • Quick overview (Models used, Total prompt tokens, Average Token/s, Average duration)
  • Total usage by model
  • Model usage historical per model
  • Total usage historical
  • Prompt Tokens per second historical
  • Response Tokens per second historical
  • Average response duration historical

Topology with all logical components

This entry is very similar to the previous ones, we will use a custom bash shell script to collect the API information from our Open WebUI, from there we will send them to InfluxDB, and visualize them with Grafana . The design would be something similar to this:

How-to download and configure openwebui_grafana.sh script

I have created a custom bash shell script so you do not need to spend your time dealing with Open WebUI API, which is very comprehensive, but no swagger at the moment.

You will need jq, so please go ahead and install it first (apt-get install jq, or yum install jq)

This shell script can be downloaded and run from the telegraf server, InfluxDB, or any other Linux. We will have to edit the configuration parameters:

# Open WebUI Configuration
webuiAPIBaseURL="http://YOUROPENWEBUIIPORFQDN:8080"
webuiEmail="YOURADMINUSER"
webuiPassword="YOURPASS"

# InfluxDB Configuration
veeamInfluxDBURL="http://YOURINFLUXSERVERIP" #Your InfluxDB Server, http://FQDN or https://FQDN if using SSL
veeamInfluxDBPort="8086" #Default Port
veeamInfluxDBBucket="veeam" # InfluxDB bucket name (not ID)
veeamInfluxDBToken="TOKEN" # InfluxDB access token with read/write privileges for the bucket
veeamInfluxDBOrg="ORG NAME" # InfluxDB organisation name (not ID)

Once the changes are done, make the script executable with chmod:

chmod +x openwebui_grafana.sh

Run it, if everything is good you should see something similar to this:

./openwebui_grafana.sh 
Last execution timestamp: 0
Successfully authenticated.
Processing updated chat: 1da5e275-aba8-4c02-8e23-e48b95d9f9f7 (updated at 1739637673)
Processing message: 88d58b12-1aa5-4825-8df5-33704f29b9d5
Extracted Stats - Chat: 1da5e275-aba8-4c02-8e23-e48b95d9f9f7 | Message: 88d58b12-1aa5-4825-8df5-33704f29b9d5 | Model: unknown | responseTokens: 0 | promptTokens: 0 | duration: 0 ms | loadDuration: 0 ms | promptEvalCount: 0 | promptEvalDuration: 0 ms | evalCount: 0 | evalDuration: 0 ms | approximateTotalMS: 0 ms | messageTimestamp: 1739637595
Data sent to InfluxDB.
Processing message: ccd3f4bd-02ac-4de9-b4f7-334507e87b45
Extracted Stats - Chat: 1da5e275-aba8-4c02-8e23-e48b95d9f9f7 | Message: ccd3f4bd-02ac-4de9-b4f7-334507e87b45 | Model: deepseek-r1_1.5b | responseTokens: 30.57 | promptTokens: 407.84 | duration: 58350341149 ms | loadDuration: 3413770001 ms | promptEvalCount: 458 | promptEvalDuration: 1123000000 ms | evalCount: 1645 | evalDuration: 53810000000 ms | approximateTotalMS: 58000 ms | messageTimestamp: 1739637512
Data sent to InfluxDB.
Processing message: f072bdf2-961f-4113-8aa7-4773ebefface
Extracted Stats - Chat: 1da5e275-aba8-4c02-8e23-e48b95d9f9f7 | Message: f072bdf2-961f-4113-8aa7-4773ebefface | Model: unknown | responseTokens: 0 | promptTokens: 0 | duration: 0 ms | loadDuration: 0 ms | promptEvalCount: 0 | promptEvalDuration: 0 ms | evalCount: 0 | evalDuration: 0 ms | approximateTotalMS: 0 ms | messageTimestamp: 1739637512
Data sent to InfluxDB.
Processing message: f4c6d2cd-84dc-4512-8f65-57912225102b
Extracted Stats - Chat: 1da5e275-aba8-4c02-8e23-e48b95d9f9f7 | Message: f4c6d2cd-84dc-4512-8f65-57912225102b | Model: deepseek-r1_1.5b | responseTokens: 27.78 | promptTokens: 47627.91 | duration: 188466162 ms | loadDuration: 59497130 ms | promptEvalCount: 2048 | promptEvalDuration: 43000000 ms | evalCount: 1 | evalDuration: 36000000 ms | approximateTotalMS: 0 ms | messageTimestamp: 1739637595
Data sent to InfluxDB.
Updating timestamp.

If so, please now add this script to your crontab, like for example every 30 minutes:

*/30 * * * * /home/oper/openwebui_grafana.sh >> /var/log/openwebui_grafana.log 2>&1

A few notes about the script. I couldn’t find a better way, so every time you ask an additional question into a chat. Imagine you had a very old chat opened, then you come back 10 days after and ask an additional question, that whole chat and the messages will be parsed again. It doesn’t matter much because InfluxDB do not save duplicated entries with the same timestamps, so we should be fine and only see the additional message 🙂

Grafana Dashboards

I created a Dashboard from scratch by selecting the best requests to the database, finishing off colors, thinking about graphics, and how to display them, and everything is automated so that it fits our environment without any problem and without having to edit anything manually. The Dashboard can be found here, once imported, you can filter by model and time:

Import Grafana Dashboards easily

I have already a wonderful Dashboard, ready just for you with everything you need to monitor our environment in a very simple way. Select the name you want and enter the ID: 22867, which is the unique ID of the Dashboard, or the URL:

With the menus above you can display and adjust the Dashboard to your different choices:

I hope you like it, and I would like to leave you the complete series here, so you can start playing with the plugins that I have been telling you about all these years:

Reader Interactions

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.