Looking for the Perfect Dashboard: InfluxDB, Telegraf and Grafana – Part XIII – Veeam Backup for Microsoft Office 365 v4

Greetings friends, almost a year ago I launched the Dashboard for Veeam Backup for Microsoft Office 365, in that case, it was the Dashboard for the product v3. With the arrival of the new version of Veeam Backup for Microsoft Office 365 v4, the time has come to update the Dashboard too. I have told you on numerous occasions all the advantages that a monitoring and the dashboarding system has.

In fact, you can find all the entries about InfluxDB, Telegraf, and Grafana here. Remember that there is a Community Edition with up to 10 free users, without paying anything).Veeam Backup for Microsoft Office 365 Dashboard

Grafana Dashboard for Veeam Backup for Microsoft Office 365

What’s new in this new Dashboard for Veeam Backup for Microsoft Office 365 v4

I already told you that among the great new functionalities we found Object Storage and at-rest encryption of what we stored in this Object Storage, which is why the new Dashboard shows us the local repositories, vs Object Storage:Besides showing us the detail of each object storage repository, such as the size over time, name, service point, and whether or not it has encryption:These are very subtle improvements but I hope you like them.

How to Enable the RestAPI in Veeam Backup for Microsoft Office 365

We’ll have to run several steps before we start downloading anything, or even touching Grafana, from our Veeam Backup for Microsoft Office 365, we’ll go in the Console to the menu icon, and then General options: We will go to the sub-menu called REST API and enable the REST service, also here we could install our own SSL, and change the timeout of the tokens and the port of SSL, in my case I have not changed anything: With this, we can go to our browser and explore the SWAGGER that Veeam Backup for Microsoft Office 365 brings as standard, we will find it at https://VBOURL:4443/swagger and must look like this: Now with this we are ready to continue towards the Perfect Dashboard for Veeam Backup for Microsoft Office 365 v4.

Veeam Backup for Microsoft Office 365 RestAPI Script

The script that can be downloaded directly from GitHub requires a brief configuration with certain parameters of our environments, but before we see this configuration, let’s see what this script gathers from RestAPI:

  • Number of Licensed Users
  • Number of new Users recently added to VBO
  • Total number of threads of the Proxies
  • Total number of objects processed in the backups
  • Summary of the latest Backup status – by job
  • Summary of the latest restoration – by job and by component
  • Statistics on reading and writing in MB/s – per job
  • Statistics on information processed in MB/s and data written in MB – per job
  • Statistics on the time it has taken each Backup – per job
  • Summary table with the status of each Backup – by job
  • Summary table with the consumption of each Backup Repository
  • Summary table with the consumption of each Object Storage
  • Disk Capacity Meter for each Backup Repository
  • Disk Capacity Meter for each Object Storage

We will make sure we have jq installed, an apt-get install jq or yum install jq if we use CentOS:

Once we download the script to our InfluxDB server, we’ll edit it and change the following:

Once we edit these parameters, make the script executable:

We will be able to execute it manually if we want to see what it does, we would have to observe an output like the next one repeated many times:

This is fine, and that’s what has to come out, now we have to program our script to launch automatically as often as we want, in my case every 30 minutes, we will use crontab -e to add it to cron, use your own path:

Verifying that we are ingesting information with Chronograf

The normal thing at this point, if we have done all the steps well, is that we are already sending information compiled by the script to InfluxDB, if we do a search using the wonderful Chronograf, we can see that we have information: All the variables of this new Veeam script for VBO are stored in veeam_office365_* so it is really easy to find them.

Grafana Dashboards

I have created a Dashboard from scratch selecting the best requests to the database, finishing off colors, thinking what graphic and how to show it, and everything is automated so that it fits our environment without any problem and without having to edit anything manually. You can find the Dashboard here, once imported, you can use the top dropdown menus to select between organizations, backups, repositories, and proxies:

How to Import easily the Grafana Dashboard

So that you don’t have to waste hours configuring a new Dashboard, and ingesting and debugging queries, I’ve already created four wonderful Dashboard with everything you need to monitor our environment in a very simple way, it will look like the image I showed you above.

Select Create Dashboard – Import

Select the name you want and enter the ID: 11286, which is the unique ID of the Dashboard, or the URL:

With the menus above you can deploy and adjust the Dashboard to your organization, backup jobs, object storage, and so on: Please leave your feedback in the comments, or in GitHub. If you want to see them working without installing anything, here is the link to my environment:

That’s all folks, if you want to follow the full Blog series about Grafana, InfluxDB, Telegraf, please click on the next links:

Author: jorgeuk

Father, writing in https://www.jorgedelacruz.es and https://jorgedelacruz.uk Blogger, Systems Engineer @veeam - vExpert 2014/2020 & NTC 2018/19

18 Thoughts

  1. Hello Jorge
    I have built a Ubuntu Server and installed Telegraf, Influxdb and Grafana
    I have modified your script as per above with login details.

    I am getting an error when the script runs
    HTTP/1.1 400 Bad Request
    Content-Type: application/json
    Request-Id: 129ecc4d-1cf0-11ea-8984-00505681dd93
    X-Influxdb-Build: OSS
    X-Influxdb-Error: unable to parse ‘veeam_office365_objectstorage,objectname=null,type=null,bucketname=null,servicePoint=null,customRegi
    onId=null,location=null,region=null,objectStorageEncryptionEnabled=null usedSpaceGB=null’: invalid number
    X-Influxdb-Version: 1.7.9
    X-Request-Id: 129ecc4d-1cf0-11ea-8984-00505681dd93
    Date: Thu, 12 Dec 2019 15:00:01 GMT
    Content-Length: 235

    This occurs twice and then the script completesOK
    It shows in Grafana OK

    Many thanks

    Best regards

    Clive

  2. Hello,
    Are you using Object Storage? Maybe not, so I will add a condition to check if you do not have it to save just it as 0.

    Thanks for the feedback

  3. I am replying to an older post so hopefully it gets to you. We have followed the directions for our Veeam environment (we are a VCSP) but aren’t Linux proficient and are trying to determine how to insert your script into InfluxDB to make the Dashboard work in Grafana. We have Grafana installed on our main Veeam VBR and Office 365 backup server and InfluxDB is setup, but no data coming into the Dashboard as the script isn’t installed. Any help would be appreciated. Thank you.

  4. Hello Matt,
    I always read all the comments. That is odd, send me an email to jorge.delacruzATveeamDOTcom so we can keep the conversation there, or if you have Teams, then add me 🙂

  5. Jorge, I’m using Ubuntu/Docker with Influxdb/Telegraf/grafana and when I run chmod +x veeam-office365.sh the output is nothing. Just runs. When I run the .sh I get:./veeam-office365.sh: line 35: jq: command not found
    ./veeam-office365.sh: line 44: jq: command not found
    ./veeam-office365.sh: line 141: jq: command not found
    ./veeam-office365.sh: line 180: jq: command not found
    ./veeam-office365.sh: line 197: jq: command not found
    ./veeam-office365.sh: line 248: jq: command not found

    Could really use some help here.

  6. Hello Jorge
    I have created a new setup to go live and get the following error running the grafana script (veeam_office365.sh)
    jq: error (at :3): Cannot index string with string “id”
    jq: error (at :3): Cannot index string with string “id”
    jq: error (at :3): Cannot index string with string “id”
    jq: error (at :3): Cannot index string with string “id”
    jq: error (at :3): Cannot iterate over null (null)

    Any ideas?

    Thanks

    Clive

  7. Jorge

    You were right
    It was to do with the password.
    On the demo network it was in a Workgroup but in “live” it is on a Domain and needed Domain\Username

    Many Thanks for your prompt response

    Best Regards

    Clive

  8. Hello Jorge – nothing to get excited about but the cron job didn’t work due to permissions for /var/log. I have changed directory path and all is well
    🙂

  9. Hi Jorge,

    first of all thank you for this big work!

    Second, I had to change your script in order to collect correctly jobs info (added uppercase letter on JobSessions at raw 202 , veeamVBOUrl=”$veeamRestServer:$veeamRestPort/v4/Jobs/$idJob/JobSessions”).

    Third..is it possible to use the single influxdb data source to create multiple dashboards in a multitenant scenario?

    Thanks!

    Marco S.

  10. hi,

    i need to add some influx tags for this, usally i do this in telegraf.conf. Example
    backupenviroment = “VBO”

    This is just run with a cronjob not with telegraf. can i put in this kind of tags?

    regards

  11. Hello,
    This specific dashboard leverages the RESTful API, but I guess you can add that tag somewhere, yes, what is the use case? We can take the Proxy Repo Name from the API, why you want to split per VBO? For your different Tenants?

  12. Thanks for the impresive respons time 🙂

    the use case is that we have a influx db centrelized and i have got one DB for backup team. so we have metric for VBR, i have to use same db and want to split the metric depending backupenviroment/platform.

    regards

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.