The Blog of Jorge de la Cruz

Everything about VMware, Veeam, InfluxData, Grafana, Zimbra, etc.

VMware: Traveling Back to the Future with Flux, InfluxDB v2.x, and Grafana 8.x – Modern vSphere Monitoring

- Written in: opensource, vmware

Greetings friends, for years I have been talking about InfluxDB, Grafana, and how to combine all of this to monitor VMware. It has been quite a journey, started in 2016 with the first blog in Spanish, then in 2018 wrote the VMware steps in English to reach a bit more audience, which it did, as at the time of writing this, the Grafana Dashboard for VMware vSphere Overview has more than 12K downloads, I can only hope all of these are real VMware Environments on Datacenters, which if they are it is a lot of happy people out there.

Farewell InfluxDB v1.x, hello InfluxDB v2 + Flux

Where we’re going, we don’t need Roads! In November 2020, InfluxData announced the GA of InfluxDB 2.0, tons of improvements but the most important one was the introduction of Flux, a next-generation data processing language. I know, I know, I do as well like traditional skateboards, traditional SQL-like queries, etc., but you know what is better?

I have been delaying myself for far too long before making the jump to Flux, of course, life get in the way, we all have jobs, families, holidays, and a Global situation were spending time at the lab, or having the minimum attention span has become a challenge.

But writing this in December 2021, what happened since March 2020 for real, where am I, what day it is? InfluxDB wrote a blog post in May 2021 sharing the news about Roadmap, and InfluxDB v1.8 is the latest release on the 1.x branch, and not just that, there will be no more patches after the end of 2021.

As the day was coming, I started having more and more comments on LinkedIn, Emails, Twitter, asking me when I was planning to upgrade all the Dashboards to InfluxDB v2.0, and Flux, so the future is now.

Introducing VMware vSphere Modern Monitoring for InfluxDB v2, with Flux, and Grafana 8.x

Before I type any further, let me thank Dennis Faucher, a dear friend who has been an excellent help testing and troubleshooting all of this process. Dennis has been always there, joining the Webinars about this topic, helping and contributing on GitHub, really active on the vExpert Slack about this, plus on his blog, he writes really nice pieces about all things VMware, Orchestration, Monitoring, Automation, etc.

Decided to upgrade my InfluxDB v1.8 to InfluxDB v2, it could not be simpler just follow the next steps. Once I had InfluxDB v2, and Flux as a default query language, it was time to upgrade the Grafana Datasource, or you can add a new one, and from there it was hard work to convert queries like this:

Into Flux queries like this:

One by one, crafting them like a blacksmith forging a new utensil, needed to rethink everything, to learn how to do mathematical operations like this one for the Consumed Datastore:

from(bucket: v.defaultBucket)
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "vsphere_datastore_disk")
  |> filter(fn: (r) => r["_field"] == "capacity_latest" or r["_field"] == "used_latest")
  |> filter(fn: (r) => r["source"] =~ /${datastore:regex}/)
  |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
  |> map(fn: (r) => ({ r with  _value: float(v: r.used_latest) / float(v: r.capacity_latest) * 100.0 }))
  |> group(columns: ["source","_field"])
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)

Or the variables, who now look much better:

So far, I have finished editing all the main four Grafana Dashboards for VMware vSphere, you can find them here, just grab the latest version of them:

How do they look like in fresh 2021, with fresh and powerful InfluxDB v2, plus Grafana 8.x? Like this:

VMware vSphere OverviewGrafana vSphere Datastore Dashboard

Grafana vSphere Hosts Dashboard

Grafana vSphere VMs Dashboard

The best thing about them is that they will work out of the box, the only small thing you need to adjust on your telegraf is to make sure you send the data to a InfluxDB v2 format:

[outputs.influxdb_v2]
   urls = ["https://YOURINFLUXDBv2IPORFQDN:8086"]
   ## Token for authentication.
   token = "YOURTOKENTHATYOUCANCREATE"
   ## Organization is the name of the organization you wish to write to; must exist.
   organization = "YOURORG"
   ## Destination bucket to write into.
   bucket = "YOURBUCKET"

That’s all folks, if you want to follow the full Blog series about Grafana, InfluxDB, Telegraf, please click on the next links:

Reader Interactions

Comments

  2. Marco Pignati says

    Hello, great job! My understanding is that, as it was in the past, Telegraf still needs vCenter to retrieve the information, and having a single instance of vSphere ESXi is not supported. Correct?

  3. jorgeuk says

    Hello Marco,
    Telegraf uses the SDK, it is my understanding that you can still point to a single ESXi, and it will give you some data, not sure how much compared to a full Center, will try and let you know.

    Best regards

Trackbacks

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.