Vehicle Fleet Management Dashboards [Grafana-Synapse | Azure]
Need to visualize vehicle fleet data in Grafana dashboards?
The CANedge lets you record 2 x CAN/LIN to an SD (e.g. from vehicles/machines) - and push it to your own server via WiFi/LTE.
In this intro, we explain how you can upload raw data to your Azure Blob Storage container, DBC decode it via Azure Functions - and visualize it via Grafana-Synapse dashboards.
Tip: See also our PowerBI-Synapse intro.
Tip: If you work in Amazon or Google Cloud, see also our intros on Grafana-Athena or Grafana-BigQuery dashboards.
Learn more below - and try our dashboard playground!
CUSTOMIZABLE
Visualize e.g. vehicle fleet data in customizable and interactive dashboards
DATA LAKE
Parquet data lake stored in Azure container - with database-like speed (but 95%+ cheaper)
YEARS TO MS
Browse through years of data - and zoom in to the exact ms of interest
SCALABLE
Seamlessly scale from 1 to 1000+ CANedge devices - and from MBs to TBs of data
FREE & OPEN
Grafana dashboards are 100% open source with a powerful cloud free tier
Open the dashboard playground - or deploy your own dashboard in < 15 min!
Grafana-Synapse: How does the integration work?
Below we briefly explain a typical Grafana-Synapse setup:
- A CANedge uploads raw CAN/LIN data to your 'input container'
- When a log file is uploaded it triggers an Azure Function
- The Azure Function DBC decodes the data to Parquet files
- The Parquet files are written to an 'output container'
- Grafana visualizes this 'data lake' via Azure Synapse
This can be deployed in <15 minutes - with zero coding.
Tip: This can also help enable more advanced architectures.
Below we provide additional details on each core element of the integration.
Tip: You may also try our step-by-step integration guide with free MF4/DBC sample data now.
Uploading CANedge data to Azure input container
The CANedge lets you upload data to any S3 bucket, incl. e.g. natively Amazon, Google Cloud, MinIO and more. Azure does not have a native S3 interface, but you can set up an S3 gateway via a service from 'Flexify'. This enables seamless integration of your CANedge with your Azure Blob Storage container. We describe how to set this up in our CANedge Intro.
Azure Function automation
Azure Functions help automate the DBC decoding of new data:
- The Azure Function runs every time a new log file is uploaded
- It loads relevant DBC files from your input container to decode the raw CAN/LIN data (using our MF4 decoders)
- The resulting Parquet files are written to an output container (Data Lake Gen2)
- Azure Functions are 100% serverless - you only pay when a log file is processed
- They are fully scalable, from 1 device to 1000+ devices
- All data processing is done automatically (and instantly) upon upload
Output container Parquet data lake
The output container stores your decoded data as a 'Parquet data lake':
- The data lake consists of Parquet files
- Parquet files are grouped by device, CAN message and year/month/day
- As a result, the data lake is compact and highly efficient
- The data lake is multi-purpose and can be used directly in e.g. Python/MATLAB
- Data lake storage is 95%+ cheaper than using a database
Example: InfluxDB Cloud storage cost is ~1.44$/GB/month vs. Azure Data Lake Gen2 storage cost of ~0.019$/GB/month.
Azure Synapse interface
Grafana can visualize your Parquet data lake via Azure Synapse (using the Microsoft SQL Server data source):
- Synapse provides a simple SQL interface to Parquet data lakes stored in Azure containers
- Further, it can extract results from GBs of data in seconds via parallelization
- Synapse is serverless - you only pay for the data scanned as a result of Grafana queries
- The cost of using Synapse is extremely low at ~5$/TB (i.e. 1-10$/year for most use cases)
- While there is no direct Grafana Synapse data source, you can use the Microsoft SQL Server data source
See also our Grafana-Athena intro for more details on why we recommend to use Parquet data lakes as the data source.
The CANedge2/CANedge3 enables you to upload recorded log files to your own server (self-hosted or cloud) - including Amazon, Google and Azure cloud. Dashboard visualization and Parquet data lakes are core tools for enabling CAN telematics at scale - and we therefore provide step-by-step integration guides for the top 3 clouds.
Below we briefly outline the three integrations:
Grafana-Athena: Most of our users connect their CANedge2/CANedge3 to Amazon S3, which is our default recommended cloud. Here, the Grafana-Athena integration can be used. It offers the simplest deployment, widest set of functionality (e.g. automated trip summary aggregation) and most efficient use of e.g. date partitioning. Data is uploaded to S3, automatically DBC decoded by a Lambda function and output to a Parquet data lake stored in another S3 bucket. This data lake is then queried via the Amazon Athena SQL interface - e.g. by Grafana.
Grafana-BigQuery: Google Cloud offers a nearly identical experience to Amazon as it provides a native 'S3 interoperability' functionality. Further, the Grafana-BigQuery dashboard integration is nearly 1:1 identical to Grafana-Athena. Uploaded data is automatically DBC decoded via Azure Functions and output to a Parquet data lake stored in another bucket. This data lake is then queried via the BigQuery SQL interface - e.g. by Grafana.
Grafana-Synapse: Azure does not provide a native S3 interface, but you can deploy an 'S3 gateway' (Flexify) to enable the CANedge to upload data to your Azure Blob Storage container. Uploaded data is automatically DBC decoded via Azure Functions and output to a Parquet data lake stored in another container. This data lake is then queried via the Synapse SQL interface. The data can then be visualized in Grafana using the Microsoft SQL Server data source.
Set up your own Grafana dashboard in <15 min
Use our step-by-step integration guide to set up your Grafana-Synapse dashboard in minutes!
Create amazing CAN data visualizations
Visualize your vehicle/machine data via beautiful graphs, GPS maps, gauges, tables, heatmaps, KPIs, histograms, pies, alert lists and more - anything is possible.
Customize your dashboard
Customize your Grafana dashboard panels including dark/light theme, chart colors, styles, pictures and logos.
Make your dashboards interactive (and scalable)
Enable users to interactively select devices and parameters via dropdowns, enabling quick custom analyses directly in the browser.
Set up OEM vehicle fleet management
Need to create dashboard visualization across a vehicle fleet?
You can easily deploy the CANedge2 or CANedge3 in a vehicle fleet to collect CAN/LIN data to your own Azure Blob Storage container via WiFi or 3G/4G. Uploaded raw data will be automatically DBC decoded via Azure Functions using your DBC files stored in the input container. The decoded data is written to your Data Lake Gen2 output container in the form of a Parquet data lake. From here, your engineers can easily deploy visualization of the data via Grafana-Synapse dashboards.
Visualize vehicle/machine data from your CANedge1
Need to visualize data recorded offline by your CANedge1?
Grafana-Synapse can also be used if you are simply logging data to an SD card via the CANedge1. Our plug & play Python scripts lets you DBC decode and upload a backlog of data from local disk to your Azure data lake. By following the Grafana-Synapse guide, you will set up an output container for storing your Parquet data lake in a few clicks - and make it accessible for visualization in Grafana.
Build white label fleet management platform
Need to create an targeted dashboard solution for your customers?
The CANedge is designed for white labeling by OEMs and system integrators. You can add your own logo and use our software/APIs to deploy your own solutions to specific target audiences - e.g. for predictive maintenance, marine telematics, heavy-duty fleet management or OBD2 car telematics. Here, Grafana-Synapse makes it easy and low cost to deploy custom dashboards for specific use cases or clients.
Democratize your data lake via Synapse SQL
Need to broaden access to your Parquet data lake?
Importantly, deploying an Azure Parquet data lake with Synapse not only enables Grafana dashboard visualization. More generally, Synapse provides a standard SQL interface to your entire Parquet data lake - enabling a far wider audience in your organization to work with your vehicle/machine fleet data. For example, your team can perform SQL queries directly in Synapse - or use e.g. Python to quickly perform scripted data extraction at scale. In particular, Synapse eliminates the significant complexity that is normally encountered when extracting data from 'beyond-memory' datasets (e.g. 1 TB of data) - returning just the insights you want via simple SQL queries.
Ready to visualize your CAN data?
Get your CANedge today!
