The best way to Combine Google Gemini into Tableau Dashboards?

Introduction

Tableau is a strong and superior visualization instrument. It covers the entire visible improvement lifecycle. Beginning with Tableau Prep Builder, you possibly can successfully clear, remodel, and supply knowledge underneath one roof. Tableau Desktop then presents this knowledge to inform a narrative, whereas Tableau Server lets you share these visuals with the supposed viewers. On this complete course of, the interpretation of the dashboard requires a guide effort. From the creator’s standpoint, the dashboard presents an ideal pattern such that it grasps all of the eyeballs however from a viewer’s perspective, it’d take a while to grasp the dashboard. The viewer would require a walkthrough of the charts, and figures after which draw some conclusions. To streamline this course of, integrating Google Gemini into Tableau can improve dashboard comprehension by leveraging AI-driven insights, lowering the time required to interpret knowledge.

What if the viewer will get an AI helper to determine the charts and figures themselves? Introducing LLMs. Because the introduction of GPTs and massive language fashions (LLMs), many guide workflows, resembling chatbots, translations, and analysis, have shifted to AI brokers. Information Evaluation is one such sector that’s steadily sliding to AI brokers. These brokers can take up automated knowledge wrangling, anomaly detection, function choice, or predictive analytics. On this information, we’ll discover methods to combine Google Gemini into Tableau dashboards. Ask it to present knowledge summaries, predictions, conclusions, and extra.

Studying Goals

• Uncover methods to develop your dashboard performance with Dashboard Extensions.
• Perceive the toolkit for establishing Gemini in Tableau Dashboards.
• Discover methods to leverage Python to work together with the Tableau Dashboard and ship data to and from the dashboards.
• Learn to allow customers to ask questions on graphs and figures and get responses in the identical window.
• Acknowledge the potential safety dangers of sending delicate data to LLMs.

This text was revealed as part of the Information Science Blogathon.

What are Dashboard Extensions?

Including an exterior service to an already fully-fledged utility is a tough job. Tableau is a pretty big utility and it’s almost inconceivable to code out a brand new function with out accessing all of the codebase and the dependencies. Subsequently we’ve Tableau Dashboard Extensions. Dashboard Extensions are net apps that may be included right into a dashboard like different dashboard parts. These net apps work together with Tableau internals utilizing Tableau Extensions API. On a excessive stage, these apps will be divided into two varieties:

• Community-enabled: These apps run on servers situated exterior the Tableau atmosphere. The app will be hosted on any platform and the client-side code is embedded instantly into Tableau by way of the extension file (trex).
• Sandboxed: The sandboxed apps run in a protected atmosphere with restricted or no entry to assets or net companies. Often developed by the Tableau workforce, they’ve restricted capabilities but are highly effective in enhancing dashboard interactivity.

There’s a complete library of Dashboard Extensions accessible on the Tableau Alternate. Some are formally developed by the Tableau and Salesforce groups and extra by well-known corporations. The extensions are an effective way to increase the performance of a base dashboard and make it extra interactive. There are extensions to export knowledge from the charts, add new data to the dashboards, or edit the prevailing knowledge!

For our use case, we’ll construct a Dashboard extension the place a consumer selects knowledge factors on the dashboard, submits a query, sends all this data to Gemini for evaluation, and shows the ends in the identical window.

Dashboard Extensions makes use of the underlying Tableau Extensions API to work together with the Tableau dashboard’s inner objects. This JavaScript library API can restrict the Python developer’s expertise. To beat this, we’ll use Anvil. Anvil is a platform for constructing full-stack net functions in Python. It has an open-source library referred to as trexjacket that permits builders to work together with the dashboard internals utilizing Python. The Anvil sits as an interface between Tableau JavaScript APIs and the Python code from the consumer facet. The trexjacket supplies the Tableau dashboard entry by way of Python and the workflow continues.

For our Dashboard Extension that works with Google Gemini, we’d like the next:

• Gemini API Entry
• Anvil undertaking preconfigured with trexjacket
• Tableau Software and a Dashboard

Gemini API Entry

Google has launched API entry to Gemini, beforehand often called Bard underneath a sub-vertical referred to as “Google AI for builders”. One can discover Gemini API, Google AI Studio, Gemma fashions, and Google AI Edge. The API entry remains to be supplied by way of the Google Cloud Mission (GCP). Head to Gemini and click on the “Get API key in Google Studio” choice. Clicking “Create API key” will immediate two choices, create a brand new undertaking or add API entry to an present undertaking. 

Select any choice and preserve the API key in a protected place.

Anvil Mission with Trexjacket

Anvil is a powerpack platform providing full-stack net improvement in Python. The primary differentiator is constructing the frontend in Python and successfully stitching it to the backend server code. The design organizes the entire utility in an object-oriented programming paradigm, facilitating simpler entry and administration of various parts within the net app. It additionally permits modifying the underlying make-up recordsdata, HTML, CSS, and JS for extra granular management over the consumer interface. 

Anvil has constructed a template undertaking preconfigured with trexjacket that fast-tracks the Dashboard extension improvement underneath Anvil X. This template has the configuration set for trexjacket as a dependency for the Anvil app. To arrange a undertaking with trexjacket as a dependency, join an account on anvil.works, choose “Create a brand new app” and choose “Tableau Extension”.

Within the template app, you possibly can delete the configure settings, and startup module and clear up the design web page of the app as these will not be wanted for our use case. Subsequent, we have to add the Google Generative AI Python library within the Anvil atmosphere. For that, go to settings from the left, choose Python variations, change to Python beta, choose the bottom package deal as customary, and add the package deal identify as “google-generativeai”

For the scope of this information, we won’t discover Anvil fundamentals and canopy solely the required sections of Anvil. If you wish to deep-dive into Anvil as a instrument, try my different article Frontend, Backend & Database Utilizing Python: Anvil. 

Tableau

Tableau Dashboard extensions assist each desktop and server/cloud situations, in addition to Tableau Desktop Public and full-suite functions. We are going to use Tableau Desktop Public to check the extension in our use case. The following factor we require is a working dashboard. You possibly can decide any dashboard of your selection for testing functions. I’m utilizing a dashboard revealed on Tableau Public concerning the Paris Olympics 2024.

Observe: The writer of this dashboard mentions the dataset is from the Paris 2024 Olympic Video games Dataset in Kaggle.

Tableau Dashboard and Anvil Connection Setup

Let’s create a dummy extension to determine the connection between the Tableau Dashboard and the Anvil. Within the Anvil app, drag a label from the add part choice to show some textual content. Upon clicking this label, we are able to change the label show textual content and tweak different properties resembling font, coloration, and measurement.

Subsequent, from the highest of the app, click on “Take a look at in Tableau” and obtain the manifest file. This file has the “trex” extension and will be loaded as an area extension within the Tableau Dashboard. You possibly can select to maintain the dashboard in Tableau Desktop for now and later publish it on the server. The server configuration for that can be talked about within the popup. 

Now drag an extension object into the Tableau dashboard and click on “Entry Native Extensions”. Right here choose the manifest file generated by Anvil.

This extension will work together with Anvil to course of the info we ship from the Tableau dashboard and subsequently falls underneath the network-enabled class of Dashboard extensions. Choose “Okay” when prompted to permit entry to the extension. The Anvil App will load within the window, establishing our connection between the Tableau Dashboard and the Anvil server! We are able to begin including parts to this app and sew the Gemini API.

Constructing the Extension UI

Anvil supplies drag-and-drop performance for designing the consumer interface of the appliance. There are numerous parts accessible within the library to select from. For our use case, the design breakdown seems like this:

• Enter field (TextBox) for taking the consumer query
• Submit button for sending the info to Anvil and Clear button to clear the query and the evaluation textual content
• A abstract label placeholder will show textual content that the Gemini API replaces with the outcomes.
• A title, colours for submit and clear button, and textual content colours to enhance consumer expertise
• You possibly can add the placeholder texts and show names by clicking on the part, opening the properties panel, and modifying the required attributes.

 Beneath is the design we can be utilizing for this extension:

The most effective half about network-enabled extensions is the power to ship adjustments to the top consumer with none reinstallation. We are able to see the up to date UI instantly within the dashboard with out regenerating the manifest file. Click on on the extension object, choose the dropdown, and click on “Reload”. It will replace the extension in real-time.

We have to present object names to those UI parts to sew them collectively, change their state within the coding half, and make them practical. To try this, open the part properties panel and add the identify within the “self”. As you possibly can guess, these parts can be accessible as objects on our predominant display screen. The names supplied for this extension are:

• TextBox: user_question
• Submit Button: btn_submit
• Clear Button: btn_clear
• Abstract Label: abstract

Coding the Logics for Extension

In Anvil, client-side coding manages the consumer interface and handles all UI interactions, whereas holding much less safe coding components. Because the shopper code runs within the net browser, it has restricted capabilities for processes on the browser stage. However, server-side coding manages duties like authentication, password checks, and sustaining database connections.

In our case, the shopper code can deal with all of the dashboard interactions and cross this knowledge to the server code to arrange and ship to the Gemini. The Gemini API must be accessed securely and would require server-side coding there. Let’s begin establishing the client-side coding after which transfer to the server facet.

Accessing Dashboard and Stitching UI

To change to the coding a part of the UI, choose the “Code” tab from the highest. The system populates it with some boilerplate code to get issues began. You possibly can take away issues like registering occasion handlers and the config web page at startup for now. Use the code beneath as a place to begin to comply with alongside.

from ._anvil_designer import client_codeTemplate
from anvil import *
import anvil.server

from trexjacket.api import get_dashboard
dashboard = get_dashboard()

class client_code(client_codeTemplate):
  def __init__(self, **properties):
    self.init_components(**properties)
    

The system robotically imports and initializes the trexjacket as a dashboard object named “dashboard,” which serves because the entry level to entry all of the dashboard interactions. We are able to register 3 kinds of interactions from the dashboard: choice, filter & parameter change. To register an occasion sort, we have to use the register_event_handler operate of the dashboard object. It takes the occasion sort and the handler operate because the arguments. The handler operate takes care of the half when the occasion is triggered. This operate can entry the occasion metadata, worksheet, and underlying knowledge.

In our case, the alternatives achieved within the dashboard have to circulate via our extension. Subsequent, to get the chosen knowledge level values, we are able to use the handler operate’s occasion worksheet object to name the get_selected_marks operate and retailer it to cross the info to the backend. For demonstration, let’s change the show worth of the abstract part to point out the chosen values. To try this, we are able to modify the textual content property of the part to take the worth as the choice.

Code Implementation

Right here is the code implementation:

class client_code(client_codeTemplate):
  def __init__(self, **properties):
    self.init_components(**properties)
    dashboard.register_event_handler('selection_changed', self.selection_changed_event_handler)
    
  def selection_changed_event_handler(self, occasion):
    user_selections = occasion.worksheet.get_selected_marks()

    if len(user_selections) != 0:
      self._data = user_selections
      self.abstract.textual content = user_selections

Subsequent, we’d like the Submit and Clear buttons practical. You possibly can view the kind of occasions supported for any part by choosing the part and switching to the properties panel. The occasions part will listing all of the occasions together with their operate mappings. For buttons, we have to map the clicking occasion to do some operations. 

• Clear button: We have to clear the query field, abstract part, and the info saved for beforehand chosen marks
• Submit button: The textual content within the query field must be despatched to the Gemini API.

The supposed work of the submit button can be mentioned within the subsequent part. For now, let’s exchange the abstract textual content with the query textual content upon clicking the submit button.

Code Implementation

The code implementation is as:

class client_code(client_codeTemplate):
  def __init__(self, **properties):
    self.init_components(**properties)
    self.abstract.seen = False
    dashboard.register_event_handler('selection_changed', self.selection_changed_event_handler)

  def selection_changed_event_handler(self, occasion):
    user_selections = occasion.worksheet.get_selected_marks()

    if len(user_selections) != 0:
        self._data = user_selections

  def btn_submit_click(self, **event_args):
    """This methodology is named when the button is clicked"""
    self.abstract.seen = True
    self.abstract.textual content = self.user_question.textual content

  def btn_clear_click(self, **event_args):
    """This methodology is named when the button is clicked"""
    self.abstract.textual content=""
    self.user_question.textual content=""
    self._data=""
    self.abstract.seen = False
    

Including Gemini API Backend

Our UI is able to settle for the consumer inputs and the parts are stitched collectively. Now we are able to add the mind of our utility, the Gemini API entry to course of these consumer inputs. The following tips describe our subsequent steps:

• Server module: So as to add server-side coding to the appliance, click on on 3-dots within the server code part on the left and choose “add server module”.
• Secrets and techniques: We must always retailer the Gemini API entry key in a safe place to forestall unauthorized requests. Anvil Secrets and techniques is a secret supervisor the place we are able to add keys, passwords, or credentials and entry them in our code utilizing the Anvil secret module.
• Pre-configure Gemini API: Though Gemini like another LLM can course of any data, we are able to present pre-context to Gemini. This context will assist us get extra refined responses and give attention to our use case. We are able to cross these directions whereas initializing our mannequin object.

The code half for the above clarification goes like this:

import anvil.secrets and techniques
import google.generativeai as genai

GOOGLE_API_KEY = anvil.secrets and techniques.get_secret("GEMINI_API")
genai.configure(api_key=GOOGLE_API_KEY)
mannequin = genai.GenerativeModel(model_name="gemini-1.5-flash", system_instruction=[
        "You are an expert analyst and know everything about data analysis",
        "You can interpret data in any form whether it's a single data point or a list of data with keys"
        "You are on a mission to provide the best data analysis report when asked",
        "You are capable of answering the question without report as well on topics that require you to answer between a finite set of possibilities",
    ],)

Subsequent, we have to create a operate that accepts the query and the info from the shopper, processes the knowledge, and returns the resultant values. We expose this operate to the client-side code to permit it to make calls. We use the Anvil callable decorator to wrap these capabilities and make them callable from the client-side code.

Closing Operate in Server Module

Right here is our ultimate operate within the server module:

@anvil.server.callable
def generateDataSummary(immediate, knowledge):
  revised_prompt = f'''
  {immediate} + "nn" + {knowledge}
  '''
  response = mannequin.generate_content(revised_prompt)
  return response.textual content

The final step in constructing our dashboard extension is to name this operate, present the consumer inputs, and show the outcomes returned by Gemini API when the consumer selects some knowledge factors and clicks on the submit button. To attain this, we have to use the anvil server module name operate.This operate takes the operate identify uncovered as callable within the server module and accepts the parameters as key phrase arguments. Moreover, we are able to add a notification for consumer acknowledgment utilizing the Notification module. This implementation is finished as:

class client_code(client_codeTemplate):
  .
  .
  def btn_submit_click(self, **event_args):
    """This methodology is named when the button is clicked"""
    msg = "Wait"
    Notification(msg).present()
    self.abstract.textual content=""
    dataSummary = anvil.server.name('generateDataSummary', immediate=self.user_question.textual content, knowledge=self._data)
    self.abstract.seen = True
    self.abstract.textual content = dataSummary
    self._data=""

Testing out the Extension

Our extension is able to take inputs from dashboards, course of them in Gemini, and return ends in the identical window! Reload the extension within the dashboard, choose some knowledge factors, enter some questions, and hit undergo see the outcomes.

Try this demo video to see issues in motion:

Potential Safety Dangers

We’ve got efficiently created a Tableau Dashboard extension that seamlessly integrates into the dashboard. This extension will be added to any dashboard with none points. The one concern with utilizing an LLM might be the sensitivity of knowledge. Datasets resembling monetary, private well being, confidential, or proprietary data would possibly require further safety measures and knowledge anonymization/redaction strategies to make sure privateness and compliance. Though LLM supplier corporations resembling OpenAI present some flexibility to not permit the info coaching of the mannequin. It’d change relying in your agency and one ought to seek the advice of the infosec workforce earlier than implementing such options.  

Conclusion

On this information, we noticed methods to combine Google Gemini into Tableau Dashboards on a deeper stage. The response high quality depends upon how the query (immediate) is supplied. Many enhancements will be made to the extension resembling including a configuration web page to offer choices for choosing the mannequin for use, updating system directions, preprocessing the info to be despatched, and way more.

On the trexjacket facet, you possibly can discover different functionalities resembling accessing the filters and parameters or instantly accessing the underlying knowledge for a worksheet. The documentation is one of the best place to test these capabilities. One can develop extensions natively in JavaScript as effectively however it’s a studying curve for the Python group, subsequently, leveraging Anvil. 

Key Takeaways

• Tableau Extension is an effective way to take advantage of your dashboards. There are tons of extensions for quite a lot of use circumstances resembling visualization extensions for including Sankey circulate charts which aren’t natively supported on the time of writing.
• One can use Gemini, an LLM, to demo the dashboard extension, nevertheless it’s additionally doable to make use of any LLM, a fine-tuned model, or a knowledge specialist LLM for sensible functions.
• Integrating an LLM in your dashboards can free an analyst from answering a few of the repetitive queries and focus extra on taking motion.
• One also needs to pay attention to the potential knowledge leakage and use such techniques moderately by avoiding passing on delicate data.

Ceaselessly Requested Questions

The media proven on this article just isn’t owned by Analytics Vidhya and is used on the Writer’s discretion.