Imagine this: you recall a document about an intriguing subject but can’t pin down a specific term. It’s a common scenario where traditional search methods in Gmail or Google Docs often fall short, relying heavily on exact terms. Enter the realm of ‘semantic’ search, powered by advanced language models. ‘Semantic’ isn’t just a fancy word; it’s about understanding the meaning and context behind your words. Instead of a frustrating keyword hunt, these models interpret your descriptions, no matter how vague, to find that needle in the digital haystack.
“Code is Poetry” is the tagline popularised by the open-source blogging platform WordPress. In this post from Riël Notermans at Zzapps.nl ‘code is FOR poetry’. It’s well worth spending the time unpick what is happening in this tutorial. To help understand the implications I would also recommend watching this short video on the Google Workspace Developers channel where Riël explains how the technique can be used for other applications like knowledge bases.
Even if you are not interested in generating poetry it’s an opportunity to see how the Vertex AI Generative Language API can be used in Google Workspace, in this instance to generate text for a Google Doc using a corpus of data from your Google Drive. Follow the source link for the code and setup instructions.
In this report, I would like to introduce the sample scripts for using Microsoft Docs files with Document service, Spreadsheet service, and Slides Service of Google Apps Script.
Users have had the ability to edit MS Word, Excel and PowerPoint files directly in Google Drive for a number of years, Google making this the default behaviour in 2020. There aren’t currently any APIs or Apps Script services that enable you to edit these particular document types. This however hasn’t stopped Kanshi Tanaike for developing and sharing the MicrosoftDocsApp library which makes it possible to edit MS Word, Excel and PowerPoint files using the same methods as you would for Google Docs, Sheets and Slides.
The library is able to do this by converting the Microsoft files into the Google equivalent, before using the MS export options in Google Drive to replace the original files. This does create some limitations highlighted in the post. In particular, the original documents are overwritten and not directly edited which means any incompatible feature as part of the Microsoft to Google conversion will be lost. Regardless of these it’s a clever approach and might be useful for users who have to keep feet in both the Google and Microsoft worlds.
Avoid downloading private service account keys by using impersonation in Apps Script to obtain access tokens.
For Google Workspace Admins you can gain super powers (and super responsibility) using Google Cloud project service accounts. A common scenario is using service accounts with domain-wide delegation to make API calls impersonating Google Workspace users. With this you can do things like set a user’s Gmail settings including signatures, authenticate as a user in Google Chat and much more.
A quick way to use a service account is to download a JSON key. The challenge then given the potential capabilities of service accounts is securely storing the JSON key. A alternative approach, which is explained in this post by Justin Poehnelt, is using Apps Script to create and fetch a short-lived credential for your service account.
Short-lived credentials are highly recommended for applications requiring robust security and precise access control for service accounts, reducing the attack surface and the risks with accidentally exposing static secrets.
You can find out more include code snippets/setup in Justin’s post which also links to the support documentation.
Create a Chatbot answering user questions based on your documents. RAG implementation with multiturn using Vertex AI Search and Apps Script
More GenAI, this time from Stéphane Giron looking at how Apps Script can be used to provide the glue for a Google Chat app powered by Vertex AI Search. In this example you can see how unstructured data like PDF documents can easily be uploaded to a Cloud Storage folder, which then become the knowledge base for the Chat app. The post includes a Google Apps Script snippet for sending messages to the Vertex AI Search API as well as instructions on how to create the chatbot, including how to import data into Vertex AI Search and how to integrate the chatbot into Google Chat.
The post is a great summary of what is possible when combining Google Chat and Vertex AI Search. If you are interested in finding out more about what is possible using Vertex AI Search with follow-ups Google provide a comprehensive guide.
Gemini LLM, now a Vertex AI/Studio API, unlocks easy document summarization and image analysis via Google Apps Script. This report details an example script for automatically creating the description of the files on Google Drive and highlights seamless integration options with API keys.
In this blog post, Kanshi Tanaike shows how you can automatically create descriptions for files on Google Drive using the Gemini Pro API with Google Apps Script. The post includes a step-by-step guide on how to set up and use the Gemini Pro API by generating a key is Google AI Studio (formerly Maker Suite). There are geographic restrictions on Google AI Studio, but you can call Gemini Pro from a Google Cloud project with a little more setup (a previous post sharing Tutorial: Respond to incidents with Google Chat, Vertex AI, and Apps Script outlines a general approach for connecting Apps Script to Google Vertex AI services.)
The post includes a couple of examples showing how Gemini can be used to provide responses to both text and visual prompts. Gemini Pro is still in public preview and as Kanshi Tanaike highlights rate limiting will likely prevent putting these solutions into production just now. At this point hopefully there is enough to start experimenting with Google Workspace integrations to Gemini.
Imagen: An photo image which has a laptop with a spreadsheet application which appears to have rays of light
This is the second part exploring the GenAI capabilities in Google Sheets. In this part learn how you can make an Enhanced Smart Fill for Google Sheets
Google recently announced the latest feature for Duet AI for Google Workspace with Enhanced Smart Fill, which uses GenAI in Google Sheets to generate content based on data and the patterns entered by the user.
Continuing a previous post exploring the PaLM 2 API and LLM capabilities in Google Sheets, this post looks provides a Google Sheet template for experimenting with LLM prompts and spreadsheet data, including how you could make a ‘Enhanced Smart Fill’-like star review generator.
The post includes everything you need to get started, with you only having to make your own MakerSuite API key.
Imagen: photo looking over the shoulder of a robot looking at a screen with chat messages and hand writing notes
This tutorial shows how to make a Google Chat app that responds to incidents in real time. When responding to an incident, the app creates and populates a Chat space, facilitates incident resolution with messages, slash commands, and dialogs, and uses AI to summarize the incident response in a Google Docs document.
Paraphrasing noted Google Workspace Developer Expert, Romain Vialard, GenAI has made Google Chat apps a tangible prospect. This tutorial from the Google Developers site is a great example of how you can use Google’s Vertex AI with Google Chat. The tutorial will help you create a Google Apps Script powered Chat app that is able to summaries the messages in a Google Chat space.
There is a lot to take away from this example, but here are some of the headlines:
Setting a Google Cloud Project to use the new Google Chat Advanced Service for Apps Script
Setup and code for making calls to Google’s Vertex AI PaLM API (LLM) from Google Apps Script
Using the responses from Vertex AI to generate new assets.
There is a lot more you can do from this starting point, but hopefully it gives you a great starting point.
Building a Support AI Assistant for Google Workspace using Apps Script, Gen AI, and Google Chat.
This post from Stéphane Giron highlights one approach for improving responses from LLMs by integrating Google Custom Search Engine responses into the prompt. In this example Stéphane used Google Apps Script to power the AI Assistant, integrating with Google Chat for the user interface and Cloud Functions to reformat data.
This post is another example of the ‘power of the prompt’ and how LLM prompting strategies are a very effective way to utilise LLMs without having to ground or fine tune. If you are interested in understanding more here is a useful notebook produced by Michael W. Sherman which illustrates two powerful LLM prompting strategies: Chain of Thought and ReAct (Reasoning + Acting).
Last month, OpenAI announced Function calling, “a new way to more reliably connect GPT’s capabilities with external tools and APIs”. Super useful ❤️ and we decided to integrate that with Google Apps Script.
Following on from yesterday’s Pulse post which highlighted Ben Collins reflections on ‘What can AI do for you as a Google Sheets user? Is the hype justified?’, here is a contribution from another Apps Script expert, Romain Vialard, announcing a ChatGPT library for Google Apps Script.
The source post highlights a number of examples, including the standard ‘prompt’ calling. Where this post gets very interesting is integration with Open AI’s function calling features:
In an API call, you can describe functions to gpt-3.5-turbo-0613 and gpt-4-0613, and have the model intelligently choose to output a JSON object containing arguments to call those functions. The Chat Completions API does not call the function; instead, the model generates JSON that you can use to call the function in your code.
To illustrate this below is a screenshot of a very slightly extended example, which has a Google Sheet with the headings email, name and tip topic. The functions are described to ChatGPT and in the case of sendMessage() the parameters it requires. The ChatGPTApp library handles the functions and in the case of the getContactsList() uses it to pass the Google Sheet data for ChatGPT to format a JSON response to correctly call the sendMessage() function.
I found it took me a while to start understanding what is going on and would recommend experimenting with the library and sample code to get a sense what is possible.
Who takes the notes ? Simple question we always ask at the begining of the meeting, maybe it will be over in a near future with Generative AI. Using Meet recording, included in the Google Workspace licences, we can generate the transcript of the meeting with Google Speech to Text API and then generate the minutes.
Lots of chatter around ‘Gen AI’, but cutting through the hype and, in particular, the Platform-As-A-Service opportunities created with generally available Large Language Models (LLM) opens lots of new doors for Workspace Developers.
This example from fellow Google Developer Expert, Stéphane Giron, is an area I think has the most potential, using LLMs in Google Workspace Add-ons such as Gmail, Docs, Calendar etc. With Google’s Vertex AI LLMs there is really opportunity to create powerful solutions whilst preserving data sovereignty. Exciting times!
