AppsScriptPulse

Library, Solutions, AI, Advanced

GeminiApp Gets a Major Upgrade: Seamless Transition, Enhanced Functionality, and More!

Tags:
| | 0 Comments |

The GeminiApp library, designed to bring Google’s Gemini AI models into your Google Apps Script projects, has just received a major update. This new release, version 2025.01, introduces significant enhancements that expand the library’s capabilities for building sophisticated AI-powered applications within Google Workspace.

Here’s a quick look at what’s new:

  • Seamless Transition: The updated library allows for a smooth transition from Google AI Studio to Vertex AI.
  • JSON-Controlled Generation: Generate content in JSON format, either by providing a schema or allowing the model to infer it from your prompt.
  • Code Execution: Generate and execute code directly within prompts using gemini-2.0 models.
  • System Instructions: Guide the model’s behavior by providing system instructions during initialization.
  • Caching: Improve efficiency and reduce token usage by caching file uploads.
  • Easier Copy/Paste: The library now supports initialization using both new GeminiApp() and new GoogleGenerativeAI(), making it easier to copy code from Google AI Studio.

The update includes over 1.5K new lines of code, primarily in src/GeminiApp.js, src/GoogleAICacheManager.js, and tests/tests.js. Existing projects can be updated by replacing the existing GeminiApp.gs code with the updated library. This update provides a broader set of tools to create more personalized and efficient workflows.

For those new to the GeminiApp library, multiple setup options are available. Check out the updated examples and test cases in the README.md file to get started.

This update marks a step forward for developers integrating Gemini into Google Workspace. With new features and the ability to move between Google AI Studio and Vertex AI, the possibilities are vast.

Feel free to share your use cases, code improvements, and feature requests!

Source: GeminiApp Gets a Major Upgrade: Seamless Transition Between Google AI Studio and Vertex AI, Enhanced Functionality, Multimodal Input, and More!

Solutions, AI, Forms, Sheets, Vertex AI, Intermediate

Using Google Forms and the Gemini API to automate creation of multiple choice questions (MCQs)

Tags:
| | 0 Comments |

This report proposes a novel learning method using Gemini to automate Q&A generation, addressing the challenges of manual Q&A creation. By integrating with Google tools, this approach aims to enhance learning efficiency, accessibility, and personalization while reducing costs.

The rapid advancement of technology has offered both opportunities and challenges to the education sector. While technology can be a valuable tool for supporting teaching and learning, concerns about its appropriate use have existed for a long time.

The education sector is witnessing an increase in AI tools, each promising to enhance teaching and learning. However, the quality of these tools varies significantly. Some are better designed and can – if used appropriately – can be beneficial, while others are poorly designed. This can make it challenging for educators to find the right solutions that meet their specific needs.

This blog post by Kanshi Tanaike sheds light on the inner workings of some commercial AI tools developed for educators. It demonstrates how these tools can use generative AI to create multiple-choice questions (MCQs) on a given topic. In this particular solution, Google Forms are used in the process, with questions and answers automatically generated by the Gemini API for the user to answer in a Google Form.

For educators who have experience with Google Apps Script, this project looks like a useful starting point to refine and create your own solution or simply used to gain insights into the functioning of similar commercial tools.

Source: A Novel Approach to Learning: Combining Gemini with Google Apps Script for Automated Q&A

Solutions, AI, Vertex AI, Intermediate

Smart replacing images in Google Slides with Gemini Pro API and Vertex AI

Tags:
| | 0 Comments |

Image credit: Ivan Kutil

Surely, you have also experienced having a presentation in which you needed to replace old content with new. Replacing text is very simple because you just need to use the Replace function, and you can either do it in the Google Slides user interface.

The problem arises when you need to replace one image with another, for example, if your corporate logo is updated to a new graphic design or if one of your favorite cloud services updates its icons (Gmail, blink blink ;-) It’s still somewhat bearable with one presentation, but what do you do when, like me, you have thousands of Google Slides files on your Google Drive?

This post explores a clever application of the Gemini API’s multimodal capabilities, created by Ivan Kutil. His code utilizes GenAI to automatically detect outdated logos within your Google Slides presentations. The original blog post (March 2024) used the Gemini Pro Vision API. As a sign of how rapidly this area evolves, Google now recommends switching to Gemini 1.5 Flash or Gemini 1.5 Pro.

Switching to the newer APIs is very straightforward as all you need to do is search the source code for models/gemini-pro-vision and replace with models/gemini-1.5-flash or models/gemini-1.5-pro.

For Apps Script projects I lean towards Gemini 1.5 Flash as it is designed for speed. I’m also always looking to opportunities to test the GeminiApp library for Apps Script and very quickly I was able to fork Ivan’s code and use Gemini 1.5 Flash using a service account.

While GeminiApp requires some initial setup, it offers significant advantages:

  • Easy Model Testing: Experiment with various models quickly.
  • Built-in Features: Includes functionalities like exponential backoff.

If you would like to explore here is a sample slide deck you can copy which has the container bound forked code (if you don’t want to use a service account here is more information on other setup options).

Source: Smart replacing images in Google Slides with Gemini Pro API and Vertex AI

Solutions, AI, Docs, Intermediate

Build a Google Drive Add-on with Gemini to rename Untitled Google Docs – Name with Intelligence

Tags:
| | 0 Comments |

Google Workspace Add-on for Google Drive, which uses AI to recommend new names for the selected Doc in Google Drive by passing the body of the document within the AI prompt for context.

Tired of staring at “Untitled document” in Google Drive? At Google Cloud Next ’24 and I/O 2024, the Google Workspace DevRel team showcased new Apps Script samples utilizing the Gemini API for AI-powered functionality.

One such sample, “Name with Intelligence” by Charles Maxson, helps developers overcome this common hurdle. This Google Drive add-on leverages the Gemini API to suggest relevant titles for your documents, saving you valuable time and effort.

This innovative Google Drive add-on tackles the common struggle of naming untitled documents. It utilizes the Gemini API to analyse the content of your Google Doc and suggest relevant titles based on its understanding of the text. This not only saves you time brainstorming titles, but also ensures your documents are clearly named for easy searchability later.

Want to see it in action? Check out this clip from Charles’ TU5.8 – Apps Script and Gemini Next ’24 Redux presentation.

Ready to give “Name with Intelligence” a try?

Head over to the GitHub repository to explore the source code.

For those interested in my GeminiApp library, you can also find the sample here.

Take Your Development Further

This is just a glimpse of what’s possible with the Gemini API and Apps Script. With a little creativity, you can develop your own AI-powered Google Workspace add-ons to streamline your workflows and boost your productivity.

I encourage you to experiment and create innovative solutions that enhance your Google Workspace experience!

Source: apps-script-samples/ai/drive-rename/README.md at main · googleworkspace/apps-script-samples

Solutions, AI, Chat, Vertex AI, Advanced

Answer questions based on Google Chat conversations with a Gemini API powered Chat app

Tags:
| | 0 Comments |

This tutorial shows how to make a Google Chat app that answers questions based on conversations in Chat spaces with generative AI powered by Vertex AI with Gemini. The Chat app uses the Google Workspace Events API plus Pub/Sub to recognize and answer questions posted in Chat spaces in real time, even when it isn’t mentioned.

Here is another great tutorial from the Google Chat DevRel team, this time showcasing how the Google Workspace Events API and some new Google Chat UI elements can be used to turn a Chat space into a Gemini Pro powered knowledge base.

The sample solution will let you consume your Chat space message history into a Firestore database. The Chat app is an intelligent agent that can then monitor for new questions and make suggestions using Gemini to generate content based on the previous messages.

There is quite a bit of setup required as part of this tutorial, but it provides a solid foundation for quickly scaling this to your needs.  A demonstration and explanation of the sample app was given as part of the Google Chat apps and APIs: Build connected workflows for the hybrid workplace session at Google Cloud Next and a recording might be available soon!

Source: Answer questions based on Chat conversations with a Gemini AI Chat app  |  Google Chat  |  Google for Developers

Solutions, AI, Sheets, Vertex AI, V8, Advanced

I challenged Gemini to a game of battleship in Google Sheets. Here’s what happened 🚢

Tags:
| | 0 Comments |

 

I tried to see if an AI that’s good at writing could also make smart moves in a game. I chose Battleship and set it up in Google Sheets to play against Gemini, the AI. The result was mixed. On one hand, yes, Gemini could play the game. It followed the basic rules and even managed to sink some of my ships. This was a big deal, especially since it took me a ridiculous number of days of coding to get there, and I nearly gave up at one point.

Dmitry Kostyuk has shared a blog post detailing his experiment pitting the Gemini API  against himself in a game of Battleship. As explained by Dmitry  while Gemini could follow the rules and even sink some ships, it needed help to avoid basic mistakes, revealing that AI still has room to grow in the realm of strategic games.

Dmitry built the game in Google Sheets and the source code is linked from the post. To guide Gemini, Kostyuk crafted detailed prompts outlining the game’s mechanics and decision-making logic. However, he encountered challenges due to Gemini’s limitations in providing strategic responses. Despite these hurdles, the project yielded valuable insights into prompting techniques for AI decision-making.

Source: I Challenged Gemini to a Game of Battleship. Here’s What Happened.

Add-on, Solutions, AI, Slides, Vertex AI, Advanced

Boost your presentations with AI: A Google Apps Script tutorial for a Google Slides reviewer assistant

Tags:
| | 0 Comments |

The Slides Advisor project is an open source, Google Workspace Add-On that uses artificial intelligence (AI) technology to review and give feedback on your presentations, whenever you need it. The project uses Google’s Gemini API image and text processing features to analyze your Google Slides against a set of guidelines and lets you know how you are doing against those guidelines. Even better, you can customize those guidelines to follow your organization’s recommendations, or remind you to improve your presentations based on rules you define for yourself.

We’ve all been there: scrambling to cram too much information onto a single slide. But what if you had an AI assistant to give your presentations a once-over?

The Slides Advisor is a free, open-source Google Workspace Add-on for Google Slides that uses the power of AI to analyze your presentations. It checks your slides against customisable guidelines, helping you ensure they’re clear, concise, and visually appealing.

Everything you need to get started is covered in the source post by Joe Fernandez and Steve Bazyl. One of the highlights for me is seeing how you can setup a service account to access the Gemini AI API with Google Apps Script. The post also covers how you can modify the prompt you use to get different responses from Gemini.

Source: Build an AI Slides Reviewer with Gemini  |  Google AI for Developers

Solutions, AI, Drive, Vertex AI, Intermediate

How to automatically rename files in Google Drive with Apps Script and the Vertex AI Gemini API

Tags:
| | 0 Comments |

Quickly rename files in Google Drive with Apps Script and Google Gemini AI. The script will automatically rename the files with a descriptive name based on the image content.

Amit Agarwal is celebrating 20 years of ‘Digital Inspiration’, one of my go to resources for Google Workspace tech tips, tutorials and how-to guides. This recent post from Amit is a great example of the types of solutions he regularly shares. This time Amit is looking at how the Generative AI capabilities of the Gemini API can be used to help rename images in Google Drive. To achieve this there Amit shares handy Apps Script snippets for finding images in a Google Drive folder, grabbing the thumbnail images as Base64 encoded strings before calling the Gemini Pro Vision API to get suggested filenames with the prompt:

Analyze the image content and propose a concise, descriptive filename in 5-15 words without providing any explanation or additional text. Use spaces instead of underscores.

For this solution an API key for Gemini Pro Vision is generated in Google AI Studio (formerly MakerSuite). There are still geographic restrictions on access to Google AI Studio, but given Apps Script runs on Google servers once you have a API key you can continue to use it without having to proxy a location.

Image credit: Amit Agarwal

Adding image capabilities to the GeminiApp Apps Script library has been on my TODO. Now I’ve got an example to play with it should be easier to do .. thank you Amit Agarwal.

Source: How to Automatically Rename Files in Google Drive with Apps Script and AI – Digital Inspiration

Solutions, AI, Docs, Drive, Advanced

Semantic poetry with Google Apps Script – Using Semantic Retrievers and Attributed Question and Answer (AQA) in Google Workspace 

Tags:
| | 0 Comments |

Image credit: Zzapps.nl

Image credit: Zzapps.nl

Imagine this: you recall a document about an intriguing subject but can’t pin down a specific term. It’s a common scenario where traditional search methods in Gmail or Google Docs often fall short, relying heavily on exact terms. Enter the realm of ‘semantic’ search, powered by advanced language models. ‘Semantic’ isn’t just a fancy word; it’s about understanding the meaning and context behind your words. Instead of a frustrating keyword hunt, these models interpret your descriptions, no matter how vague, to find that needle in the digital haystack.

“Code is Poetry” is the tagline popularised by the open-source blogging platform WordPress. In this post from Riël Notermans at Zzapps.nl ‘code is FOR poetry’. It’s well worth spending the time unpick what is happening in this tutorial. To help understand the implications I would also recommend watching this short video on the Google Workspace Developers channel where Riël explains how the technique can be used for other applications like knowledge bases.

Even if you are not interested in generating poetry it’s an opportunity to see how the Vertex AI Generative Language API can be used in Google Workspace, in this instance to generate text for a Google Doc using a corpus of data from your Google Drive. Follow the source link for the code and setup instructions.

Source: Semantic Poetry with Google Apps Script – Zzapps

Solutions, AI, Chat, Advanced

Create a Google Chat Bot on your own data with Vertex AI Search and Google Apps Script

Tags:
| | 0 Comments |

Create a Chatbot answering user questions based on your documents. RAG implementation with multiturn using Vertex AI Search and Apps Script

More GenAI, this time from Stéphane Giron looking at how Apps Script can be used to provide the glue for a Google Chat app powered by Vertex AI Search. In this example you can see how unstructured data like PDF documents can easily be uploaded to a Cloud Storage folder, which then become the knowledge base for the Chat app. The post includes a Google Apps Script snippet for sending messages to the Vertex AI Search API as well as instructions on how to create the chatbot, including how to import data into Vertex AI Search and how to integrate the chatbot into Google Chat.

The post is a great summary of what is possible when combining Google Chat and Vertex AI Search. If you are interested in finding out more about what is possible using Vertex AI Search with follow-ups Google provide a comprehensive guide.  

Source: Create a Google Chat Bot on your own data with Vertex AI Search and Google Apps Script