Last week at Google Cloud Next ’25 was packed with announcements, but one that particularly grabbed my attention was the unveiling of Google Workspace Flows. As Google Apps Script developers, many of us are familiar with automating simple tasks. However, the Flows demo hinted at a more accessible approach for tackling complex business processes.
Think about those workflows that go beyond basic if-this-then-that and into a world where you can easily configure Gemini to be your virtual assistant. Updating specific spreadsheet entries based on nuanced analysis, or finding and summarising information scattered across different files before replying to a customer. Traditional automation often hits a wall here because these tasks require context, reasoning, and sometimes even creative generation – capabilities standard automation tools lack.
Workspace Flows: AI Agents Joining the Workflow
What Google presented with Flows is a new solution for Google Workspace designed to automate these kinds of multi-step processes with AI providing assistance. Instead of just triggering actions, Flows uses AI models, including Gemini, as agents within the loop. This means the AI isn’t just kicking off a process; it’s actively participating – researching, analysing, creating, and reasoning to help get work done more efficiently and intelligently.
Having used tools like IFTTT it feels like a conceptual shift from simple automation to building intelligent, agentic workflows directly within the Workspace environment, without writing a single line of code.
Why Flows Looks Interesting for Developers: Current Capabilities and Future Vision
Beyond the core AI capabilities shown in the demo, Google outlined a vision for Flows that is particularly relevant for developers, indicating where the platform is heading:
No-Code/Low-Code Interface (Current): The initial preview allows configuring triggers (like new emails, form responses, etc.) and actions across core Workspace apps and integrating Gemini and Gems for AI-driven steps, removing the need to find alternative solutions (code you write or third party solutions).
Apps Script Extensibility (Future Vision): Google announced plans to allow developers to build their own custom triggers and actions using Apps Script. This creates the opportunity to integrate your own systems or add specific logic to get more out of Flows. The presentation briefly showed an example appsscript.json manifest snippet for declaring these elements around 13:08).
Workspace Connectors Platform (Future Vision): A dedicated platform for third-party integrations was also announced in the presentation as part of the roadmap. The plan is to enable connections to tools like Jira, Asana, Salesforce, HubSpot, etc., allowing them to be used as triggers or actions. The stated goal is to include the ability to build end-to-end workflows spanning beyond Google Workspace, with connectors built once potentially working in both Flows and Gemini Extensions.
Bring Your Own Models via Vertex AI (Future Vision): For advanced AI needs, Google shared the vision for integrating your own custom or fine-tuned models hosted on Google Cloud’s Vertex AI. The concept shown involved an ‘Ask an LLM’ step where you could select a ‘Custom Model’ directly within the flow builder, pointing towards future capabilities for incorporating highly specialized AI into Workspace automations.
Looking Ahead
Google Workspace Flows is definitely a platform I’ll be watching closely. The initial preview, focusing on AI agents-in-the-loop and core Workspace automation, is already compelling. But the announced roadmap for developer extensibility – adding Apps Script support, a robust connector platform, and the ability to call custom Vertex AI models – is what makes Flows truly exciting from a development perspective.
Flows is currently in alpha with its initial feature set. If this vision sounds as interesting to you as it did to me, you can sign up for the early access waitlist.
Getting started with generative AI in your Google Apps Script projects just got a whole lot easier! Google AI Studio has introduced a handy new feature allowing you to directly export your AI interactions as ready-to-use Apps Script code. If you’re new to Apps Script or integrating AI, this is a fantastic way to quickly add powerful features to your automations. Here’s how you can grab the code:
Click the Get code icon (</>) above the chat prompt.
In the ‘Get code’ window, click the language dropdown (this might initially show ‘Python’).
Select Apps Script from the dropdown list.
Click the Copy button to copy the generated Apps Script code to your clipboard.
If you are using AI Studio with your enterprise data, make sure you’re using a billable account so that your data is protected. This addition is perfect for rapid prototyping and understanding the basic API interaction. However, for applications needing more features essential for production environments, including robust error handling like exponential back-off, then you might want to look at my open source GeminiApp library.
Gemini API now generates images via Flash Experimental and Imagen 3. This report introduces image evolution within conversations using Gemini API with Google Apps Script.
The Gemini API recently gained the ability to generate images. Taking this a step further, Kanshi Tanaike has explored how to create evolving images within a conversation using Google Apps Script.
Often, you might want to generate an image and then iteratively add or modify elements in subsequent steps. Kanshi’s approach cleverly uses the chat functionality of the Gemini API (gemini-2.0-flash-exp model). By sending prompts sequentially within a chat, the API uses the conversation history, allowing each new image to build upon the previous one. This enables the generation of images that evolve step-by-step based on your prompts, as demonstrated in the original post with examples like drawing successive items on a whiteboard.
This technique is particularly useful because, as noted in the post, using chat history provides better results for this kind of sequential image generation compared to generating images from isolated prompts.
Kanshi Tanaike’s original post includes a detailed explanation, setup instructions (including API key usage and library installation ), and complete sample code snippets that you can adapt for your own Google Workspace projects.
As a Google Workspace Developer Advocate, I’m always exploring innovative ways to leverage technology. I’m thrilled to share my new Jupyter Notebook that showcases the power of Gemini and the Google Calendar API for productivity coaching. 🚀✨
This notebook dives into:
– Using Gemini’s multimodal capabilities to analyze calendar data. 📅
– Leveraging function calling to connect Gemini with Google Workspace APIs. 🔗
– Developing a personalized AI productivity coach. 🧑🏫
It’s amazing to see how generative AI can transform the way we work and optimize our time. ⏳
Explore the notebook on GitHub and discover how to build your own AI-powered productivity tools! 🛠️
Are you struggling to manage your time effectively? Mohammad Al-Ansari, Google Developer Advocate, has recently shared how the Gemini API can act as a personal productivity coach. The solution uses a Google Colab notebook, which is connected to the Google Calendar API. The Gemini API is used to provide personalized insights and recommendations to boost your productivity and improve your work/life balance. For those who don’t know, Google Colab is a free cloud-based platform for running Jupyter Notebooks, which are interactive coding environments that allow users to write and execute code and can be a useful tool to have in the toolbox when exploring data.
Some of the key features in Mohammad’s ‘Productivity Coach’ are:
Function Calling: This notebook uses Gemini’s ‘function calling’ capabilities, allowing it to dynamically interact with the Google Calendar API and retrieve real-time data. This ensures that the analysis and recommendations are always up-to-date and relevant to your current schedule.
Google Workspace Integration: By integrating with the Google Calendar API, this notebook shows how the Gemini API can be seamlessly integrated with other Google Workspace services. This opens up exciting possibilities for quickly experimenting with other Workspace data sources, such as Google Docs, Sheets or Drive.
Personalized Coaching: As a bonus you can see how Gemini can act as your personal productivity coach, so if nothing else you can get some tailored guidance and support about your own calendar
Imagine that you write in plain English what you want to do in Google Workspace (eg. workflows) and it happens just like magic. Insert text prompt, Gemini will generate the code for you and run it immediately. A dream? No, reality, thanks to my conceptual and practical idea of how to implement AI Agents in Google Apps Scripts to leverage the V8 runtime.
Ivan Kutil has explored the concept of AI Agents in Google Apps Script, enabling Google Workspace automation via plain English descriptions. Users describe their automation needs in natural language, which is then processed by Gemini API to generate the necessary code. The generated code is then executed in your Google Apps Script project.
Ivan’s solution uses the gemini-2.0-flash-thinking-exp-01-21 model, an experimental model within Vertex AI specifically designed to reveal its ‘thinking process’, resulting in more reliable code generation. The enhanced reasoning capabilities of this model are particularly beneficial for complex automation tasks, making it a powerful tool for Google Workspace customisation.
To ensure that the agent is doing the right thing, the clever bit is you can test the execution via a dry-run, where the code created with Gemini Flash Thinking is sent to an internal ‘Tester’ agent, which uses Gemini to comment on the code and summarises it in a log. It’s important to review the script before running it, as Ivan accepts no responsibility for the results of the script. Another nice feature is the generated code is stored in the Cache, so after running a dry-run and then a run, the same version will be executed within the Cache limit (currently set to 5 minutes).
This solution, which mirrors Gemini for Workspace’s ability to generate and execute basic Python code, suggests a future where Gemini for Workspace could write and execute Apps Script code for basic tasks. This has the potential to transform how users interact with and automate their Google Workspace environments.
If you are interested in a version of Ivan’s solution that incorporates my GeminiApp library, follow this link. For additional information on Ivan’s solution including setup instructions follow the source link.
If you have domain-specific knowledge that you want an LLM to leverage, you probably have a use case for fine-tuning. Fine-tuning can significantly improve how well the model understands and responds to your queries, whether it’s legal documents, medical texts, financial reports, or niche industry data.
The most crucial step in this process is structuring your data correctly. If your dataset is well-organized and formatted properly, the rest of the workflow becomes much more manageable. From there, it’s just a matter of setting up a few configurations and automating parts of the process with Apps Script. That’s where things get interesting and surprisingly efficient.
Tired of manually processing invoices? I recently built a demo that automates this tedious task using the power of Gemini, AppSheet’s new Gmail integration, and a custom Apps Script library. Let me show you how it works!
Here’s the setup:
Gmail Integration: AppSheet’s new Gmail integration allows you to trigger automations based on incoming emails. I’ve set it up to monitor a specific Gmail label called “Invoices”.
Apps Script Library: Using my “GeminiApp” (available on Github) which simplifies the interaction with Google’s Gemini AI models directly from Apps Script. This library handles the heavy lifting of making API calls and managing responses.
Controlled Generation: Gemini’s “Controlled Generation” feature lets me define a JSON schema that structures the AI’s output. This is key for extracting invoice data in a consistent, machine-readable format.
The Workflow:
Invoice Received: When an invoice email arrives and is labelled “Invoices”, AppSheet’s Gmail integration kicks in.
Automation Triggered: AppSheet triggers an automation that calls a custom Apps Script function called jsonControlledGeneration.
Data Extraction: The jsonControlledGeneration function uses the GeminiApp library to send the email body to Gemini with a predefined JSON schema for invoice data.
Structured Output: Gemini processes the email content and extracts the relevant invoice details (e.g., invoice number, supplier name, date, amount) in a JSON format that adheres to the schema.
Downstream Processing: The structured JSON output can then be easily returned to the AppSheet automation for further actions, such as automatically populating your data table, updating a database, or triggering a payment process.
If you would like to try this yourself, instructions are provided below.
Want to try it yourself?
To set this demo up you will either need a Google AI Studio API key or a Google Cloud project with Vertex AI enabled. Information on both these setups is included in the GeminiApp Setup Instructions. Once you have these you can follow these steps:
Open the Invoice Tracker template and click ‘Copy and Customize’, then click OK on the Error Creating App: Apps Script function is invalid error
Navigate to appsheet.com and from your ‘recent’ apps open the ‘Invoice Tracker’ app
Open Automations and for the ‘New Invoices’ event under the Gmail Event Source, click Authorize, then
In the ‘Add a new data source’ enter a name e.g. Invoices Trigger, click the Gmail button and follow the authentication steps
Once complete in the AppSheet Automation settings select your Gmail account and a Label to watch
In the Process section click on the GeminiApp task and click on the blue file icon, then select your copied version of the Apps Script project and click Authorize
Once Authorize, from the Function Name select jsonControlledGeneration
Step 7: Click on the blue file icon, then select your copied version of the Apps Script project
The Power of Controlled Generation
Controlled Generation is a powerful way for extracting information from unstructured data like emails. By defining a JSON schema, I can specify exactly what information I want Gemini to extract and how it should be structured. This ensures that the output is consistent and reliable, eliminating the need for manual cleanup or post-processing.
Here’s an example of a simple JSON schema for invoice data:
const schema = {
"type": "object",
"properties": {
"Invoice Reference": {
"type": "string",
"description": "Unique identifier for the invoice"
},
"Supplier Name": {
"type": "string",
"description": "Name of the supplier"
},
"Invoice Date": {
"type": "string",
"description": "Date the invoice was issued",
"format": "date"
},
"Due Date": {
"type": "string",
"description": "Date the invoice is due",
"format": "date"
},
"Invoice Amount": {
"type": "number",
"description": "Total amount due on the invoice"
},
"Notes": {
"type": "string",
"description": "Additional notes related to the invoice",
"nullable": true
}
},
"required": ["Invoice Reference"]
}
Creating JSON Schemas with Gemini
Creating JSON schemas can seem a bit daunting at first, but Gemini can actually help you with that too! If you have sample data in a Google Sheet, you can use the Gemini Side Panel to generate a schema automatically. Just highlight the data and ask Gemini to create a JSON schema for you. You can even provide a description for each property to make your schema more understandable. Below is a prompt you can use in the Gemini Sheet Side Panel to generate a schema for your own data:
I'm using Controlled Generation with the Gemini API as described in the document https://drive.google.com/file/d/1ETKHlEUDQzJ-f2fmAzsuDjcwdt1D7R2y/view?usp=drive_link
I need help creating a JSON schema to capture data from a screen.
Could you generate a JSON schema suitable for using Controlled Generation with the Gemini API? I want to extract specific information from what's displayed on my screen.
Here are my requirements:
* **Comprehensive Schema:** The schema should be designed to capture a variety of relevant data points from the screen.
* **Detailed Descriptions:** Please include a clear and concise \`description\` for each \`property\` in the schema. This will help me understand the purpose of each field.
* **Format Specification:** If any columns contain date or datetime data, please use the \`format\` field to specify the appropriate format (e.g., "date", "date-time"). This is crucial for accurate data parsing.
* **Output Example:** Please provide the schema in the following format:
```
const schema = {
description: "Description of the data",
type: "array", // or "object", depending on the structure
items: { // If type is array
type: "object",
properties: {
propertyName: {
type: "string", // or other appropriate type
description: "Description of the property",
format: "date", // or "date-time", if applicable
nullable: false, // or true
},
// ... more properties
},
required: ["propertyName"], // If any properties are required
},
properties: { // If type is object
propertyName: {
type: "string", // or other appropriate type
description: "Description of the property",
format: "date", // or "date-time", if applicable
nullable: false, // or true
},
// ... more properties
},
required: ["propertyName"], // If any properties are required
};
```
Limitations and future developments
While the beta Gmail integration in AppSheet marks a significant new feature, it’s important to note a current limitation is support for processing email attachments. Currently, the integration focuses on metadata such as sender name, subject, and message body, but the AppSheet team have acknowledged attachment support will be added in the near future.
Looking ahead, at Google Cloud Next 2024 the AppSheet team announced an upcoming “Gemini Extract” feature, currently in private preview. This feature intends to include a native Gemini ‘controlled generation’ feature which would let app creators select the data fields they would like populated from sources including images and text. This should be a more intuitive approach to data extraction, directly integrating Gemini capabilities into AppSheet. The Next video includes a Google URL to sign up to the Gemini Extract feature https://goo.gle/appsheet-previews?r=qr
Summary
The Invoice Tracker example hopefully highlights the opportunity for streamlined solutions with data extraction with AppSheet’s Gmail integration, Gemini, and Apps Script.The GeminiApp library also simplifies the integration of Google’s Gemini AI models into Google Workspace, providing developers with tools to create sophisticated AI-powered applications.
Using the structured JSON output with Controlled Generation can help AppSheet creators by making it easier to ensure you get the data back in a suitable format including the type of data you need, such as dates. With the GeminiApp library, rapid prototyping is achievable, in the ‘Invoice Tracker’ example I was able to get a functional prototype up and running in under 30 minutes.
AppSheet’s Gmail integration, generally available to all AppSheet and Google Workspace Core users, can trigger automations directly from incoming emails without requiring app deployment. Combined with Apps Script functions this opens the door to some powerful opportunities for AppSheet creators. Integrating Gemini-powered AI extraction with AppSheet and Apps Script provides an innovative solution for automating data extraction from emails. By taking advantage of these capabilities, citizen developers can create efficient and user-friendly solutions.
Discover how to combine Imagen 3 and Google Sheets for rapid image creation. Automate your workflow and generate visuals in bulk.
Have you ever wished you could create a bunch of images without the hassle of typing in each prompt one by one? Well, Stéphane Giron has shared a clever solution using Google’s Imagen 3, Google Sheets, and Apps Script to streamline this process. While Gemini for Workspace is rolling out to more users and includes Imagen 3 for image creation in apps like Slides and the Gemini side panel, it doesn’t offer an easy way to create images in bulk. That’s where this method comes in. Instead of making images one at a time, this approach allows you to generate multiple images at once, saving you time and effort.
To get started with this method, you’ll need a Google Cloud Project and creating a service account (see the Build an AI slides reviewer with Gemini tutorial for instructions on setting this up). Once that’s set up, you create a Google Sheet with a column for your text prompts, and another for the links to the generated images. The images are automatically saved into a specific folder in your Google Drive.
The real magic happens with Apps Script. It uses Vertex AI to connect with the Imagen 3 model. The script reads each prompt from your sheet, sends it to Imagen 3 to create an image, and then stores the image in Google Drive. It also helpfully adds the image’s link back into the Google Sheet.
Check out the source post for the required code and instructions.
The GeminiApp library, designed to bring Google’s Gemini AI models into your Google Apps Script projects, has just received a major update. This new release, version 2025.01, introduces significant enhancements that expand the library’s capabilities for building sophisticated AI-powered applications within Google Workspace.
Here’s a quick look at what’s new:
Seamless Transition: The updated library allows for a smooth transition from Google AI Studio to Vertex AI.
JSON-Controlled Generation: Generate content in JSON format, either by providing a schema or allowing the model to infer it from your prompt.
Code Execution: Generate and execute code directly within prompts using gemini-2.0 models.
System Instructions: Guide the model’s behavior by providing system instructions during initialization.
Caching: Improve efficiency and reduce token usage by caching file uploads.
Easier Copy/Paste: The library now supports initialization using both new GeminiApp() and new GoogleGenerativeAI(), making it easier to copy code from Google AI Studio.
The update includes over 1.5K new lines of code, primarily in src/GeminiApp.js, src/GoogleAICacheManager.js, and tests/tests.js. Existing projects can be updated by replacing the existing GeminiApp.gs code with the updated library. This update provides a broader set of tools to create more personalized and efficient workflows.
For those new to the GeminiApp library, multiple setup options are available. Check out the updated examples and test cases in the README.md file to get started.
This update marks a step forward for developers integrating Gemini into Google Workspace. With new features and the ability to move between Google AI Studio and Vertex AI, the possibilities are vast.
Feel free to share your use cases, code improvements, and feature requests!
This report proposes a novel learning method using Gemini to automate Q&A generation, addressing the challenges of manual Q&A creation. By integrating with Google tools, this approach aims to enhance learning efficiency, accessibility, and personalization while reducing costs.
The rapid advancement of technology has offered both opportunities and challenges to the education sector. While technology can be a valuable tool for supporting teaching and learning, concerns about its appropriate use have existed for a long time.
The education sector is witnessing an increase in AI tools, each promising to enhance teaching and learning. However, the quality of these tools varies significantly. Some are better designed and can – if used appropriately – can be beneficial, while others are poorly designed. This can make it challenging for educators to find the right solutions that meet their specific needs.
This blog post by Kanshi Tanaike sheds light on the inner workings of some commercial AI tools developed for educators. It demonstrates how these tools can use generative AI to create multiple-choice questions (MCQs) on a given topic. In this particular solution, Google Forms are used in the process, with questions and answers automatically generated by the Gemini API for the user to answer in a Google Form.
For educators who have experience with Google Apps Script, this project looks like a useful starting point to refine and create your own solution or simply used to gain insights into the functioning of similar commercial tools.