Bypassing Google Apps Script Limits: A Zero-Cost Pipeline for Large Datasets

Google Apps Script is easily my favorite runtime, so much so that I wrote a book about it. Yet, being a low-code environment it comes with its quirks, quotas and limitations. In a recent project I needed to process 300 spreadsheets totalling over 5 million cells in over 80,000 rows. I could eithr abandon the platform and spin up a “real” backend, or get creative. Really creative.

Dmitry Kostyuk has published a detailed guide on bypassing Google Apps Script limits to process very large JSON datasets. Rather than completely abandoning Apps Script, Dmitry shares an example zero-cost pipeline to process over 5 million spreadsheet cells without timing out. Here’s the breakdown of his architecture:

Extract and compress the master data snapshot locally using Node.js and fflate (Dmitry highlights this could be hosted in Cloud Run).
Encode to Base64 and slice it into text chunks.
Automate multipart file uploads to Google Drive using the open-source Google Workspace CLI (gws).
Use Apps Script as a thin proxy to fetch and return the text strings to the client browser.
Reassemble, decompress and parse the chunks in the browser using Alpine.js.

A key component of this workflow is the efficiency of the open-source Google Workspace CLI (gws). Rather than writing custom API upload scripts, it uses the gws +upload helper to automatically detect MIME types and execute multipart Drive uploads in a single command.

Crucially, this solution moves heavy computation to the developer machine and the user browser, keeping the Apps Script runtime focused purely on serving data. This relies on HtmlService for client-side processing and you can deploy it in standalone web apps or custom sidebars and dialogue boxes for editors like Google Docs and Sheets.

Dmitry also uses his open-source Apps Script Engine Template, which provides a pre-configured Vite, clasp and environment management setup to speed up development and allow you to standardise your build workflow.

Read Dmitry’s full post for the complete code snippets and a look at the technical trade-offs of this approach. It is a fantastic example of using platform constraints to define a more efficient architecture!

Source: How I Shipped 341 MB of JSON Through Google Apps Script — and Lived to Tell the Tale