Tuesday, May 7, 2013

Automatically Process Cloud Storage Uploads with Change Notifications

Do your customers upload files to Google Cloud Storage for your applications to process? For example, a photo app may want to create thumbnails of new images as soon as they are uploaded. Normally you would have to poll for updated objects which can be a resource waste or cause you to react slowly. Most times writing and deploying custom scripts to trigger your application is cumbersome.

Today, we're releasing object change notification as a preview feature, allowing you to watch your Google Cloud Storage buckets for new, modified, or deleted objects with a webhook you provide. Now your application can be automatically triggered when an important change happens and start processing data immediately. We've also updated gsutil with a notifyconfig command. A Google App Engine webhook can be as simple as the following:

We're also releasing an update to the Google Cloud Storage JSON API, bringing it into parity with our existing XML API, including exposing new methods such as Copy and Compose. As a part of this release, we are making the API available to everyone without requiring an invitation.
Enjoy, and as always, we watch StackOverflow.

- Posted by Dave Barth, Product Manager


Larry White said...

This is nice, but for my use, the maximum file open time limits in Cloud Storage have made me go back to using EC2 wrapped S3 buckets from AppEngine

Google won't have a competitive product until people feel they're not going to bump into an arbitrary limit and have to restart development on another platform.

Dave Barth said...

Larry, what file open time limits are you referring to? If AppEngine Files API then there are some constraints there we definitely will be fixing. If something else then I'd like to understand. thanks.