Python protobuf on Google App Engine

June 18 2013

Today, if you try to use Google Protocol Buffers on App Engine, you’ll run into an error like this one:

from google.protobuf import descriptor 
ImportError: No module named protobuf

The problem is that Google’s own App Engine apis also use the google package namespace, and they don’t include the protobuf package.

Thankfully, there’s a simple way to fix this. First, vendorize the library as you normally would. I just ripped the site-packages folder from a virtualenv into the application root:

app.py
app.yaml
vendor
├── google
│   └── protobuf...
└── httplib2...

Then, just add your google directory to the google namespace, and add your vendor directory to the system path. I used this bit of code:

import os
import sys

import google  # provided by GAE

# add vendorized protobuf to google namespace package
vendor_dir = os.path.join(os.path.dirname(__file__), 'vendor')
google.__path__.append(os.path.join(vendor_dir, 'google'))

# add vendor path
sys.path.insert(0, vendor_dir)

That’s it! There’s no need for weird hacks.