Manage long-running external webservice requests from Rails apps (on cloud servers), howto

Case: (as long as Rails is synchronous) requests to external webservices take the use of server resources to impossible levels, even when webservices behave normally—let alone when they are long delayed.

Plan: two web apps (one Rails, the other async Sinatra) can fairly easily manage the problem of external web service requests by minimizing use of server resources—without abandoning normal, threaded, synchronous Rails. The async Sinatra web app can be a separate business, even a moneymaking one.

This solution uses RabbitMQ, Memcache and PusherApp.

The async Sinatra web dynos (on the one hand) comprise external webservice request brokers. Also they have browser-facing functionality for signing up webmasters.

The Rails web dynos don't wait (on the other hand) for external webservices and they aren't short-polled by browsers.

This attempts to be efficient and robust. It should speed up heavily loaded servers while remaining within the mainstream of the Rails Way as much as possible.

E.g. it tries hard not to Pusherize browsers more than once for the case that a cached response to an external webservice was missed, but relies on browser short-polling after perhaps a 10-second timeout to cover these and other unusual cases.

But in the normal case browser short-polling will be avoided so Rails server response time should be peppy.

It tries to delete its temporary work from memcache but even if something is missed, memcache times out its data eventually so too much garbage won't pile up there.

Note: this is for web services without terribly large responses (thus appropriate for memcaching). Very large responses and non-idempotent services should be handled another way such as supplying them directly to the browser.

Method: the Rails web app dynos immediately use memcached external webservice responses if the URL's match.

Otherwise they push the URL of each external webservice request and an associated PusherApp channel ID (for eventually informing the browser) to a RabbitMQ Exchange.

For security purposes, minimal information is passed through PusherApp to the browser (only suggesting a short-poll now, not where).

The Rails web dynos (if necessary) return an incomplete page to the browser as usual (for completion with AJAX).

To cover cases where something got dropped the browser should short-poll the Rails app after a longish timeout—its length should be set by an environment variable and may be shortened to half a second when the Rails website is not terribly active, or when the async Sinatra web dynos are scaled down to off.

Each async Sinatra web dyno attaches a queue to the Rails app's RabbitMQ exchange for accepting messages without confirmation.

With each queued message, an async Sinatra web dyno:

1. Checks the memcache for the external webservice request (with response)—if present, it:
• Drops the message. (Some may slip through and be multiply-processed, but that's okay.)
• Frees memcache of the request (without response) if it still exists (see below).
Otherwise it checks the memcache for the external webservice request—without response. If recently memcached (perhaps within 10 seconds) it drops the message. (Some may slip through and be multiply-processed, but that's okay.)
Otherwise it makes the request to the external webservice, setting a generous response timeout (maybe 60 seconds).

2. Memcaches the external webservice request (without response) with the current time (not in the key).
3. If the request times out, drops it in favor of letting the browser handle the problem, but leaves the memcached external webservice request (without response) for later viewing by async Sinatra web dynos.
4. (Usually) receives a response from the external webservice request.
5. Again checks memcache for the external webservice request (combined with the same response). If it's not there:
• Pusherizes the appropriate browser. (Some requests may be multiply-processed, but that's okay.)
• Memcaches the external webservice request (with response).
• Clears from memcache the external webservice request without response.

The browser then asks the Rails web dyno to supply all available AJAX updates.

The Rails web dyno returns (usually incomplete: whatever is memcached—some may have been dropped, but that's okay) a set of still-needed AJAX responses to the browser (for further completion with AJAX).

Or (if all were memcached) the Rails web dynos return the complete set of outstanding AJAX responses to the browser.

I'm starting to implement this here, now.
Copyright (c) 2012 Mark D. Blackwell.