What Do You Care What Other People Think?

In the cloud, scale means concurrency

In enterprise computing, scale has traditionally meant "lots of transactions per second."  On Wall Street for many years, "20,000 TPS" was the magic number as it was the rate of a typical market data feed.  Infrastructure like TIBCO's UDP-based information bus and then IBM's MQSeries became the base platforms for much of this scale of computing, and are still heavily used alongside modern JMS and MSMQ implementations.
 
Relatively little attention was paid to concurrent connections.  Enterprise environments tend to be well-regulated, and most applications will have under 1000 simultaneous users (whether human or machine driven).  As a result, application servers and related technologies evolved to support high transaction throughput at limited concurrency.
 
The web on the other hand brought in much higher concurrency requirements, and platforms like WebLogic became default components of web computing environments for sites serving 1,000s people at the same time.  This was a breakthrough and led to significant market success in a short time period.
 
With the rise of cloud computing, two things change.  First, mobile applications and the API economy are driving an order of magnitude increase in the number of simultaneous users.  Second, these users are often machines rather than people, and therefore aren't limited to the demand patterns of humans users clicking links or refreshing their pages.
 
This produces a new set of demand patterns which increase both total throughput and peak concurrency.  As an example, travel sites like Kayak.com and Bing.com/travel issue hundreds of API requests to airline reservation system backends as a result of a single human-driven query.  Furthermore, these requests are being made not just by desktop or web applications but by mobile applications – especially iPhone applications.  As most people are aware, the next 10 billion devices that come online will not be PCs or netbooks but will be mobile devices (phones, MIDs, GPS, game units, media players).  Each of these is prized for its native application experiences.  Each of these devices will be making user-driven and automated calls to cloud services in order to deliver those experiences.
 
Where backend systems are not protected from this demand, they are being penalized in performance and load management.  This causes either outright outages, "web brownouts" where the core website that uses the same backend slows down, or erratic performance across both the web and cloud properties.  Again, mobile access exacerbates the issue due to the intermittent nature of mobile internet connectivity, which multiplies the number of connections that need to be set up and torn down as the device comes on and off the network.
 
So the explosion of concurrent usage is already beginning, as the traffic and backend impact is expanding.  To manage this and maintain stability of existing infrastructure, a new layer of infrastructure is emerging, much as HTTP load balancers have evolved to serve the needs of web computing.  What we're seeing is the rise of cloud service controllers, a category of infrastructure that works well with existing systems and builds on top of the strengths of application servers, enterprise messaging systems, and application delivery controllers.

Business Logic vs. Business Policy in Cloud Services and APIs

As companies move from websites to cloud APIs - we're seeing them separating business logic from business policy.

When you think about business logic, you probably imagine an application server or stored procedures, or custom code that accesses stored data and exposes it in a useful and manageable way – such as “show me all of a given customer’s purchases in this timeframe”.  

These pieces of code reflect the understanding of the fundamentals of the business - like customers, orders, tickets, requests – and how they interact.  In any business you build skill over time in understanding your domain and then build on top of that understanding to take the business further.  Business logic is generally stable, consistent across requests, and subject to careful modification.  After all, you wouldn’t want to put partly tested logic about bank balance transfers into production.

When you think about business policy, what comes to mind? 

Maybe a stack of papers documenting HR or accounting guidelines, or maybe the elements of Sarbanes-Oxley requirements.  Policy is not about how you compute a specific result – like the customer’s purchase history – but about other factors like who can access it, how frequently, from what locations, how it’s rendered, how long the result is considered valid, and other “meta-logic” issues.  In any business you make changes to your policies frequently, as a way to meet the changing face of the market environment.  The nature of business policy is therefore to be dynamic, varied according to the request context, and subject to frequent experimentation.  For example, product managers need to continually test the value of different offers to given segments without having to redevelop the product.

When policies like security (“who can access this and what credentials do they have to show?”) and service level (“how many requests can be made in a given time interval?”) are collapsed into code responsible for business logic, the result is loss of agility, high cost to adopt new policy implementations (e.g. for identity and access schemes or to keep pace with increased hardware capacity), and confusion of concerns (rather than separation of concerns).

In the current era of cloud APIs, where these interfaces are being built as a direct link to what started as the backend for the website, many developers are realizing that what was really policy was built into their business logic layer.  They’re also seeing that adding new clients – going from enabling partners to enabling rich clients to enabling mobile device (such as iPhone or Android) applications – require many adjustments to how the request and response of an API are rendered, but should require no changes to the stable core of business logic.  

Supporting an API with a set of policies that activate based on the user (such as preventing or enabling access to different parts of the API – for example, only the development & testing methods but not production methods, or query methods only, or limiting requests to a certain transaction volume or financial value) and type of client (such as providing pagination and format translation for a mobile device) means that the risk of delivering the API is reduced, and the risk of negative impact to existing users of the API (typically the original website) are reduced.   Also,  the computational load for the policy processing can be moved to a low-cost, scale-out tier and away from the high-value application servers that host the business logic.

So what this seems to indicate is a movement towards a separation of concerns between logic and policy.  Responsibility for business logic processing lies in a stable and slowly changing layer, and policy is processed in a tier that allows for agile modification and enables the total computational result to meet the shifting needs of the business.

Why modern applications need an API proxy

Structures of control are spontaneously generated in every environment and every wave of computing.

Today on the web we have a model where browsers are the single point of control for much of what happens, not just at the level of applications, but at the meta-application level as well.  Not simply usage (“point-click-type”), but things about usage – who is the user (browser cookie), what are they using the app through (user agent), where did they come from (referrer), what can we infer about their behavioral state, and so on – as well as modifications of usage (browser add-ins, content filters, security modes, local caching for performance).  To be sure, some of these things can be and are performed using infrastructure between the browser and the website (such as content filtering, security, and caching), but the guaranteed component is the browser.

This is one of the reasons that Google Analytics is so popular and useful – you can rely on it to tell you useful things about your traffic because it can rely on the browser as a predictable point of control.  Including an invisible piece of content on your web page makes the browser fetch data from Google, implicitly sending information that enables Google to report on your usage.

For web and cloud APIs, what is the equivalent structure of control?

Currently there is no one point like the browser.  This is for great reasons – APIs are all about reusing application or service logic and rendering it to different form factors: pure logic (built into an internal application computation), web UIs (part of a mashup), and most notably, client applications on a wide range of devices (from PCs to mobile phones, set-top boxes, and tablets like the iPad).  These devices are in the early part of a boom that will see over 10 billion individual units in use, representing at least hundreds of unique hardware/software designs.  The sheer utility of these internet-connected devices predicts that their usage will drive high demand for APIs rather than standard websites.  There are initial specifications like BONDI that suggest a standard contract across all of these for “mobile web applications” that include interaction with the features of the local device (such as a camera or GPS) but they are years from broad adoption and don’t attempt to unify all API access down to a common control point.

Given that APIs are to application logic what RSS is for content, we know they will be very important; at least as important as the visible web that we use today and possibly more important.  This suggests that the other things that are spontaneously generated in value-exchange environments like user/customer management, behavior analysis, content filtering, caching, and security – will show up for APIs as well.

The web API equivalent of the browser’s control structure is an API proxy.

This is a control point which unlike a web proxy is fully aware of API content, communications patterns, and able to drive the meta-application controls discussed above.  An architecture like Google Analytics which is founded on a browser’s predictable algorithms cannot work in an API setting.  The same rule applies to add-ons that modify usage – they can’t do so relying on the local device if they are to be widely adopted.  But an API proxy – a server or service on the internet, sitting between the client (regardless of type) – is able to be that point of control.  As traffic runs through it, meaningful data can be captured for immediate outcomes (block access, change the message, or respond from a cache) and later used for behavior analysis and business planning.  Add-ons that modify usage of the API can be installed at this point (content filtering, adding new information such as advertising, or identity management).  All of this can be done while adhering to the contracts of the APIs and supporting the web architecture and rules of HTTP-based applications, and without attempting to solve the logarithmically complex problem of modifications to all the world’s clients.

So API proxies are likely to be necessary for the sustained growth of web and cloud API usage.  There are likely to be several nuances that end up differentiating the different implementations and providers of API proxies.  The key is to start experimenting with them now in order to build better apps and stay ahead of the competition.