Blog

Fun With HTTP Pods

June 8, 2014 / Posted By: wotio team

A little while ago I rolled out the HTTP Pod for BipIO which provides a basic webhook channel (called ‘request’) that can drop onto any hub to do some work. Think fetching a file, or posting form data or making some ad-hoc API call as part of a pipeline. It’s the kind of irresponsibly powerful tool you might want to leverage for your app to fan out web requests or temporarily make available a hidden or authenticated resource to (un)trusted 3rd parties.

A more interesting feature of Bips (the graphs that do work) and in particular the HTTP flavor is that they support the concept of rendering dynamic content from any available channel. Bips themselves have the characteristics of ephemerality, authentication, name, transport encapuslation and graph processing. They’re public facing endpoints that anyone can call, with a graph behind them that potentially can process requests. Without a renderer, HTTP Bips will just sit there accepting data and all going well respond with a happy ‘200 OK’ message. With renderers, we add the ability to respond in a custom way beyond the generic ‘200 OK’, per endpoint. This makes them a very powerful tool for backing parts of a web application or adding ‘app-like’ characteristics to simple URL’s in something as simple as email.

That said, I want to demonstrate a few of the funky things I was able to get going in no time with HTTP Bips and some of the new renderers the HTTP Channel provides. Using both HTTP Bips and Channels is a good starting point because they share the same protocol, but keep in mind that any Pod/Channel which has a renderer can also serve content or have its protocol encapsulated via HTTP Bips. The demo’s were active workflows during the beta pre-launch period for user onboarding and segmentation which turned out really useful. I hope you find so, too!

Simple Proxying

When the launch key emails were being sent out, I wanted to get a rough guage of how many people were opening the message for their invite code, grouped per day. There’s a range of different techniques for surfacing this kind of simple metric but I went for simply tracking when an image was loaded - specifically the bipio logo embedded in the email’s HTML.

The endpoint is https://beta.bip.io/bip/http/logo.png and it looks and renders like just any other image. Because HTTP Bips don’t know anything about serving images by themselves it was the perfect time to try out the ‘proxy’ renderer supported by the new ‘Request’ Channel in the HTTP Pod. The logic simply being, ‘When the image is served, record a count grouped by day’.

To get started, the ‘Channels’ section of the  bipio dashboard is where the channel itself will get created. Clicking on the HTTP Request icon will start the channel creation process. Channels will generally just sit there doing nothing until added to a Bip’s graph, or enabled as a renderer. Lets have this ‘http.request’ channel GET the logo image file :

To create this with the API, just POST a channel structure like so :

POST /rest/channel
{
  "name": "BipIO logo",
  "action": "http.request",
  "config": {
    "url": "https://bip.io/static/img/logo.png"
  }
}

Building the web hook which will serve this channel’s content publicly is then pretty simple. Under ‘Bips’, click ‘Create Web Hook’ and make sure :

- It has a name that looks like an image file,

- Has authentication disabled

- A renderer set as a the ‘HTTP Proxy’ renderer of the logo image channel which was created earlier.  You can find renderer setup in its own tab during Bip configuration :

So just enable the ‘HTTP Proxy’ renderer for your new channel.

Via the API, it’s something like :

POST /rest/bip
{
    "name": "logo.png",
    "type": "http",
    "config": {
      "auth": "none",
      "renderer": { // renderer object
        "channel_id": "01ded262-4150-4041-bcea-6727bd46960e",
        "renderer": "proxy"
      }
    },
    "hub": {
      "source": {
        "edges": []
      }
    }
}

And that’s it! The named endpoint which is created

(https://beta.bip.io/bip/http/logo.png) will proxy any requests it receives via the renderer and serve up the file like magic. To handle hitcounts on the endpoint and build a basic report, its just a matter of filling out the hub with two extra channels for time formatting (time.format) and count (flow.counter). I’ve split it out here so you can wrap your head around the structure but it can replace the empty ‘hub’ in the previous POST, also :

"hub": {
    "76ba8c52-7161-4e80-aa06-146b45da75b9": {
        "transforms": {
            "c9a7c8ad-bd43-447a-9eea-fac54a9e1ebc": {
                "_note" : flow.counter channel",
                "increment_by": "1",
                "group_by": ""
            }
        },
        "edges": [
            "c9a7c8ad-bd43-447a-9eea-fac54a9e1ebc"
        ]
    },
    "source": {
        "transforms": {
            "76ba8c52-7161-4e80-aa06-146b45da75b9": {
                "_note" : "time.format channel",
                "format": "MMDDYYYY",
                "time": "1402247179"
            }
        },
        "edges": [
            "76ba8c52-7161-4e80-aa06-146b45da75b9"
        ]
    }
}

The ‘flow.counter’ channel has a renderer itself which dumps out all the data it has collected. I could either call that renderer directly to get at the data, or encapsulate it in a bip like the previous example to run a report etc. Pretty neat!

Request Redirection

To redirect a user directly to a target resource rather than proxy it, its just one small change to the renderer structure above - just set ‘renderer’ from ‘proxy’ to ‘redirect’ in the config section.

PATCH /rest/bip/{bip-id}

{
    "config": {
        "auth": "none",
        "renderer": {
            "channel_id": "01ded262-4150-4041-bcea-6727bd46960e",
            "renderer": "redirect"
        }
    }
}

A couple of cases for using the HTTP Pods redirect renderer might be as a link shortener, or to segment users in MailChimp by who clicks through via the generated link!

SSL Encapsulation

A common problem for people running content sites or apps with SSL is that any non-encrypted content being rendered into a browser raises a bunch of security warnings as it undermines the integrity of the content. On the flipside there is significant infrastructure overhead in downloading every piece of content onto your server or cdn for serving over SSL, simply to make browsers happy. By using the ‘http.request’ proxy renderer its possible to encapsulate insecure content in SSL instead.

Here’s a simple SSL bridge which you can use via the BipIO website (which forces SSL connections). Creating a bridge is very similar to the ‘http.request’ channel defined earlier which served a logo, however a ‘url’ config parameter is not defined - its injected by the Bip, instead :

POST /rest/channel

{
    "name": "SSL Bridge",
    "action": "http.request",
    "config": {
    }
}

POST /rest/bip

{ "name": "anonymousbridge", "type": "http", "config": { "auth": "token", "renderer": { "channelid": "91ded262-4150-4041-acea-6727bd46960e", "renderer": "proxy" } }, "hub": { "source": { "edges": [ ] } } }

To test it out just call the endpoint like so, authenticating with your username/API key :

https://{username}.bip.io/bip/http/anonymous_bridge?url=http://example.org

Voila!

A quick note on security …

Given the ability to proxy, redirect and encapsulate web requests with abandon, its generally a bad idea to accept and process any URL a web client throws at you. Be sure to always authenticate clients using the authentication config attributes in HTTP bips! Additionally, if you’re running the server on your own infrastructure, you may notice ‘blacklist’ errors returning to your connecting clients. This is because by default all local network interfaces are blacklisted by the HTTP Pod out of the box. To whitelist local network interfaces, add their IP or Hostname to the HTTP Pods ‘whitelist’ section in your server config.