Blog

Easy Analytics from Web-Sourced Data

Apr 2015/ Posted By: wotio team

<p>Let&rsquo;s create a Bip that Tracks some Stock Prices and saves it for us out to an Analytics Service. We'll use <a href="https://bip.io/docs/pods/keenio">keen.io</a> for this example, but you could just as easily use <a href="https://bip.io/docs/pods/mongodb">mongodb</a> or other pods and run the data through whatever services you want as well.</p>
<p>This data could be anything that&rsquo;s found on the web. So although this example is polling for a current stock price, <em>any</em> data that can be scraped from the web is fair game. We&rsquo;ll show how that&rsquo;s done in a bit.</p>
<p>First lets define which stock symbol we&rsquo;d like to track. Big Blue is as good as any, so lets pick the (1) Flow Controls and (2) &lsquo;Generate a Payload' with (3) &ldquo;IBM" as the payload.</p>
<p>(1) <img title="" src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-8-41-01-AM.png" alt="" /></p>
<p>(2)<img title="" src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-8-41-52-AM.png" alt="" /></p>
<p>(3) <img src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-1-16-38-PM.png" alt="" /></p>
<p>This tells our bip to start out <em>emitting</em> "IBM" into whatever we want to connect to it. <br /> We'll want to bind that "IBM" payload to a specific HTML DOM element on a specific page, so let's go ahead and use the <a href="https://bip.io/docs/pods/html">HTML pod</a> DOM Selector action (4) to make a web request and use the jQuery Selector to get the data we want. We'll add that action the same way we added our initial payload. (Create New Action -&gt; HTML pod -&gt; DOM selector) <br /> (4) <img src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-3-35-40-PM.png" alt="" /></p>
<p>(5) <img src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-8-45-11-AM.png" alt="" /> Go ahead and <em>add</em> our new <em>DOM Selector</em> action onto the graph (5) and double-click to edit the properties. (6) <br /> (6)<img title="" src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-8-45-57-AM.png" alt="" /> Here we&rsquo;re going to go out to the web to grab the latest Stock Price. We'll use Google Finance for that as you can see in the URL field. We're interested in a specific DOM element, so we're going to enter <code>#price-panel .pr span</code> to select the page element we're interested in (the price!)</p>
<p>Do you see how we used the <em>generated payload</em> of "IBM" in the first step to build out our URL query in this step? That's how we grab a piece of dynamic data from one pod and feed it into another in whichever way we want!</p>
<p>Now that we're grabbing some data (IBM listed price), we need a place to put it. Well, that's exactly what Keen.io was built for, so let's use the <a href="https://bip.io/docs/pods/keenio">keen.io pod</a> to send our data over to our keen.io account!</p>
<p>Again, adding a Keen.io action to our bip is done the same way we added our Payload Generator &amp; DOM Selector actions. (Pods -&gt; Keen.io -&gt; Add an Event)</p>
<p>Once we've added Keen.io, go ahead an add the action to the graph and its simply a matter of <em>connecting the dots</em>! <br /> (7) <img src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-8-45-42-AM.png" alt="" /></p>
<p>By connecting the actions in this way, <em>as if by magic</em> we now have available to us the <em>transformed</em> or <em>collected attributes</em> from the other actions. <br /><img src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-8-46-57-AM-1.png" alt="" /></p>
<p>Keen.io expects to recieve a well formed key:value object, so we'll marshal the Event Object field to send over a JSON object keyed on "IBM" with the value dynamically set to the <code>Text</code> field from our DOM Selector pod action.</p>
<p>With those things set, let's run our bip and then head over to our keen.io dashboard to see that everything is flowing correctly.</p>
<p><img src="http://idfiles.leveelabs.com/55bd0288af0b0930ba599bd0c4b7ca38/resources/img_new/blog_bip_io/Screen-Shot-2015-04-06-at-3-19-01-PM.png" alt="" /></p>
<p>Viewing the latest data event we sent to keen.io (via our bip!) and sure enough we see that our stock price value is getting sent correctly.</p>
<p><code> { "keen": { "timestamp": "2015-04-06T14:00:01.257Z", "created_at": "2015-04-06T14:00:01.257Z", "id": "5522916196773d1d96613471" }, "IBM": "159.08" }</code></p>
<p>Success!!</p>
<p>Now we can take advantage of all of the rich analytics and monitoring that Keen.io provides on that data we've sent.</p>
<p>To recap: <br /> 1. Pods -&gt; Create Event -&gt; Flow -&gt; Generate Payload <br /> 2. My Bips -&gt; Create a Bip -&gt; Add Generate Payload <br /> 3. Add an Event -&gt; HTML -&gt; DOM Selector <br /> 4. Add an Event -&gt; Keen.io -&gt; Add an Event <br /> 5. Click-n-drag to connect Payload -to- DOM-Selector <br /> 5a. Click-n-drag to connect DOM-Selector -to- Keenio-Event <br /> 6. Double Click Each Pod Icon To Configure the Actions. <br /> 7. Hit Run!</p>
<p>Feel free to follow these steps to build up a tracker for your own portfolio.</p>
<p>And of course with bip.io we can connect many <a href="https://bip.io/docs/pods/">different services</a> to store and mix-n-match whatever data we want, coming from whatever website or web-service we want, pretty easily.</p>
<p>So get out there and start bipping!</p>
<p>&nbsp;</p>
<div class="bipio-share" data-share-id="a9026f11-2148-4289-a84f-4f8d01cfb9ed">
<script src="https://bip.io/static/build/js/widget-loader.js" async="" type="text/javascript" data-embed-host="https://api.bip.io"></script>
</div>