Top 10 Inventions at Browserling

Posted: 24 Mar 2014 10:51 AM PDT

Today is 4 years since I and my co-founder James Halliday have been working at Browserling Inc. I thought I'd share what I think are the top 10 inventions at our company.

The first two choices are obvious. They're our products Browserling and Testling. The next eight choices are useful and popular open source projects - browserify, dnode, ploy, seaport, airport and upnode, bouncy, hyperglue and hyperspace and hyperstream, cipherhub.

1. Browserling

Browserling is our first and most successful commercial product. Browserling lets you interactively test websites in all the browsers - IE, FireFox, Opera, Chrome, and Safari.

We successfully managed to monetize the vnc-to-browser technology that we developed four years ago (see the announcement). Our vnc-to-browser technology don't use Flash or Java applets, it uses HTML5 canvas instead.

Try it out at www.browserling.com! (Use Chrome)

2. Testling

Testling is our second product. It lets you run cross-browser JavaScript tests on every git push. After the tests run, you get a badge that shows their status in all the browsers:

Learn more about Testling at www.testling.com!

3. Browserify

Browserify lets you use node.js modules from npm in your browser. It's a game-changer for front-end development.

Browsers don't have the require method defined, but node.js does. With browserify you can write code that uses require in the same way that you would use it in node.

Here's a tutorial on how to use browserify on the command line to bundle up a simple file called main.js along with all of its dependencies:

var unique = require('uniq');    var data = [1, 2, 2, 3, 4, 5, 5, 5, 6];    console.log(unique(data));

Install the uniq module with npm:

  $ npm install uniq

Now recursively bundle up all the required modules starting at main.js into a single file called bundle.js with the browserify command:

  $ browserify main.js -o bundle.js

Browserify parses the AST for require calls to traverse the entire dependency graph of your project.

Drop a single <script> tag into your HTML and you're done!

  <script src="bundle.js"></script>

We use browserify everywhere at Browserling. Browserling and Testling are built out of hundreds of small modules. They're bundled using browserify. If you look at Browserling's or Testling's source code, you'll see that there is just one source file bundle.js. All your Testling tests are browserified before run in the browsers as well.

Learn more about browserify at www.browserify.org.

4. Dnode

Dnode is an asynchronous RPC system for node.js that lets you call remote functions. Dnode is safe. No code gets passed along when you call a remote function. Only function references and their arguments get passed along.

Here's a tutorial on how to use dnode. First, create server.js that listens on port 5555 and exports transform function that converts its arguments to uppercase:

var dnode = require('dnode');    var server = dnode({      transform : function (s, cb) {          cb(s.toUpperCase())      }  });  server.listen(5555);

Then create client.js that connects to the server over dnode and calls the transform function:

var dnode = require('dnode');    var d = dnode.connect(5555);  d.on('remote', function (remote) {      remote.transform('test', function (s) {          console.log('test => ' + s);          d.end();      });  });

When you run this, you get test => TEST as output.

We use dnode heavily at Browserling. All the processes communicate between each other using dnode. For example, we've a centralized authentication service that Browserling and Testling use. When someone signs up at Browserling, they can use the same login at Testling. Dnode makes that work behind the scenes.

Learn more about dnode at dnode's github page.

5. Ploy

Ploy is our node.js deployment system. It includes a http(s) router, a git endpoint and a process manager all in one. You just git push your code at ploy, and it deploys it and runs the necessary processes. You can also manage staging branches and monitor process logs with it.

I recently wrote an in-depth article about how we deploy code at Browserling and Testling. It starts as a tutorial and then explains how we deploy our code in-depth. Read it!

Learn more about ploy at ploy's github page.

6. Seaport

Seaport is a service registry. Seaport stores (host, port) combos (and other metadata) for you so you won't need to spend so much effort keeping configuration files current as your architecture grows to span many processes on many machines. Just register your services with seaport and then query seaport to see where your services are running!

Here's a tutorial on how to setup a seaport server, register a web server at seaport and connect to the web server from another service. First spin up a seaport server:

  $ seaport listen 9000

Then from your web server, connect to seaport and register your web service at seaport by calling ports.register():

var http = require('http');  var seaport = require('seaport');  var ports = seaport.connect('localhost', 9000);    var server = http.createServer(function (req, res) {      res.end('hello world\r\n');  });    server.listen(ports.register('web@1.2.3'));

Now that your web server has been registered at seaport, just ports.get() that web service from another program:

var seaport = require('seaport');  var ports = seaport.connect(9000);  var request = require('request');    ports.get('web@1.2.x', function (ps) {      var url = 'http://' + ps[0].host + ':' + ps[0].port;      request(url).pipe(process.stdout);  });

This program gets the web server's port, connects to it through request, gets response, and prints hello world. Port 9000 is the only port that you need to know in all of your programs.

We also use seaport heavily at Browserling and Testling. Browserling is built using many small services, such as stripe service for taking payments, stats for usage statistics, monitor for monitoring services, status for status.browserling.com, auth that's a centralized point for authentication, and many others. We don't even know what ports these services run on, seaport takes care of it.

Learn more about seaport at seaport's github page.

7. Airport and Upnode

Airport provides seaport-based port management for upnode. What's upnode? Upnode keeps a dnode (see invention #4 above) connection alive and re-establishes state between reconnects with a transactional message queue.

Here's a tutorial on upnode. First create a server.js that exports the time function:

var upnode = require('upnode');    var server = upnode(function (client, conn) {      this.time = function (cb) { cb(new Date().toString()) };  });  server.listen(7000);

Now when you want to make a call to the server, guard your connection in the up() function. If the connection is alive the callback fires immediately. If the connection is down the callback is buffered and fires when the connection is ready again.

var upnode = require('upnode');  var up = upnode.connect(7000);    setInterval(function () {      up(function (remote) {          remote.time(function (t) {              console.log('time = ' + t);          });      });  }, 1000);

This program will connect to upnode server on port 7000, and keep printing the time every second. If you take the upnode server down, it will buffer the callbacks and they'll fire when the server is available again.

Learn more about upnode at upnode's github page.

Airport provides an upnode-style dnode connections using service names from a seaport server (see invention #6 above). Instead of connecting and listening on hosts and ports, you can .connect() and .listen() on service names.

Here's a tutorial on airport. First start a seaport server on port 7000:

  $ seaport listen 7000

Then write a service called fiver that exports the timesFive function that multiplies its argument by five:

var airport = require('airport');  var air = airport('localhost', 7000);    air(function (remote, conn) {      this.timesFive = function (n, cb) { cb(n * 5) }  }).listen('fiver');

Now write a client that connects to the fiver service and calls the timesFive function:

var airport = require('airport');  var air = airport('localhost', 7000);    var up = air.connect('fiver');    up(function (remote) {      remote.timesFive(11, function (n) {          console.log('timesFive(11) : ' + n);      });  });

This program outputs timesFive(11) : 55. In case the connection between the client and fiver goes down, upnode will buffer the callbacks until the connection is back.

Learn more about airport at airport's github page.

8. Bouncy

Bouncy is a minimalistic, yet powerful http(s) router that supports websockets.

Here's a bouncy tutorial. Let's say you want to route route requests based on the host HTTP header to servers on ports 8001 and 8002. For every http request, bouncy calls function (req, res, bounce) { }, so you can inspect the req.headers.host and then call bounce(8001) or bounce(8002), like this:

var bouncy = require('bouncy');    var server = bouncy(function (req, res, bounce) {      if (req.headers.host === 'status.browserling.com') {          bounce(8001);      }      else if (req.headers.host === 'browserling.com') {          bounce(8002);      }      else {          res.statusCode = 404;          res.end('no such host');      }  });  server.listen(80);

The bounce(PORT) function redirects the connection to the service running on PORT.

I personally don't see myself using anything but bouncy for routing http(s) requests, including websockets. It's rock solid and has been used in production for many years.

Learn more about bouncy at bouncy's github page.

9. Hyperglue, Hyperspace, and Hyperstream

Hyperglue lets you update HTML elements by mapping query selectors to attributes, text, and hypertext both in the browser and node.js.

Here's a tutorial on hyperglue. Let's say you've an article template in article.html and you want to fill a .name, span .author, and div .body with data:

<div class="article">    <div class="title">      <a name="name"></a>    </div>    <div class="info">      <span class="key">Author:</span>      <span class="value author"></span>    </div>    <div class="body">    </div>  </div>

With hyperglue you can write the following code that will do just that:

function createArticle (doc) {      return hyperglue(html, {          '.title a': {              href: doc.href,              _text: doc.title          },          'span .author': doc.author,          'span .body': { _html: doc.body }      });  }    document.body.appendChild(createArticle({      author: 'James Halliday',      href: '/robots',      title: 'robots are pretty great',      body: '<h1>robots!</h1>\n\n' +            '<p>Pretty great basically.</p>'  }));

The createArticle function will fill the article HTML template with data at the right selectors. As a result it will produce the following HTML, and it will append it to document.body:

<div class="article">    <div class="title">      <a name="name" href="/robots">robots are pretty great</a>    </div>    <div class="author">      <span class="key">Author:</span>      <span class="value author">James Halliday</span>    </div>    <div class="body">  <h1>robots!</h1>    <p>Pretty great basically.</p>    </div>  </div>

The object returned by hyperglue also has an innerHTML property that contains the generated HTML, so you can also use it on the server side to get the resulting HTML:

console.log(createArticle({      ...  }).innerHTML);

Learn more about hyperglue at hyperglue's github page.

Hyperspace renders streams of HTML on the client and the server. Here's a tutorial on hyperspace. First pick a stream data source that will give you records and let you subscribe to a changes feed. Let's start with the rendering logic in file render.js that will be used on both the client and the server:

var hyperspace = require('hyperspace');  var fs = require('fs');  var html = fs.readFileSync(__dirname + '/row.html');    module.exports = function () {      return hyperspace(html, function (row) {          return {              '.who': row.who,              '.message': row.message          };      });  };

The return value of hyperspace() is a stream that takes lines of JSON as input and returns HTML strings as its output. Text, the universal interface!

Template row.html used is just a really simple stub thing:

<div class="row">    <div class="who"></div>    <div class="message"></div>  </div>

It's easy to pipe some data to the renderer:

var r = require('./render')();  r.pipe(process.stdout);  r.write(JSON.stringify({ who: 'substack', message: 'beep boop' }) + '\n');  r.write(JSON.stringify({ who: 'pkrumins', message: 'h4x' }) + '\n');

Which prints:

<div class="row">    <div class="who">substack</div>    <div class="message">beep boop</div>  </div>  <div class="row">    <div class="who">pkrumins</div>    <div class="message">h4x</div>  </div>

To make the rendering code work in browsers, you can just require() the shared render.js file and hook that into a stream. In this example we'll use shoe to open a simple streaming websocket connection with fallbacks:

var shoe = require('shoe');  var render = require('./render');    shoe('/sock').pipe(render().appendTo('#rows'));

If you need to do something with each rendered row you can just listen for 'element' events from the render() object to get each element from the data set, including the elements that were rendered server-side.

Learn more about hyperspace at hyperspace's github page and the html streams for the browser and the server section of Stream Handbook.

Hyperstream streams HTML into HTML at css selector keys. Here's a tutorial on hyperstream. Let's say you've the following HTML template in index.html:

<html>    <body>      <div id="a"></div>      <div id="b"></div>    </body>  </html>

And you've two more files, a.html that contains:

<h1>title</h1>

And b.html that contains:

<b>hello world</b>

And you want to stream HTML from files a.html and b.html into selectors #a and #b. You can write the following code using hyperstream:

var hyperstream = require('hyperstream');  var fs = require('fs');    var hs = hyperstream({      '#a': fs.createReadStream(__dirname + '/a.html'),      '#b': fs.createReadStream(__dirname + '/b.html')  });  var rs = fs.createReadStream(__dirname + '/index.html');  rs.pipe(hs).pipe(process.stdout);

And it will do just that! You'll get the following output:

<html>    <body>      <div id="a"><h1>title</h1></div>      <div id="b"><b>hello world</b></div>    </body>  </html>

Learn more about hyperstream at hyperstream's github page.

10. Cipherhub

Cipherhub is our secure communications tool. It can be frustrating and annoying to communicate with somebody using public key cryptography since setting up PGP/GPG is a hassle, particularly managing key-rings and webs of trust.

Luckily, you can fetch the public ssh keys of anybody on GitHub by going to:

  https://github.com/USERNAME.keys

If you just want to send somebody an encrypted message out of the blue and they already have a GitHub account with RSA keys uploaded to it, you can just do:

  $ cipherhub USERNAME < secret_message.txt

And it will fetch their public keys from GitHub, storing the key locally for next time, and output the encrypted message. You can now send this message to your friend over IRC and they can decode it by running:

  $ cipherhub <<< MESSAGE

Just recently we used cipherhub to send company's credit card information over IRC and I loved how easy that was.

Learn more about cipherhub at cipherhub's github page.

4 Years of Browserling

Happy birthday to Browserling Inc!

You can follow Browserling and Testling on Twitter at @browserling and @testling, and co-founders at @pkrumins and @substack.

We can also be contacted over email at hello@browserling.com.

Hooray!

An Ignoble Experiment

Pages

Sponsor

2014/03/25