Category: Code

CSS Naming Conventions that Will Save You Hours of Debugging

Ohans Emmanuel:

I have heard lots of developers say they hate CSS. In my experience, this comes as a result of not taking the time to learn CSS.

CSS isn’t the prettiest ‘language,’ but it has successfully powered the styling of the web for over 20 years now. Not doing too badly, huh?

However, as you write more CSS, you quickly see one big downside.

It is darn difficult to maintain CSS.

Poorly written CSS will quickly turn into a nightmare.

Ohans makes some interesting points about about CSS conventions such as the use of BEM naming convention (Body, Elements, Modifiers).

Definitely worth a share and a read and a few claps.

Source: https://medium.freecodecamp.org/css-naming-conventions-that-will-save-you-hours-of-debugging-35cea737d849

Scan your projects for crossenv and other malicious npm packages

Dominik Kundel, writing for the Twilio Blog:

On August 1st, Oscar Bolmsten tweeted about how he found a malicious npm package called crossenv that scans for environment variables and POSTs them to a server.

This is particularly dangerous considering that you might have secret credentials for different services stored in your environment variables.

Apparently it’s also not limited to just crossenv, but a series of packages — all of them are names of popular modules with small typos such as missing hyphens.

Check your project for malicious packages

These packages have been taken down by npm, but since credential theft happens upon installation, you should check if you have installed one of them. Ivan Akulov was so kind to compose and publish a list of (at least some of) these malicious packages on his blog. He also wrote a small one-liner that you can execute to check if these packages are installed in your current project:

npm ls | grep -E "babelcli|crossenv|cross-env.js|d3.js|fabric-js|ffmepg|gruntcli|http-proxy.js|jquery.js|mariadb|mongose|mssql.js|mssql-node|mysqljs|nodecaffe|nodefabric|node-fabric|nodeffmpeg|nodemailer-js|nodemailer.js|nodemssql|node-opencv|node-opensl|node-openssl|noderequest|nodesass|nodesqlite|node-sqlite|node-tkinter|opencv.js|openssl.js|proxy.js|shadowsock|smb|sqlite.js|sqliter|sqlserver|tkinter"

Search for infected projects on Mac/Linux

If you are like me a person who regularly develops Node.js applications you might have a series of projects and not just one project to check for. I extended Ivan’s command for that reason using find and xargs to actually scan all subdirectories of the folder that contains my projects and execute Ivan’s command there. You can run it by simply copy pasting this command into your command-line:

find . -type d -maxdepth 4 -name node_modules -print0 | xargs -0 -L1 sh -c 'cd "$0/.." && pwd && npm ls 2>/dev/null | grep -E "babelcli|crossenv|cross-env.js|d3.js|fabric-js|ffmepg|gruntcli|http-proxy.js|jquery.js|mariadb|mongose|mssql.js|mssql-node|mysqljs|nodecaffe|nodefabric|node-fabric|nodeffmpeg|nodemailer-js|nodemailer.js|nodemssql|node-opencv|node-opensl|node-openssl|noderequest|nodesass|nodesqlite|node-sqlite|node-tkinter|opencv.js|openssl.js|proxy.js|shadowsock|smb|sqlite.js|sqliter|sqlserver|tkinter"'

Search for infected projects on Windows

That command works when you are on Mac or Linux.. Corey Weathers wrote a small PowerShell script for that will do the same thing on Windows:

Get-ChildItem $directory -Directory -Recurse -Include "node_modules" | foreach { cd $_.FullName; cd ..; npm ls | Select-String -Pattern "babelcli|crossenv|cross-env\.js|d3\.js|fabric-js|ffmepg|gruntcli|http-proxy\.js|jquery\.js|mariadb|mongose|mssql\.js|mssql-node|mysqljs|nodecaffe|nodefabric|node-fabric|nodeffmpeg|nodemailer-js|nodemailer\.js|nodemssql|node-opencv|node-opensl|node-openssl|noderequest|nodesass|nodesqlite|node-sqlite|node-tkinter|opencv\.js|openssl\.js|proxy\.js|shadowsock|smb|sqlite\.js|sqliter|sqlserver|tkinter"} -ErrorAction Ignore

What if a malicious package was detected?

You should immediately rotate all secrets that you have stored in the environment variables. If it’s a project that is shared with other folks don’t forget to alert them to do the same. Don’t forget that Continuous Integration systems and cloud hosts like to use environment variables as well. So if you shipped one of these projects into production or used a system that uses environment variables don’t forget to rotate them there as well.

Read the rest of the original post for more information, I wanted to share this here so you can check your code for any packages that shouldn’t be there.

Source: https://www.twilio.com/blog/2017/08/find-projects-infected-by-malicious-npm-packages.html

Introducing outgoing webhooks

Today we’re announcing outgoing webhooks – a long awaited feature that will notify your websites and apps whenever data changes occur.

This opens up a new range of possible integrations

Adding outgoing webhooks means you can get notified directly in slack or elsewhere when new records are added, when records are updated or deleted.

To create an outgoing webhook, just click webhooks in the dropdown:

Then edit the form:

Hit Save and when you save a new record with our API, we will send notifications to your webhook instantly.

We’ve got some options for how we send the data, you can have it sent using a standard form post, send as a json string using payload as the containing variable, or just send as a JSON request.

If you are sending to Slack, we automatically detect the slack url and format it accordingly so it shows up nice looking.

All outgoing webhook notifications also include two extra fields, one for event to say what happened (new record, updated record, deleted record) and a url to the record in your dashboard.

You can delete the webhook at anytime from your webhooks management page.

Build a “Serverless” Live Blog System with Flybase, Twilio and StdLib

Flybase and Twilio work great together, we’ve posted several posts before about using these two services in one place, but what about if you want to go serverless?

StdLib.com is excellent for that. In their own words, StdLib is essentially a package manager for APIs that’s built upon new “serverless” architecture, meaning you never have to worry about managing servers or allocating resources for scale.

All you have to do is write a function or two (or three), deploy, and you’re done, and this is perfect for quickly building services with Flybase.

We’ve covered building a Live Blogging tool before, so I wanted to use that tool here and show how to make it work inside StdLib.

Setup

To start, you need an account with our three services:

  • Twilio to handle receiving text messages.
  • Flybase to store our data and output it to viewers in real-time.
  • Stdlib to handle the backend processing between Twilio and Flybase.

First install StdLib:

npm install lib.cli -g

Now, create a folder called stdlib and login:

$ mkdir stdlib
$ cd stdlib
$ lib init

Ok, time to create the liveblog service:

$ lib create liveblog
$ cd <username>/liveblog
$ lib function:create sms -n
$ mkdir pages
$ npm install --save async ejs flybase twilio

Where <username> is the username you are logged into StdLib with.

You will now have the following structure:

  • functions/__main__.js
  • functions/sms/__main__.js
  • pages/

Let’s give this structure some code and make it work for us.

Open the file called env.json in the root of the project, it was created automatically for you, and copy the following:

Populate these variables with your Twilio and Flybase information.

Now, open functions/__main__.js and copy the following code:

This will set up our index, which will output your live blog posts.

Next, open functions/sms/__main__.js and copy the following:

This is our webhook to receive text messages. It will take the text message and any photos from Twilio and store it in your Flybase app, it will then send a reply to your phone to tell you the message was received.

Finally, we need to create our pages/index.ejs file:

This template file will be read by our index and used to display any useful information.

Get online

To put this app online, all we have to do is type:

lib up dev

This will upload your app and make it available at the following url:

https://<username>.lib.id/<service>@dev/
https://<username>.lib.id/<service>@dev/sms/

Where <username> is your StdLib username and <service> is the service you’ve created, in this case that would be liveblog.

Once you’re done developing, you can also use

lib up -r

To publish your app in release (prod) mode. The URLs would then be:

https://<username>.lib.id/<service>/
https://<username>.lib.id/<service>/sms/

Let’s stick to just dev mode for now.

Once this has been published, you can go to the https://<username>.lib.id/<service>@dev/ URL and see your live blog, there won’t be any content right now, so let’s create some using Twilio.

Go to your Twilio account and create a phone number, then in the settings for that number, add your /sms/ URL:

https://<username>.lib.id/<service>@dev/sms/

Remember to update <username> with your username and <service> with the name of the service.

And now you are ready to send SMS messages to your live blog and see them on the screen.

Finishing Up

You can see the full code here at the repo, download it, extend it, play with it, do whatever you wish.

This is a pretty basic example of using StdLib, Flybase and Twilio together, but it shows some of what you can do.

You can also just drop the index file into any webpage and have it output your live blog posts, since Flybase works great that way, so it’s really up to you on how you want to use it.

How to fix errors in production with GitHub and Sentry

Sometimes bugs slip through even the most diligent of code reviews. Putting out fires can disrupt your flow, forcing you to comb through logs and user-submitted screenshots. In the meantime, your users are left with an enigmatic 500 page. To help your team get from error to fix as efficiently as possible, try Sentryan open source project under the BSD license. Sentry alerts you to the problematic line of code, pinpoints the commit and author likely responsible, and lets you resolve the error by including fixes ISSUE_ID in your commit message.

It starts with an overlooked edge case during an otherwise routine deploy. When a new deploy introduces an error, your team gets alerted and kicks off the familiar workflow of triaging, assigning, reproducing, and, finally, fixing the error. Sentry integrates neatly into your GitHub development flow, providing you with timely information and the rest of your team with visibility into every step.

We make extensive use of both Sentry and GitHub here at Flybase so when I came across this blog post on the GitHub blog, I wanted to share it.

Source: https://github.com/blog/2388-how-to-fix-errors-in-production-with-github-and-sentry

Healthchecks and Heartbearts

Most web apps set up a healthcheck, it’s usually an endpoint that tells you that your databases are up, and websites are up.

I’ve even got a repo ready to go that I install on servers and enable quickly, based on another middleware healthcheck but with a few changes to make it easier to use, and with endpoints already in place for mongodb, redis and elasticsearch if needed.

We use healthchecks heavily here at Flybase to monitor all our services and let us know in case of any issues.

These work great, if one thing goes down then your monitor will alert you.

But what about all those third party APIs your app uses?

Or if you use a microservice architecture, how do you ensure everything continues working as expected?

This is where heartbeats come into play, and are a slightly different take on healthchecks.

For example, let’s look at an e-commerce site that sells spoons:

  1. User picks an item to purchase
  2. Item goes to cart
  3. User logs in (or signs up) and pays for item
  4. System registers purchase, and signals warehouse and shipping company about shipment
  5. Shipment gets picked up
  6. Users gets his new spoon.

So there are several APIs involved here, we have the payment processor (maybe Stripe), we have the shipping processor, we also have internal APIs such as user management, inventory control, order management and notifications.

In this scenario, let’s say the shipping processor never gets notified? or the inventory control doesn’t adapt the numbers and we end up showing more in stock than there actually is?

Nobody likes ordering a sold out item thinking they’ll have it in a week, and actually waiting a month or two.

Or worse, ordering an item and the shipping system never gets the pick up request.

So we set up a heartbeat. Which is a simulated order that is run every so many hours.

In this case, we can create a special item that never gets seen anywhere else, and a user account that is called heartbeat.

Then you would have your heartbeat user walk through the order system using automated scripts and if any errors occur, for example an email never arrives in the heartbeat box, or the order page never shows sucessful order.

You could add some extra checks to make sure the db updates and check responses back from the various end points, heartbeats are allowed to be slower than the actual order process as they are checking everything involved, but the end result…

You get a snapshot of your system every so many hours and can get a heads up in cases of any issues.

Also, most important, perform clean ups once done, this is also why you should use a dedicated user (or even users).

The End?

This wasn’t a code heavy post, I shared a repo I use on several projects for healthchecks and mostly explained ways to do a heartbeat.

It’s hard to code a demo heartbeat as the code is so different based on projects.

So all I can recommend in building a heartbeat is monitoring every step you can. It’s the heartbeat of your app and an unhealthy heartbeat is very dangerous.

This post was originally published on Coded Geekery, my blog on various topics from cooking to work life balance.

Build a Status Page powered by Github and Uptime Robot

This post was originally posted on Coded Geekery, my personal blog about finding a work-life balance, but I wanted to share it here as well since it’s about the Flybase status page.

Overview

We built our own status page for Flybase, you can see it here.

It was built as a static HTML page on Github Pages. It uses Github Issues to report any incidents and Uptime Robot to monitor our sites.

This idea is based loosely on the statuspage repo created by @pyupio, but simplified, as I wanted this to be pretty much automated, plus we already use Uptime Robot for monitoring, so combining Uptime Robot with Github Issues works great.

To get started, you’ll want two things:

  1. An Uptime Robot account
  2. A GitHub repo where you can throw your site up and use the issues system.

Create a branch in your repo called gh-pages, this is where your files will sit.

Ok, let’s build our status page:

1. create index.html

First, create our index.html file:

2. Create script.js

Next, we’ll create script.js, this is the file that talks to our services.

Replace the following variables with actual lines:

YOUR-UPTIME-ROBOT-API-KEY-1 and YOUR-UPTIME-ROBOT-API-KEY-2: Uptime Robot’s default API key is universal, but is also read and write. We want this monitor to be read-only so we have to create an API Key for each site we are creating. This is an array of keys.

To add more sites, just add a new line and add a new monitor key.

YOUR-GITHUB-USERNAME: Your Github username where the repo you created lives.

YOUR-GITHUB-REPO: The Github repo you created to use.

2. Create style.css

Finally, we want to create our style.css file. Just copy this entire block into the file.

4. Git it Going

Once you’ve created your files, you want to put them on your repo, you can either create them directly from the github.com interface, or you can create them locally and commit them.

If you want to add this to a specific domain, then create a file called CNAME and store your domain or subdomain in there.

Finally, create a file called .nojekyll which tells Github Pages that this is a strictly static site.

To customize Github Issues, we set up labels to identify issues:

  • operational means all systems good.
  • investigating means under investigation.
  • outage to identify an outage.
  • degraded to identify an issue causing degraded performance.

On top of that, you can add labels that start with system: and they will show what system the issue is related to. For example system:blog would show an issue with our blog.

Labeling an issue with any of these tags will reflect on the status page.


This status page works pretty well, and was useful last week with the AWS outage that happened. It showed the status of our various services, and let us push updates via Github Issues that showed up below.

I do plan on making an update at some point to take into account comments inside issues.

This is a basic status page, but it helps show people what is happening with your sites, and keep everything nice and transparent.

Originally posted on Coded Geekery, my personal blog about finding a work-life balance

Build a real-time SMS group chat tool with Flybase, Twilio, Node.js and Heroku

This is an update to our original Post about building a group chat tool. Some things remain the same, but we’ve updated the library to Flybase (finally) and also removed the ngrok section for Heroku.

Overview

Last November, I went to a conference with several co-workers, and we wanted to keep everyone organized so we could keep track of what our plans were.

We set up a group chat system that let one of the members send an SMS and everyone else get the message, then if someone replied, we’d all see the reply.

That was handy, and today, I’m going to show you how to build a similar web app. The app will consist of a simple control panel where you can manage who is part of a group, and a backend that will handle incoming and outgoing text messages and route them to the proper group members.

You will also be able to send and receive messages from a page on the site in real-time, for when you may not have you phone on you but want to send a message to the group, and vice versa.

We’ll use Flybase to handle the data-storage and real-time aspects of the app, Twilio to handle the actual SMS work, and Node.js for the system itself.

We’re going to build this particular app for one single group, but it wouldn’t be hard to extend it for multiple groups.

Then finally, we’ll host this app on Heroku as a free app.

Ingredients

We’ll be using a few tools to build this app. You’ll want to have these set up before you continue on:

  • Twilio: To send and receive SMS messages.
  • Flybase: A real-time database API. We’ll be using it to store a record of incoming and outgoing messages, and also people who are part of our group.
  • Node.js: A platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications.
  • Heroku: A handy hosting platform for getting your projects up and running quickly, especially handy when combined with Flybase and Twilio.

Node.js will be the backend portion of our app, it’s where we will build our listeners for Twilio to talk to when ever we send or receive a text message.

Flybase is a real-time app platform and will be our datastore of choice for our app, it will be used to manage who is a member of a group, and to store incoming and outgoing messages and who they came from. If you haven’t already, Sign up for a free Flybase account now, then create a new app from inside your dashboard. You’ll use this app for your group chat system.

Twilio is our every handy phone API, which lets us build services like a group chat app, or even a call center. Don’t have a Twilio account? yet Sign up for Free

Getting Started

We first need to set up our Node.js app.

Besides the Twilio and Flybase modules, we’ll be using the Express framework to set up our node web server to receive the POST request from Twilio so we’ll need to install the express package. We’ll also be using the body-parser module so we are going to install that as well.

Let’s create our package.json file:

{
  "name": "group-chat",
  "version": "0.0.1",
  "description": "SMS Group Chat powered by Flybase, Twilio and Node.js",
  "main": "app.js",
  "repository": "https://github.com/flybaseio/group-chat",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "twilio",
    "flybase",
    "sms"
  ],
  "author": "Roger Stringer",
  "license": "MIT",
  "dependencies": {
    "body-parser": "~1.16.0",
    "ejs": "~2.5.5",
    "express": "~4.14.0",
    "flybase": "^1.8.2",
    "less-middleware": "~2.2.0",
    "method-override": "~2.3.7",
    "moment": "~2.17.1",
    "node-buzz": "~1.1.0",
    "twilio": "~2.11.1"
  }
}

Save this file, and from the terminal run the following command:

npm install

This will create a node_modules folder containing all of the modules we want to use.

Let’s set up our folder structure, create a folder called views, this is where we will keep our frontend.

Now, create a folder called public, this will host our static files, inside that folder, create a css folder and a js folder, we’ll come back to these later.

The first file we want to create is config.js, this will hold our configuration information:

module.exports = {
    // Twilio API keys
    twilio: {
        sid: "ACCOUNTSID",
        token: "AUTHTOKEN",
        from_number: "YOUR-NUMBER"
	},
    flybase: {
		api_key: "YOUR-API-KEY",
		app_name: "YOUR-FLYBASE-APP"
    },
    un: 'admin',
    pw: 'password'
};

This file is for our configuration, we can access anything in here at anytime by referencing the file and calling the keys, for example, to get our Flybase API Key, we would call:

var config = require('./config');
console.log( config.flybase.api_key );

Replace ACCOUNTSID, AUTHTOKEN and YOUR-NUMBER with your Twilio credentials, and a phone number in your Twilio account that you’ll be using.

Then, replace YOUR-API-KEY, and YOUR-FLYBASE-APP with your Flybase API Key to use.

At the beginning of our app.js file we’ll need to require express and initialize it into a variable called app. We’re also going to use the bodyParser middleware to make it easy to use the data we’ll be getting in our POST request.

Create a new file called app.js and require the twilio, express and flybase packages:

var express = require('express');
var bodyParser = require('body-parser');
var methodOverride = require('method-override');
var path = require('path');
var config = require('./config');

var app = express();
app.set('views', path.join(process.cwd(), 'views'));
app.set('view engine', 'ejs');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({	extended: true	}));
app.use(express.static(__dirname + '/public')); // set the static files location /public/img will be /img for users

var port = process.env.PORT || 8080; // set our port

var twilio = require('twilio');
var client = twilio(config.twilio.sid, config.twilio.token );

var flybase = require('flybase');
var messagesRef = flybase.init(config.flybase.app_name, "messages", config.flybase.api_key);
var groupRef = flybase.init(config.flybase.app_name, "groups", config.flybase.api_key);

Flybase uses collections to organize data inside apps, so one app could have several collections. If you’re familiar with relational databases, this is the equivalent of a table.

We’ll be using two collections for our project, one will contain messages, the other will contain groups. With that in mind, we’ve created two different references to our Flybase app, one for messages, and one for our group.

This is the start of our app, next we’ll build our web interface to manage group members and also allow for sending and receiving messages.

After that, we’ll build our Twilio interface and you’ll have a fun app to play with.

Sending and receiving texts

We’ll need to add a few things to send and receive texts, our first step is to add a listener for Twilio.

Twilio uses webhooks to let your server know when an incoming message or phone call comes into our app. We need to set up an endpoint that we can tell Twilio to use for the messaging webhook.

We’re going to add a route for /message that responds with some TwiML. TwiML is a basic set of instructions you can use to tell Twilio what to do when you receive an incoming call or SMS message. Our code will look like this:

//	listen for incoming sms messages
app.post('/message', function (request, response) {
	var d = new Date();
	var date = d.toLocaleString();

	groupRef.where( {"memberNumber":request.param('From')} ).limit(1).on( "value", function ( data ){
		if( data.count() ){
			data.forEach( function( snapshot ){
				var member = snapshot.value();
				messagesRef.push({
					sid: request.param('MessageSid'),
					type:'text',
					tstamp: date,
					fromName:member.memberName,
					fromNumber:request.param('From'),
					message:request.param('Body'),
					fromCity:request.param('FromCity'),
					fromState:request.param('FromState'),
					fromCountry:request.param('FromCountry'),
					groupNumber:request.param('To')
				});
			});
		}
	});
	var resp = new twilio.TwimlResponse();
	resp.message('Message received.');
	response.writeHead(200, {
		'Content-Type':'text/xml'
	});
	response.end(resp.toString());
});

This will listen for any incoming sms messages and store them inside your Flybase app, specifically inside the messages collection.

As part of storing the message, we perform a look up to find the groups member with the same phone number the message was sent from, we then use this lookup to verify the member was part of the group, and also to get the member’s name.

If no member was found, then no message gets sent.

Once a message has been received, we use the Twilio node library to initialize a new TwimlResponse. We then use the Message verb to set what we want to respond to the message with. In this case we’ll just say “Message received”.

Then we’ll set the content-type of our response to text/xml and send the string representation of the TwimlResponse we built.

Listening For Changes

As part of our app.js code, we also want to add some asynchronous listeners to listen for changes to our Flybase apps.

//	when a new message is added to the Flybase app, send it via Twilio...
messagesRef.on("added", function (data ){
	var snapshot = data.value();
	sendMessage(
		snapshot.groupNumber,
		snapshot.fromName,
		snapshot.fromNumber,
		snapshot.message
	);
});

groupRef.on("added", function ( data ){
	var snapshot = data.value();
	var msg = snapshot.memberName + ' has joined the group';
	messagesRef.push({
		sid: "",
		type:'',
		tstamp: new Date().toLocaleString(),
		fromName:"Admin",
		fromNumber:"",
		message:msg,
		fromCity:"",
		fromState:"",
		fromCountry:"",
		groupNumber:snapshot.groupNumber
	});
});

groupRef.on("removed", function ( data ){
	var snapshot = data.value();
	var msg = snapshot.memberName + ' has left the group';
	//	send broadcast that a group member has been removed
	messagesRef.push({
		sid: "",
		type:'',
		tstamp: new Date().toLocaleString(),
		fromName:"Admin",
		fromNumber:"",
		message:msg,
		fromCity:"",
		fromState:"",
		fromCountry:"",
		groupNumber:snapshot.groupNumber
	});
});

//	broadcast a message to the group
function sendMessage( group_number, from_name, from_number, message ){
	var msg = from_name + ": " + message;
//	loop through the group members and get list of people to message:
	groupRef.where( {"memberNumber":{"$not":from_number}} ).on( "value", function ( data ){
		if( data.count() ){
			data.forEach( function( snapshot ){
				var member = snapshot.value();
				client.sendMessage( {
					to:member.memberNumber,
					from:group_number,
					body:msg
				}, function( err, data ) {
				});
			});
		}
	});
}

We’ve set up three asynchronous listeners, one for the message collection, which listens for any messages being added to it, and when it receives a notification of a new message, calls our sendMessage function and sends the message on to the other members of the group.

The other two asynchronous listeners are for our groups collection, the first one, listens for any new members being added to a group and then sends an announcement that the member has joined the group.

The last listener will listen for any members being removed from a group, and sends an announcement that the member has left the group.

Finally, our sendMessage function is used for sending messages on to the other group members, it will perform a query to return all members of the group, excluding the person who sent the message, and send the message onto each member.

Messages will appear formatted with the member’s name followed by the message.

	John: How about pizza after work?

Finally, let’s set our server to listen on port 8080, and tell it what to do when we view it from a brower:

// frontend routes =========================

// Create basic auth middleware used to authenticate all admin requests
var auth = express.basicAuth(config.un, config.pw);

// route to handle all frontend requests, with a password to protect unauthorized access....
app.get('*', auth, function(req, res) {
	res.render('index', {
		api_key:config.flybase.api_key,
		app_name:config.flybase.app_name,
		group_number:config.twilio.from_number
	});
});

var server = app.listen(port, function() {
	console.log('Listening on port %d', server.address().port);
});

This is the backend portion of our group chat app it listens for incoming text messages, stores them in our Flybase app, and then sends broadcasts to the other members of the group.

Now, we need to build our control panel, where the admin can manage group members, and also send and receive messages.

We’ll build that now.

Managing Your Group

We’re going to build a simple web interface to manage our group members.

The data we store for our group members will consist of the following three pieces of data:

  • Group phone number (the Twilio number we stored in the twilio_number variable in the Getting Started section)
  • Member name
  • Member phone number

We’ll also display a basic chat box which will let our admin send messages and see what messages are being sent.

First, let’s create our view, in the /views folder, create a file called index.ejs:

<!doctype html>
<html>
<head>
	<link href='//fonts.googleapis.com/css?family=Lato:400,300italic,400italic&subset=latin,latin-ext' rel='stylesheet' type='text/css'>
	<link rel="stylesheet" type="text/css" href="//angular-ui.github.com/ng-grid/css/ng-grid.css" />
	<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">
	<link href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css" rel="stylesheet">
	<link rel="stylesheet" type="text/css" href="/css/style.css">

	<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
	<script src="https://cdn.flybase.io/flybase.js?20150217"></script>
	<script src="https://cdn.flybase.io/libs/phone.js"></script>
	<script src="/js/group.js"></script>

	<title>Group Chat, powered by Flybase and Twilio</title>
</head>
<body>
	<div class='container'>
		<div class="row">
			<div class="col-md-6">
				<h3>Group Members</h3>
				<div id="group_wrapper"></div>
				<hr />
				<h2>Add new member</h2>
				<div class="well">
					<form id="group_form" method="post" accept-charset="utf-8" class="form-inline">
						<div class="form-group">
							<div class="input-group">
								<div class="input-group-addon"><i class="fa fa-pencil"></i></div>
								<input type="text" class="form-control" id="name" name="name" placeholder="name">
							</div>
						</div>
						<div class="form-group">
							<div class="input-group">
								<div class="input-group-addon"><i class="fa fa-mobile"></i></div>
								<input type="tel" class="form-control" id="phone" name="phone" placeholder="+11112223333"/>
							</div>
						</div>
						<button type="submit" class="btn btn-primary">Save</button>
					</form>
				</div>
			</div>
			<div class="col-md-4 col-md-offset-1">
				<div id="chatBox" class='chat'>
					<header>Chat Log</header>
					<ul id='messagesDiv' class='chat-messages'></ul>
					<footer>
						<form id="msg_form" method="post" accept-charset="utf-8" class="form-inline">
							<input type="text" id="messageInput" placeholder="Type a message..." />
						</form>
					</footer>
				</div>
			</div>
	</div>
	<script>
		$(function(){
//			inititialize our Flybase object
			var myGroupManager = new groupManager( "<%= api_key %>", "<%= app_name %>", "<?%= group_number %>");
			myGroupManager.start();
		});
	</script>
</body>
</html>

This will display out control panel, which will be split into two panes, the left side for viewing group members, the right side for viewing the chat log.

At the bottom of the page, we’re initializing our groupManager class. We’ll create that file shortly.

Next, let’s create our style sheet. In the public/css folder, create a file called style.css:

body{font-size:12pt;font-family:helvetica}
.chatWindow{float:left;margin:20px;border:1px solid #000;width:300px;background:#e5e5e5;border-radius:5px}
.chatName{margin-bottom:10px;background:#666;color:#fff;padding:4px}
.messages{padding:4px}
.message_outbound{color:blue;text-align:right}
.tstamp{font-size:9px;padding:2px;margin-bottom:10px;border-bottom:1px dotted #666;color:#666}
.error{color:red;text-align:center}
.messageForm textarea{float:left;width:220px;margin:5px}
#phone{width:140px;}
#chatBox{background-color: #f8f8f8;background: rgb(229, 228, 228);margin:10px;}
.hide {display: none; }
.chat {font-family: "Helvetica Neue Light", "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;border-radius: 3px;-webkit-box-shadow: 0px 8px 20px rgba(0, 0, 0, 0.2);box-shadow: 0px 8px 20px rgba(0, 0, 0, 0.2);background-color: #dfe3ea;border: 1px solid #CCC;overflow: auto;padding: 0px;font-size: 18px;line-height: 22px;color: #666; }
.chat header {background-color: #EEE;background: -webkit-gradient(linear, left top, left bottom, from(#EEEEEE), to(#DDDDDD));background: -webkit-linear-gradient(top, #EEEEEE, #DDDDDD);background: linear-gradient(top, #EEEEEE, #DDDDDD);-webkit-box-shadow: inset 0px 1px 0px rgba(255, 255, 255, 0.9), 0px 1px 2px rgba(0, 0, 0, 0.1);box-shadow: inset 0px 1px 0px rgba(255, 255, 255, 0.9), 0px 1px 2px rgba(0, 0, 0, 0.1);border-radius: 3px 3px 0px 0px;border-bottom: 1px solid #CCC;line-height: 24px;font-size: 12px;text-align: center;color: #999; }
.chat input {-webkit-box-sizing: border-box;-moz-box-sizing: border-box;box-sizing: border-box;-webkit-box-shadow: inset 0px 1px 3px rgba(0, 0, 0, 0.2);box-shadow: inset 0px 1px 3px rgba(0, 0, 0, 0.2);border-radius: 3px;padding: 0px 10px;height: 30px;font-size: 18px;width: 100%;font-weight: normal;outline: none; }
.chat .chat-toolbar {background-color: #FFF;padding: 10px;position: relative;border-bottom: 1px solid #CCC; }
.chat .chat-toolbar label {text-transform: uppercase;line-height: 32px;font-size: 14px;color: #999;position: absolute;top: 10px;left: 20px;z-index: 1; }
.chat .chat-toolbar input {-webkit-box-shadow: none;box-shadow: none;border: 1px solid #FFF;padding-left: 100px;color: #999; }
.chat .chat-toolbar input:active, .chat .chat-toolbar input:focus {color: #1d9dff;border: 1px solid #FFF; }
.chat ul {list-style: none;margin: 0px;padding: 20px;height: 200px;overflow: auto; }
.chat ul li {margin-bottom: 10px;line-height: 24px; }
.chat ul li:last-child {margin: 0px; }
.chat ul .chat-username {margin-right: 10px; }
.chat footer {display: block;padding: 10px; }
.chat footer input {border: 1px solid #ced3db;height: 40px; width:75%;}

Now, let’s move onto the brains of our system, inside the public/js folder, we’ll create a file called group.js:

var groupManager = function(api_key, app_name, group_number) {
//	store the group number
	this.group_number = group_number;
//	reference to our messages collection...
	this.messagesRef = new Flybase(api_key, app_name, "messages");

//	reference to our group collection...
	this.groupRef = new Flybase(api_key, app_name, "groups");

	this.group_members = [];
};

This is the first part of our groupManager class, so far we’ve told it to start up two Flybase references, one called messagesRef and one called groupRef, we also stored our group number as a variable called group_number.

Now, let’s set up our actions:

groupManager.prototype.start = function(){
	var _this = this;

//	list group members if any
	this.groupRef.on("value", function( data ){
		if( data.count() ){		
			data.forEach( function( snapshot ){
				var member = snapshot.value();
				_this.group_members[member._id] = member;				
			});
		}
		_this.displayGroup();
	});

//	listen for new members being added
	this.groupRef.on("added", function( snapshot ){
		var member = snapshot.value();
		_this.group_members[member._id] = member;
		_this.displayGroup();
	});

//	save new group member to our app
	$("#group_form").submit( function(e){
		e.preventDefault();
		var member = {
			'groupNumber': _this.group_number,
			'memberName': $("#name").val(),
			'memberNumber': clean_phone( $("#phone").val() )
		};
		_this.groupRef.push( member );
		$("#name").val('');
		$("#phone").val('');
		return false;
	});

//	listen for members being removed
	$('div').on('click','a.delete', function(e){
		var _id = e.target.id;
		_this.groupRef.remove(_id);
		return false;
	});

	this.groupRef.on("removed", function( snapshot ){
		var member = snapshot.value();
		_this.group_members[member._id] = undefined;
		_this.displayGroup();
	});

//	list any existing chat message
	this.messagesRef.on('value', function (data) {
		if( data.count() ){
			data.forEach( function(message){				
				_this.displayChatMessage(message.value() );
			});
		}
	});		
//	listen for incoming chat messages
	this.messagesRef.on('added', function (data) {
		var message = data.value();
		_this.displayChatMessage( message );
	});

//	listen for outgoing chat messages
	$('#msg_form').submit( function(e){
		e.preventDefault();
		var message = {
				"tstamp": new Date().toLocaleString(),
				"fromName": "Admin",
				"fromNumber": "",
				"message": $('#messageInput').val(),
				"fromCity": "",
				"fromState": "",
				"fromCountry": "",
				"groupNumber": _this.group_number
		}
		_this.messagesRef.push( message );
		$('#messageInput').val('');
		return false;
	});
};

Our start function sets up our asynchronous listeners, as well as listeners for form submissions and members being deleted by pressing the delete button.

If a group member is added, then the member will be added to the groups collection and a notification will be sent to the other members of the group. The listing of group members will also show the new member.

If a person is removed, then they will vanish from the list, and a message will be sent to the remaining group members.

The other side of our groupManager class is the actual chatting side of things, when the admin types in a message, it will get broadcast to the other group members, at the same time, when another group member sends a message, the admin will see the message in the chat box.

We have two functions left, one is to display all members of a group, and the other displays chat messages.

For our groups, we stored information in a class-wide variable called group_members, this lets us quickly add, update or remove members as we receive notifications about it.

//	Display group members
groupManager.prototype.displayGroup = function(){
	$('#group_wrapper').html('');
	for (var i in this.group_members ) {
		var member = this.group_members[i];
		if( member !== undefined ){
			var html = '';
			html = '<span>'+member.memberName+' ( ' + member.memberNumber + ' )</span> <a href="#delete" class="delete" id="' + member._id+'">[remove]</a>';
			$('<div/>').prepend( html ).appendTo($('#group_wrapper'));
		}
	}
};

Our last function, displays each chat message as it is received:

//	Display chat messages
groupManager.prototype.displayChatMessage = function( message ){
	var _this = this;
	$('<li/>')
		.attr("id",message._id)
		.text(message.message)
		.prepend(
			$("<strong class='example-chat-username' />").text(message.fromName+': ')
			).appendTo( $('#messagesDiv') );
	$('#messagesDiv')[0].scrollTop = $('#messagesDiv')[0].scrollHeight;
};

One last thing to do, Let’s fire up our app:

node app.js

We’ve told our app to run on port 8080, so if you go to your web browser and type in http://localhost:8080/ you should see your call center.

Hosting it on Heroku

Heroku is great for making server configurations easy and painless. We can build faster and worry about the things that matter to us instead of trying to configure our own servers. This works perfectly with our philosophy here at Flybase and lets us build things quickly.

Let’s look at how we can deploy our group chat app to Heroku in mere seconds.

Go ahead and go to Heroku.com and create your free account. At first glance, the dashboard is incredibly simple and user friendly.

Next, you’ll want to install the Heroku Toolbelt. The Heroku toolbelt will give us access to the Heroku Command Line Utility.

After we install the Toolbelt, we’ll have access to the heroku command.

Now, you’ll want to perform the following operations:

  1. git init inside the folder you created your group chat app in to create a new git repository.
  2. heroku login to log into Heroku.
  3. heroku create to create the application within Heroku.
  4. git push heroku master to push your group chat repository to Heroku.
  5. heroku ps:scale web=1 to tell Heroku to create a dyno (a worker, to respond to web requests).
  6. heroku open to open Safari at your new, custom URL.

And that’s it. Your app is now running on Heroku.

Assigning your group chat to a phone number in Twilio.

Now, we want to go back to our Twilio account, and open the phone number we were using to send messages.

When you created your app on Heroku, it gave you a unique URL. For example, let’s say it’s:

https://my-group-chat.herokuapp.com/

Our URL to receive messages via SMS will now be:

https://my-group-chat.herokuapp.com/message

So with that in mind, we need to tell Twilio to use this messaging url as a our Message Request URL:

Go to the area in this picture and add your Heroku URL with /message appended to it.

Now, send an SMS message to your Twilio number and you should get a response back. If you don’t, take a look at the Twilio App Monitor to help determine what went wrong.

Finishing Up

We’ve built a real-time group chat app using Flybase and Twilio

You can find our group chat app here at Github.

This app can be used for a group of people to carry on a conversation. This can be handy when attending events.

You could use this to notify even attendees of upcoming talks, for example, a conference could add their attendees to a group and then send a broadcast when it is time for a talk to begin, or when it is lunch time.

You can extend this to include support for multiple groups, by simply giving each group it’s own phone number for example.

Manage Your Data With Indexes

We’ve mentioned before about our roll out of nextgen features, and we are
starting to roll these out now.

We’ve rolled out the queues, and now this week we’ve launched our schema builder.

This is a very basic schema builder, after you’ve added a few records to a collection, you can set rules for each field, such as
field types (number, string or mixed), index rules such as unique or required (or unique and required).

This gives you more control over your data, and also cuts down on doing things like searching to see if a record exists before adding it, now you just save it and that was it.

Some other rules will be coming shortly, followed by our new GraphQL API which is shaping up beautifully and is in testing with a few users now. Then we’ll be launching the rest of our nextgen features, but we are rolling these out one by one so that people can play with them as they roll out and not have a ton of new features hit them at once.

Queuing It Up With Flybase Simple Queues

Queues are handy for processing items in the background, you just send an item to the queue and have a worker in the background to do something with it and remove it from the queue. You can even have multiple workers running

Flybase is great for setting up queues, but we wanted to make it even better (and simpler) so we built our Queue system.

Adding items to a queue is simple, you just have to use the enqueue function, available in our latest javascript and node libraries:

var ref = new Flybase("YOUR-FLYBASE-APIKEY-OR-TOKEN", "web", "posts");
ref.enqueue({
	username: 'Joe',
	message: 'Hello',
	published: new Date().getTime()
});

Then to get items out of a queue, you just dequeue it:

var ref = new Flybase("YOUR-FLYBASE-APIKEY-OR-TOKEN", "web", "posts");

function worker(){
	//	check if there are pending jobs..
	ref.getLength(function(row){
		if( row.jobs ){
			//	grab the first item from the queue and process it...
			ref.dequeue(function( data ){
				console.log( data.username + " said: " + data.message + " on " + data.published);
				worker();
			});
		}else{
			worker();
		}
	});
}
worker();

The getLength function returns a count of pending tasks in your queue, Then the dequeue function returns the next item in your queue. If there are no pending jobs, then it just performs another check to see.

You could run the worker portion of your tasks in a backend node.js app, while the enqueue portion can be handled from anywhere, even direct from the browser.


RESTful Queues

You can also use our REST API to enqueue and dequeue items:

curl -X GET -H "X-Flybase-API-Key: YOUR-FLYBASE-APIKEY-OR-TOKEN" \
	  https://api.flybase.io/queue/web

Will return the next item in the queue.

curl -X GET -H "X-Flybase-API-Key: YOUR-FLYBASE-APIKEY-OR-TOKEN" \
	  https://api.flybase.io/queue/web/count

Will return the number of pending jobs currently in the queue.

And finally:

curl -X POST -d '{ "username": "baileys", "date_of_birth": "June 9, 1978", "full_name": "Bailey Stringer"}' \
	  -H "X-Flybase-API-Key: YOUR-FLYBASE-APIKEY-OR-TOKEN" \
	  https://api.flybase.io/queue/web

will store a new task to our queue for your Flybase app called web


Queued items are removed from the queue once retrieved either via the Javascript library or from the REST API. We’ve kept our queue system simple to jump into and use, and you can have as many workers working as you want.

There are various ways you can use the queue system, and we’ve kept it as simple as possible on purpose, we don’t want to overcomplicate something as simple as queues.