Using Twilio With Image Filters
Photo filters are a big thing today. Everyone seems to love Instagram. So this post is going to show you how to get filters working via MMS with Twilio and a Node.js backend. Disclaimer - This definitely works with jpegs, but I’m not sure about pngs, as everything I send to twilio ends up as a jpeg.
This assumes you already have a Twilio account, which you can get at Twilio. The next thing you’ll need after that is a couple of node modules installed via npm.
npm install hapi twilio node-uuid jpeg-js png-js file-type image-size inert
Hapi is the framework we’re using to handle our requests. node-uuid is just to create filenames that we are sure are unique. jpeg-js lets us encode and decode jpegs files and png-js does the same for png. file-type and image-size just allow us to apply filters to our images. inert is the file handler for Hapi.
Make sure your Twilio credentials are stored in environment variables.
const twilio = require('twilio');
const client = new twilio.RestClient();
Next we create a few helper methods.
This method just downloads the file for us and stores it on the file system.
let download = function(uri, filename, callback){
request.head(uri, function(err, res, body){
request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);
});
};
This method decodes our image into an array of pixels. It will handle jpeg or png.
let fileData = fs.readFileSync(fileName);
// Get file type
let metaData = fileType(fileData);
console.log(metaData);
let imageData;
if (metaData.ext == 'jpg') {
imageData = jpeg.decode(fileData);
callback(imageData);
} else if (metaData.ext == 'png') {
png.decode(fileName, function(pixels) {
var size = sizeOf(fileName);
// Convert to jpeg. Writing back out to png was a pain.
imageData = {
data: pixels,
width: size.width,
height: size.height
};
callback(imageData);
});
}
Once we have an array of pixels, we just pass those pixels to our filter; in this case grayScale.
imageData.data = filter.grayScale(imageData.data);
This is the Hapi route to handle the incoming message:
server.route({ method: 'POST', path: '/receiveMessage', handler: function(request, reply)
This is the function that runs upon receiving a message:
// Download media
download(message.MediaUrl0, fileName, function() {
// Apply the filter to the media
applyFilter(filterName, fileName, function(filteredFileName) {
// Create the response and return the filtered media
const twilioMessage = {
body: "Filter applied!",
to: message.From,
from: message.To,
mediaUrl: 'http://curs.es:8088/'+filteredFileName
}
createMessage(twilioMessage);
});
This of course isn’t doing any sort of error handling which would need to be added. This is the function to actually send the message.
function createMessage(twilioMessage) {
client.sendMessage(twilioMessage, function(err, twilio) {
console.log(twilio.sid);
});
}
This is all we need to do to get the processing setup for the messages. Now we want to add some filters. I got a lot of this data from http://www.html5rocks.com/en/tutorials/canvas/imagefilters/ so take a look if you want to understand how these filters work.
I moved my filters to a different file just to make things a little bit cleaner.
This is the grayscale filter.
// Go pixel by pixel and change to grayscale
for (let i = 0; i < imageData.data.length; i += 4) {
let r = imageData.data[i];
let g = imageData.data[i+1];
let b = imageData.data[i+2];
let v = 0.2126 * r + 0.7152 * g + 0.0722 * b;
imageData.data[i] = imageData.data[i+1] = imageData.data[i+2] = v;
}
This is the threshold filter.
// Go pixel to pixel and change to pure black and white
for (let i = 0; i < imageData.data.length; i += 4) {
let r = imageData.data[i];
let g = imageData.data[i+1];
let b = imageData.data[i+2];
let v = (0.2126 * r + 0.7152 * g + 0.0722 * b >= threshold) ? 255 : 0;
imageData.data[i] = imageData.data[i+1] = imageData.data[i+2] = v;
}
The last thing we need is a route to serve up our altered images to Twilio to send back.
server.route({
method: 'GET',
path: '/images/{file}.jpg',
handler: function(request, reply) {
reply.file('images/'+request.params.file+'.jpg');
}
});
That’s it! Feel free to try it out by sending an MMS message to 702-666-0736.
Dig in and take a deeper look at https://github.com/slooker/twilio-image-filter.