Overview
This app demonstrates the server side caching ability of the platform. It also demonstrates the ability to easily get data from external web services and return it to the client in a small and usable format.
Grab the Code on GitHub
Front End
The app front-end is very simple. When the 'Get Mashup from Server' button is pressed, a call to the server is made. The server responds with some data and information about the data. The UI is updated with this data and information. The information headings are:
- My Hash - the latest hash value that the client knows about. This changes whenever the data from the server has updated. No value will be shown if the client has never gotten any data before
- Servers Hash - the hash value of the data returned from the server. This value is taken by the client and sent to the server next time it requires data
- Local Cache Used - boolean value representing whether or not the clients local cache of the data was used. If the clients hash and the servers hash are the same, this value should be true
- No. Entries - the number of data entries
- Data - stringified version of the entire data object
Mashup
A mashup a combination of data from different sources. In this app, the data is retrieved 2 sources,Flickr and Yahoo Local Search. These 2 sources provide JSON API's that return locations based on the passed parameters. The code that makes these external calls can be seen in /cloud/util.js getFlickrData() and getYahooData().
function getFlickrData() {
var response = [];
var param = {
url: "http://www.flickr.com/services/rest/",
method: "POST",
charset: "UTF-8",
contentType: "application/x-www-form-urlencoded",
body: "method=flickr.photos.search&api_key=" + FLICKR_API_KEY + "&sort=interestingness-desc&place_id=2367105&extras=geo%2Cmedia&per_page=" + FLICKR_LIMIT + "&format=json&nojsoncallback=1"
}
var res = $fh.web(param);
//$fh.log('debug', res);
var data = $fh.parse(res.body);
if (data.stat == 'ok') {
response = data.photos.photo
}
return response;
}
function getYahooData() {
var response = [];
var param = {
url: "http://local.yahooapis.com/LocalSearchService/V3/localSearch",
method: "POST",
charset: "UTF-8",
contentType: "application/x-www-form-urlencoded",
body: "appid=YahooDemo&query=" + YAHOO_QUERY + "&location=boston,ma,USA&results=" + YAHOO_LIMIT + "&output=json"
}
var res = $fh.web(param);
var data = $fh.parse(res.body);
response = data.ResultSet.Result;
return response;
}The data format returned from each of these sources is verbose and more than required for this app, so is stripped down to a simpler format containing the latitude, longitude, title and type for each entry. The code for this can also be seen in /cloud/util.js doMashUp().
function doMashUp() {
var mashup = [];
// Add some test placemarks so cache can be invalidated if required
//mashup.push({type: 'test', title: 'Test 1', lat: 57.5, lon: -7.5});
//mashup.push({type: 'test', title: 'Test 2', lat: 57, lon: -7});
//mashup.push({type: 'test', title: 'Test 3', lat: 57.3, lon: -7.3});
// Get placemarks from flickr
var flickr = getFlickrData();
// Iterate over the results, adding them to the mashup
for (var fi = 0, fl = flickr.length; fi < fl; fi++) {
var entry = flickr[fi];
mashup.push({
type: 'flickr',
title: entry.title,
lat: entry.latitude,
lon: entry.longitude
});
}
// Get placemarks from Yahoo Local Search
var yahoo = getYahooData();
// Iterate over the results, adding them to the mashup
for (var yi = 0, yl = yahoo.length; yi < yl; yi++) {
var entry = yahoo[yi];
mashup.push({
type: 'yahoo',
title: entry.Title,
lat: entry.Latitude,
lon: entry.Longitude
});
}
return mashup;
}More information on mashups can be found on wikipedia http://en.wikipedia.org/wiki/Mashup_(web_application_hybrid)
Hashing
Hashing is the process of converting a large amount of data into a much smaller amount using a predetermined algorithm. This process is one-way, meaning it is unlike compression algorithms (such as zip) or encryption algorthims. The resulting smaller data is a representation of the larger data. The usefulness of hashing data becomes apparent when the size of the original data is quite large and can be time consuming to process.
Consider the following scenario. If the mashed up data returned from the server above was hashed, and this hash stored, the hash would represent the state of the data. If the data was to then update, the code would have to figure out that the data changed. There are 2 ways to do this:
- Either iterate over the new data, comparing it to the old data and make a note if a change is spotted
- or (much faster way) create a hash of the new data and compare this to the old hash. If the hashes are different, the data has changed. If the hashes are the same, the data is the same and there is no need to send the data to the client again.
The second method is what this app actually does. Every time the client sends a request to the server for data, it also sends along the latest hash it has. If the server responds saying the hash the client has is the latest, the client uses the locally stored data. If the server responds with a new hash and updated data, the client updates it's stored hash and data locally. The code for this process can be seen in /client/default/script.js:
// Call the server side function to get the mashup data
$fh.act({
act: 'getMashup',
req: {
hash: hash,
timestamp: new Date().getTime()
}
}, function(res) {
var mashup = res;
// Check if our hash's match, so we can use the locally cached data
if (hash === mashup.hash) {
// we already have the latest data, lets use our cachedData
updateMashup(hash, cachedMashup);
} else {
// Hash's don't match, so server has included the newest data to use
// Lets save it, along with the new hash value
$fh.data({
act: 'save',
key: 'mashup_data',
val: JSON.stringify(mashup)
}, function(res) {
// mashup data saved, no need to do anything else
}, function(error) {
alert(error);
});
// No need to wait for asynchronous saving above to finish. Lets go ahead
// and show the latest mashup data
updateMashup(hash, mashup);
}
}, function(code, errorprops, params) {
// Something went wrong with server side function call, let's alert the user
alert('Failed to get mashup from server. Error:' + code);
enableMashupButton();
});The server process for checking the hash and updating it as necessary can be seen in /cloud/main.js getMashUp():
function getMashup() {
var response = {};
// Check the cloud cache to see if we have data
var cached = readCache();
// No cahced data in cloud
if ("" === cached) {
// Get data from remote web services
var data = doMashUp();
// Create MD5 hash of data
var hash = createHash($fh.stringify(data));
// Store data and hash in cloud cache
doCache(hash, data);
// Build response object
response = {
'data': data,
'hash': hash,
'cached': false
};
} else {
//transform cached data from string type to object type
cached = $fh.parse(cached);
// Check if client hash value present & correct
if ($params.hash && $params.hash === cached.hash) {
// Don't need to send data back to client as hash is up to date
response = {
'hash': $params.hash,
'cached': true
};
} else {
// Hash value from client missing or incorrect, return cached cloud data
response = cached;
}
}
return response;
}More information on hashing can be found on wikipedia http://en.wikipedia.org/wiki/Hash_function
Caching
Caching is a mechanism used throughout software applications, in particular, web applications. It allows data that would otherwise take a while to retrieve or process, to be cached in a usable state, therefore increasing the performance of the application. The easiest example of this is browser caching of web pages. The first time a web page is loaded by the browser, it takes a while. This may be noticed moreso with images on the web page. Subsequent times the page is visited, it loads up quicker because resources are cached locally by the browser.
In this app, caching is used on the server to store the results of the mashup so they don't have to be retrieved from the external web services every time they are required. This increases the performance of the app, and improves the user experience.
The mashup caching is done in the doCache() function in /cloud/util.js. This function is very simple. It creates the mashup object, then uses the $fh.cache API call to cache the mashup for some time. The expiry time is set to 10 seconds to help demonstrate the caching.
function doCache(hash, data) {
var obj = {
"hash": hash,
"data": data,
"cached": true
};
$fh.cache({
"act": "save",
"key": "_cache",
"val": obj,
"expire": CACHE_TIME
});
}Reading from the cached is handled by the readCache() function in /cloud/util.js. It loads the data that corresponds to the mashup key.
function readCache() {
var ret = $fh.cache({
"act": "load",
"key": "_cache"
});
return ret.val;
}Invalidating the cache is a 2 step process. The first thing that must happen is the last data cached on the server must have expired. So every 10 seconds, the data expires.
[javascript]var CACHE_TIME = 10[/javascript]
The second thing is the hash of the data must change in order for client to use the new data. As the hash value is based on the data itself, the easiset way to get the hash to change is to change the data. In the doMashUp() function at the top of /cloud/util.js, you will see the following commented out lines:
// Add some test placemarks so cache can be invalidated if required
//mashup.push({type: 'test', title: 'Test 1', lat: 57.5, lon: -7.5});
//mashup.push({type: 'test', title: 'Test 2', lat: 57, lon: -7});
//mashup.push({type: 'test', title: 'Test 3', lat: 57.3, lon: -7.3});Commenting out any one of these 3 lines will add a new entry to the mashup, therefore changing the hash.