Why the world needs another Offline HTML5 App Tutorial

There are plenty of great resources already written for offline HTML5 websites, but just getting a website to work offline is not enough. In this tutorial we will build two versions of an offline website in order to demonstrate how to add functionality to an existing offline website in such a way that existing users won't get left behind using an old version. Many existing tutorials tend to focus on a single technology at a time. This tutorial intentionally avoids going into detail on particular technologies and instead attempts to give a high level overview on how, with the fewest lines of code and in the shortest amount of time, various technologies can be brought together to create an real (and potentially useful) working web app that is structured in a way that makes further development on it in the future easy.

Introduction

We are going to make a simple RSS feed reader capable of storing the latest news items for offline reading. The completed project is ready for forking on github. Requirements for the demo app This demo will use PHP and jQuery because we want the best combination of ubiquity and brevity for demo purposes. Introducing the application cache The appcache can be used to enable websites to work offline by specifying a discrete, markup-less list of files that will be saved in case the user's internet connection is lost. However, as is widely documented on the internet, the app cache is a bit rubbish. 31/08/2012 Edit: Read about our efforts to fix app cache! So this is what we do instead We use the appcache to store just enough Javascript, CSS and HTML to get the web app started (we call this the bootstrap) then deliver the rest through an ajax request, eval() it, then store it in localStorage*. This is a powerful approach as it means that if, for whatever reason, a mistake or corruption creeps into the Javascript code which prevents the app from starting then that broken Javascript will not be cached and the next time the user tries to launch the app their browser will attempt to get a fresh copy of the code from the server. * This is controversial because localStorage is synchronous, which means nothing else can happen - the website will be completely frozen - whilst you save or retrieve data from it. But in our tests on the platforms we target it is also fast, much faster than WebSQL (the client side database available on platforms such as iOS and Blackberry, which can sometimes be slower than the network). When we come to storing and retrieving articles from our RSS feed, we will use client side database technology WebSQL.

1. The bootstrap

In order to make a simple bootstrapped Hello World web app, create the following files.
/index.html The bootstrap HTML, Javascript & CSS
/api/resources/index.php This will concatenate our Javascript & CSS source files together and send them as a JSON string.
/css/global.css
/source/application/applicationcontroller.js Start off by making a single Javascript file for our application, & create others later
/jquery.min.js Download the latest version from jquery.com
/offline.manifest.php The app cache manifest file.
/index.html Start by creating the bootstrap html file. [html] News
Loading…
[/html] To summarise, this file does the following:- /api/resources/index.php Now to make the server side response to api/resources/ (which we requested on #47 of the previous file, /index.html):- [php] /css/global.css At this stage, this is just a placeholder to demonstrate how we can deliver CSS. [css] body { background: #d6fab2; /* garish green */ } [/css] /source/application/applicationcontroller.js This will be expanded later, but for now this is the minimum Javascript required to inject our CSS resources, remove the loading screen and display a Hello World message instead. [js] APP.applicationController = (function () { 'use strict'; function start(resources, storeResources) { // Inject CSS into the DOM $("head").append(""); // Create app elements $("body").html('
Hello World!
'); // Remove our loading splash screen $("#loading").remove(); if (storeResources) { localStorage.resources = JSON.stringify(resources); } } return { start: start }; }()); [/js] /offline.manifest.php Finally, the appcache manifest. This is where other tutorials will tell you to edit your apache config file to add the content-type for *.appcache. You would be right to do this but I want this demo web app to be as portable as possible and work by simply uploading the files to any standard PHP server without any .htaccess or server configuration file hassle, so I will give the file a *.php extension and set the content type by using the PHP header function instead. The *.appcache extension is a recommendation, not a requirement, so we will get away with doing this. [php] CACHE MANIFEST # 2012-07-14 v2 jquery.min.js / NETWORK: * [/php] As you can see, in line with our app cache usage recommendations we've only used the app cache to store the bare minimum to get the web app started:- jquery.min.js and / - which will store index.html. Upload these files to a standard PHP web server (all the files should go in a publicly accessible folder either in public_html (sometimes httpdocs) - or a subfolder of it) then load the app and it should work offline. Currently it doesn't do anything more than say Hello World - and we needn't have written a single line of Javascript if that were our aim. What we've actually created is a web app capable of automatically upgrading itself - and we won't need to worry about the app cache for the rest of the tutorial.

2. Building the actual app

So far we’ve kept the code very generic - at this point the app could feasibly go onto become a calculator, a list of train times, or even a game. We’re making a simple news app so we will need:- We'll use a standard Model-View-Controller (MVC) approach to organise our code and try to keep it all as clean as possible. This will make testing and future development on it a lot easier. With this in mind, we'll make the following files:-
/source/database.jsSome simple functions to make using the client side (WebSQL) database easier.
/source/templates.jsThe V in MVC. View logic will go in here.
/source/articles/article.jsThe model for articles - in this case just some database functions.
/source/articles/articlescontroller.jsThe controller for articles.
/api/articles/index.phpAn API method for actually getting the news.
We will also need to make changes to api/resources/index.php and /source/application/applicationcontroller.js. /source/database.js The client side database technology which we will use to store article content will be WebSQL even though it is deprecated because its replacement, IndexedDB, is still not supported on iOS - our key target platform for the demo web app. We will cover how to support both IndexedDB and WebSQL in future posts. [js] APP.database = (function () { 'use strict'; var smallDatabase; function runQuery(query, data, successCallback) { var i, l, remaining; if (!(data[0] instanceof Array)) { data = [data]; } remaining = data.length; function innerSuccessCallback(tx, rs) { var i, l, output = []; remaining = remaining - 1; if (!remaining) { // HACK Convert row object to an array to make our lives easier for (i = 0, l = rs.rows.length; i < l; i = i + 1) { output.push(rs.rows.item(i)); } if (successCallback) { successCallback(output); } } } function errorCallback(tx, e) { alert("An error has occurred"); } smallDatabase.transaction(function (tx) { for (i = 0, l = data.length; i < l; i = i + 1) { tx.executeSql(query, data[i], innerSuccessCallback, errorCallback); } }); } function open(successCallback) { smallDatabase = openDatabase("APP", "1.0", "Not The FT Web App", (5 * 1024 * 1024)); runQuery("CREATE TABLE IF NOT EXISTS articles(id INTEGER PRIMARY KEY ASC, date TIMESTAMP, author TEXT, headline TEXT, body TEXT)", [], successCallback); } return { open: open, runQuery: runQuery }; }()); [/js] This module has two functions that other modules can call:- * See our article on offline storage for more details on database size limits. /source/templates.js We will keep all view or template type functions together here. [js] APP.templates = (function () { 'use strict'; function application() { return '
'; } function home() { return '
'; } function articleList(articles) { var i, l, output = ''; if (!articles.length) { return '

No articles have been found, maybe you haven\'t refreshed the news?

'; } for (i = 0, l = articles.length; i < l; i = i + 1) { output = output + '
  • ' + articles[i].headline + '
    By ' + articles[i].author + ' on ' + articles[i].date + '
  • '; } return ''; } function article(articles) { // If the data is not in the right form, redirect to an error if (!articles[0]) { window.location = '#error'; } return 'Go back home

    ' + articles[0].headline + '

    By ' + articles[0].author + ' on ' + articles[0].date + '

    ' + articles[0].body; } function articleLoading() { return 'Go back home

    Please wait…'; } return { application: application, home: home, articleList: articleList, article: article, articleLoading: articleLoading }; }()); [/js] In this file we'll just put some simple functions that (with as little logic as possible) generate HTML strings. The only slightly odd thing here is: you may have noticed the database.js runQuery function always returns an array of rows even if you're only expecting a single result. This means the APP.templates.article() function will need to accept an array that contains a single article to be compatible with that. A new method could easily be added to the database function which could run a query but only return the first result, but for now this will do. As our app grows we might like to split this file up, the article functions could go into /source/articles/articlesview.js, for example. /source/articles/article.js This file will deal with communication between the article controller and the database. [js] APP.article = (function () { 'use strict'; function deleteArticles(successCallback) { APP.database.runQuery("DELETE FROM articles", [], successCallback); } function insertArticles(articles, successCallback) { var remaining = articles.length, i, l, data = []; if (remaining === 0) { successCallback(); } // Convert article array of objects to array of arrays for (i = 0, l = articles.length; i < l; i = i + 1) { data[i] = [articles[i].id, articles[i].date, articles[i].headline, articles[i].author, articles[i].body]; } APP.database.runQuery("INSERT INTO articles (id, date, headline, author, body) VALUES (?, ?, ?, ?, ?);", data, successCallback); } function selectBasicArticles(successCallback) { APP.database.runQuery("SELECT id, headline, date, author FROM articles", [], successCallback); } function selectFullArticle(id, successCallback) { APP.database.runQuery("SELECT id, headline, date, author, body FROM articles WHERE id = ?", [id], successCallback); } return { insertArticles: insertArticles, selectBasicArticles: selectBasicArticles, selectFullArticle: selectFullArticle, deleteArticles: deleteArticles }; }()); [/js] There are complexities to be dealt with here:- /sources/articles/articlescontroller.js Now create the articles' controller. [js] APP.articlesController = (function () { 'use strict'; function showArticleList() { APP.article.selectBasicArticles(function (articles) { $("#headlines").html(APP.templates.articleList(articles)); }); } function showArticle(id) { APP.article.selectFullArticle(id, function (article) { $("#body").html(APP.templates.article(article)); }); } function synchronizeWithServer(failureCallback) { $.ajax({ dataType: 'json', url: 'api/articles', success: function (articles) { APP.article.deleteArticles(function () { APP.article.insertArticles(articles, function () { /* * Instead of the line below we *could* just run showArticeList() but since * we already have the articles in scope we needn't make another call to the * database and instead just render the articles straight away. */ $("#headlines").html(APP.templates.articleList(articles)); }); }); }, type: "GET", error: function () { if (failureCallback) { failureCallback(); } } }); } return { synchronizeWithServer: synchronizeWithServer, showArticleList: showArticleList, showArticle: showArticle }; }()); [/js] The article controller will be responsible for:- /api/articles/index.php This file will download then parse an RSS feed (using xpath). It will then strip out all the HTML tags from each article's body (except for <p>'s and <br>'s) and output this information using json_encode. [php] xpath($xpath); if ($items) { $output = array(); foreach ($items as $id => $item) { // This will be encoded as an object, not an array, by json_encode $output[] = array( 'id' => $id + 1, 'headline' => strval($item->title), 'date' => strval($item->pubDate), 'body' => strval(strip_tags($item->description,'


    ')), 'author' => strval($item->children('http://purl.org/dc/elements/1.1/')->creator) ); } } echo json_encode($output); [/php] Although we've finished adding all the new files we're not quite done yet. /api/resources/index.php We need to update the resource compiler to let it know the locations of our newly added Javascript files, so /api/resources/index.php becomes:- [php] /source/application/applicationcontroller.js And finally we will need to update applicationcontroller.js so that all the new functions we've added can actually be used by our users. [js] APP.applicationController = (function () { 'use strict'; function offlineWarning() { alert("This feature is only available online."); } function pageNotFound() { alert("That page you were looking for cannot be found."); } function showHome() { $("#body").html(APP.templates.home()); // Load up the last cached copy of the news APP.articlesController.showArticleList(); $('#refreshButton').click(function () { // If the user is offline, don't bother trying to synchronize if (navigator && navigator.onLine === false) { offlineWarning(); } else { APP.articlesController.synchronizeWithServer(offlineWarning); } }); } function showArticle(id) { $("#body").html(APP.templates.articleLoading()); APP.articlesController.showArticle(id); } function route() { var page = window.location.hash; if (page) { page = page.substring(1); if (parseInt(page, 10) > 0) { showArticle(page); } else { pageNotFound(); } } else { showHome(); } } // This is to our webapp what main() is to C, $(document).ready is to jQuery, etc function start(resources, start) { APP.database.open(function () { // Listen to the hash tag changing $(window).bind("hashchange", route); // Inject CSS Into the DOM $("head").append(""); // Create app elements $("body").html(APP.templates.application()); // Remove our loading splash screen $("#loading").remove(); route(); }); if (storeResources) { localStorage.resources = JSON.stringify(resources); } } return { start: start }; }()); [/js] (Working from bottom to top) this file will handle the following functionality:-

    Ideas for further development

    Wrapping Up

    Clearly our demo web app leaves a lot of room for improvement. However, by organising our code in a clean and structured way, we've created a platform that almost any kind of application could be built upon and by using a short script (which we called the bootstrap) to download and eval the application's code, we don't need to worry about dealing with the app cache's problems. This leaves us free to get on with building great web applications. Finally, if you think you'd like to work on this sort of thing and live (or would like to live) in London, we're hiring! MA - @andrewsmatt on Twitter & Weibo.

    Continue to part 2 - Going cross platform with an FT style web app