A Simple and Effective Backup Strategy for Mac OS X

Disk is inexpensive compared to the value of your time and data. My personal backup configuration consists of three types of backups. The following combination has proven itself over the last several years and I recommend it. It includes 1) A full disk clone, 2) an incremental backup, and 3) an online backup service. This setup is redundant, quick to configure, needs little maintenance, and allows for rapid recovery of data, even with a catastrophic failure.

Details of the three part backup strategy:

  1. A clone is a replica of your disk. One great feature of Mac OS X is that you can boot directly from a clone. This means if your hard drive dies, you can reboot from a clone on an external drive and be back to work in minutes rather than hours. I recommend SuperDuper ($28) as the user interface is very well done. Carbon Copy Cloner is an excellent alternative that is free to use, although the author encourages donations. Both applications support scheduling backups for a time when your system is not in use. Both applications also support incremental updates to substantially reduce the amount of time needed for subsequent backups. The hard drive for your clone must be as large as the amount of data you wish to back up.
  2. An incremental backup application called Time Machine ships with every copy of Mac OS X that archives any file changes every hour. Time Machine has a unique time-based interface that allows you to easily find and restore previous versions of files. Overall, Time Machine is simple to use and works well unattended, but it does have several detractors. First, if you have a hard disk crash, you must manually reinstall the base operating system from the DVD and then use Time Machine to a restore the rest of your data. This makes time machine most useful in cases of accidental file deletion or data corruption. Time Machine works very well when combined with a clone as you can quickly restore from a clone and use Time Machine to restore any files more recent than the clone version. Time Machine is far less useful on drives with FileVault enabled. I recommend giving Time Machine at least two times as much hard drive space as the amount of data you want to back up.
  3. An online backup service allows you to have offsite backups for cases of theft, natural disaster, or large mugs of coffee. Online services also allow laptop users to continue to make backups in any place that has a network connection. I have used the CrashPlan service for about 18 months and I find the service reasonably priced and reliable. CrashPlan automatically archives file changes in real-time and encrypts all backups. This is nice if you use it on a laptop because it means that you have backups even when you travel. CrashPlan also allows online restores from a web-based interface. The unlimited service is $25 a year for a 10GB service, $50 a year for unlimited service for one computer, and $120 a year for a family unlimited plan for up to ten computers. Multiyear subscriptions are discounted.

CrashPlan has a backup seeding service for $125 where they send you a 1TB drive. You then run the initial backup locally and ship the drive back to CrashPlan. Depending on the size of your disk and the speed of your network connection, the initial backup can easily take weeks. Companion emergency recovery services are also $125. Expedited shipping is extra. CrashPlan also offers a computer-to-computer backup mode. This means you could backup to another machine in your house or to a computer in a friend’s house. The computer-to-computer backup feature is free. The paid version provides real-time versioning with fine-grained control over the versioning settings, stronger encryption, the ability to restore from the web, and the client is ad-free. CrashPlan works with Mac OS X, Microsoft Windows, and Linux operating systems

I last wrote about backup options in We Need Simple Backup Solutions for Complicated Data.

No Frills SSL Certificates are Inexpensive and Useful

SSL De Facto for Securing Connections

SSL, short for Secure Socket Layer, is a cryptographic protocol for securing network traffic that is the de facto mechanism for securing transactions on the web and many other protocols including email (SMTP/IMAP/POP), IM (Jabber/XMPP), VoIP (SIP), and SSL-based VPNs. The topic of SSL certificates is a bit arcane, but the much of security of our everyday online purchases depends on SSL. Yet, fewer services use SSL than one might hope. It is possible to buy a basic no-frills SSL certificates from a universally accepted certificate authority very inexpensively–less than $15 a year–if you shop around. In most cases, it makes no sense to use a self-signed certificate, to purchase a certificate from a second tier provider, or to purchase a chained certificate. This article is a substantial revision of an article in Messaging News from a few years ago. I receive some requests for an update and have also found an even more inexpensive provider in the meantime, which make the update worthwhile.

Securing a connection requires that at a minimum both the client and server application support SSL and that the server application must have a digital certificate with a digital signature from a Certificate Authority (CA). This is the most basic and the most common form of SSL Public Key Infrastructure (PKI), which a client to securely authenticate a server. Nearly every online shopping transaction uses this form of SSL to secure the payment details from the user’s browser to the merchants servers. One quick aside, the Transport Layer Security (TLS) protocol released in 1999 superseded the last version of SSL released in 1996, but nearly everyone still calls the protocol SSL.

The January 2009 Netcraft SSL Server Survey found nearly 2.1 million sites that responded to a request for a SSL certificate, but only about 40% of those were valid third-party certificates. Netcraft has been collecting SSL certificates since 1996 and reports that in recent years, use SSL has been growing at a rate of 30% a year. Still the August 2010 Netcraft Web Server Survey found over 210 million sites, which means the number of SSL enabled sites is a small percentage overall.

Why Is Server-Side Adoption of SSL So Low?

Given that nearly every consumer web browser and email client is SSL-enabled, why is server side adoption of SSL so low? In addition there are many reasons why businesses and even technically inclined individuals would want SSL certificates. There is substantial debate around the efficacy of the security provided by SSL for many common configurations, especially with its ability to prevent phishing and man in the middle attacks. Still, the security of an endless number of services such as small webmail providers, dashboards for managing blogs, and web-based router configuration consoles would all benefit from SSL. The majority of high volume ecommerce vendors use SSL, but I regularly see services that ask for credit card numbers over (shudder) unencrypted connections.

The relatively low use of SSL is due in part to the expense and difficulty of purchasing SSL certificates, the complexity of installing them, and the need for a static IP address. For small and medium businesses and individuals no-frills SSL certificates are affordable, especially if you are willing to shop around. The inexpensive certificates provide the same level of functional security for network traffic as the inexpensive certificates. The no-frills certificates are typically domain validated meaning someone just needs to be able to receive and email or possibly respond to an automated phone call in order to validate the domain, which makes the process fast but does not offer any particular assurance the certificate owner is who they say they are.

Other features beyond the level of security provided to network traffic are important for some business. For example, a business handling large numbers of consumer transactions may consider the branding of the certificate or the site seal important, or they may want the green bar shown by sites with Extended Validation (EV) certificates, or a Unified Communications (UC) certificates for an Exchange server. In these cases, then the no-frills route is probably the best one. No matter what kind of SSL certificate you want the process of purchasing them is frustrating and it is difficult to make any sense of the actual differences between the certificates by reading the marketing literature.

Certificate authority certificates, any intermediate certificates, and server certificates form a certificate chain that are verifiable through the SSL Public Key Infrastructure (PKI). It is possible for anyone to set up a private certificate authority and produce a “self-signed certificate.” This is often done for personal use or development purposes.

Inexpensive Certificates

Self-signed certificates require the same amount of effort to install and configure as a commercial certificate, they also require additional work to install and configuring a local certificate authority to sign the certificate. Self-signed certificates are not verifiable through the public PKI chain and most applications will produce warning messages that the certificate is not valid unless the user explicitly loads the credentials for the private certificate authority into each browser. Many second tier SSL providers offer chained SSL certificates, which are more complicated to install in many configurations and are typically less compatible on older browsers and mobile browsers. This said, chained certificates theoretically offer the certificate authority more security as they may revoke a compromised intermediate certificate with far less disruption than the root certificate.

RapidSSL is one of most economical of the top tier SSL certificates. RapidSSL has a bit of a convoluted history, but it is part of the GeoTrust family of certificate authorities, which is far and away the largest digital certificate vendor. GeoTrust was purchased by Verisgin in 2006 and in May 2010 VeriSign’s sold its certificate authority business to Symantec. Luckily, for the purposes of my argument the history is not important. What is important is that the GeoTrust family of certificates is recognized by nearly every browser.

For example, most recently I purchased certificates from a reseller called Revolution Hosting Pricing, Their pricing SSL certificates follows:

Type 1 Yr  2 Yrs 3 Yrs 5 Yrs
RapidSSL  $14  $24  $33  $50
RapidSSL Wildcard $135 $260 $360 $550
QuickSSL  $45  $86 $126 $300
QuickSSL Premium  $75 $140 $195 $300
True BusinessID $105 $190 $270 $425

Problems Purchasing Certificates

For many organizations, SSL certificates are moderately expensive, complicated to purchase, and even more complicated to install. In my own personal experience, the process of purchasing certificates has not improved greatly over the last decade. Going through the process, it is easy to see why so few sites, especially smaller ones, use SSL certificates. Clearly, there is great room for improvement in the user experience of the purchasing process. Unfortunately, I don’t see the process improving any time soon.

It can be surprisingly difficult to get a list of the certificate authority roots (often called a CA bundle) included in specific browsers and even more difficult to get the root certificate bundles included in most mobile devices. Unless the vendor provides a public list of included certificates, it is difficult to determine what CA’s are supported without extracting the CA bundle and analyzing it, which is a major pain. The lack of detailed information about the root certificates substantially complicates the problem for businesses that wish to determine which certificate may meet the needs of their users.

Because there is effectively no standard CA bundle for applications, operating systems, or mobile devices, each vendor has its own bundle of “trusted” certificates. This means, every application that employs SSL may use a different bundle, even if they are on the same machine. For example, both Windows and Mac OS X have a system-wide list of root certificates, but Firefox will use its own list of root certificates regardless of the platform.

To make matters worse many certificate authorities offer multiple types of certificates that may be signed with different roots. I looked at GeoTrust, Comodo, and GoDaddy, and Network Solutions web sites. Only GeoTrust clearly listed which root certificate signed each type of certificate on the main part of their site and not buried in a support document. The situation with GeoTrust was not always so simple, last time I checked a bit more than a year ago, I had to do quite a bit of work digging around the site to determine which root would sign the certificate I wanted to purchase.

Previously, a quick side project to SSL enable and IMAP server turned into an annoying extended detour after I realized that one of the older smartphones did not include the root certificate used on the IMAP server. While, it was possible to load the certificate manually, the process is too complicated for multiple users, although it could be handled in a bulk provisioning process. I ended up spending a significant amount of time searching for certificate authority lists and extracting certificate bundles for several smartphones to figure out which certificate to purchase that would cover them all.

Some Improvements in Purchasing Certificates

SSL certificate compatibility is gradually improving as applications, systems, and devices with out of date certificate bundles are gradually retired. As root certificates and intermediate certificates begin to time out and certificate authorities issue new root certificates. This means that if you have a server with a multi-year SSL certificate issued several years ago, its root certificate may differ from the current one. This is important if you are trying to connect to your SSL server from machines or devices with out of date certificate bundles.

Unfortunately, a market for automatic certificate installation in common machine configurations never developed. Both Microsoft and Apple have made strides with better GUI administration tools for SSL certificates. A number of web hosting services sell SSL certificates with installation for users who pay for the certificate and a static IP address. Another improvement on the horizon is RFC 3546–the Server Name Indication (SNI) extension for TLS. SNI will effectively allow name-based virtual hosting to use SSL similar to the name-based virtual hosts in HTTP 1.1. One major benefit is that this will allow multiple SSL enabled hosts on the same IP address. These are welcome improvements, but we still have a long way to go.

Appendix: A Brief History of RapidSSL and GeoTrust

GeoTrust became a certificate authority in 2001 when it purchased Equifax Digital Certificate Services from Equifax, which is why many of the GeoTrust root certificates are Equifax. FreeSSL launched in 2001 and offered free SSL certificates with its own single root certificate. These were popular, but only had 92% browser compatibility. In 2002, FreeSSL began to offer chained SSL certificates under the ChainedSSL brand for $35 a year, which was a very low price at the time. In 2003, FreeSSL relaunched and temporarily offered free one year ChainedSSL certificates and ChainedSSL wildcard certificates. In February 2004, FreeSSL launched a new brand called StarterSSL, which was a single root certificate. Also February 2004, FreeSSL relaunched the FreeSSL brand as a 30-day free trial certificate. The FreeSSL root certificate signed both the FreeSSL and StarterSSL certificates. Later in 2004 FreeSSL launched another brand called RapidSSL, which combined the StarterSSL single root certificate and included support.

In 2005 FreeSSL formally changed it’s name to RapidSSL. VeriSign purchased Thawte in 2003 and GeoTrust in 2006. At this point some of the details are fuzzy and involve a number of subsidiaries in Europe and Japan, but GeoTrust now apparently owns RapidSSL. In May 2010 Symantec purchased VeriSign’s Security Certificate Business and now controls all roots from all the prior acquisitions.

You should follow me on Twitter.

How to Email a Complete Web Page From Any Browser

Email is still one of the most convenient ways to quickly share links to friends and colleagues. Unfortunately, there are two major problems. First many people’s browsers are not configured to work correctly with their email client, especially for webmail. Second, many browsers only support emailing a link to the web page and not the entire web page. Furthermore, native support to email links is inconsistent and often formatted in a way that may break links for the recipient. I my Messaging News article a Better Way to Share Links in Email described these problems as well as a solution based on the free Readability bookmarklet that should work in nearly any browser and typically produces better results.

Native Options

This article looks at your options for emailing full web pages from nearly any browser. Unfortunately, there are few native options for emailing full web pages. If your primary email client is Outlook 2007 you can select to View -> Toolbars -> Web then open your web page in the built-in browser and finally select “Send Webpage by Email” from the Actions menu. In Internet Explorer version 6 and higher you can click on the “Send Page by Email” button. If you use both Apple Mail and the Safari browser you can select the “Mail Contents of This Page” from the File menu.


The next most simple option is to use the EmailTheWeb service, The service requires that you sign in with Google Account and uses your Gmail account to send out the message. The service is free for up to 25 messages a day. Email the web will also archive your pages for a limited time and mirror the original web page for the recipient in cases where the HTML was too difficult for the application to send correctly. Paid plans range from $20 to $80 a year. Paid plans include longer archiving and mirroring periods. You can use the service by entering your URL on the web site, with a browser bookmarklet, as a Google Toolbar button in IE, or as a Firefox extension.

Limitations of Email Web Pages

All of the above methods of email a full HTML page have limitations. In particular, complex HTML pages will likely look different to the recipient as the application sending the web page may modify contents when sending and the recipient’s email client may further modify the page when rendering it. Web mail clients typically have strict limitation on style sheets in email and many block images by default. The Campaign Monitor Guide to CSS support in email clients is an excellent overview of the limitations. Campaign Monitor has more details on other aspects of HTML in email in their resources on designing and building emails. In some cases it is possible to simply copy and paste the entire email message, but the results are typically far from satisfactory, especially since the style sheet is often not copied along with the HTML. Some pages have a print link that produces a simplified version that works better with cut and paste.

Readability Offers a Better Solution

In general I recommend that people first use the Readability bookmarklet to clean up the page and send the new version via email. Unmodified web pages will often not look like the original and may in fact be far less readable if an essential element is modified or removed. I regularly see pages that have text which becomes mashed together, hidden beneath images, and is otherwise unreadable. The page may also contain many unnecessarily elements such as page navigation and embedded items such as Flash that will not typically arrive correctly. Web pages that processed by Readability often fare much better.

Readability is an excellent tool from Arc90, that reformats web pages, strips out extraneous elements/ads, turns the text into a single column, and generally improves the typography. I find it makes nearly any web page significantly easier and more pleasant to read. I find several advantages to forwarding pages processed by readability. First, Readability inserts a reload button into each page so the recipient only needs to click on the button to see the original in the browser. Second, Readability includes a print link with a stylesheet customized for printing. Third, the pages greatly simplified, easier to read, and have less HTML for any email client to screw up. From all reports, it is also very helpful for people with limited vision as it increases accessibility. Pages processed with Readability make it far easier for recipients with mobile phones to read the content and typically load faster. I tested reading emailed pages on both iPhone and Android devices. Finally, since you are mailing the entire page to the recipient the well be able to read it offline.

To use Readability, just drag the bookmarklet to your toolbar and click on the bookmarklet for any page you want to improve. Readability offers a selection of fonts including two licensed from TypeKit, options to change the size of the text, modify the width of the margins, and optionally convert all links to footnotes. You can find more information about readability in the Arc90 blog posts Introducing: Readability 1.5 and Readability Updated: An End To The Yank Of The Hyperlink. Finally, the most recent update to Readability includes the long-awaited feature to automatically stitch together multi-page articles, which is a feature that none of the native clients offer. The service is free and the Readability source code is available under the Apache license. For users of Safari 5 on the Mac, Safari Reader is based on Readability and offers much of the same functionality, but does not have any customization options. The “Mail the Contents of This Page” option works from Safari Reader.

There are a few limitations, first Readability will not work on every web page. It is specifically designed for longer articles and does not fare well on complex home pages. Second, the process adds an extra step, which is decidedly less convenient. Finally, in testing I found that ad blockers caused Readability to over block images in some cases. In cases where Readability fails, I find that the Instapaper Mobilizer service is a good alternative, but it is not designed for high volume use.

iPhone Screenshot and Photo Smart Album Hack

I take a lot of screenshots when I research products, both on the desktop and on the iPhone, so having some way to automate organizing my collection is important. The problem is that screenshots images taken with the iPhone have no EXIF metadata. This means there is no straightforward way to produce a list of all your screenshots.

After a little bit of experimentation, I found a workable but not ideal solution. You can use the lack of EXIF metadata as conditions to group all the images. Screenshots are saved as PNG files on the original iPhone and the iPhone 3GS (the two models I had access to) and have no EXIF records. The only other metadata fields available are filename, file size, and modified, and imported dates. The PNG extension for the filename is the one existing feature you can search for, all others have to be unknown. I selected two features aperture and ISO, even though one would work in the hopes that this would reduce any false positives.

A Smart Folder recipe for iPhone Screenshots

  • Match all of the following conditions
  • Aperture is Unknown
  • ISO is Unknown
  • Filename contains PNG

iPhone Screenshot 3 Item Smart Folder.png

Photos taken on the iPhone are saved as JPEGs and contain EXIF metadata. The iPhone 3GS embeds many more fields than the original iPhone. The easiest feature to select is “Camera Model.” The field type must be is or is not, there is no option for contains, so you will have to specify each phone separately.

A Smart Folder recipe for iPhone Pictures

  • Match any of the following conditions
  • Camera Model is Apple iPhone
  • Camera Model is Apple iPhone 3GS

iPhone Pictures Smart Folder.png

Searching for Screenshots from the command line

All iPhone screenshot images have a width 320 pixels and height 480 pixels in portrait or landscape. It is possible search for these files using the Spotlight command line tool mdls to integrate them into other scripts. There are many other options for searching for images with the full Spotlight syntax and it is possibly to execute these as Raw Querys in the Finder or use a Spotlight front end such as HoudahSpot, but that is a topic for another post.

mdfind -onlyin $HOME/Pictures 
  'kMDItemKind == "Portable Network Graphics image" && 
  kMDItemPixelHeight == 480 && kMDItemPixelWidth == 320'

Preparing Your Site for the iPad

The Apple iPad does an excellent job of displaying most web sites. However, there are a few obstacles you may want to avoid. There are also a few customizations that will make your site look even better on the iPad. I will summarize the most important issues you should start to plan for and the differences between the iPad browser, the iPhone browser, and desktop browsers. As an added benefit, most improvements made for the iPad will also benefit users with an iPhone or an iPod Touch. There is list of resources to find more information and a list of tools to help you test your site at the end of the article.

Differences in Mobile Safari on the iPad

The primary differences you should account for first are:

  • No support for plugins such as Adobe’s Flash or Sun’s Java for ads, navigation, and multimedia
  • The fixed viewable screen size (viewport) may affect your layout
  • The touch screen is the primary means of interaction and offers different modes of user control

Unlike most desktop browsers, the iPad does not support plugins such as Flash or Java. Any navigation elements, embedded audio and video, or banner ads written in Flash or Java will not appear. Based on public statements, Apple is unlikely to support either language in the future. This means you will need to provide alternative or fallback navigation elements and multimedia embedding options. Apple’s official recommendation is to avoid plugins entirely and use HTML5 elements across your site. Navigation elements may be implemented with standard AJAX techniques. If your revenue depends on banner advertising delivered via Flash or Java, you will need to need to make some changes. If your ad server supports mobile devices, you can turn this on for iPad users. An alternative is to treat mobile users the same as email campaign advertisements. Today at the iPhone OS 4.0 press event, apple announced its own mobile ad platform and ad network called iAd, implemented entirely in HTML5. The mobiThinking Guide to Mobile Advertising Networks in the references surveys most of the available mobile ad network options.

The standards and implementations of HTML5 audio and video tags are still evolving and making your content available in all browsers is still complicated. Supporting HTML5 H.264 encoded video with a fallback to Flash for browsers that do not support it is likely your most straightforward solution. In the references, I have linked to some of John Gruber’s articles on H.264 and Flash that explain the problem in more detail. Video for Everybody from Camen Design and the upcoming SublimeVideo from Jilion are two options for hosting HTML5 friendly video on your site.

The iPad has a 9.7-inch touch-sensitive screen, a fast processor, and fast network connectivity. It provides a web browser experience that is much closer to the desktop experience than a smartphone. This means you should avoid sending iPad users to versions of your site optimized for mobile phones if you are sniffing for iPhone or mobile user agents. If you look at the user-agent strings for the iPad and the iPhone, you will notice that the iPad user-agent lists “like Mac OS X” rather than “iPhone OS.” Both browsers include the “Mobile” in the user-agent string. Most browsers have mechanisms to change the user agent string. I’ve listed some of these in the references.

The current version of iPhone OS (version 3.1.3) uses the following user agent string (line artificially wrapped):

Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_3 like Mac OS X; en-us)
    AppleWebKit/528.18 (KHTML, like Gecko)
    Version/4.0 Mobile/7E18 Safari/528.16

While the iPad with iPhone OS 3.2 uses the following user agent string (line artificially wrapped):

Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us)
    AppleWebKit/531.21.10 (KHTML, like Gecko)
    Version/4.0.4 Mobile/7B367 Safari/531.21.10

The iPad viewport is set to 980 pixels wide, in portrait mode the iPad is 768 pixels wide, but the content will scale to 980 pixels. If you have content that wider than the viewport that uses fixed CSS positioning, that content may end up off screen and your users will not see it since they can not resize the window in Mobile Safari.

Users control the iPad with a multi-touch interface and a touch screen keyboard. The “Apple iPhone Human Interface Guidelines: Introduction” is a great document for starting to think about multi-touch user interaction as the metaphors and modes of physical interaction differ. For example, a flick action rather than a mouse controls scrolling and a pinching action controls how a page scales up and down.

There are other issues, some of which Apple may resolve in a future update. In John Gruber’s review of the iPad, he points out that often only a single page is held in memory at one time, subsequent pages often take all the memory available for web pages. This means that if you could loose form data on a page that you have not submitted if you open another page. The memory problem could also appear on AJAX heavy pages.

iPhone OS User Base

Apple announced the iPad at then end of January and released specifications, documentation, and a software development kit (SDK) for those paid members of the iPhone developer program under an non-disclosure agreement. The WiFi only model of iPad began shipping this week and Apple released the SDK to everyone registered in the Apple Developer Program. Apple announced that it sold more than 300,000 iPads on the first day and more than 450,000 as of April 8th. The iPhone OS platform user base is significant. Steve Jobs announced that there were 75 Million iPhones and iPad Touch devices running iPhone OS at the iPad launch in January. The Apple’s 2010 Q1 filing said that it had sold more than 42 million iPhones total. Today at the iPhone OS 4.0 launch Jobs announced that there were 85 million iPhone OS devices.

Mobile Safari on the iPad uses the open source WebKit rendering engine as do iPhone, and iPod Touch devices. Testing your site with the WebKit rendering engine is now essential. Desktop versions of the Safari browser, Google’s Chrome browser, all iPad, iPhone, and iPod Touch devices, Android devices, Palm webOS devices, Symbian Series 60 (S60) devices all use WebKit. RIM has stated that future BlackBerry devices will use WebKit. This means that every major smartphone browser aside from Windows Mobile will be WebKit-based in 2010.

Testing Your Site on the iPad

Testing your site directly on an iPad is the only way to guarantee that your experience will match your visitors with iPads. There are numerous reports by developers of minor differences between the iPad and the iPad in a simulator.

However, next to owning an iPad, the iPhone simulator comes closest to rendering your site as an iPad would. The iPhone simulator that ships with the iPhone SDK 3.2 has an iPad mode under the device option. Anyone can register as an Apple Developer for free and then download the SDK. The iPhone SDK includes the XCode development environment and is nearly a 2.5 gig download, it also only works on Mac OS X 10.6.2 (Snow Leopard) or higher.

The paid iPhone Developer Program is $99 a year. The subscription allows developers to submit native iPhone and iPad applications to Apple’s App Store. Apple also allows paid developers early access to upcoming versions of its SDK such as the iPhone OS 4.0 SDK announced today.

iPad Peek by Pavol Rusnak is a web service that allows you to see what your web site will look like on an iPad. It is free and the source code is available under an open source license. Three things will make your experience with iPad Peek closer to than of an actual iPad.

  • Use a browser with a WebKit-based rendering engine, preferably Safari, since it is the most similar to the iPad browser. Chrome will works too.
  • Disable all plugins in your browser. Otherwise your browser will still load the plugins even though an iPad would not.
  • Change your user agent string in your browser to match the iPad one listed earlier.

Apple’s Official Developer Documentation

Other Resources


The easiest way to change your user agent in Safari is to use the option in the developer menu. The easiest way to change the user agent in Chrome and Firefox (uses the Gecko rendering engine, not WebKit) is to use an extension.

Further Reading

John Gruber at Daring Fireball has written a series of posts about Flash, HTML5, and H.264 video. They are really worth reading for background on the technical and political issues related to HTML5.

* This article originally appeared as Preparing Your Site for the iPad in my Messaging News “On Message Column.”

Why Does My Text Look Funny? Adventures in Character Set Encodings

Character set encoding

Character encoding is the low-level representation of the letters, numbers, and symbols we see in our daily interactions with computers. Common encodings for documents in English are ISO-8859-1 (a superset of ASCII), UTF-8 (an 8 bit Unicode character encoding), and Windows-1252. There are a great number of character set encodings in use and a long and complicated history of how they came to be. This complexity often leads to problems. Typically, these problems are caused when the document is encoded with one encoding, but is interpreted as another.

If you don’t ever have to deal with character encoding issues, then consider yourself fortunate, as it can be a royal pain to decipher and correct large numbers of character encoding issues.

Why you might care

It is likely that you see character set encoding problems all the time. If you have ever opened an email, a web page, or document and some of the letters looked wrong then there this is a good chance this is due to a character set encoding mismatch. You are mostly likely to notice problems with curly quotes, bullets, and accented characters. If you are interested in learning more, there are some excellent sources at the end of this article.

Just to illustrate the extent of the problem–A composite approach to language/encoding detection](http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html) is the original research paper by the Netscape employees who wrote the character detection algorithm that is still used in Firefox. The page is encoded as ISO-8859-1, but the meta tags in the page are set to UTF-8. In most browsers, you should see the resulting funny looking characters due to the character encoding mismatch. Email can have character set encoding problems as well. RFC 2047 defines MIME extensions for non-ASCII text and HTML email has the same problems as web pages.

The best tools I have found are primarily open source command line-based utilities. Specialized GUIs are hard to come by although as I will describe a browser and some text editors will work for many basic tasks. I only tested the command line tools under Mac OS X, Linux, and FreeBSD Unix variants, although most can be compiled under Windows with Cygwin or similar systems. Some of the tools are available as pre-compiled Windows binaries.

Detecting character set encodings

The absolute quickest way to check to see if you have a character encoding problem is to open the web page or file in Firefox and go to the Character Encodings option under the view menu. You can experiment by changing to a different character encoding and see if your document displays correctly.

If you are unsure of which character set your document is encoded in then that is a good place to start. I would first try the file command. It is a standard utility in every modern Unix system I have used. The program attempts to determine many characteristics about the file including types of line ending and the text encoding of the file.

If you need more sophisticated tests for character encoding than the file command offers, then chardet, the Universal Encoding Detector, is your most sophisticated option. The software is a Python port of the code from Mozilla/Firefox code base that includes multiple character encoding auto-detection mechanisms. The most recent version now has a limited command line interface. Previously, it was only accessible to developers willing to wrap their own code around the library. rchardet is a Ruby variant.

Converting between character set encodings

It is possible to use a text editor many character encoding conversions, if you know or can guess the original encoding. Simply open your text file in your favorite editor such as the built in TextEdit or TextMate on(Mac OS X, TextPad or the E – TextEditor on Windows, Yudit on Unix systems with X-Windows, and GNU Emacs on most systems. Then simply select a different encoding in the editor and re-save the file.

Uni2ascii can perform both ends of the conversion between UTF-8 and a large number of encodings and formats including many ASCII variants, quoted printable, HTML, XML, and escapes for POSIX and many programming languages. I like many options to decompose UTF-8 into other encodings. The -B flag creates best effort ASCII by decomposing UTF-8 characters into a reasonable plain ASCII alternative. For example, the copyright symbol becomes (C). In my experiments, there were minor problems where the following characters were not converted middle dot (0x00B7/U+00B7), next line (0x0085/U+0085), and line separator (0x2028/U+2028). Aside from these the program did a tremendous job.

iconv/libiconv is the standard for character set conversion. The application needs to be used as a filter so it can be less convenient if you would prefer to operate on files directly.

I have used GNU Recode for a number of projects. Recode relies on libiconv and can process files directly. The release version of Recode has not been updated in many years, however it is under active development and a recent beta of Recode can be found on the author’s site.

convmv converts the character encoding of filenames (not the contents of the files) and can work on entire directories of files.

The Commetdocs service (formerly known as the iconv.com) allows you convert between many character sets and files types. The service is currently free.

I have not tried either extensively, but Enca the “Extremely Naive Charset Analyzer” and UTRAC the “Universal Text Recognizer and Converter” both provide extensive support for conversion between non-Western character encodings.


Example – Convert files to UTF-8:

iconv -f original_charset -t utf-8 oldfile.txt > newfile.txt

recode UTF-8 file.txt

Example – Convert UTF-8 into readable 7-bit ASCII. The -B option is equivalent to the flag combination -cdefx.

uni2ascii -B file.txt

find . -type f -exec recode utf8..ascii {} ;

Example: use convmv to convert the filenames of a directory of files from IS0-8895-1 to UTF-8. The –notest flag is a dry run feature that can be very useful for testing.

convmv -f iso-8859-1 -t utf8 --notest  directory/

The Future

In general, I recommend that people use the UTF-8 for all new documents. UTF-8 is capable of representing the vast majority of alphabets and is a mature internationally accepted standard. More than a year ago, Google found that the majority of the pages on the web used UTF-8 character encoding.


If you want to learn more about character encoding, the following sources are good places to start.

* This article originally appeared as Why Does My Text Look Funny? Character Set Encoding Detection and Conversion in my Messaging News “On Message Column.”

Simple Package Tracking with TrackMyShipments

The web-based interfaces offered by the shipping services allow you to schedule shipments, manage billing, store addresses, and track packages online. Some third-party services offer simplified interfaces and allow you to track shipments from multiple shipping carriers at once. Still, the process of entering multiple tracking numbers into multiple services can be cumbersome. I prefer the email-based input method used by the TrackMyShipments service.

TrackMyShipments is an email-based online package tracking service I used for more than year and half to as a streamlined method to track packages. TrackMyShipments takes advantage of the fact that you already have the tracking numbers sent to you in email. I wrote about another email based interface in my review of how TripIt Shows the Value of Combining Email, Web and APIs. The signup process is very quick. After registration, you simply forward an email messages with tracking numbers to track@trackmyshipments.com and the service will send you a notification when the shipping status of you package changes.

Say you want to see when the new hard disk you ordered will arrive so that you could finally get around to your New Years resolution to make regular backups. The most common way to find out the status of your package is to search through your email to find the confirmation email from the store that has the tracking number for your drive. If you are lucky the store has formatted the message so you can simply click on a link and it will take you directly to the page on the shippers site that has information about the state of your package.

Unfortunately, many stores do not give their customers such an easy path and so must copy the number from the email and paste it into the web form for your package carrier. You might even already have an account on the package carriers web site that lets you save the number for future reference or set up email or SMS alerts to let you know when there is progress or problems. So you sign into the service and paste in the tracking number you found. This somewhat cumbersome process is the norm.

TrackMyShipments has a few options to configure the level of detail about the status of the shipment. If you choose, the service will notify you about every hop the package takes along the route, but in my experience this is far too much information. I configure the service to notify me on the day of delivery and for any exceptions. This means I get notified that the package is out for delivery and when it is delivered or if there are any problems with the delivery. All of the package carriers have pretty significant lag in their delivery status information and TrackMyShipments can not give you any more information than the carriers have, it’s just more convenient.

The TrackMyShipments iPhone and iPod Touch application allows mobile users to see the current status of all packages tracked and the ability to remove any packages from tracking. Previously the service offered both free and paid versions of iPhone application. TrackMyShipments for the iPhone is now free and advertising supported with iAds. The application includes push notifications, unlimited shipments and the ability to associate users, which were previously paid add ons. The iPhone application works with both free and pro accounts.

Overall, I find TrackMyShipments is the most convenient way to track packages online. The service is simple to use and in my experience it just works. While neither the TrackMyShipments web site nor the iPhone application will win any design awards, there is little reason to use either unless you want an overview of all shipments at once. TrackMyShipments supports tracking DHL, FedEx, UPS, and US Postal Service packages. The basic TrackMyShipments service is free for tracking up to 10 shipments at a time. You will receive email updates about that status of your package or you can log on to TrackMyShipments to see the status and location of all of your shipments. TrackMyShipments Pro costs $20 a year and gives you the ability to track unlimited packages and receive notifications about the shipping status via SMS. I suspect most people will find the basic more than adequate, although those with greater package tracking needs will find the pro service a bargain.

* A version of this article originally appeared as TrackMyShipments Offers Simple Email-Based Package Tracking in my Messaging News “On Message Column.” Revisions and iPhone application updates on September 13, 2010.

You should follow me on Twitter.

Validating Email Address in Web Forms – The Hazards of Complexity

Validating data in web forms reduces the likelihood of inadvertent submission of data that is incorrectly formatted, inconsistent, or incomplete. It is often useful to validate email addresses, especially if the addresses are going to be used for receipts or other types of follow up. Validation (and basic bounds checking) can also reduce the chance that email address field could be used as an attack vector.

It is important to note that email addresses can be significantly more complicated than commonly thought. This means that it is important to consult the most current RFCs for email standards and ICANN announcements for new types of Top Level Domain names otherwise valid email addresses may be blocked. For example, the plus character is a valid within the local portion of an email address. The plus is typically used as an optional feature for sub-addressing and is supported in many mail servers, Cyrus IMAP installations, and in Gmail. However, the plus sign is frequently rejected as invalid by many web forms.

Unless there is a specific need for sophisticated email address validation, I recommend that sites limited themselves to very basic validation such as simply checking for an @ sign and possibly characters to either side of it. When sophisticated validation is used, it is important to test the algorithm and make sure it is kept up to date. This Stack Overflow thread, How far should one take e-mail address validation?, details many of the problems with being too clever when validating addresses. There will always be users who purposefully submit incorrect data and while this can be limited somewhat by validation, simply sending a verification email is a far more effective method.

Dave Child’s early posts from 2004, Email Address Validation and Email Address Validation Updated, laid out many of the complexities of more sophisticated email address validation. The comments to the posts brought up edge cases where the script resulted in both false positives and false negatives. Child has continued to revise the script and it is available as a Google Code project php-email-address-validation.

Douglas Lovell’s 2007 Linux Journal article Validate an E-Mail Address with PHP, the Right Way attempted to present and even more complex email validation algorithm along with detailed notes on the requirements relating to the various updated RFCs. The comments to this article also bring up many edge cases, which demonstrate the complexity of accurately validating email addresses. Jochen Topf’s articles, the Anatomy of a Mail Address and Characters in the local part of a mail address, are good introductions to the problem as well.

Dominic Sayers wrote a series of posts that iterated on a further refined algorithm that resulted in the RFC-compliant email address validator. Sayers also produced a set of unit tests with a large collection of email addresses in order to compare his own algorithm against others. His PHP code is regularly updated and is also available on Google Code. Cal Henderson (formerly of Flickr) wrote his own RFC (2)822 & 3696 Email Address Parser in PHP, which also passes 100% of Sayers Unit tests.

The examples above are all in PHP. Unfortunately, I could not find a client-side only validation option in JavaScript that was anywhere near as complete as an of the PHP examples. Hopefully, someone will write one or port one of the PHP versions to JavaScript. Les Hazlewood released a Java-based application for Email Validation using Regular Expressions (the Right Way) and Casey Connor of Boxbe updated Hazlewood’s EmailAddress.java code.

The chapter on inline validation from Luke Wroblewski’s excellent book Web Form Design: Filling in the Blanks describes how inline validation can improve the usability of web forms. He suggests that users should receive immediate feedback on whether or not a given input will be accepted as well as suggestions for correcting invalid input. His blog post Web Form Design: Boingo shows a real world example where inline validation would improve the user experience for a registration form. A recent report Web forms design guidelines: an eyetracking study from cxpartners’ Chui Chui Tan provides even more suggestions on how to best handle inline validation.

In this article, I primarily discuss server-side validation, rather than validation by SMTP commands such as looking for 250 and 550 SMTP response codes as presented in How to check if an email address exists without sending an email?. If the email address is to be used in a mailing list I recommend that systems send an email with a URL that must be clicked for verification so that the address qualifies as double opt-in for compliance with CAN SPAM and most major Email Service Provider requirements.

* This article originally appeared as Validating Email Address in Web Forms – The Hazards of Complexity in my Messaging News “On Message Column.”

Using a Plus to Simplify International Dialing

When I travel out of the country, I usually test out new VoIP services both for calling back to the states and receiving calls while I am traveling. I consistently find that while the rates for VoIP services are very attractive, the user experience and flexibility is often lacking when I traveling, particularly with limited network connectivity.

Depending on the length of my stay, I purchase a prepaid SIM to use in a spare unlocked mobile phone so that I can make and receive local calls at local rates. Since most countries outside of the US offer free inbound calls, having a local SIM is even more attractive, although navigating voicemail prompts in another language can be challenging. I often use one of the VoIP services to forward a my United States phone number to my international cell phone number so that I can let friends and family reach me without incurring international charges on their part. Rates for calling international mobile phones range from $0.15 to $0.30 a minute, so be careful who you give your forwarded number to if you try this method.

Calling from multiple devices and multiple services is where plus dialing standard becomes important. People who make regular calls overseas from a mobile phone or a VoIP service will likely be acquainted plus dialing. However, I find that there is often confusion about what plus dialing is and how it works from people who only dial international numbers using landline phones. For those interested in the details, the official specification dialing using the international prefix symbol (commonly known as a plus) is the ITU specification E.123 : Notation for national and international telephone numbers, e-mail addresses and Web addresses

If you want to dial a phone number in another country using a standard landline phone, you need to dial extra digits. Let’s pick an imaginary number in the Netherlands 011 31 20 00012345 as an example. The breakdown for this number follows: The international dialing prefix is 011 for the US, The country code for the Netherlands is 31, the city code for Amsterdam is 20, and the remainder is the local number. We are now used to dialing ten digits for long distance call in the US. For example, (415) 555 1212, which corresponds with the country code, area code, local prefix, and last four digits.

The problem is that the number you call depends on what country you are calling from. Dialing the same number in the Amsterdam from Brussels requires a slightly different number 00 31 20 00012345. Also, in many countries there is a local digit added to the numbers for in country dialing, so the number might look like this 31 0 20 00012345.

Plus dialing is a more straightforward option for mobile phones, VoIP phones, and newer business phones. With plus dialing the phone network can assume that the number is a complete international number and treats the same way no matter what country you are in when you dial the number. The one constraint is that your phone must me able to dial a plus.

Under the new system, you dial a plus, the country code, city code, and then the local number. For example, +31 20 00012345 as opposed to 011 31 20 00012345 or 31 0 20 00012345. Dialing a phone number in the United States would take the form of +1 415 555 1212. The nice thing is that once you have your numbers in plus dialing format you don’t have to worry about variations when you travel. You can dial the same number from your cell phone and a VoIP call such as from Skype. Most cell phones can dial using the plus symbol, although the correct key combination is not always obvious. Most landline phones cannot dial a plus.

I internationalized my entire address book, so the number will be correct independent of my current location. Since I synchronize my address book with my mobile phone and my VoIP soft phone address book, I only need to store and use one form of the phone number.

* This article originally appeared as Using a Plus to Simplify International Dialing in my Messaging News “On Message Column.”

A Better Way to Share Links in Email

I regularly share links with friends and colleagues. I use several social bookmarking services, but the vast majority I share via email. Firefox, Safari, and Internet Explorer have a function to create a new message with an email link. The main disadvantage of sending links using the built-in browser methods is that the links they generate are prone to breaking unless the whole message is converted to HTML rather than plain text.

I prefer a bookmarklet for emailing links. Bookmarklets are bits of JavaScript saved into a bookmark that act like simple browser plugins. BetterExplained’s How To Make a Bookmarklet For Your Web Application is an excellent introduction. Opera and Google Chrome browsers do not currently offer functionality for emailing a link, so a bookmarklet is your only option. My version of the bookmarklet has a few improvements over the built-in browser methods, extensions, and other similar bookmarklets I’ve seen. Previously, I constantly made minor adjustments when emailing URLs to reduce the chances that the link would work correctly for the recipient. The new version of the bookmarklet has improved my workflow for sharing links.

The bookmarklet:

    javascript: function trim12(str)
        var str=str.replace(/^ss*/,''),
        return str.slice(0,i+1);

To use this bookmarklet yourself, simply drag the following link to the bookmark bar in your browser. [Email link]

The most common way that browsers implement the email a link feature is to create a new email message with the document title in the subject and the link in the body. Most email clients will recognize these links as URLs and turn them into clickable links that you can open in your browser. There are several problems with the built in methods for emailing links.

  • The first problem is that the algorithms that recognize the URLs do not always recognize valid URLs or may only partially recognize the URL leaving the recipient with a broken link. In these cases you have to copy the URL from your email client and paste it into your browser. Certainly not a difficult or time-consuming task, but every bit of additional effort reduces that chance that the recipient will click on the link
  • The second problem is that if a URL is long the email client may wrap the line with the URL. There are several ways that email clients wrap long lines and conflicts arise periodically between different email client line wrapping styles. When links with URLs are wrapped this creates additional complexity for the receiving email client to correctly parse the URL and increases the chance of failure. Dan’s Mail Format Site has a nice description of the URL wrapping issues in his Body: Line Length article.
  • The third problem is that some web pages include additional whitespace in the document title to visually offset the text. The extra whitespace is not desirable when mailing the link as it can make the title line less readable and add unwanted blank lines before the URL.

Safari, works around the first two problems by creating a rich text/HTML email message and turns the URL into a real hypertext link. Safari and Internet Explorer both have the option to email an entire webpage. In this case, the browser simply copies the HTML from the webpage into an email message. This method solves the problems of URLs being incorrectly recognized or poorly wrapped, as the recipient’s email client does not need to parse the text to discover the link. I generally avoid emailing entire web pages as most mail clients parse a very restrictive subset of HTML to reduce security vulnerabilities and the page formatting may not appear as intended.

Most bookmarklets I have seen to email a link mirror the built-in browser functionality and thus have the same problems. Bookmarklets often have an additional problem where ampersands and other meta characters are not properly escaped and may cause the bookmarklet to fail or truncate the titles.

My modified bookmarklet presented above includes the document title in subject and the document title, a carriage return, and the URL in the body. The URL is wrapped in angle brackets < >, which most allows the majority of email clients to correctly interpret complex and wrapped URLs. The World Wide Web Consortium (W3C) recommends this method in Wrappers for URIs in plain text as does RFC2396 Uniform Resource Identifiers (URI): Generic Syntax, the specification that defines the modern URL. Finally, I use the JavaScript function encodeURIComponent to protect page titles from ampersands and other characters breaking them. The n version of the escaped newline character must be used as opposed to %0A. In later revisions of my bookmarklet, I included trim12 from Steven Levithan’s Faster JavaScript Trim, which will trim excess whitespace from the document title so that the subject line is not surrounded by spaces and there are no blank lines before the URL.

The bookmarklet relies on the browser’s default mailto: handler to determine which email application should be used to creat the message with the link. Each browser and operating system has multiple ways to set the mailto: handler. Changing the default mailto: handler is generally straightforward for most desktop email clients, but can be more complicated for webmail clients such as Gmail and Yahoo! Mail. Here are a few options to start with. Changes in Firefox 3.5 greatly simplified the procedure for changing the e-mail program used by Firefox. The Affixa application is a simple way to change the mailto: handler for Windows users to set popular desktop clients, Gmail and Yahoo! Mail as the default handler. The Internet Options settings for Internet Explorer make it straightforward to change the default to desktop another application or Windows Live Hotmail. Clint Talbert’s page Click Testing for Protocol Handling is a quick and easy way to test if your mailto: handler is working correctly.

Updated August 2nd, 2009 to add trim code and including information on mailto: protocol handlers. Fixed problem with CMS HTML corrector breaking bookmarklet code.

* This article originally appeared as A Better Way to Share Links in Email in my Messaging News “On Message Column.”