The value of over-the-counter service

My primary computer is a dual 2GHz PowerMac G5 until I can upgrade it with a Nehalem Mac Pro, most likely around the end of the year or early next year. I bought it in 2004, along with a 23" Apple Cinema HD (the old pinstripe plastic bezel kind with an ADC connector). Unfortunately, about a year ago the CCFL backlight on the monitor started turning pink from old age, and thus unusable in a properly color-managed photographic workflow.

I used that as an excuse to splurge on a humongous (and agoraphobia-inducing) HP LP3065 30 inch LCD monitor after reading the glowing reviews. The two features that sold me were the enhanced color gamut (the only way to improve that would be to get a $6000 Samsung XL30, something I am not quite prepared to do), and the fact it has 3 built-in DVI ports, so it can easily be shared by multiple computers (assuming they support dual-link DVI, which unfortunately my basic spec Sun Ultra 40 M2 does not). The fact it was 25% less expensive than the Apple 30" Cinema Display helped, of course.

About 6 months ago, I discovered there was a fine pink vertical line running across the entire height of the monitor, roughly 25 centimeters from the left. Since I primarily use that monitor for photo (the primary monitor for Mail, web browsing or terminals remains the Apple), at first I worried there was a defect with my camera. I managed to reproduce the problem with my MacBook Pro (they have dual-link DVI, unlike lesser laptops), and called HP support (the 3 year HP warranty was also an important consideration when I purchased).

My first support call in November 2007 went well, and the tech told me I would be contacted to arrange for an on-site exchange. This is a seriously heavy monitor and I did not relish the idea of lugging it back to FedEx, so getting premium support for a business-class monitor sounded an attractive proposition. Unfortunately, they never did call back, and as I had other pressing matters to attend to involving international travel, I just put it out of my mind (it is a very subtle flaw that is not even always visible).

I only got around to calling them back a few weeks ago. Unlike in November, I was given the run-around with various customer service reps in India until I was finally routed to a pleasant (and competent) tech in a suburb of Vancouver (the US dollar going in the direction it is, you have to wonder how much longer before HP outsources those call centers back to the US). The problem is not with Indian call centers, in any case, all but one of the CSRs were very polite (I suspect Indians learn more patience as they grow up than pampered Americans or Europeans would). The problem is poorly organized support processes and asinine scripts they are required to go through if they want to keep their jobs. In any case, the Canadian rep managed to find the FRU number and also told me someone would call to schedule an appointment. Someone did call this time, to let me know the part was back-ordered and they would call me when it becomes available.

This morning, as I was heading for the shower, my intercom buzzed. It was a DHL delivery man with the replacement monitor. I had to open the door to him in my bath robe. Naturally, nobody at HP bothered to notify me and had I left earlier, I would have missed him altogether.

One of the great things about Apple products is that if you live near an Apple store, you can just stop by their pretentiously-named Genius bars and get support for free (though not free repairs for out-of-warranty products, obviously). I now have a fully working HP monitor again, so I suppose I can't complain too loudly, but the Apple monitor with the sterling support looks like the true bargain in hindsight.

17:42 - permalink [IT, Macintosh, Soapbox]

Backing up is hard to do (right)

You can never overstate the importance of backups. Over the last year I have put quite a bit of effort in making sure my data is backed up properly. The purpose of this article is not to describe backup best practices (that is a vast subject, there are other, better resources available on the web, and in any case there is no one-size-fits-all solution). I am just documenting my setup, the requirements that drove it, and possibly give readers some ideas.

The first part in planning for backup is to do an inventory of the assets you are trying to protect. In my case, in order of priority:

• 1.5GB of scans of important documents: birth certificates, diplomas, invoices, legal documents, bank statements, and so on. This data is very sensitive, and should be encrypted.
• 150GB of digital photos and scans
• My address book, which lives on my laptop
• My source code repositories
• My personal email, approximately .75GB
• The contents of this website, about 5GB
• 190GB of music (lossless rips of my CD collection)
• My Temboz article database

Thus the total storage capacity required for a full backup is reaching the 400GB mark. This in itself precludes DVD-R or even tape backup (short of buying an expensive LTO-4 tape drive or an autoloader, that is).

The second step is to devise your threat model. In my case, by decreasing order of likelihood:

1. Human error
2. Hard drive failure
3. Software failure (e.g. filesystem corruption)
4. Silent data loss or corruption, e.g a defective disk
5. Theft
6. Fire, earthquake, natural disaster, etc.

Third, some general principles I believe in:

• Do not use proprietary backup formats. The best format is plain files on a filesystem identical in structure to the original.
• Do not rely on offline media for backups. The watched pot does not boil over, online data is much less likely to go bad without my noticing until it is too late.
• A backup plan needs to be effortless to be successful. Plugging in external drives when backups are needed, or rotating drives between home and office is something I have tried, but not stuck to.
• Backups should be verified — they should generate positive feedback, so that the absence of feedback can alert to problems
• For all types of data, there should be one and only one reference machine that holds the authoritative copy. Multi-master synchronization and replication is possible using tools like Unison, but is much harder to manage and increases the risk of human error.

With these preliminaries out of the way, here is my system:

• My primary backups reside on my home server, a Sun Ultra 40 M2 workstation, running Solaris 10. This machine is very quiet, so I can keep it running in the room next to my bedroom without disturbing my sleep. It is also relatively power-efficient at 160W with seven hard drives.
• One of the seven drives is the 160GB boot drive, and the other six are 750GB Seagate drives configured in a 3TB ZFS RAID-Z2 storage pool.
• With large SATA drives, reconstruction after a drive failure is long and the risk of another drive failing due to the stress of rebuilding is not negligible. RAID-Z2 can tolerate two drives failing, unlike RAID 5 which can only tolerate a single drive failure. This level of data protection is higher than RAID 1 since RAID 1 won't protect you if two drives that are the mirror of one another fail. You can get the same level of protection in RAID 6 or RAID-DP.
• I have scripts to take ZFS snapshots daily. The daily snapshots are kept for the current month, then I keep only monthly snapshots. Snapshots are the primary line of defense against human error.
• Snapshot technology consumes only as much disk space is required to store the differences between the snapshot and current versions of a file, and is much more efficient than schemes like Apple's Time Machine where a single byte change to a mult0gigabyte file like a Parallels virtual disk image will cause the entire file to be duplicated, wasting storage. Because snapshots are taken near instantly and cost almost nothing, they are an extremely powerful feature of a storage subsystem.
• I backup from my various machines to the Sun via rsync over ssh. An incremental backup of my PowerMac G5, which has most of the 400GB in my backup set, takes less than 5 minutes over Gigabit Ethernet, despite the ssh encryption.
• ZFS is probably the best filesystem, bar none, but it is not perfect, as demonstrated by the Joyent outage and you still need another copy for backup in case of ZFS corruption.
• Every night at 2AM a cron job on my old home server (2x400GB, ZFS RAID 0), that I now I keep at work, pulls updates from the Sun using rsync over ssh (the company firewall won't let me push updates to it from the Sun). Another cron job at 8AM kills any leftover rsync processes, e.g. if there are more data changes to transfer than fit in the 1-2 GB that can be transferred in 6 hours over my relatively pokey 320-512kbps DSL uplink (no thanks to AT&T's benighted refusal to upgrade its tired infrastructure).
• A last line of defense is to back up my server at work to a D-Link DNS-323 NAS box using rsync over NFS. This cute little unit holds two Western Digital Green Power 1TB drives in RAID 1, which slide right in, no tools required. It consumes next to no power or desk space. Since it runs Linux and is easy to extend using fun-plug, I could conceivably run the cron and rsync from there. As a bonus, the built-in mt-daapd server streams my entire music collection to iTunes over the LAN so I can listen to any of my CDs at work.
• It can take a few days for this data bucket brigade to catch up with a particularly intense photo shoot, but it will eventually and is never too far behind. This provides me with near continuous data protection and disaster recovery.

16:45 - permalink [Macintosh]

Push recruiting

As I was debugging why feedparser is mangling the GigaOM feed titles, I found this easter egg on the Wordpress hosted site:

zephyr ~>telnet gigaom.com 80
Trying 72.232.101.40...
Connected to gigaom.com.
Escape character is '^]'.
GET /feed HTTP/1.0
Host: gigaom.com

HTTP/1.0 301 Moved Permanently
Vary: Cookie
X-hacker: If you're reading this, you should visit automattic.com/jobs and
apply to join the fun, mention this header.
Location: http://feeds.feedburner.com/ommalik
Content-type: text/html; charset=utf-8
Content-Length: 0
Date: Thu, 20 Mar 2008 23:36:17 GMT
Server: LiteSpeed
Connection: close

Connection closed by foreign host.

Knowing how to issue HTTP requests by hand is one of my litmus tests for a web developer, but I had never thought of using it in this creative way as a recruiting tool...

16:33 - permalink [IT]

Macworld 2008 round-up

The MacBook Air was what I was waiting for (I pre-ordered the SSD version just before the online Apple Store buckled under the load). I have a MacBook Pro 15", and because of its weight I end up leaving it at work and not carry it with me at all times (the MacBook is hardly any lighter). Sure, the Air has drastically limited connectivity (the lack of Gigabit Ethernet is probably what I will regret most, even though I clocked my Airport Extreme at 90 true Mbps throughput). Other minuses include the glossy screen (instead of an anti-reflective one), the MacBook-like chiclet keyboard rather than the much nicer MacBook Pro keyboard), or the sealed non user-replaceable battery.

I suspect people deriding it are people whose main machine is a laptop. My main machine is a tower desktop, and no laptop is ever going to compete in terms of capacity and expandability. The drive on the laptop is merely a cache for the desktop where the real data lives. The compromises the Air makes are acceptable ones in exchange for a machine that is light enough for me to carry all the time. I was considering getting an Asus Eee PC prior to the show, and the MacBook Air is a vastly more capable and polyvalent machine.

Apart from that, the show was a relatively quiet one with few truly noteworthy new products. Here are the main highlights:

• Matias did not have the Tactilepro 2.0 keyboard on display. I love mine (a version 1 with the ALPS keyswitch) and would like to get a spare, but apparently they have parted ways with the manufacturer of the new Matias-designed keyswitches and are working on a 3.0 version for later this year.
• Fujitsu were demonstrating an ultra-small, bus-powered document scanner, the S300M. Unfortunately, once again for reasons due to licensing of the bundled software, they could not release a single SKU that would work with both PCs and Macs.
• The German company Project Wizards was demonstrating Merlin, a project management program similar to Microsoft Project. The scheduling and load-leveling algorithms look at least as capable as Project 2000, and they told me the next version will allow team members to report on task advancement by simply contacting a built-in web server. Looks like a promising product.
• Samsung showed the CLP-300 which they bill the world's smallest color laser printer. Indeed it looks roughly the same size as my monochrome HP LaserJet 1320, and much smaller than my bulky HP 2605dn, that's quite an achievement. I am wary of Samsung lasers since buying the CLP-500 for Kefta a few years back. The print quality was fine, but it was ludicrously slow, taking something like 5 minutes per color page to print. The CLP-300 seems reasonably fast, faster than the 2605dn at any rate.
• Samsung was also showing off the gorgeous XL30 30" LED-backlit LCD monitor. LED backlight is more environmentally friendly, does not shift colors as it ages unlike a TFT backlight, and gives a wider color gamut. Unfortunately, its price is a princely "between $6000 and $7000".
• Microsoft was showing off Office 2008, emphasizing ease of use and productivity rather than features for features' sake for a change.
  
  
  
  They even set up a bloggers-only salon to curry favor, complete with Internet cafe and snacks.
• I tried Nikon's humongous AF-S VR Nikkor 200mm f/2G IF-ED lens. Very heavy but impressive piece of gear.
• Canon was showing off the new Flash-based HD camcorders they introduced at CES. They are not that much smaller than the HDV ones. The HV30 replaces the excellent HV20, but the only real improvements are 1080p30 mode and an articulating LCD.

22:29 - permalink [Macintosh]

A San Francisco local's advice to Macworld attendees

I have been living and working in downtown San Francisco for almost eight years now. Until a month ago, my office window (right) used to overlook Third Street and the Moscone center. San Francisco is a popular convention destination (one wonders why proctologists seem to prefer it to, say, Detroit) but Macworld Expo is definitely the biggest show in town. Restaurants and hotels are taken by storm, taxis become scarce, traffic gets even snarlier and the lines at Metron eateries cross the threshold of ludicrousness. So here are a few tips for Macworld attendees to have a better time and not caught in tourist traps.

Transportation

Driving in San Francisco is a non-starter. Traffic is horrendous, parking is scarce and you would lose far too much time just getting around. SF Muni is a pretty good public transport system (at least by admittedly paltry US standards) and their 1, 3 or 7 day Passport passes are good value.

Cars are mostly useless inside the city, but nice if you want to drive to make a Fry's run or a day trip to Marin across the Golden Gate. If you must drive, the friendly folks at Reliable Rent-a-Car will give you decent rates on Toyotas. Until I bought a car last month, they were my go-to place for when I needed a car.

Lunch

San Francisco has the best food in the United States, but you wouldn't know if from the overpriced eateries in a three block radius. The Firewood Cafe and Buckhorn Grill in the Metron are actually reasonably decent, but the throngs of convention-goers mean long lines. Mo's Grille has excellent burgers (I recommend the aptly named "Belly Buster"), and since access to it is a little tortuous, you have a fighting chance (it is literally just above the Moscone South).

Ranging a little further, Nova has decent burgers and a lovely lobster quesadilla, and the new Westfield Mall three blocks to the west has a decent food court. Some good local chains are Bistro Burger, S.F. Soup Co. or Café Madeleine (official birthday cake purveyor to Kefta).

That said, the best lunch experience is to take the historic F line streetcar to the Ferry Building Marketplace with its wide variety of gourmet food stores and eateries. I heartily recommend the clam chowder at Ferry Plaza Seafood (it used to be my Friday lunch of choice) or the eclectic fare at Boulette's Larder. Chocolates from Michael Recchiuti or fresh-pressed olive oil from Stonehouse make for great (and edible) souvenirs.

Staying hydrated is important when you expect to spend an entire day on the show floor. There is a Whole Foods store a mere block away where you can buy any required provisions.

Dining

Dining in San Francisco is an embarrassment of riches, it would be a shame to settle for overpriced hotel food. A word to the wise: most of the better places are hooked into the OpenTable reservation system which makes finding a good place with availability a much less hit-and-miss affair. This year Macworld coincides with the annual Dine About Town event where participating restaurants will offer specially discounted menus.

Equipment

Murphy's law will strike at the worst possible moment. If you need help with your Mac, the geniuses at the San Francisco Apple Store (or the smaller Chestnu St or Stonestown locations) can help. It's also good to keep in mind the Apple stores all offer free WiFi connectivity.

If you need commodity spare parts like a USB hub in a hurry, Central Computers is a mere block away and carries a wide assortment, albeit PC-centric.

If you are an attendee and have questions I have not answered, please feel free to email me, my contact info is at the right.

18:38 - permalink [Macintosh, San Francisco]

She said Yes

I flew into London today. I took the adorable Shaheen B. to dinner at The Ledbury.

Between courses, I asked her to marry me. She said Yes. The only plausible explanation for this lapse in judgement is temporary insanity. I couldn't be happier.

P.S. the ring is guaranteed to be De Beers-free and not a blood diamond, courtesy of the nice folks at Brilliant Earth, who by the greatest of coincidences are just across the street from my office.

19:00 - permalink [About Me]

iPhone first impressions

I thought I would escape the frenzy of iPhone hype by filtering out any mentions of it from my feed reader. In fact, I was quite resentful of the way the iPhone launch pushed out the release of OS X 10.5 Leopard to October 2007. On my way to my cousin's wedding on Friday June 29th, I passed by the San Francisco Apple Store and saw the line. It was actually fairly tame, as it only went halfway around the block (when the store originally opened, the line went all the way around and spilled over into Market Street).

Of course, when I came back, I had to see one. One of the petty annoyances with my Nokia E62 was how it would take several seconds for the address book to load. The iPhone, despite having a much more computationally intensive user interface, still manages to have lightning-quick responsiveness to user input. That itself convinced me to buy one.

The iPhone mostly meets or even exceeds the hype. The user interface is exceptionally good, let alone for a version 1.0 product. Some quick notes from a Nokia E62 switcher (my previous phone was also using Cingular/AT&T):

• Email and web are very snappy. The SSL implementation on the E62 would take forever to negotiate with my home IMAP server (as in several minutes), the iPhone's is instant. The E62's web browser, despite being based on the same WebKit code base as Apple's Safari, could not run two concurrent AJAX XMLHttpRequest concurrently, Safari has no such problems.
• The battery life is very short, well under 2 days, and it takes a long time to fully charge.
• The glass screen does not scratch, but it does show fingerprints and smudges.
• The virtual keyboard is surprisingly effective. This was the single biggest area where I thought it would fall short, but it actually performs far better than the E62's chiclet keys. Part of the reason is that the E62's keys actually wobble when you press them, which doesn't make for precise typing, and they are so tiny anyway that it's hard to type accurately without pressing other keys in the process. The iPhone's magnification effect as well as the fact you can slide your finger to correct a misregistered virtual keypress, makes for much faster typing. The predictive text engine is also far superior to schemes like Symbian's, or T9. T9 is unbearably annoying in the same vein as Microsoft Word's noxious autocorrect functionality or Clippy, I always disable T9 on any phone that has it, the iPhone's system is unobtrusive and eminently usable in comparison.
• The sound quality on the iPhone is not at the same level as the E62, specially for the speakerphone.
• No voice recorder. A rather silly omission.
• The calendar does not support To-Do list items from iCal. This is ridiculous.
• You cannot use iTunes music files as either the ring tone or alarm sound. This was probably to appease AT&T and the RIAA, who seem to believe they have a divine right to make you pay over again and again for the same music. Even if I were prepared to accept their racketeering and pay the obscenely expensive charge for a ring tone, I seriously doubt they would have what I used for mine on the E62, the finale theme from Sibelius' Kullervo op. 9.
• The recessed phone jack is incompatible with most earphones like my ER-4P, but it works just fine with B&O A8, whose jack is actually a fairly thin molded connector wrapped in a rubber jacket that easily slips off to accommodate the iPhone jack.
• Safari has no option to remember passwords for you, unlike the desktop version, and it does not recognize the standard http://login:password@site/ convention either, which makes logging onto Temboz harder than it has to be.
• The Bluetooth functionality in the iPhone is pretty minimal, limited to using Bluetooth wireless headsets and not much more. You cannot beam business cards or photos. Unlike the E62, I cannot use it as a modem for either my MacBook Pro or my Nokia N800. Since there is no SSH client on the iPhone, this could bite me when I need emergency access. Then again, the $20 unlimited data plan for iPhones is half the price of my previous $39.99 unlimited data plan.
• Not supporting Java or Flash is a feature, not a bug.
• The camera, as could be expected, is mediocre. We all know the only purpose is to snap facsimiles of notes, billboards, flyers. whiteboards and the like.
• The calculator is minimal. It does not support RPN and does not have either scientific or financial capabilities.
• You can specify 24-hour time format, but there is no way to specify ISO date format.
• The iPhone seems incompatible with my SendStation PocketDock Line Out USB, and thus I cannot connect it to my Ray Samuels Hornet pocket headphone amplifier and full-size Sennheiser and AKG headphones. It is also incompatible with Apple's own universal AV dock, and displays a warning message telling you so. Then again, since it is a GSM phone, the annoying pulsating buzz induced by GSM would make such an arrangement impractical.

Update (2007-07-13):

It must be the Friday 13th effect at work... My iPhone seems to have developed a defective proximity sensor. The phone works as a speakerphone, but no longer turns the headset speaker on when I bring it to my ear. Resetting and even restoring the phone does not help, it's probably a hardware issue.

Fortunately, the SF Apple Store Genius Bar let me in this evening without an appointment, and swapped it for a new one. This was the first time they had seen this particular problem, and they told me Apple's policy for the first month is to do full replacements and collect field failures for analysis. The repair process afterwards seems to be still up in the air. I would recommend they have swap or loaner units on hand, as people are less likely to tolerate not having a phone than not being able to listen to music for a week.

12:47 - permalink [Macintosh]

MacBook Pro 3G first impressions

I upgraded my MacBook Pro to the third-generation model so I can bump up my RAM to 4GB. Aperture and CS3 are very resource-intensive, and the 2GB upper limit of my first-generation MBP was somewhat constraining.

I just transferred my files over using Apple's migration utility and target firewire mode. The process, while not 100% automated (it did not transfer X11, for instance, or some of the preferences), is far smoother than any Windows equivalent. Here are my first impressions on the new model:

• The new, environmentally friendly mercury-free LED backlight is definitely more blue in tone than the pinkish cold-cathode fluorescent backlight on the old model. The default ColorSync monitor profile does a good job of compensating for this, however. There is some vignetting on the 15" screen (darkening in the corners). I wonder how the 17" model fares, and whether they had to add additional LEDs for a more even backlight illumination.
• This machine is fast. It blows my dual-2GHz G5 PowerMac out of the water in all benchmarks other than disk I/O. Unsurprisingly, it is also much faster than the first-generation machine, specially on graphics but also on disk I/O.
• It does not heat up quite as much as the older Core Duo model, the heat level, while high, never reaches a potentially dangerous temperature. This is probably due to improved power management, as running two Parallels virtual machine will bring it up to the same pant-scorching levels as the Core Duo.

19:35 - permalink [Macintosh]

Adobe "Creative" Suite 3, a mixed bag

I installed Adobe Creative Suite 3 on my home PowerMac and my MacBook (the license allows you to install it on two computers as long as they are not in simultaneous use). The only real reason I upgraded is to get a native Intel version. I have barely started using it already and I already have peeves:

• Bridge looks butt-ugly, is even slower than before and with a more amateurish interface than ever
• The install procedure is incredibly annoying and Windows-like. There is no justification for an install procedure that chokes if the beta was not uninstalled officially (although I have to give some brownie points due to the fact the cleanup script is written in Python).
• The icons are aesthetically bankrupt. What kind of credibility does Adobe think it has with creative people with such an astoundingly lackluster effort?
• Barely installed and already in need of software updates. The widespread availability of fast Internet connections is no excuse for shoddy release management or a "we'll patch it post-release" mentality. Speaking of which, the only proper time to interrupt users with a software update dialog is as they are quitting the application, not by getting in the way of whatever task they are trying to get done by starting up the app.
• Don't clutter my hard drive with legal drivel in twenty different languages. It's called "Creative Suite", not "Boilerplate Suite".
• All the tie-ins to paid add-on services like Adobe Stock Photos or Acrobat Conferencing are incredibly obnoxious, just like those for MSN or .Mac.
• JavaScript in Acrobat is a big security and privacy risk, and should be disabled by default.
• On the plus side, thanks for making a "Design Basic" edition without all the despicable Flash garbage in it. I would actually pay more for the Basic version than for the supposedly premium one infected with Flash and Dreamweaver.

Update (2008-01-01):

It seems Adobe has also crossed a serious ethical line by building in spyware to track on whenever a user starts a CS3 application.

As far as I am concerned, this is the last straw and I will actively start looking for substitutes for Adobe products as soon as I return from my vacation.

Update (2008-01-02):

It seems Adobe does not collect the serial number after all. The apps should nonetheless never call on the Internet except possibly to check for updates. For people like myself who have static IPs, the IP address itself could be used to correlate the analytics with personal information.

21:53 - permalink [IT, Soapbox]

Slava Rostropovich, 1927-2007

Legendary cellist and all-around good guy Mstislav "Slava" Rostropovich passed away in Moscow today. He was a friend and supporter of Shostakovich, Prokofiev, Britten and many others like Dutilleux, and many of the greatest works for cello of the 20th century, indeed of all time, were dedicated to him.

I had the opportunity to hear him conduct Shostakovich's "Babi Yar" a year ago (when I took this photo) and a few years earlier as a cellist the Dvořák Cello Concerto (sadly in replacement of the far superior Shostakovich First Cello Concerto that was dedicated to him).

The world of music has suffered a grievous loss. None of the current generation of cellists (Ma, Gastinel) is of the same caliber. As a conductor, his legacy is more mixed, as his Shostakovich interpretations often lack fire, but his Prokofiev cycle with Erato is marvelous, specially the Fourth in its original version.

03:53 - permalink

Eery similarities

Is it just me or is the Finale of Sibelius' Second Symphony, Op. 43 (1902) eerily reminiscent of the first movement of Saint-Saëns' Third Piano Concerto, Op. 29 (1869)?

13:31 - permalink

Acxiom acquires Kefta

I guess it's official now. Acxiom acquired my company, Kefta, last week. Acxiom is very discreet, but influential company, with a strong technical focus — how many public company CEOs do you know who are listed as inventors on their company's patents? The other founders and myself came to the conclusion a merger will allow us to serve our customers better, ramp up our sales to capitalize on an exploding market and enhance our infrastructure, something that would have beeen much harder if we stayed independent.

Due to confidentiality reasons, I cannot give much more specifics, but Kefta was my first startup. I thought I would have mixed feelings letting it go (I am staying on board, of course, but in a different role now). That has not turned out to be the case, however. We started in 2000, a mere two months before the bottom fell out of the market, and managed not just to survive, but to recover and thrive. At the moment, I am too excited considering the possibilities to wax nostalgic. There might be a few bumps down the road for this blog as I now have to extricate my personal web presence from Kefta's machines (my new hosting platform is a Solaris-powered Joyent accelerator).

Update (2007-05-16):

It seems I was acquired again. Once is good fortune, twice is negligence...

10:00 - permalink [About Me]

Yet another AppleTV article

Ever since my Panasonic PVR died and I switched to an Elgato EyeTV 250 for my PVR needs, I hardly ever use my 32" Sharp Aquos LCD HDTV, and do most of my watching on my Mac's 23" Cinema HD display.

To rectify this, I purchased an AppleTV yesterday at the San Francisco Apple Store, where they are prominently displayed, hooked up to Sony Bravia LCD TVs. While their choice of TV is questionable (remember, Sony is a four-letter word), the demo is effective for those who did not get to see it at MacWorld Expo 2007.

In all likelihood, I will cancel my Comcast cable subscription in a few days. The only TV shows I watch are:

• Battlestar Galactica (iTunes season pass: $34.99)
• South Park (iTunes season pass: $23.99)
• The Simpsons (not available on iTunes yet)
• Family Guy (although the show has become stale and probably on its way out)

I stopped watching live TV seven years ago when I bought my first PVR (a TiVo Series 1). My monthly Comcast bill is $56.20 (basic extended analog cable, no premium channels). Purchasing an iTunes season pass for Galactica and South Park would cost me just slightly more than one month of Comcast's "service". This also means the AppleTV will have paid for itself in less than 6 months (the famous "return on investment" or ROI metric used by IT departments to estimate whether a project is worthwhile or not). The Fox shows I can get over ATSC HDTV because I have an Elgato EyeTV 500 ATSC DTV/HDTV to Firewire tuner (broadcast flag free), and direct line of sight to Sutro Tower, where the San Francisco digital TV over-the-air signals are beamed from.

Of course, the satisfaction of firing the cable company, with its tendency to jack prices up much faster than inflation for ever degrading service, is in itself priceless. As a bonus, the iTunes shows are fully digital, and without ads.

The limiting factor is of course the abysmally slow standard of what passes for broadband in the US. Ironically, I left Europe for California in 2000 because I thought the epicenter of the Internet industry was here, but nowadays the US lags badly behind even formerly dirigiste France in terms of optical broadband and high speed DSL.

Broadband prices are much higher in the US — I pay $70 per month for 2.5 Mbps downstream and 384 kbps upstream, when in France I would get 18 Mbps for half that price (or 70 Mbps for the same price as in the US in the many areas that are getting optical coverage). This is despite the fact my former colleagues at France Telecom face labor costs and Internet transit costs double those of US carriers (the US' central role in terms of connectivity means US carriers can impose peering terms where non-US carriers pay the lion's share of the transoceanic cable costs, even now that Euro or Asian Internet traffic is beginning to eclipse US traffic). The reason for high prices is of course the coddling of the AT&T-Verizon-Comcast oligopoly by a FCC overly influenced by the doctrinaire Chicago School of economics, which refuses to accept even the theoretical possibility of a monopoly...

AppleTV is the second key product in Apple's digital hub strategy, and like the iPod, it is also available to Windows users. Apple did learn from its mistakes in the 1980s, where it lost potential dominance of the desktop PC market to Microsoft by having unrealistically maximalist designs on the market. In some way, this is akin to the virtualization phenomenon shaking corporate IT: like the browser or Parallels, iTunes is another middleware layer that makes the operating system almost irrelevant - Windows users can switch painlessly to the Macintosh, once they realize the elegance and simplicity of the iPod and AppleTV also apply to the Mac and they do not have to settle for the inferior Windows experience.

Now, AppleTV is a semi-closed environment like the iPod. I refuse on principle to buy low-quality, DRM-infested music tracks from the iTunes store. Switching to DRM-infested video tracks from the Apple store is not very consistent. For my defense, I must say:

• Unlike music, video is something you see once and usually never again. Thus, the DRM restrictions are less onerous (still outrageous, but less unacceptably so).
• There is no legal non-DRM alternative, unlike CDs for music.
• Cable companies are really, really evil...

Last but not least, just as you can load your iPod with high-quality, non-DRM music ripped from good old CDs and SACDs, you can load video into iTunes from various sources other than Apple, such as the excellent Elgato EyeTV PVR software, a DVD ripper like Handbrake, podcasts and probably all sorts of other mechanisms in the future (I would be surprised if YouTube did not come out with an AppleTV compatible service soon). Since Apple refuses to license its DRM, that effectively forces other players to use non-DRM video. Who said two wrongs do not make one right?

In any case, I fully expect the AppleTV to be reverse-engineered and alternative operating systems made available for it, just as Rockbox provides FLAC support and gapless playback on the iPod, or how people managed to get Linux running on the original Xbox. Apple is probably not subsidizing the AppleTV the way Microsoft does with its game consoles, so they probably do not have a strong incentive to prevent repurposing with mechanisms like the encrypted boot loader on the Xbox. Less than a week after initial availability, there are already reports of people upgrading the internal hard drive...

Flat-panel HDTVs were the star of the 2006 holiday shopping season, thanks in no small part to free-falling prices. There is now a critical mass of people in the US who are starting to realize just how lousy standard definition TV is, like my friend and colleague Frank who can't bear to watch his TiVo Series 1 any more now that he has a humongous rear-projection 1080p screen, and is mulling building his own MythTV or Freevo box.

The iPod is already a mass-market phenomenon. I believe Apple has a real shot of taking a huge chunk of the cable companies' business away from them. Hollywood will be cheering, because Steve Jobs is one of them, and they can make much more profit from iTunes Store sales than from the crumbs the cable distribution monopolies grant them. Of course, there will be collateral damage like TiVo (not that I would particularly mind), and possibly NetFlix. Presumably Microsoft will do the same by adding equivalent functionality to the Xbox 360. Sony will try, but will fail utterly because of its insistence on polluting everything with proprietary yet unusable pseudo-standards and unredeemably horrid software. All in all, the television industry is in for some mighty interesting times.

Update (2007-03-24 10AM):

I have just cancelled my Comcast subscription. The guy handling the cancellation was actually very friendly, and we talked a little about South Park, TiVo, digital TVs and DVR options. They did not put any hurdles or unnecessary hoops to jump through in the cancellation process, you have to grant them that. Contrast this with scumbags like AOL who have been repeatedly been slammed by state attorney-generals for fraudulently keeping on charging users after cancellation. The cable company's pricing policies may be evil, but their customer service seems pretty good.

02:56 - permalink [Macintosh]

Glimpsed at Chicago O'Hare airport

10:09 - permalink [Photo]

Is Vista a piece of unalloyed garbage?

As far as I can see, the answer is yes.

About a month ago, my two-year old Windows PC game machine started crashing every two minutes in NWN2. This proved the last straw, and I decided to upgrade. One of the games I have, but seldom play is Oblivion, which is graphically gorgeous, but chokes on anything but the most powerful hardware at ordinary resolutions, let alone my Apple Cinema Display HD's 1920x1200, and cutting-edge video cards are no longer available for the AGP bus in any case.

I looked around for packaged solutions from systems integrators, specialized gaming PC companies like AlienWare, and Dell. Contrary to conventional wisdom, it is still much cheaper to build a PC from components than to buy one from a major vendor, $1500 vs. $2500 minimum. Part of the reason is that the vendors flag anyone wanting the absolute best video card as a "cost is no object" customer, add all sorts of expensive components that make no sense in a machine that will only ever be used for games, like fancy DVD burners or flash card readers to jack up the profit margins. As if anyone in his right mind would use a Windows computer for serious work like digital photography...

My configuration is the following: a relatively quiet Antec Sonata II case, an Abit KN9 Ultra motherboard, an AMD Athlon x2 5200, 2GB of Kingston DDR-800 RAM, a humongous nVidia GeForce 8800GTX video card, a 500GB hard drive and a basic DVD-ROM drive.

When it came to choosing the OS, after much trepidation I opted for Vista Home Premium because the 8800GTX is one of the few cards that support DirectX 10, which is a Vista-only feature. I knew Vista would embezzle half the processing power of one core in DRM code that is actually working against my interests, but then again nobody in his right mind would use DRM-ed formats, whether Microsoft or otherwise, to store their music library, so the damage would be limited. Also, Vista comes with "downgrade rights" which allow you to legally install the previous version of Windows.

Vista comes in an attractive copper-colored DVD that is actually quite elegant. Its color scheme is also far superior to the molten Play-Skool set monstrosity that is XP. When I started the Vista installer, I was pleasantly surprised by how quickly it dealt with hard drive formatting (the previous Windows I installed myself is Windows 2000, which will insist on a time-consuming full format instead of the quick format used by the XP or Vista installers). The good impression lasted for all of five minutes. After the inevitable restart to complete installation, the screen promptly dissolved into a scrambled red-and-white screen of doom (I did glimpse a blue screen of death shortly before it rebooted). The diagnostics were completely unhelpful, as could be expected. When the operating system cannot even install itself, you have got to wonder...

Dejectedly, I fished out a Windows XP install DVD. it would not accept the Vista serial number. So much for downgrade rights. Of course, since the package was now opened, no hope for a refund either. I ended up buying a copy of Windows XP, which installed without a hitch. Of course, I still had to install the video drivers, but it did not crash half-way through the install procedure. And Oblivion is now playable without agonizing stutters every two paces.

The 8800GTX is very recent hardware, which did not even have non-beta Vista drivers when I installed it, so I could understand the OS falling back to SVGA mode. There are no other really exotic components here, certainly nothing than XP SP2 could not deal with and therefore Vista should as well. The machine is also well within the recommended minimum configuration (although some experts now advise 4GB of RAM as a realistic minimum for Vista). Crashing during install, when a five year old OS like XP handles it just fine, is simply unacceptable in my book. Even Solaris 10 Update 3, an OS notorious for its limited hardware support, installed without a hitch. Despite the ten man-millennia Microsoft invested in this lemon, they apparently could not be bothered to test the installer.

Conclusion: unless you buy a computer with Vista pre-installed, avoid it like the plague until SP1 is out, just like Intel.

Post scriptum:

Actually, I would not even recommend a PC with Vista preinstalled, as it has terrible backward compatibility. It will not run Office 2000, which is what my company has, for instance. Joel Spolsky has an excellent article on how the new, bloatedly bureaucratic Microsoft lost its way by sacrificing backward compatibility on the altar of useless marketectures. Perhaps they are just trying to force-upgrade people to Office 2007. They should beware: unlike 2002, people have credible alternatives now.

Update (2007-08-30):

The paper about how Vista eats up CPU on DRM has been criticized by the generally reliable George You. My point about the inability to even install on a modern machine that XP has no problems with remains. In any case, having the operating system constantly eat up CPU on tasks I do not want it to, whether it is 7% or 100% of one core, is still morally no better than a parasitic botnet.

00:14 - permalink [IT, Soapbox]

The operating cost of a home server

Like many people, I keep a server running at home 24/7. In my case, it's an old but relatively quiet Compaq Evo D315 AMD Athlon XP2000 PC with 1GB of RAM, 750GB total disk space and running Solaris 10. It serves as my personal email server (Postfix and Dovecot), to run Temboz, and miscellaneous auxiliary services like DNS, SNMP or being a staging point for off-site backups via rsync. All in all, very light usage, less than 5% average CPU utilization.

I have a Kill-a-Watt power meter measuring the load on that shelf, and the server, along with other devices on standby power, consumes about 160W. At PG&E's marginal rate of $0.13 per KWh, that comes to $180 a year, or half the cost of the machine itself. I am thinking of upgrading to a machine with 6 750GB or 1TB drives in a 4+2 redundant RAID-Z2 configuration for reliable backups (the current setup runs on ZFS for snapshots but has no provisions for drive failure). I will definitely look at power consumption more closely in my decision process

Update (2007-08-25):

I ended up getting a Sun Ultra 40 M2 dual-core AMD Opteron workstation with 6 additional Seagate 750GB drives. It is remarkably quiet and consumes only 160W, which is pretty good since it does have 7 drives spinning inside. ZFS benchmarks at 160MBps sustained disk I/O...

23:35 - permalink [IT]

MacWorld SF 2007 round-up

One of the perks of living in San Francisco is easy access to MacWorld Expo. I can literally see the Moscone center a mere two blocks from my new office window. This year's show spanned both North and South halls, but in some ways was a let-down compared to the last two.

Of course, all the buzz was about the iPhone. The amazing thing is not that Apple should make one, but rather that not a single cell phone manufacturer has a clue about design and ergonomics. Nokia used to, but they have backslid badly with their sluggish and over-complex Series 60 allegedly smart phones.

The prototypes were securely held under glass bells, presumably to preserve them from the salivating legions of the Mac faithful. From the demos, it looks pretty snappy compared to the incredibly sluggish Symbian or Windows Mobile equivalents, but I have serious doubts as to whether even Apple can make on-screen virtual keyboards work.

The other marquee product is the Apple TV, essentially a severely anorexic Mac mini without an optical drive or separate power brick, and running an unspecified embedded OS with the Front Row user interface. Pity it is limited to 720p (the 1080i support is interpolated). At a time when CompUSA sells a top of the line 42 inch Sharp Aquos 1080p LCD flat panel for under $2000, the lack of 1080p support is puzzling.

I haven't seen that much innovation among the third party vendor stands either. Here is what I did find at least somewhat noteworthy:

• Fujitsu came out with a new model of its ScanSnap document scanner line, the S500M, the only document scanner with official Mac OS X support. They claim the new model is slightly faster, and has a much improved paper feed. Indeed, the 5110EOX2 I have is annoyingly prone to double-feeding. The new model is also bundled with ReadIRIS Pro and Acrobat 7 Standard, a pretty good bundle all in all since those two programs together retail for nearly the same price as the scanner.
• Speaking of PDF, viewing the PDFpen demo makes me regret even more shelling for that piece of bloatware that is Acrobat. Simple, inexpensive software to manage and edit your PDFs. They have a show special, 20% off if you follow the link www.smileonmymac.com/macworld.
• Invisible Shield was demonstrating its self-healing protective plastic film for various gizmos by shaking an iPod mini in a box filled with screws and bolts, and showing how it survived unscathed. They also make protective films for digital camera LCDs, this looks like an interesting option since DSLR LCDs are very easily scratched.
• A number of stands were using the Logitech 3DConnexion SpaceNavigator controller. Ovolab (makers of the excellent Phlink answering machine peripheral, were demoing a photo geocoding application Geophoto, with lightning-fast Google Earth style navigation (oddly enough, the Google stand did not use this nifty human interface device). The controller has six degrees of freedom and is remarkable easy to pick up.
• Logitech has a fairly subdued stand. There were no real demonstrations of their NuLOOQ controller for Photoshop users, nor of their newly acquired SlimDevices Transporter, or Harmony programmable remotes. The emphasis was on their laser mice. SlimDevices was a popular draw at previous MacWorlds, I am not sure whether Logitech has gotten a grip on how to market that product line yet.
• Infrant had a small stand with a ReadyNAS NV+. I had never seen this NAS before, it is much smaller, quieter and more solidly built than I expected. The rep at the counter was a new recruit and not all that knowledgeable about the product (I asked whether they expect to support iSCSI soon, which would make it a killer expansion option for my Solaris 10 home server with ZFS). Infrant has a partnership with SlimDevices, and the bundle of a Squeezebox with a ReadyNAS is one of the most attractive networked digital music options available, far superior to the flashy but ultimately unsatisfying Sonos.
• Matias was demonstrating a prototype of their new TactilePro 2.0 keyboard. They now make their mechanical keyswitches by themselves instead of buying them from Alps (as with the version 1.0 Tactilepro I am using to type this blog entry). I like the original version so much I bought a spare when Alps announced it was discontinuing the keyswitches. The feel of the 2.0 is slightly different from the old one, but it still has that honest-to-goodness clickety-clack feel, albeit with a more subdued sound. The other differences involve upgrading the built-in hub to USB 2.0 and adding the Optimizer feature, which turns the useless Caps Lock key into a shortcut key instead. I remap the Caps Lock key to Control anyways on Macs, Windows and Solaris, so this last feature is of dubious interest to me.
• Intelliscanner was selling rebadged Symbol CS1504 scanners for $250. Save your money, buy the OEM Symbol version for under $100 and use my free Python driver instead.
• Canon was out in force, as was HP. Nikon and Epson had smaller stands this year. I got to handle the excellent new Canon HV10 HD camcorder, the new 70-200mm f/4L IS lens (a version of the excellent 70-200mm f/4L lens I already own, with gyroscopic optical Image Stabilization added), and the upcoming new Pixma Pro 9500 pigment ink printer that should compete with the Epson R2400 and the HP B9810.

22:41 - permalink [Macintosh, San Francisco]

M8, a missed opportunity

Last Saturday, I became the proud owner of a Leica M8. Then, a not-so-proud owner. As of yesterday, an ex-owner...

I returned it and sprung for an Epson R-D1 instead, saving almost 50% in the process. I had already previewed one at MacWorld SF two years ago.

Most people interested in a M8 know by now about its problems with sensitivity to near infrared, which manifests itself as a magenta cast in certain situations. There is a work-around (buy costly Heliopan or B+W IR filters for your lenses, although there are rumors Leica will provide two free filters), but many are legitimately angry at Leica for having rushed the M8 launch despite such a fairly obvious flaw. It's not an ideal situation but I could deal with it, as long as Leica stood behind its product and committed to a free upgrade to the corrected model once a definitive fix becomes available.

The straw that broke this particular camel's back was quality control, however, or the lack thereof. My M8 exhibited almost an entire column of dead pixels (the bottom 3/4 at x=2888). If you must, see this jpeg or the original DNG. This kind of flaw would be unacceptable in a sub-$1000 Canon or Nikon, it is simply outrageous in a camera as expensive as the M8.

The magenta cast is not an edge condition visible in limited conditions, by the way (Leica claims it only affects black synthetics under tungsten light), the photos I took last Sunday indoors in available light are completely unsalvageable, with a strong magenta cast everywhere that cannot be corrected by any amount of custom white balancing. Here is an example: JPEG, DNG.

Last, but not least, noise levels are excessive at ISO 1250, let alone 2500, with smearing in rows where bright highlights are present. Essentially, this camera as it stands today is utterly useless outside broad daylight conditions (I don't have an IR filter, so I can't comment on how effective they are). Of course, pretty much all cameras do reasonably well in daylight, even cheap and nasty point-and-shoots with too many megapixels crammed in a sensor too small. Rangefinders give you a two stop advantage due to the absence of mirror slap, but even with a Noctilux, the M8 has no edge over a Canon DSLR because of the noisy sensor. Then again, it is one of my rules of photographic thumb that Kodak stands for poor quality, and since they make the sensor in the M8, I should have expected the worst.

It's interesting to note how the reviews published so far managed not to mention any of these problems, which are completely obvious, even with the most cursory of inspections. In at least one case (Michael Reichmann of The Luminous Landscape), the reviewer found out about the IR issue, informed Leica about it but neglected to mention it in the review. This confirms me in my belief Phil Askey's reviews at DPReview are the only reliable online reviews of digital cameras.

Leicaphiles seem to be mostly in denial, or minimize the extent of the problem. I am as big a fan as any of Leica's optics and their rangefinder cameras, but the flaws in my M8 were so glaring I can't even begin to fathom the levels of cognitive dissonance required to sustain a positive opinion of this train wreck in the making.

All the reviews I have read so far have been raising hallelujahs and claiming the M8 feels like a real Leica M. It most certainly does not:

• The body feels much thicker than the MP, and is just as thick as the R-D1, in fact, despite not having a flippable LCD like the R-D1.
• The lightweight magnesium body does not have the same level of robustness as the R-D1, let alone a MP, and feels more like a CM. It's not even to the same grade as the original Digilux.
• The lens mount lock does not snap positively and reassuringly as it should, and the release button feels cheap compared to my MP or M6TTL.
• The shutter release is mushy and unpleasant. The shutter sound itself is a loud thunk followed by a noisy motorized re-cocking.
• Setting ISO is buried in a menu and you need even more keystrokes to change it than on a Rebel XT (the R-D1, in comparison, has a genuine knob to set it quickly with direct feedback).
• The rangefinder on mine was slightly misaligned vertically, something one can tolerate in a $300 Bessa, but certainly not in a M (to be fair, rangefinder patch vertical alignment is an endemic problem with the R-D1 as well).
• In another sign of sloppiness and poor quality control, the copy of Capture One LE included in the box was missing the serial number required to activate the program.

After using the R-D1 for a few hours, the superiority of the design over the M8 is readily apparent (with the sole exception of the taller body and short rangefinder base length):

• The R-D1 has perfectly acceptable ISO 800 and 1600, unlike the M8, making suitable for available light shooting.
• The LCD screen pivots and can be turned around to protect it from scratches (or resist the temptation of chimping).
• The viewfinder has an honest to goodness magnification of 1.0x like the original M3, not one that panders to jaded wide-angle junkies (I never shoot wider than 50mm and my MP is a 0.85x mag, so yes, I am biased)
• The power supply is a manageable size and even has a cord, unlike the bloated wall-wart type Leica supplies with the M8.
• The shutter speed dial goes in the traditional direction, not the M6TTL/M7 direction...

One bright light in this fiasco: Doug Thacker at Calumet Photo San Francisco (above) went well above the call of duty to help me with my purchases, all with unfailing good humor (he once sent me an email at 11PM to let me know of the IR sensitivity problem before they started receiving theirs). He even set one M8 aside for me even though I had cancelled my initial pre-order (they are in short supply and are reportedly going for over $6000 on eBay right now, so the opportunity costs are considerable). I think I will switch from B&H to Calumet for the bulk of my photo purchases in the future.

Update (2007-08-25):

I must be a glutton for punishment, as after reading Phil Askey's M8 review, remarkably thorough as usual, I decided to give it another chance and get one for my birthday. The first one I ordered (from Amazon) had a severely misaligned rangefinder - points at infinity would not coincide at all when the lens was at infinity focus. It had a low serial number, suggesting an early model with teething problems. Presumably Amazon does not sell that many, so I returned it and ordered another one from a place with much higher turnover, B&H. That one was a recent vintage (they have an orange sticker on the body cap), but its rangefinder was also misaligned, if not as severely.

In frustration, I went to my local Calumet and finally found one that focuses correctly. Wonder of wonders, it even seems like there are no dead pixels or highlight streaks. Conclusion: Leica's M8 quality control is still spotty, your best bet is to buy locally and test the rangefinder in the store itself.

13:47 - permalink [Photo]

Inserting graphviz diagrams in a CVStrac wiki

CVStrac is an amazing productivity booster for any software development group. This simple tool, built around a SQLite database (indeed, by the author of SQLite) combines a bug-tracking database, a CVS browser and a wiki. The three components are fully cross-referenced and build off the strengths of each other. You can handle almost all aspects of the software development process in it, and since it is built on an open database with a radically simple schema, it is trivial to extend. I use CVStrac for Temboz to track bugs, but also to trace changes in the code base to requirements or to bugs, and last but not least, the wiki makes documentation a snap.

For historical reasons, my company uses TWiki for its wiki needs. We configured Apache with mod_rewrite so that the wiki links from CVStrac lead to the corresponding TWiki entry instead of the one in CVStrac itself, which is unused. TWiki is very messy (not surprising, as it is written in Perl), but it has a number of good features like excellent search (it even handles stemming) and a directed graph plug-in that makes it easy to design complex graphs using Bell Labs' graphviz, without having to deal with the tedious pixel-pushing of GUI tools like Visio or OmniGraffle. The plug-in makes it easy to document UML or E-R graphs, document software dependencies, map process flows and the like.

CVStrac 2.0 introduced extensibility in the wiki syntax via external programs. This allowed me to implement similar functionality in the CVStrac native wiki. To use it, you need to:

1. Download the Python script dot.py and install it somewhere in your path. The sole dependency is graphviz itself, as well as either pysqlite2 or the built-in version bundled with Python 2.5
2. create a custom wiki markup in the CVStrac setup, of type "Program Block", with the formatter command-line:
   path/dot.py --db CVStrac_database_file --name '%m'
3. Insert the graphs using standard dot syntax, bracketed between CVStrac {dot} and {enddot} tags.

For examples of the plugin at work, here is the graph corresponding to this markup:

{dot}
digraph sw_dependencies {
style=bold;
dpi=72;

temboz [fontcolor=white,style=filled,shape=box,fillcolor=red];
python [fontcolor=white,style=filled,fillcolor=blue];
cheetah [fontcolor=white,style=filled,fillcolor=blue];
sqlite [fontcolor=white,style=filled,fillcolor=blue];

temboz -> cheetah -> python;
temboz -> python -> sqlite -> gawk;
temboz -> cvstrac -> sqlite;
python -> readline;
python -> db4;
python -> openssl;
python -> tk -> tcl;

cvstrac -> "dot.py" -> graphviz -> tk;
"dot.py" -> python;
"dot.py" -> sqlite;
graphviz -> gdpng;
graphviz -> fontconfig -> freetype2;
fontconfig -> expat;
graphviz -> perl;
graphviz -> python;
gdpng -> libpng -> zlib;
gdpng -> freetype2;
}
{enddot}

Another useful plug-in for CVStrac I wrote is one that highlights source code in the CVS browser using the Pygments library. Simply download pygmentize, install it Setup/Diff & Filter Programs/File Filter, using the string path_to/pygmentize %F. Here is an example of Pygment applied to pygmentize itself:

#!/usr/bin/env python
# $Log: pygmentize,v $
# Revision 1.3  2007/07/04 19:54:26  majid
# cope with Unicode characters in source
#
# Revision 1.2  2006/12/23 03:51:03  majid
# import pygments.lexers and pygments.formatters explicitly due to Pygments 0.6
#
# Revision 1.1  2006/12/05 20:19:57  majid
# Initial revision
#
"""
CVStrac plugin to Pygmentize source code
"""
import sys, pygments, pygments.lexers, pygments.formatters

def main():
  assert len(sys.argv) == 2
  block = sys.stdin.read()
  try:
    lexer = pygments.lexers.get_lexer_for_filename(sys.argv[1])
    out = pygments.highlight
    block = pygments.highlight(
      block, lexer, pygments.formatters.HtmlFormatter(
      style='colorful', linenos=True, full=True))
  except ValueError:
    pass
  print unicode(block).encode('ascii', 'xmlcharrefreplace')

if __name__ == '__main__':
  main()

18:43 - permalink [IT, Python]

Not-so-pop cultural references

Cartoons can be the most subversive of genres. The Simpsons has been serving acerbic social commentary from deep within the bowels of the reactionary Fox network for over 10 years, yet they manage to stay fresh when other, more edgy shows like Family Guys struggle for relevance.

One of the main characters in Futurama, Matt Groening's other animated show, is named Turanga Leela, a transparent reference to Olivier Messiaen's Turangalîla Symphonie. The number of people who have even heard of this fairly esoteric work is quite limited, and those who actually like it (as opposed to professing admiration for it out of conformity) can probably be counted on the fingers of one hand. Thus we have established that Groening or his staff are intimately familiar with 20th century classical music.

Last week's episode of The Simpsons featured the US Army showing a recruitment commercial at the Simpson kids' school. It starts with a rousing martial score. I have extracted a 4MB clip of the scene (Quicktime 7 required) for those who missed it.

The piece in question is from Prokofiev's Alexander Nevsky cantata. The music was composed for the epnymous anti-german propaganda movie by Eisenstein (although, in a little known consequence of the Molotov-Ribbentrop pact, Stalin made Eisenstein produce Wagner's Die Walküre for the Bolshoi in atonement for the newly inconvenient Nevsky). The lyrics heard can be translated in English as follows:

Arise, ye Russian people,
to glorious battle, to a battle to the death:
arise, ye free people,
to defend our beloved country!
All honour to the warriors who live,
and eternal glory to those slain!
For our native home, our Russian land,
arise, ye Russian people!

Needless to day, the delicious irony is unlikely to be mere coincidence.

00:47 - permalink [Soapbox]

iTunes 7, one step forward, one step back

I upgraded to iTunes 7 yesterday. This new version adds a feature opera and classical fans were clamoring for, gapless playback. Often, two or more consecutive tracks must be played without interruption. CD players do that effortlessly, surprisingly few MP3 players can. A pause, however slight in the middle of an aria can be incredibly jarring. Apple took its sweet time, but they eventually delivered (although it is not clear if all older iPod models will support this beyond the 5G video-capable iPod). You have to explicitly flag albums that must be played without gaps by selecting the "gapless album" option.

I still have to get used to the cosmetic changes in the new version. I liked the green musical note icon better than the new blue one. The new style looks a little flat and one-dimensional, just like many Windows XP applications have a flat web page inspired look compared to the beveled buttons and controls of the CUA-based Windows 2000 interface.

iTunes 7 is also markedly different in behavior from previous versions in how it handles albums. To keep the user interface easily understandable for the general public, all tracks are shown as a single flat database, but behind the scenes, iTunes implicitly normalizes this into something that looks like the following relational data model:

iTunes 7 added a few new fields, including "Album Artist", which seems to be of dubious utility. I described in a previous article my coding conventions to work around the fact iTunes was obviously designed by people with absolutely no clue about classical music. Unfortunately, iTunes 7 does not rectify this situation, but what's worse, they also changed the key for the album to include both Artist and Album Artist:

I can understand why they did this - if you have several alternative versions of the same work by the same composer, but by different artists, they would be played in intermingled order, which is why I append (2), (3) and so on to multiple interpretations of the same work (by decreasing order of preference).

On the other hand, the new behavior means that if you assigned different artists to tracks in an album (in a classical piece or an opera, you seldom have all the tracks played by the same artists), iTunes mistakenly thinks you have several albums, one for each possible value for artist, and this breaks libraries that worked with previous versions. iTunes 6 and earlier would not care and play them all in the order implied by the Track# field. iTunes 7 will see them as different albums and you will get a nasty surprise when you play the album in shuffle mode, as it will skip to a completely different album halfway through.

Let me illustrate with the album I first encountered the problem with. The last two tracks have Peter Seiffert credited, whereas the first two do not. The list view shows the albums sorted properly, like previous versions of iTunes:

The album view makes it clear what iTunes thinks is happening:

The only work-around this today is to force all tracks in an album to have the same Artist field. I will need to write an AppleScript to identify all tracks that need to be normalized thus, even if this means a loss of information. A further quirk is that the implicit normalization is apparently done at iTunes startup, so you need to restart iTunes for the changes to have an effect.

Update (2006-11-03):

I auto-updated to 7.0.2 today and finally figured out something that has been puzzling me for a couple of weeks now. My copy of Poulenc's Stabat Mater (Battle/Ozawa/Boston/Tanglewood) would not play as a single album, even though it is displayed as such in the album view. I have "shuffle by album" activated, and it would skip from any track in that specific album to a completely unrelated one. Even if I pinned it down by entering "poulenc stabat" in the search box so no other album could be selected, iTunes would skip from track 1 to a completely random track, instead of moving on to track two.

Apparently iTunes shuffles using a different algorithm than the album grouping. I have another album that has the exact same title "Stabat Mater", albeit one by Pergolesi, not Poulenc (and four others by Dvořak, Szymanowski and Vivaldi, but the opus or RV number differentiates the album titles). Having two albums with the same title seems to confuse iTunes no end. I don't want to mangle my album names just to appease iTunes 7, but I found a work-around: just change the mostly useless "Disc Number M out of N" field so the two albums with conflicting titles have different disc numbers, and they will collide no more. Better yet, use the CD number field to uniquely identify each CD in your collection, instead of just within a boxed set. Once again, you need to restart iTunes for this to kick in.

01:17 - permalink [Macintosh]