Temboz - interesting items
 
avid imovie ipad linksBenBrooks

Leanna Lofte:

Both iMove and Avid Studio can do the basics that you would expect from any video editor. You can insert videos, photos, and music, trim clips, add titles, and export to YouTube. Unfortunately for iMovie, this is where the similarities end.

Glad she wrote this up — I have been really curious how the two stack up. Sounds like Avid is the real deal and should only get better.

leica lenses ms-optical perar super tripletLR admin

MS Optical Perar Super Triplet 28mm F4 ens MS Optical Perar Super Triplet 28mm F/4 lens

I have covered the MS Super Triplet Perar 3.5/35 Mark II lens on this blog in the past, but I was not aware that there is also a 28mm f/4 version until I came across this listings on japancamerahunter.com:

This lens has been produced after over a year of research by Miyazaki san, and he has learned a lot about how to overcome problems that he faced with previous models of the perar he produced. The lens will be produced in batches, the first batch being of 180 lenses.

Specs:

  • Triplet Formula lens with high optical stability
  • Ultra compact and lightweight, only 45grams (55 grams with the hood)
  • Triplet design with 3 elements in 3 groups for refined sharpness
  • Premium multicoated lens with a 97% light transmission rate
  • 10 blade aperture diaphragm with smooth selection for a clean bokeh effect:Minimum focal distance of 0.8m
  • Unique exposed front aperture element

Some sample images taken with the MS-Optical Perar Super Triplet 28mm F/4 lens can be found on flickr.

Check also what other product are currently for sale on japancamerahunter.com.

Posted by LeicaRumors.com
This RSS feed is copyrighted and cannot be republished in any way without permission.
Follow (LR) on Facebook and Twitter.
we buy used cameras MS Optical Perar Super Triplet 28mm F/4 lens

Related posts:

  1. New MS Super Triplet Perar 3.5/35 Mark II lens announced
  2. Leica Super Elmar M 21mm f/3.4 ASPH lens now shipping in the US
  3. The design of Leica Super Elmar M 21mm f/3.4 ASPH lens was slightly changed
  4. Leica Super Elmar M 21mm f/3.4 ASPH lens now available for pre-order
  5. Leica 28mm Summicron-M ASPH silver version now discontinued

Vladimir Koifman (noreply@blogger.com)
IHS iSupply Image Sensor Market Q1 2012 report states that CMOS sensors took 73% of the revenue and 92% of the units shipped in 2011. "CMOS technology is expected to climb even higher in future years, to 90 percent of revenues for sensors and 97 percent of shipments.
CMOS sensors already dominate cameras in notebook PCs cameras with 100% penetration as a result of their lower power and cost advantages.
While CCD manufacturers are attempting to compete with price reductions due to yield improvements, we believe this will not help in the long run as CMOS continues to improve with offering technologies such as back side illumination (BSI) improving low light conditions.
Even in one of CCDs largest segments – digital still cameras – the technology is waning projected to drop to 25 percent of CCD revenues and 27 percent of all CCD units by 2015.
"
general random random thoughtNate

I hate it when people want to push code on a Friday. Here it is, Friday, I was working to wind up a few last tasks before going home when my phone went off saying part of my company’s site was not working right.

After some investigation with a developer we discovered it was an issue with Facebook (ugh, how I hate thee) and their code was breaking because Facebook was broken (they were not aware this would happen I am sure they will fix it going forward).

So while they are working to work around the issue I ran a search for it, and it seems to be a more wide spread problem caused by a Facebook software deployment done today, Friday at nearly 7PM!

My monitor first detected the failure at 6:49 PM so they weren’t even done deploying by the time it failed. It was intermittant for a few minutes then went hard down at around 7:11PM.

@$#$ facebook. Thanks for screwing me, and who knows how many others by deploying code on a Friday night.

Of course the developers on our end deserve some of the heat as well. But I am nit picking about code deployments on a Friday, not code bugs..

health itdidabodybad milk rawmilk recalls and safety unacceptable food unacceptablefood unpasteurizedmilkChris Morran

Unpasteurized, aka "raw," milk is illegal to sell in a number of states because of concerns about possible pathogen contamination. Of course, those bans also tend to make raw milk a sought-after delicacy for those who believe that pasteurization has a negative effect on the taste and nutritional value of milk. But in the last few weeks, at least 35 people in four states have become ill after consuming the unpasteurized stuff.

Health officials in Pennsylvania say that the 28 known cases of campylobacter bacterial infection in that state -- along with four in Maryland, two in West Virginia, and one in New Jersey -- all appear to be linked to milk purchased from one farm in Chambersburg, PA, sometime after January 1.

That farm, one of 153 in PA licensed to sell unpasteurized milk, has voluntarily stopped production of raw milk. It also had its latest batch tested by an independent lab and claims that the test came back pathogen-free.

35 cases of illness tied to Pa. farm's raw milk [Philly.com]

20 Campylobacter Cases Now Linked to Raw Milk Dairy [Food Safety News]

apple filevault 2 links patchBenBrooks

According to this support document (you need to be logged in to view it) Apple says that 10.7.2 and Security Update 2011-006 did the following:

Impact: A person with physical access may be able to access the user’s password

Description: A logic error in the kernel’s DMA protection permitted firewire DMA at loginwindow, boot, and shutdown, although not at screen lock. This update addresses the issue by preventing firewire DMA at all states where the user is not logged in.

CVE-ID

CVE-2011-3215 : Passware, Inc.

That sounds an awful lot like it patched the previously report security vulnerability of FileVault 2.

[via reader Matt S]

chef linuxNate

First off, sorry for being away for so long, I’ve been really, really busy preparing a new data center deployment to migrate my company out of the cloud into. The last time I did anything remotely resembling this was in 2007, though this time there are some extra layers involved that I didn’t have back then. It is certainly an interesting experience though configuring the software and infrastructure from the absolute ground up, having nothing to base it off of (other than past experience obviously!). I mean we have our stuff in a public cloud now but there are so many things that are different from an infrastructure perspective that little of it transfers over.

I wanted to write about a sort of topic that I haven’t really written about before. It’s about a systems management tool named Chef from a Seattle-based company named Opscode. It’s supposed to be a next generation tool that is supposed to make your life easier, more advanced than older tools like Puppet and Cfengine.

I’ll start off by saying I have a very strong background in Cfengine, having had used it since late 2004, at three different companies. My techniques and approaches evolved significantly over the years, and my last deployment was quite good in my opinion considering I had to adapt an existing Cfengine deployment made by folks who didn’t know what they were doing into something that worked well, and doing so in a 4 nines environment. That was not easy, as you know one wrong command or config in one of these tools can wreck havoc as I know first hand. I grew to like Cfengine a lot, and there was really nothing that I needed it to do that it couldn’t do for me. I knew it’s limitations well and it was simple to use.

I was introduced to Chef in the summer of 2010 when I went to the headquarters of Opscode and met their senior staff including one of the co-founders I believe. They gave us their powerpoint presentation on what Chef was, how it worked, what it could do, why it exists.

It certainly came across as a very impressive tool, being able to do tons of things that Cfengine could not do, had a lot of concepts that sounded like they could be useful. At the same time however it looked incredibly complicated.

I raised my concerns with their senior staff on that very first day and we had about a 15 minute discussion on it. I’m not a programmer, nor do I ever intend to be. I have a very big line that I refuse to cross from scripting tools in perl & bash to help make my life easier to full on code. A developer at my company constantly jokes that I say I am not a programmer yet I come up with complicated regexes and scripts to do things they don’t understand how to come up with on their own.

They tried to re-assure me that learning Chef is no different than learning the syntax of an Apache configuration file, or DNS or something like that. I didn’t really buy it, but was still willing to give the tool a shot since it sounded like a nice level of systems management that you could achieve with it. I still joke with my co-workers and current boss( who was my boss at the time too) on this very topic, they all remember that conversation to this day.

Chef is written in Ruby, and is very Ruby-centric. I guess you could say I am very biased against Ruby given my past experience supporting Ruby (on Rails) applications.

So here I am, almost 18 months later and things haven’t changed much. My dislike of Ruby continues, and is perhaps even stronger now having used Chef.

My first chef implementation about a year ago was fraught with frustration at almost every turn. I could(and still can) see the promise in the tools it provides the user with but it’s just so difficult to work with especially coming from a Cfengine background(and lack of programming experience) that for my first iteration I dumbed it down a whole bunch, making the logic very Cfengine like, at least as much as I could. I didn’t use any data bags, any attributes, no templates, nothing like that. I had (and still have) a very hard time finding usable examples for many things in Chef. They have a big repository of sample cookbooks – but to me for the most part those are not usable, because while examples they don’t go into details as to specifically, literally what each line of code does. Chef apparently uses this for it’s template language, I looked at it a couple of times – and really I could not make heads or tails of it.

I like to tell people that Chef makes the easy things hard, and the hard things possible. It seems very clear to me that they attacked the hard things in system management first before addressing the easy things. I remember seeing something in their documentation around the concept of the holy grail in the single instance copy, which fits along those lines well. The idea is you have one small bit of code that can be adapted to (m)any environments and situations, using the templates to pull attributes and values from data bags or other sources to make something on the fly.

The concept is novel for sure, coming from a Cfengine background I am very used to duplicating config stanzas, for different environments, making static config files, one for each environment or something like that. I’ve been doing it so long it’s second nature.

Where the opscode folks and I seem to part ways is our priorities. Their priority is to turn the system management into code and automate it to the point where it scales to a million systems. Mine is less ambitious, I want it to be easy to manage and it can scale to a few thousand systems at the most, since going beyond that gets so cookie cutter that it’s not fun anymore. I can certainly see the value of such an approach when dealing with massive environments that are changing all the time. Most companies though this situation doesn’t exist – most companies things are fairly static, you get a new system here and there, you get a new environment maybe once a quarter at the most. Maybe some big project comes along that increases your system count by a large amount for some special purpose.

I have absolutely no problem in maintaining separate config files for each environment and having different config stanzas in the config management tool to push those files out. Not only is this approach simpler (in my view) it gives much more, insight – perhaps is a good word into what is actually happening. I mean if you have a template filled with things that are pulling values dynamically from a half dozen or more different sources you really have no idea what that file really looks like until it lands on the server in question. I like to be able to open the file and look at the settings rather than hunt down the various flags and values that can come from these various sources chef provides.

I’m not building new environments every day, the level of change in general is quite small (as it has been over the past decade at companies I have worked at), I don’t need the level of dynamic abilities that Chef provides because it doesn’t help me that much.

I came up with a new saying a few months ago after dealing with Chef. If it’s not friends with sed, awk and grep then it’s not friends with me. Chef, being very developer-centric uses a lot of JSON to store and manage it’s various configurations. JSON is very much not friendly to sed, awk and grep, and so it frustrates me greatly whenever I have to deal with it.

Because we are moving into a self managed data center environment we needed a way to provision systems. My background is Red Hat/CentOS, Kickstart and Cfengine. We have Ubuntu, <nothing>, and Chef. I came up with a system that for now uses VMware templates (my first ever use of VMware templates) and some custom scripting to integrate with Chef and do other provisioning tasks. It works, it’s not as nice as Kickstart but it works. So speaking of this, and JSON there is a bootstrap process Chef needs to do in order to get itself registered and stuff with the Chef service. This involves creating a bit of JSON that Chef can read. The standard way of Chef bootstrap is a sort of push approach, where there is a management agent that waits for a system to be provisioned, then ssh’s to the system and runs a bunch of stuff. I wanted a pull approach, where the system is provisioned and boots up and configures itself. So I came up with this little bash snippet to construct this JSON file

echo -n "Making first-boot.json ..."
echo -n "{ \"run_list\": [ ">/etc/chef/first-boot.json;
export ROLES=`grep ROLE /root/00-50-* |head -n 1 | sed s'/.*=//'g | sed s'/,/ /'g` &&
for ROLE in $ROLES; do echo -n \"role[${ROLE}]\",;done | sed s'/\,$//'g >>/etc/chef/first-boot.json;
echo -n " ] }" >>/etc/chef/first-boot.json

That /root/00-50-* file is a configuration file named after the MAC address of the VM. This is based on my older kickstart stuff which has been extended to support Chef. It stores things like IP address, Host name, default gateway, for the network, then Chef environment, Chef Role(s), and Chef Organization. It’s a simple text file format, that looks like VARIABLE=value, one VARIABLE per line.

My point with pasting that code is the ugly length I have to go through to simulate valid JSON output using my own regular tool set. Remember I am NOT a programmer!

The scripting works fine(at least so far, built a dozen or so different roles and systems), but it shouldn’t be that complicated.

For those of you more experienced with Vmware templates I noticed there is the ability to customize a template so that Vmware can set the IP address, host name etc of the guest OS. When I saw this I spent a good two hours trying to get it to work, but no matter what I tried Vmware said my configuration was not supported and it would not let me customize. I have read conflicting reports as to whether or not it is possible on Ubuntu. I am running ESX 4.1 with vCenter 5.0. I think if I was running vCenter 4.x it would work fine, but Ubuntu and other “non tier 1″ operating system support for template customization is no longer supported in the 5.0 products. Often times when I see “not supported” especially when something used to work, it means that it might work but don’t ask us for help if it blows up. Maybe coincidence or not but as I said no matter what I did, the customization boxes were greyed out and I could not get vCenter 5.0 to work with Ubuntu.

At the end of the day it doesn’t matter though, I had, what was to me at least a good provisioning process I could adapt from my Kickstart days, a process that works well on both physical as well as virtual machines. Something that leverages the MAC address or the serial number(in the case of physical machines) for unique identification.

With regards on how I used to do things with Cfengine, it was simpler than Chef. Cfengine operates more on trust than Chef. Chef uses public/private keys to authenticate systems, and these keys have to be in the right place in order for a system to get registered. This is good for untrusted networks, like public clouds(ugh). Cfengine works more on trust, where you can (or at least I did) assign network ranges where the IPs are trusted, and a new system could just register itself without any special configuration. The keys would be generated automatically and exchanged between cfengine client and server. I had my cfengine configuration, for the most part dynamic based on the host name of the server. Most of my major Cfengine classes ran a simple grep on the file name that had the host name in it, if the host name matched a particular pattern it was automatically included in the right classes. With Chef life is different, I can’t do that. I have to specifically define which role(s) or recipes a system has up front. Because the system will only download cookbooks that it is specifically configured for using. This isn’t a big deal but is an extra step that I’m not used to having to do.

Sample CFengine class defitition:

ENV_CORPDMZ     = ( ReturnsZero(/bin/egrep -q "^HOSTNAME=corpdmz" /etc/sysconfig/network) )

With Cfengine, prior to implementing the hostname-based approach, adding a new server with Cfengine involved manually editing the master cfengine configuration so that it was aware of the new system that was about to come online. I still had to edit this file on occasion, if there were special configs needed for a server, but for the most part, for like systems, web servers and the like I did not.

Which sort of brings me to the next topic – recruiting talent that can use Chef.  I’ve been managing server systems for about 17 years now, wow has it really been that long.  It’s clear to me after 18 months of chef I lack the knowledge to be able to effectively use the tool (though it hasn’t stopped me from using it at this point), but knowing that, and working with people at my previous company with Chef and seeing the tool present them with a similar level of frustration (if not more), I can see Chef being a real sticking point finding talent that is capable of managing it. My company is actively recruiting senior systems people(well one person) and the candidates that I have spoken with so far, along with candidates I have spoken to in the past, I honestly can think of perhaps one or two people over the years that I know that could handle Chef, and one of them is a full time programmer now (when I met him he was hired to be on my operations team back in 2003).

Well short of the co-worker I have now who does quite a wonderful job in deploying and managing Chef, who wrote the vast majority of Chef stuff at my current company. It’s really well done, but even now that a lot of the hard work was done by him, in a very chef-like way I constantly struggle to add new stuff in, or to change existing things because it’s so dynamic. I see a value for something – where is it coming from? is it from the node? environment ? data bag? attribute? something else?

So I see Chef somewhat like I see Hadoop as far as what skill sets are needed and who can provide them. One of my previous companies was working on migrating towards Hadoop and a big complaint I heard from them about Hadoop (and I have heard it from others since) is finding talent that knows the product. With the likes of Yahoo, Google, and other big companies with very deep pockets and big data aspirations they can afford to pay out the wazoo for Hadoop talent, something small companies just can’t compete with. The number of people qualified to do Hadoop right vs the number of people that can do SQL, well it’s obvious, right.

I see the same with Chef. It’s a powerful tool but it’s just not there yet with regards to usability, I can see it being a very useful tool for the likes of those same kinds of companies who manage very huge fleets of systems and have a very dynamic environment. One such place is HP, whom someone I know is going to work for HP Cloud, because he knows Chef. I assume he is probably pretty good at Chef by now, though the caveat with him is he has a strong Ruby programming background. So it’s no real surprise that he could pick Chef up.

I filed several feature requests and bug reports on the Chef support site about a year ago when I was first interacting with it, though I don’t think much made it through. One thing I’d really like is a good way to do in-line editing of text files. At least at the time the Chef mantra was “find another way to do it”, which a friend of mine says is the same thing Puppet people say. So how do I go about adding an entry to /etc/hosts?

Another thing I’d like to be able to do is bulk file copies from the cookbook and preserve ownership and permissions from the source files(e.g. having a directory tree with various owners/groups/permissions and copying it all at once), I don’t think that is possible still. At the time the Opscode people suggested I use rsync for that.

Another thing I’d like is to be able to host cookbooks internally while using the external service for other things. This is mainly for security purposes I feel more at ease when my core data stays within the confines of my network, on systems under my direct control.

Another thing I’d like to see which I have mentioned to Opscode in one way or another as well is a more abstracted configuration language. I think I called it idiot mode or something. The Ruby syntax they use, while I’m sure it’s great for ruby people really sucks for people like me. I’m fine with a reduced subset of functionality that may be provided by idiot mode, because it’s likely that I won’t use that functionality to begin with(at least not initially). Make the learning curve to actually using the tool less steep.

At one point Opscode was interested in talking to me about a full time position being an advocate for their platform. I just couldn’t go through with it, I just can’t get excited about the platform after all the frustration it has given me. I certainly see the promise and will continue trying,  but I think some fundamental things need to be done to the system in order to make it more usable.

So, in the end, I see Chef as a very powerful tool, a very useful tool for those with the skills that can handle the power it gives you. If I were deploying a new environment today I would certainly NOT use Chef, I would use Cfengine. I don’t want to discourage people from using Chef, it is a good tool, just realize the much higher level of investment you need in order to properly leverage it and try to weigh that against the benefits. For me, the hard things that are made possible by Chef really involve a trivial amount of time. I dare say I have spent FAR more time trying to work with Chef on these hard things (understanding the concepts, code etc) than just flat out doing it by hand the old fashioned way.

You might want to ask – why haven’t I tried Puppet? My answer would be – to-date I haven’t had a reason to. I’ve had a few brief discussions with people who use Puppet over the years(including those who have used Cfengine as well) and asked them why should I use Puppet over Cfengine. For the most part the response was there’s nothing really revolutionary in Puppet so if your happy with Cfengine then stick to it. There are a few things Puppet apparently does better (What they are I don’t remember), but in my talks with people there wasn’t anything — anything that made me want to jump on Puppet. There was things that sounded nice (like Chef has), but not enough return to justify the investment in time to make a migration when, as I mentioned earlier Cfengine does pretty much everything I need it to do.

With Cfengine I could probably train a systems person up on the basics in literally an afternoon. My Cfengine configurations were not complicated. With Chef, well here I am at 18 months and still lost.

3,400 words, I think that’s a record for me for a published blog post. Should get back to sleep now, started writing this at about 3:30AM.

hi-fi amplifiersAndy Clough
NotePad 250 and NotePad Air amps designed to be driven by any Apple iOS device using AirPlay

read more

Just about a month ago, PHP 5.3 ...(more)...
...(more)...
articles ipad pdfpenBenBrooks

PDFPen for iPad was released just before I hopped on a plane to head down to Macworld|iWorld in San Francisco. I read a quick review of it and then purchased the app, closed down my iPad, and jumped on a plane.

While airborne I got an email from my real estate agent saying that he needed some paperwork signed for the home my wife and I are purchasing. I used iMessage to tell my wife to sign the paperwork on her Mac and then send me the file. At this point I could have pulled out my MacBook Air and paid another Wi-Fi fee to connect it, but I had PDFPen on my iPad — why not give it a go?

I did and it worked fantastically well.

You might be thinking that there are hundreds of apps on the App Store that can do this same thing, and there likely are, but PDFPen has some really great things that made me love it.

A few of those things are:

  • iCloud Sync
  • Stored Signatures
  • Email a “Printed” PDF

iCloud

If you have PDFPen on your Mac, then PDFPen for the iPad will sync the files between the two applications over iCloud. Unless you are a heavy PDF user this likely doesn’t seem to be all that life-changing of a feature, but it still is pretty great.

The iCloud support in iOS and Mac applications has become a fantastic Dropbox like utility, but unlike Dropbox it is something that is built-in at the system level.

You don’t have to think about it and that is key.

So when I sent out 4-5 signed PDFs from PDFPen on my iPad and a few days later was sitting at my MacBook Air with a need to resend a couple of those documents — I didn’t need to go find my iPad. All I had to do was grab those PDFs in PDFPen and resend them. They were just there.

It’s these really small moments that add up to a product that just fits in your life. More developers need to add iCloud syncing — it really is great — and I am glad PDFPen has it.

Stored Signatures

I didn’t know this when I was signing all those PDFs, but there is a fantastic feature of PDFPen that allows you to store your own signatures and other scribbles so that they can be added to any document with one tap. Here’s how you do that (from a Smile newsletter):

I love that feature and have already added not just my signature, but my initials as well. It’s a nice little touch and I can imagine there being some really great uses for this (including storing company logos).

Emailing a “Printed” PDF

One of the most annoying problems that I run into on a weekly basis is filling out a PDF, sending it to a Windows user only to get an email back saying that the document isn’t filled out. I don’t know why this is a problem on Windows, but the layered approach that Preview seems to take on the Mac is not compatible in a universal manner.

The solution has always been to fill out the PDF and then print the file to a PDF — thus flattening the document. The fine folks at Smile must have had this problem too because when sending out the PDF in PDFPen you can choose to send it as a flattened file (printed PDF) if you want.

This is fantastic.

Two Criticisms

There are two things about this app that I don’t care for:

  1. The icon. I have never been a fan of the styling that Smile uses for its icons and PDFPen is no exception. I know that I pick on icons a lot, but a good icon is a good icon. A bad icon is one that I never want on my home screen — so if my home screen is your goal, you better make your icon good.
  2. Highlighting PDFs is a bit awkward. I could see this being pretty good with a Cosmonaut, but with my finger I felt like I never learned how to highlight before. If some sort of tracking could be built-in so that you can make relatively straight lines then we would really have something here. Until then, if your primary use case is highlighting, you might want to look elsewhere. 1

One Step Closer

Like I said before, I am not a PDF guru. However I am a real estate professional and PDFs are a norm in my business. There’s nothing missing from PDFPen for my needs, which takes me just one step closer to not needing my Mac at all.

In fact, I didn’t even need my MacBook Air at Macworld until I recorded a podcast — a large part of not needing the Air was because of PDFPen for iPad.

  1. Also, why are you highlighting so many PDFs?
business malware news security smartphonesdan.goodin@arstechnica.com (Dan Goodin)

Google engineers have unveiled a cloud-based service that scours the Android Market for malicious smartphone apps.

Bouncer, as the scanner is called, automatically checks each title in the Google app bazaar to make sure it doesn't match signatures of known malware, Hiroshi Lockheimer, vice president of Android Engineering, told Ars. It also looks for clues that apps contain surreptitiously abusive behavior by running them through a system that simulates an Android device. The scan happens when developers first upload an app to the Market and then periodically after that.

For years, critics have said Google doesn't do enough to police its own servers for apps that steal user data, rack up expensive charges, and carry out other undisclosed abuse. Google's guidelines for Android developers promise they have "complete control over when and how they make their applications available to users." While many developers and users welcome the freedom, it has also allowed malware purveyors to install their titles on tens of thousands of Android phones.

In December, for instance, researchers unearthed at least 22 malicious Android apps, some that were downloaded more than 10,000 times. The titles advertised themselves as popular games such as Angry Birds and Cut the Rope, but once installed they sent text messages that accrued hefty charges for users who fell for the ploy.

"We really designed this in a way to maintain the flow the users and developers are familiar with," Lockheimer said. "Android has been a comfortable place for users to download and purchase apps from."

Bouncer has been up and running for about six months, he said. Google saw a 40 percent decrease in the number of potentially malicious downloads in the second half of 2011 compared to the first half. Google blogged about the scanner here.

Read the comments on this post

Since the news of Passware being able to decrypt FileVault over FireWire connections, Mac users might be concerned about the integrity of the security on their computers.
enterpriseLyle Smith

Infortrend has announced the addition of new high-density storage solutions in its EonStor DS family. These new solutions aid customers meet extreme capacity requirements with 48 drive bays in a 4U form factor, in addition to offering outstanding cost efficiency. The new systems are equipped with hybrid Fibre Channel (FC)/iSCSI host connectivity for cost-effective remote replication or consolidated SAN tiering.

read more

economics educationTyler Cowen

The puzzle is why firms pay huge sums to big name consulting firms, when their advice comes from kids fresh out of college, who spend only a few months studying an industry they previous knew nothing about. How could such quickly-created advise from ignorant college students be worth the millions paid? Why don’t firms just ask their own internal recent college grads?

Some say that consulting firms use their access to collect data on best practices, data that other firms are eager to pay for. But while this probably contributes, I find it hard to see as the main effect.

My guess is that most intellectuals underestimate just how dysfunctional most firms are. Firms often have big obvious misallocations of resources, where lots of folks in the firm know about the problems and workable solutions. The main issue is that many highest status folks in the firm resist such changes, as they correctly see that their status will be lowered if they embrace such solutions.

The CEO often understands what needs to be done, but does not have the resources to fight this blocking coalition. But if a prestigious outside consulting firm weighs in, that can turn the status tide. Coalitions can often successfully block a CEO initiative, and yet not resist the further support of a prestigious outside consultant.

To serve this function, management consulting firms need to have the strongest prestige money can buy. They also need to be able to quickly walk around a firm, hear the different arguments, and judge where the weight of reason lies. And they need to be relatively immune from accusations of bias – that their advice follows from interests, affiliations, or commitments.

All three of these functions seem to be achieved at a low cost by hiring good-looking kids from our most prestigious schools. These are the cheapest folks you can buy with our most prestigious affiliations, they are smart enough to judge where reason lies, and they have few prior affiliations to taint them with bias. They can not only “borrow your watch to tell you the time,” but can also cow you into submission in accepting that time.

Yes the information contained in consulting advice can be obtained elsewhere at a lower cost. Firms could hire most any smart independent folks, or set up a prediction market. But alas those sources don’t have the raw strength of status to cow opponents into submission, opponents who in practice can block changes no matter what a CEO declares.

So mine is a signaling and status story (surprise surprise). The weight of status often decides outcomes, no matter what the CEOs commands, and so CEOs often need to bring out status ringers, to cow opponents into submission.

Here is a bit more.

shirtsWill (noreply@blogger.com)

According to Stu Bloom at RAVE FabriCare, about 80% of men get their shirts back from the laundry on hangers, and this is certainly the wisest course since they are free of the creases that come from having them folded. The challenge with the practice comes when it is time to pack for a trip, and the shirts must be folded anyway.
The usual way to prepare shirts for packing is to fold them in thirds, replicating the commercial laundry folding machine (see the shirt on the right in the photo).  Whoever designed that machine was apparently not very clothes conscious as that fold leaves the vertical and horizontal creases it imparts placed so that they can be visible under a jacket, which might not be not a terrible thing with some cloth as it will hang out in an hour or two but heavier shirtings like oxfords and twills can remain creased for much of the day, contributing to a messier look than a man ought to aspire to (here we deliberately ignore no-iron shirts on the grounds that the well dressed man eschews them).
Now, it is only natural that a man would assume that folding meant visible creases and that there is nothing to be done about it, at least until like me he noticed that RAVE's clean by mail shirts are folded so that any creases that might occur in parts of the shirt front are not visible when a man has his jacket on (the shirt on the left was folded by RAVE). The secret is to fold the shirt in half rather than in thirds. In other words, when the shirt is on its front laid out for folding, turn the sleeve sides over only a quarter of the way, leaving a space between them. Then fold the bottom up so the shirt is roughly halved into a square. Leaving all the folds loose will also help the shirt's appearance, but only marginally as the state of being packed will inevitably press it to a certain extent.
I will be the first to admit that the square shirt fold is fairly obscure advice, and has the downside that otherwise useful suitcase accessories like Eagle Creek's folders and cubes seem to all be designed to accommodate shirts folded into thirds. Nonetheless, a supply of heavy duty polyethylene bags makes for a reasonable substitute and having a supply of pressed looking shirts when one unpacks is worth a little one-time trouble.
garage simple living travelErin Doland

Today’s edition of Britain’s Daily Mail includes an article, photo gallery, and impressive infographic describing London’s newest clutter-free street, which officially opened earlier today. The piece “No kerbs, pavements or nanny-state signs: Britain’s longest clutter-free street is unveiled to make things SAFER” explains the initiative to improve safety on this stretch of road by removing visual distractions:

Britain’s longest ‘clutter-free’ street was opened today with the aim of making cars and people co-exist harmoniously — without the need for hectoring signs and protective steel barriers.

Indeed, the newly revamped Exhibition Road in the heart of London’s museum quarter in Kensington, visited by millions of people from around Britain and the world, doesn’t even have kerbs or pavements.

The idea underlining the project is that when nannying rules and orders — in the form of countless signs, traffic signals and barriers — are removed, motorists take more personal responsibility for their own actions and drive more attentively, making more eye contact with pedestrians.

In addition to taking on projects in London, two years ago national officials in Britain formally began encouraging city council leaders to decrease road signage to improve road safety. This specific decision to rework Exhibition Road came in 2003 and is based on popular urban design and engineering concepts from Dutch traffic engineer Hans Monderman. Monderman’s engineering ideas are implemented in many areas of Europe and Asia and are referred to as “shared space” planning design.

More about the clutter-free road from the Daily Mail article:

Councillor Daniel Moylan, deputy chairman of Transport for London (TfL), said: “… The psychology of this scheme is fascinating. Experience seems to show that when you dedicate space to traffic and control it with signs and green traffic lights, motorists develop a claim on it. It becomes ‘my space.’ Drivers become annoyed if people move into it.

They get angry if a mother pushing a buggy moves across the crossing just as the lights are about to change.

This new scheme is more like the behaviour in a supermarket car park. Drivers know there are people around pushing shopping trolleys and so drive more cautiously. They are looking out.

They don’t feel that pedestrians are invading their space. They don’t therefore get annoyed.”

Image from Britain’s Daily Mail. Thanks to reader Samantha for bringing this post idea to our attention.

Like this site? Buy Erin Rooney Doland's Unclutter Your Life in One Week from Amazon.com today.

os xmatty
I just came across a reference to ZEVO tonight. This appears to be an add-on package for OS X that is built on top of ZFS. I’m going to have to keep an eye on this. Snapshots, data checksumming, de-dup, compression and zfs send/recv would be pretty cool on my Laptop. :)
medicalEvan Ackerman

Science has finally gone and done something useful for a change by inventing a mouthwash that they say can completely eliminate the bacteria responsible for tooth decay. Refined sugar, you and me have a brand new (and sparkly white) future to look forward to.

newsJoe Brockmeier

redmonk-1.jpgOnce upon a time, the default stack for a lot of developers consisted of the LAMP stack. Linux, Apache, MySQL and one of the P triumvirate: PHP, Python or Perl. Those days, however, are over. Sure, Linux is still powering a lot of servers. But above that, almost everything is up for grabs. Today at the Monki Gras conference in London, Simon Willinson of held forth on the new Web stack.

Willinson was part of the day's last talk, a conversation with Matt Biddulph, formerly Nokia's head of data strategy for location and commerce applications.

Willinson and Biddulph talked a lot about the history of Lanyrd and how technology choices could give developers or a project an enormous lead on competition. What was particularly interesting, though, was the list of tools that Willinson recommends for building new Web-based applications.

The New Tools

The biggest decision used to be which Web framework to choose. While that's still important, Willinson said that infrastructure is much more important.

First, Willinson says that you need a message queue and workers. He suggests the Python-based Celery distributed task queue. "Once you have it," says Willinson "all sorts of things become super-easy."

Next, you need a full-text search engine. Here, Willinson suggests Solr.

Willinson also talked glowingly about the Redis key-value store. Redis, he says "is in a category all its own." Not a database, exactly, Willinson calls it a "data structure server" that is "so screamingly fast, things you thought of doing that would have a performance impact, you don't have to think about" at all.

Finally, Willinson recommends Varnish, which is billed as a Web-application accelerator on the front end.

Of course, things are changing rapidly. What's crucial today may not be that relevant tomorrow. The important thing is to keep an eye on open source projects and evaluate them for use in your projects. Willinson says "any tech advantage means you can iterate faster."

How does Willinson evaluate tools? He says he typically gives them 30 minutes. If he can't have a tool up and running within 30 minutes, it goes by the wayside. Sometimes this is a mistake. He cited Puppet as a tool that Lanyrd finally embraced after incurring "a huge technical debt" because it couldn't be set up easily in 30 minutes. Note that the 30-minute rule is not to put something into production, but simply to have working and see what a tool is capable of.

By necessity, Willinson and Biddulph's talk was a bit breezy and not in great depth. However, it does provide some insight into what some of the more cutting-edge developers are using. What's your suggested Web stack?

Discuss
algae biofuels biology biotechnology energy news sciencekyleniemeyer.ars@gmail.com (Kyle Niemeyer)

Biofuels may hold the key to reducing our dependence on foreign oil and cutting down on our greenhouse gas emissions. Ethanol is currently the biofuel of choice, with almost all gasoline bought at the pump in the United States containing 10 percent ethanol. Right now, though, most ethanol comes from corn and sugarcane, and there are concerns that growing our fuel from these crops could drive up food prices (“food versus fuel”).

Biofuels made from macroalgae, aka seaweed, avoids this problem. Seaweeds do not require arable land, fertilizer, or fresh water, and they are already cultivated as food (though not a staple crop like corn), animal feed, fertilizers, and sources of polymers. Traditionally, scientists ignored seaweed as a biofuel source because its main sugar component was too difficult to process. A recent paper published by Science describes how researchers genetically-engineered a microbe that is capable of producing ethanol from seaweed.

The so-called second generation of bioethanol is derived from inedible crops like wood and switchgrass, or the inedible portions of food crops like corn (the leaves and stalks). However, this cellulosic material is difficult to process due to the presence of lignin in the cell walls—although we reported on some attempts to genetically modify switchgrass to make this easier. Seaweed doesn’t contain lignin, making processing a lot easier and enabling higher yields: a Department of Energy study showed that, under ideal conditions, seaweed could produce twice the ethanol that we get from sugarcane and five times the amount from corn.

You may be asking “This sounds great, why aren’t we making ethanol from seaweed?” Well, there is a catch. Seaweeds contain three primary sugars: alginate, mannitol, and glucose. Right now, existing industrial microbes can’t metabolize alginate, so ethanol yields are severely limited.

At this point, most people would stop and say “Well, maybe seaweeds aren’t the best way to produce biofuels.”

On the other hand, if you were Adam Wargacki and a team of 13 others from the Bio Architecture Lab, you wouldn't stop. Instead, you’d look to the well-known bacterium Escherichia coli (E. coli), which has a natural ability to metabolize mannitol and glucose. Since we know of enzymes that can process alginate (alginate lyase and oligoalginate lyase), Wargacki et al apparently thought “We can make this work.”

There are several bacteria species with known alginate metabolic systems, but only one where we've identified an alginate transport system to get it inside of cells: Sphingomonas sp. A1. Unfortunately, this system is too large and complex to be incorporated into E. coli.

Instead, the team searched the National Center for Biotechnology Information genome database and found a 30,000 base-pair (30 kbp) section of DNA from the bacterium Vibrio splendidus 12B01 that looked like it might hold all the genes needed for alginate degradation, transport, and metabolism. (I pity the researcher that had to search the database for this.)

Now, a 30-kbp DNA fragment is too long to directly clone into E. coli, and the function of the genes in that DNA hasn’t been described yet. To get E. coli with the right DNA, the authors created a library of random DNA fragments from the V. splendidus genome. Each fragment is carried by DNA called a fosmid, which will stably integrate a 40-kbp piece of DNA into a target’s genome. After inserting the fosmid library into E. coli, they placed the bacterial colonies into a medium where alginate was the only food source. Only colonies with a particular fosmid (designated pALG1) grew, suggesting that this section of DNA contained the 30-kbp piece they identified earlier.

After this, they checked the individual protein coding sections of pALG1 to determine the function of each. By deleting them one at a time and testing for the ability to grow on alginate, they were able to identify an alginate transport system that hadn’t previously been described.

After inserting these genes into a strain of E. coli, they took genetic pathways for ethanol production from Zymomonas mobilis—through enzymes called pyruvate decarboxylase and alcohol dehydrogenase B, for those interested—but deleted some pathways that produced undesired byproducts. Finally, they tested their engineered E. coli strain (which they named BAL1611), in a five percent sugar mixture containing alginate, mannitol, and glucose at a ratio of 5:8:1, which represents the typical ratio in brown macroalgae (seaweeds). They found that it produced ethanol at a yield of about 20 grams per liter.

For a final demonstration, they used Saccharina japonica, otherwise known as kombu, a common edible kelp. Their microbe produced ethanol at a ratio of 0.281 grams of ethanol for each gram of algae—which is over 80 percent of the theoretical yield. In addition, 83 percent of the yield was obtained within 48 hours.

This study may have opened the door to using seaweed as a source of ethanol—there is certainly a lot of potential here. Even more fascinating is the approach to engineering a microbe to give it the characteristics we desire. 

Hopefully, microbes can eventually be engineered to make more than just ethanol. It is much less energy dense than gasoline, so current vehicles can’t burn a fuel mix that contains more than about 15 percent ethanol. Butanol, on the other hand, is much more similar to gasoline, so perhaps a future microbe may produce that.

Science, 2012. DOI: 10.1126/science.1214547 (About DOIs)

Read the comments on this post

bruce@momjian.us (Bruce Momjian)

In my previous blog entry, I analyzed how various tools (ps and smem) report memory usage. In summary:

  • ps columns TRS, DRS, and VSZ report virtual address space allocated, not actual RAM allocated.
  • smem's USS reports a process's private (unshared) memory allocated.
  • smem's PSS is a sum of process's private memory allocated and a proportional amount of shared memory (both System V shared memory, like Postgres's shared_buffers, and shared libraries).
  • RSS shows actual RAM allocated, private and shared.

With these issues understood, let's look at a running Postgres cluster:

Continue Reading »

cartography crowdfunding happy mutants kickstarter maker origami post san francisco travelCory Doctorow

Shan sez, "Our guide/map of SF is printed on a single sheet of A3 Tyvek, and is then folded up according to a technique originally developed at Tokyo University for satellite solar panels. The bistable nature of the fold means that it can be fully opened or closed in one smooth motion, and that there is no way to fold it 'wrong.' The places we included are a mix of overlooked gems, classic restaurants, and other things like hidden parks, games played across the city, and interesting shops and markets. We just launched our project on Kickstarter yesterday evening, and as of today we're almost 10% funded!"

TOC Guide to SF (Thanks, Shan!)

productsScott M. Fulton, III

SGI (on InfiniteStorage brick, 150 sq).jpgThe "G" in its name used to stand for "Graphics." A few decades ago, the most delightful room for one to be in during a computer conference was the one where Silicon Graphics was showing a demo. It was like one of those dreams where you knew you weren't really on-board the Starship Enterprise, but you forced yourself to ignore that fact and look at the pretty lights and colors. When SGI ceased to be a company unto itself in April 2009, most folks wrote off the SGI brand as an historical remnant.

Wrong. It's wonderful to see a brand that never says die. Ever since Rackable Systems adopted the SGI name, it's been lucky. It's finding its way back as a high-density storage provider. This afternoon, the company is introducing a very high density storage server platform designed, its engineers tell us, to pack the maximum number of terabytes into a 19-inch rack while staying cool.

120131 SGI drive bricks 03.jpg

The result is what SGI describes as a module full of "drive bricks." Each brick can be loaded with up to nine 3.5-inch SATA or SAS drives, or 18 2.5-inch SAS or solid-state drives. If you've ever washed dishes in a cafeteria, you may have experienced something similar to the situation of hot-swapping drives in a storage rack. So SGI used a little something called "computer-aided design" to engineer a solution.

"Any time you get a system that is this dense, data center managers need to be able to access it," SGI's director of storage products, Floyd Christofferson, tells RWW in an interview. "If you've ever pulled out a large, very dense system, typically they only allow access from the front. The weight of those trays pulling all the way out adds a lot of strain on the internal cables. In large data centers, people would like to be able to access these hot-swappable parts, but really don't want to do so in a way that either puts strain on the rack itself or on the internal components."

120131 SGI drive bricks 02.jpg

So the slider on SGI's Modular InfiniteStorage (MIS) units can be pulled from the back or front on a nice drawer, all without tripping up a cable or maybe tripping off the power.

Each 4U MIS chassis may have one of two configurations. One is as a storage server with one or two motherboards, each with dual-socket Intel Xeon E5-2600 "Sandy Bridge" processors clocked at up to 3.3 GHz. The remaining space can be populated with either 72 3.5-inch or 144 2.5-inch drives. Alternately, the unit can be maxed out with 81 3.5-inch or 162 2.5-inch drives.

120131 SGI drive bricks 01.jpg

So with that many drives packed that close together, how do they keep from becoming a virtual radiator unit? We asked SGI's chief storage architect Lance Evans, who told us there's a secret in being tight but not too tight. When you push air into certain spaces under pressure, it's like breathing in through your mouth with your teeth shut tight.

"Imagine a cross-sectional slice through the chassis. At any given point along the front-to-back axis of the chassis is a certain cross-sectional air channel," explains Evans. "The smaller those channels are, the higher the pressure that the air channeling system has to be able to generate to pull more air through. Second, you have to be very careful about where that air flows. It needs to be able to flow over the components that are generating heat. If we have airflow through the machine, but it's not passing over components that are creating heat, then it's really not doing us much good. Every last little bit of air that you pull in, you need to be able to use effectively to cool the machine."

So SGI designed a high-pressure air movement system, comprised of six 60mm twin-axial, high-RPM fans positioned in the middle. They're stationed in such a way that, if one of the fans fails, it doesn't result in backflow.

Doing some math on the fly, Evans estimated the total power requirements for a fully populated SGI MIS rack at about 20 kW. That's on the high side of normal, compared to recent analysts' estimates, and perhaps a bit above normal for rack requirements circa 2009. But with firms like Dell now warning customers to expect as much as 30 kW per rack, MIS may be a viable solution for fitting big clouds into tight spaces.

Discuss
art belgiumEDW Lynch

Lichtfestival Gent

Lichtfestival Gent

Luminarie De Cagna, an Italian company specializing in illuminated installations, converted a street into a monumental facade and hall lit by 55,000 LEDs for the 2012 Light Festival Ghent in Belgium. The light installation is visible in the beginning of this light festival video by Lieven Vanoverbeke.

via Colossal

photos by Sacha Vanhecke

clairefontaine comparison exaclair fountain pen ink interesting moleskine quo vadis review rhodia testStephanie

Our UK friends at the Pens and Paper blog tested a number of notebook papers with four different fountain pens – each filled with a different ink. If you ever wanted to know how water based inks react on a certain paper, this is a great series of photos and commentary to help you choose the best product for your needs. The papers that were tested:

  1. Whitelines (grid);
  2. Rhodia Webnotebook (lined and dot grid);
  3. Quo Vadis Habana (lined);
  4. Monseiur (plain);
  5. Moleskine (lined);
  6. Leuchtturm 1917 (lined);
  7. Rhodia ePure (plain);
  8. Jottrr (lined/plain);
  9. Rhodia Exabook (lined);
  10. Clairefontaine (lined and Séyès ruled);
  11. Smythson Featherweight (lined); and,
  12. Archie Grand (plain).

Read the full post here.

baddeals groceryshopping letdowns shoppingPhil Villarreal

Sometimes items on the store shelves jump into your cart with the promise of better things than they deliver. Examples include food stuffs that look nothing like their glamour shots on the box and big bags of chips that are only half-full.

Len Penzo dot com points out some other grocery store letdowns:

* Anything that claims to have berries. Many products that claim to include blueberries -- such as cooking mixes -- actually use apples or a combination of maltodextrin and food coloring. Manufacturers figure that if something looks small and blue and tastes fruit-like then that's good enough for consumers.

* Vanilla flavored stuff. The spice is rather expensive, so food makers tend to go with a synthetic vanillin made with "coal tar derivatives."

* Juices that boast extra vitamins and antioxidants. Since the pasteurization process destroys natural vitamin C, juice factories replace it with ascorbic acid. You may as well get the vitamin in pill form.

The 5 Most Misleading Grocery Items Shoppers Waste Money On [Len Penzo dot com]

Dean Bubley (noreply@blogger.com)

One of the largest trends in enterprise IT right now is “BYOD” – standing for “bring your own device”. This is just a snappy acronym for what’s been happening for a while – employees using their own mobile phones, tablets or other products for work as well as in their personal life. Previously, it was given the less-cool name of “consumerisation of the enterprise”, although pedantically that also implies the use of consumer-grade services (eg Skype) as well as hardware.

BYOD is important for a whole host of reasons relevant to enterprise CIOs and their suppliers – security, management, fit with IT applications and so on. In the past, IT departments would have typically had a proscriptive “Thou shalt use company-approved Device X if thou wisheth to receive support”. Or in other words “Use your BlackBerry or E71 for business email, not your iPhone or Galaxy S”. But over time, the pushback has become more solid – often starting with C-level executives or top staff ignoring those edicts and demanding that IT support their favoured products. It’s a brave IT manager that will tell the CEO or top salesperson that they can’t use their iPad when they’re with clients.

But this post is not about those practical issues – it’s about how this impacts telcos.

The first point is that this potentially doesn’t just mean BYOD – it also implies BYOSP (bring your own service provider). Employees’ own devices, if used for business, are likely to be connected via a broad array of mobile network operators, or if WiFi-only, perhaps no SP at all. This is a completely different model to the idea of a corporate “fleet” of mobile devices all provided by the same company, with a bundled device+SIM deal. Instead, BYOD means that employees will have various SIMs, and various operator-customised versions of phones, plus some that are “vanilla” bought through retail.

This is a major problem for operators that have been trying to develop and sell enterprise mobility applications such as mobile PBX clients, or dedicated middleware for connection to back-end corporate applications. If a company’s IT department now has a mix of users with iOS, Android, Windows and other devices, connecting via Vodafone, O2, Orange and WiFi, it makes it much less likely that they will want (say) Vodafone OneNet or an Orange VPN client .

Instead, they will want applications that can work on any device (and OS/firmware build), running on any network. In other words, OTT-style functions using generic data connectivity – probably via the public Internet, but perhaps also via a dedicated connection like BlackBerry’s BES.

If you’re a regular reader of the blog, you can probably see what’s coming next:

If mobile operators seriously want to offer advanced mobile enterprise services, they are going to need to run them over their competitors’ networks, at least part of the time. Maybe they will be better when integrated with their own optimised device and network, but to reach the BYOD community they will need to push towards an OTT model themselves.

MNO services + BYOD + WiFi-only devices = mandatory Telco-OTT

That, needless to say, is easy neither for operators to accept, nor execute upon. Yet it will be essential, unless operators want to confine their enterprise exposure to the dwindling group of corporate-provided homogenous fleets of users.
This is one of the themes covered in Disruptive Analysis’ new report on Telco-OTT Strategies. If you're interested, contact information AT disruptive-analysis DOT com
cafe chocolate dining & travel evier france french fries frites garlic gaufre lille market meringue merveilleux mussels north parmentier pommes frites potatoes sausage sink snails three weeksDavid

Merveilleux

“Three weeks?! Is that all?” they laughed uproariously, as a response to my telling folks at a dinner party the other night about how much trouble I was having finding things like sinks, tiles, light fixtures, and so forth, for the renovations of my apartment. I literally spent weeks and weeks combing plumbing catalogs, scoping out different stores devoted to kitchen fixtures, and relying heavily on our friend, the internet, in search of a plain, large, white sink.

I don’t want swoops and swirls, (and I only have one more Ikea visit left in me, and I’m banking that for something really important) – I want a generous basin that’s large enough to hold a few pots and pans. And I’m not interested in a purple or green one. You wouldn’t think it would be all that hard – and neither did I – but after three solid weeks (and I mean, twenty-one days and twenty-one nights), I finally found one in France. The only problem? It was in Lille.

Merveilleux Windmill in Lille

So we decided to make a day trip up to the city in the North, just a few hours from Paris, and while there, eat some of the local fare. Because things are so frantic right now — imagine if I took me three weeks to find a sink…then I really need to get cracking on a toilet, a towel bar, kitchen cabinet handles, a soap dish, and light bulbs — so I don’t have a huge amount of time.

Continue Reading Lille, Aux Moules, and a Sink...

apple community mw2012 newschris.foresman@arstechnica.com (Chris Foresman)

For 2012, the long-lived Macworld Expo changed its branding to "Macworld|iWorld" and began billing the conference as the "ultimate iFan event." The annual show, which has been running continuously for the last 27 years, has had a bit of a rocky transition since Apple announced in 2009 that it would no longer attend. Though the show no longer features Apple keynote presentations and some of the larger vendors like Microsoft and Adobe haven't had a significant show floor presence for a few years now, the event still somehow manages to get consumers lined up around the block to see new products or apps, talk to product managers and developers in person, or attend a series of events that cater to Apple fans of all stripes.

At the last Apple-attended Macworld in 2009, the floor actually covered two of the three expo halls at the Moscone Center in San Francisco. Since Apple left, the show floor has shrunk down to just cover Moscone West. That has made it easier for the press to cover the show's myriad booths without feeling overwhelmed, but it also means that some of the larger vendors have shied away, and companies have much smaller booths than they might have in the past.

Macworld|iWorld also typically takes place within a few weeks of the annual Consumer Electronics Show, the behemoth conference held in early January in Las Vegas. But without Apple anchoring Macworld, many companies that used to make use of Macworld to launch new products are instead opting to launch products at CES. Much of what we saw on the floor was a repeat of what we saw just a couple weeks ago in Las Vegas.

As a tech journalist, Macworld has gradually become less interesting. As a user of Apple's products and a geek in general, however, the show has become perhaps more interesting.

A cultural experience

The Macworld All Star Band, which performs at the annual Cirque du Mac after party, consists of long-time Apple journalists and writers including (l to r) Chris Breen, Dave Hamilton, Chuck La Tournous, Bob LeVitus, Macworld|iWorld organizer Paul Kent, and Bryan Chaffin.
The Macworld All Star Band, which performs at the annual Cirque du Mac after party, consists of longtime Apple journalists and writers including (l to r) Chris Breen, Dave Hamilton, Chuck La Tournous, Bob LeVitus, Macworld|iWorld organizer Paul Kent, and Bryan Chaffin.

In the past, the Macworld Expo consisted of the trade show floor as well as a $300-per-person conference that featured different tracks such as "content creation" or "IT." Macworld|iWorld has done away with that to a certain extent. On the first floor of Moscone West this time around was the usual trade show exhibits, and consumers could opt for a $45 pass just for the show floor. Unlike CES, vendors are allowed to sell their wares, and they usually offer special show-only discounts directly to consumers.

However, an extra $80 netted users an "iFan" pass, giving access to a variety of "tech talks" and panels on the second floor. Tech talks were generally shorter presentations covering a variety of topics like music production on a Mac, getting the most out of HTML5, the best iPad apps for work, and playing PC games on an iPad via desktop sharing. Full-day workshops (which did cost extra) were also available the day before the show, covering iOS device deployment in enterprise, using Apple technology in education, automating Mac OS X, and becoming a better photographer.

Additionally, IT professionals could opt for an $800 MacIT conference pass to attend the MacIT conference on the third floor. Enterprise-focused vendors, such as FileMaker and JAMF, had exhibits here. There were also IT-focused workshops and tech talks throughout the show's three days.

Augmenting the tech talks and panels were a number of other cultural events, including an electronic music show with artists that produce their songs on Macs or iPads, galleries of digital art produced using Macs, and two nights of film screenings consisting of short films shot (and sometimes edited) entirely on iPhones.

The show attracted several celebrity speakers, including actor Jonathan "Number One" Frakes, author Susan Orlean, hip hop producer Hank Shocklee, and comedian Rob Corddry. The popular jamband Moe played a show which included an entire set of songs played entirely with iPads, and discussed how iPads and iPhones have aided their songwriting. (Regular attendee Sinbad, who gave a keynote presentation at last year's Macworld, was also spotted on the show floor.)

Popular jamband moe. performed an iPad-only set during one of the many iFan events throughout the weekend.
Popular jamband Moe performed an iPad-only set during one of the many iFan events throughout the weekend.

In this way, Macworld|iWorld has morphed from more of a traditional trade show to more of a festival. It's sort of like a Burning Man for Apple nerds, but with far less heat and dirt.

"This is a year that has been so different from other years," Vice President and General Manager of Macworld|iWorld Paul Kent told Ars. "We've done so many things differently than before—the way we marketed, the events and speakers we booked."

Against all odds, it seems to be working. Traffic was light but steady throughout Thursday and Friday, but Saturday morning had attendees lined up out the door and all the way down the block for several hours. Though official numbers won't be released until Tuesday of this week, Kent told us that attendance was at least 10 percent higher than 2011—roughly 25,000 attendees (not including media or exhibitors).

"A lot of people came just for the show floor, and when they got here heard about the iFan events and upgraded," Kent said. "We've made that pass crazy affordable compared to past Macworld shows. When you line up some of the events, like Hank Shocklee's brilliant presentation on dealing with copyright and other issues for musicians in the digital age, it's like, 'holy cow what a deal.' This isn't an industry insider only event—anyone can come and see all this great stuff."

Bringing it back

Though the show didn't have as much appeal for the media this year as it has in the past, Kent believes that it doesn't have to stay that way. "Journalists still attend the event in large numbers, because Macworld has always sort of had this reputation that some small company off in the corner might be the next big thing," Kent said. "There's something like a two-to-one ratio of journalists to companies that are exhibiting, so there's a real opportunity for vendors to connect with journalists."

For instance, The Omni Group CEO Ken Case took time to sit down with us and give us a full overview of its upcoming software releases for 2012. HP's marketing manager for the Mac segment also gave us an extensive tour of its products, detailing compatibility with Macs and iOS devices.

Companies like Omni, long Mac OS X platform supporters, continue to have a large presence at the show.
Companies like Omni, longtime Mac OS X platform supporters, continue to have a large presence at the show.

Beyond that, though, companies have an opportunity to connect directly with users that few other forums permit. Again using Omni as an example, the company held demos and workshops all day long at its booth showing users how to take advantage of its Mac OS X and iPad apps. Smaller Mac and iOS developers had booths in the Mac OS X Zone and iOS Zone where they gave demos to individual users. IK Multimedia let users try out prototypes of an upcoming iOS DJ mixer which won't ship until later this quarter. And Seagate was selling a handful of Thunderbolt adapters for its GoFlex line of portable hard drives before that product ships in the next few weeks.

"We have shaped the show to appeal to a younger audience that maybe hasn't come to every Macworld for the last 25 years," Kent explained. "That's an opportunity to reach a whole new demographic for some companies."

The annual San Francisco tradition will continue on next year, as Macworld|iWorld 2013 is already scheduled for January 31 through February 2. Kent hinted that we may see more companies taking better advantage of the opportunity to reach out to both the press and the growing community of Mac and iOS users. And he told us that there are more ideas for more and varied events for next year, which he expects will bring back alumni as well as attract new visitors.

Macworld|iWorld has certainly become more of an "experience"—something you need to see first-hand—and as a journalist, that's hard to translate into words. For us at Ars, that means rethinking the way we cover this show, because "the ultimate iFan event" isn't likely to go back to the olden days of Macworld Expo yore.

Read the comments on this post

Pushing up to 384GB per socket

It looks like Advanced Micro Devices is first to market with support for load reduced DIMM DDR3 main memory for x86 and quite possibly all kinds of servers, and is trotting out Inphi, the maker of the isolation memory buffer chip that is at the heart of this technology.…

accessoriesAndrew Everard
BHA-1 has both balanced and single-ended headphone outputs, dual mono design and selectable gain, will sell for £1425

read more

leather goodsWill (noreply@blogger.com)

In 1973, divers off England's Plymouth Sound found the wreck of the Catharina von Flensburg, an eighteenth century brigantine that sank in 1786 with a cargo of reindeer hides. They had been cured in baths of rye or oat flour and yeast, hand embossed before being soaked in wood liquor and finally hand curried and soaked in seal oil and birch tan oil. The result is a unique finish that cannot be replicated.

Though covered with mud for centuries, the hides proved to be water resistant and still very serviceable. Bundles have been periodically brought to the surface and sold by the divers who discovered them. They are dried, cleaned and sorted in a small workshop in Cornwall where some are made into attaché cases, belts and other leathergoods on the spot. Others are sent to London to be made into shoes in London by bespoke shoemakers G. J. Cleverley .

There is some question as to how long the supplies of hide will be available. I have heard it estimated that half of them still lie in the mud of the seabed, but the diver who was given rights to them has retired and there is no successor in sight. For now, Cleverley continues to deliver a small supply of products from two hundred year old Russian leather.
Photos: G. J. Cleverley
Ivan Pepelnjak (noreply@blogger.com)

Most interesting article in this batch: Ethernet Taps - Don't Get Me Started by Chris Marget, focusing on Ethernet taps: passive, active, aggregators, L1 switches ...

And here are the other interesting links I found in somewhat random order:

Read more ...
gamesTyler Cowen

Here is an excellent and varied article on that topic, by Ryan Dancey.  Excerpt:

The more segmented those brains became, the weaker the overall social network was. Every new game system, and every new variant to those systems, subdivided that network further, making it weaker. Between 1993 and 1999, the social network of the TRPG players had become seriously frayed. Even if you just looked at the network of Dungeons & Dragons players you could see this effect: People self-segmented into groups playing Basic D&D, 1st Edition, 2nd Edition, and within 2nd Edition into various Campaign Settings that had become their own game variants. The effect on the market was that it became increasingly hard to make and sell something that had enough players in common that it would earn back its costs of development and production.

We looked around the industry and saw the same problem at virtually every company that had become successful: White Wolf had 5 World of Darkness games which were all slightly different, surrounded by a more diffuse constellation of games somewhat related to the Storyteller system but designed to be mutually incompatible. FASA had 4 games, none of which shared anything in common. Palladium & Steve Jackson Games both had “house systems” that they tried to use across their entire product lines, but they had ended up with the “Campaign Setting” issue that was bedeviling TSR; the variant rules at the edges of their games were creating independent game networks despite the shared DNA of the core. And we knew that inside every one of those companies they were seeing the same financial information we were seeing: Each new release was selling fewer and fewer copies, and in response, the companies were increasing the pace of releases trying to sustain planned revenues by volume of titles, not by volume of units. And it was killing everyone.

…My opinion is that the hobby gaming industry is going to transform into a very small niche business. It will cater primarily to an aging group of players who have made TRPGs their lifetime hobbies. As those players age, they’ll need less and less support in the form of commercially produced products. They will instead seek out community support tools to help them remain in touch with their hobby even as the social network they’re directly connected to becomes ever more frayed.

The piece is interesting throughout, and for the pointer I thank Will Koenig.

card payments google wallet mobile payments mobile technology mobile wallets point of sale (pos) purchase details russ jonesRuss Jones

Russ Jones - Glenbrook Partners

Who knows what in payments? Sounds like a question begging for a snappy one-liner comeback.

But this is a question we hear more and more from our clients. Particularly those that see the increasingly close relationship between payments and marketing — and want to better understand how the targeting of ads and offers can be enhanced with insight from payment transaction data. The real question, then, is who knows what in payments and do mobile wallets change anything?

I’ve discussed the underlying purchase “visibility” issue before and argued that few players in the card ecosystem have a complete view of the consumer and what they buy. But before we look at how mobile wallets might change things, let’s review purchase visibility in the pre-wallet world.

Of the various stakeholders in the four corner card model, it is the issuer and the merchant who are closest to the consumer’s purchase behavior. While card networks, merchant acquirers, and the various processors have some insight into the transaction, they don’t necessarily know what the transactions represent or have the right to unilaterally use or repurpose what they do know.

From the consumer’s perspective, card issuers know who you are (know-your-customer laws in the U.S. actually require them to know this) and have some insight into your financial situation, depending on the type of card they issue. They know where you shop by merchant category and merchant name. Of the two, merchant category is a lot more important than merchant name. There are several hundred big “name” brands that drive a lot of transactions but there are millions and millions of merchants in the card system. And besides, sometimes the merchant name is truncated or nonsensical.

Beyond the categories, which can be helpfully narrow or frustratingly broad, issuers have no idea what consumers actually buy. Was that $34.75 purchase from Walmart for baby diapers or shotgun shells? And while they know where you shop, they don’t have a view of your overall purchase behavior — just where you you shop when you buy something using their card.

Retailers know what you buy, but they don’t know who you are. Unless you tell them. And they are really good at getting you to tell them. Want the special discounted price on select items? You need to sign up for the members club or reward program. Don’t want to carry another loyalty card? No problem, just share your telephone number with them on each purchase. That way you’ll be guaranteed every benefit you have coming!

In addition to helping them understand who you are, these types of programs help retailers correlate your purchases across visits, across payment methods, and across shoppers in the same household. Online retailers know even more than physical retailers, as they can use cookies to see how often you visit, track your movement through their online store, and analyze what is added and removed from your shopping cart. But online or offline, they don’t know everything you buy, they only know what you buy from them.

It’s tempting to say that only the consumer has a full view of their own purchase behavior. But that may be overstating it. I’m not sure, for example, consumers always remember where they shopped or what they bought. That comes up a lot in the chargeback process. How many times a day do issuers hear a cardholder say, “I’ve never heard of this merchant and I’ve certainly never bought anything from them.”

That’s who knows what in payments. Now, how do mobile wallets change things?

It would seem at first blush that the mobile wallet might know a lot, as it is extremely close to the consumer and is involved, after all, in the transaction. But what will it know?

  • Does it know who you are? There are no anonymous wallets. As a result of the registration and activation process, the mobile wallet knows who you are at least in terms of name, phone number, email, bill-to address, etc. Maybe it doesn’t know if you rent or own your home, but it certainly has enough information to target you (oops, I meant contact you.)
  • Does it knows where you shop? Well, it kind of knows where you shop. It might know from the GPS location that you are making a purchase in Redwood City, but it doesn’t have the accuracy to figure out if you standing in PetSmart or next door in Bed, Bath, and Beyond. There’s an outside chance, however, that it might be able to figure out which store based on available WiFi hotspots and signal strength.
  • Does it know how you pay? Because most mobile wallets will be multi-card and multi-brand, in theory, the wallet will also see purchase behavior across issuers and across card brands depending on what cards the consumer uses. From this, it will know which card is “top of wallet” in terms of usage frequency, but not necessarily in terms of “spend”. But it only knows how you pay when you use it (and not another wallet) to make a purchase. You might have multiple wallets on your phone, none of which would have a complete view of how you pay.

So the mobile wallet knows a lot, but does it know what the consumer buys? That’s the million dollar question. Actually, probably a multi-hundred million question. What the wallet knows about what you buy is circumstantial. I’ll use Google Wallet as an example, because it’s in the market now, but think the story will likely be true of all the major mobile POS wallets.

The Google Wallet has three real usage modes at the POS:

  • Tap and Pay. This is for merchants that have contactless terminals, but haven’t agreed to change their POS environment to accept Google offers. In this mode the wallet emulates a passive contactless card — passive in the sense that the wallet doesn’t have a dialog with the merchant about what is happening at the POS. The wallet knows that you just made a purchase, but other than the GPS location of the transaction, the wallet doesn’t know the merchant name, merchant category, purchase amount, or what the purchase represents. The issuer knows, but Google Wallet doesn’t know.
  • SingleTap. This is for merchants that have contactless terminals, and have agreed to customize their POS environment to support the Google Wallet initiative. In this mode the wallet has a peer-to-peer dialog with the merchant’s POS terminal as it hands over coupons to be redeemed, the card data to pay for the purchase, and any merchant-specific gift cards or loyalty card. We don’t know the specifics of what is included in the peer-to-peer dialog, but hypothetically you can imagine this being a pretty data “rich” exchange. Even if the transaction is not especially data rich, part of the dialog will inevitably involve the merchant handing the mobile wallet the purchase receipt. Talk about line-item purchase data!
  • Offer Presentation Mode. This is for merchants that haven’t upgraded their POS devices to support contactless payments, but want to accept Google offers. In this mode, the payment is handled outside the wallet (most likely with a swipe) and the wallet is just used to visually present the offer for redemption. The merchant can scan the offer barcode or key in the offer code. Here the wallet doesn’t even know that a purchase was just made; it just knows that an offer has been presented. Perhaps somewhere in the Google cloud they know that an offer has been redeemed, but there is no guarantee at this point that it is automated or information rich. Either way, the mobile wallet doesn’t get its hands on the receipt and there is no line-item data provided by the merchant.

So does the mobile wallet know where the consumers shops and what they buy? Using Google Wallet terminology, in SingleTap mode it very well might. Especially if they can get merchants to send the purchase receipt back to the wallet. In Tap-And-Pay and Offer Presentation Mode, it won’t. And while its premature to know for sure, my gut tells me that most of the transactions will be tap-and-pay transactions.

So, as you can see, no one party knows everything that a marketer would ideally like to know. But some information is better than none, and successful campaigns have been built around partial data sets like card transactions or purchases made at a single retailer.

But if the great promise of mobile payments really lies in mobile marketing, then the next generation of killer apps will likely result from creative business partnerships that meld specific purchase information, analytical insights, and message delivery capabilities in a way that balances the interests of multiple participants and produce a comfortable and valued experience for the consumer.

green techEvan Ackerman
Solar power cheaper than fossil fuels for 1.3 billion people

Here in the U.S., we're used to thinking about solar power as one of those happy eco-friendly things that we'd all totally be using except for the fact that it's so much more expensive than fossil fuels. In the developing world, though, it's exactly the opposite: solar power is gaining ground with 1.3 billion people simply because it's the cheapest way to go.

foodRusty Blazenhoff

By simply using one of those all-edge brownie pans, Adam Kuban at Slice, the pizza-centric blog at Serious Eats, has created a mighty fine all-edge sicilian-style pizza. Abbondanza!

You’ve seen the all-edge brownie pan, right? The first time I saw it, one thought popped into my twisted mind: “F*** brownies. Hello, pizza!”* After all, on a square pizza, be it Sicilian or grandma or Detroit-style, the corner is king followed closely by the edge…The center is for jokers…

photos by Adam Kuban

concurrency hardware opinion & editorial software developmentHerb Sutter

With so much happening in the computing world, now seemed like the right time to write “Welcome to the Jungle” – a sequel to my earlier “The Free Lunch Is Over” essay. Here’s the introduction:

Welcome to the Jungle

In the twilight of Moore’s Law, the transitions to multicore processors, GPU computing, and HaaS cloud computing are not separate trends, but aspects of a single trend – mainstream computers from desktops to ‘smartphones’ are being permanently transformed into heterogeneous supercomputer clusters. Henceforth, a single compute-intensive application will need to harness different kinds of cores, in immense numbers, to get its job done.

The free lunch is over. Now welcome to the hardware jungle.

From 1975 to 2005, our industry accomplished a phenomenal mission: In 30 years, we put a personal computer on every desk, in every home, and in every pocket.

In 2005, however, mainstream computing hit a wall. In “The Free Lunch Is Over” (December 2004), I described the reasons for the then-upcoming industry transition from single-core to multi-core CPUs in mainstream machines, why it would require changes throughout the software stack from operating systems to languages to tools, and why it would permanently affect the way we as software developers have to write our code if we want our applications to continue exploiting Moore’s transistor dividend.

In 2005, our industry undertook a new mission: to put a personal parallel supercomputer on every desk, in every home, and in every pocket. 2011 was special: it’s the year that we completed the transition to parallel computing in all mainstream form factors, with the arrival of multicore tablets (e.g., iPad 2, Playbook, Kindle Fire, Nook Tablet) and smartphones (e.g., Galaxy S II, Droid X2, iPhone 4S). 2012 will see us continue to build out multicore with mainstream quad- and eight-core tablets (as Windows 8 brings a modern tablet experience to x86 as well as ARM), image_thumb99and the last single-core gaming console holdout will go multicore (as Nintendo’s Wii U replaces Wii).

This time it took us just six years to deliver mainstream parallel computing in all popular form factors. And we know the transition to multicore is permanent, because multicore delivers compute performance that single-core cannot and there will always be mainstream applications that run better on a multi-core machine. There’s no going back.

For the first time in the history of computing, mainstream hardware is no longer a single-processor von Neumann machine, and never will be again.

That was the first act.  . . .

I hope you enjoy it.


Filed under: Concurrency, Hardware, Opinion & Editorial, Software Development
att taking it seriously verizon voip youcanthearmenowMary Beth Quirk

Now that telephone and cable companies have increasingly moved away from using the old tried and true copper lines to provide landline service, you might find yourself without a phone in a power outage. Our seriously smart siblings at Consumer Reports looked into the drawbacks of landline fiber optic and VoIP telephone systems.

During a power outage, the newer systems aren't able to maintain landline phone service indefinitely like the old copper lines can. Today's systems usually provide around eight hours of standby service, says Consumer Reports, and then only if they happen to have an in-home battery backup.

That means if cell phone lines also fail, no dialing out for emergency supplies of cheese or letting your mom know you're okay and have plenty of bottled water, or more importantly, no 911.

Even as many companies switch to VoIP and fiber systems that keep the copper lines next to the new lines, customers aren't always aware they can opt for the traditional service.

Consumer Reports found that companies like Verizon, AT&T and others include warnings to consumers in their terms of service, which entails that the customer knows that in cases of service disruptions or power outages, they know they will not be able to place or receive calls to 911.

One reason copper may be falling out of popularity with phone companies -- federal law requires them to share copper lines with competitors, but there is no such rule for fiber.

For more info on the pros and cons of copper lines, and whether you should ask your provider about using them, check out ConsumerReports.org.

Surprise! Your high-tech home phone system could go dead in an emergency [Consumer Reports]

Michael Johnston

Bluemarble

NASA released this "blue marble" picture two days ago. It was taken by the Visible/Infrared Imager Radiometer Suite (VIIRS) aboard NASA's most recently launched Earth-observing satellite, Suomi NPP, named in honor of the late Verner E. Suomi of the University of Wisconsin.

Granted, there's a fair amount of distortion in this—Mexico isn't actually as big as South America—but it's beautiful. There's a very large version available for download at the NPP page at nasa.gov.

Vern Suomi, considered the father of satellite meteorology, invented the device that for many years showed us those moving cloud- and weather-progression images on the evening news.

Mike
(Thanks to Doug Dolde)

Send this post to a friend
Please help support TOP by patronizing our sponsors B&H Photo and Amazon

Note: Links in this post may be to our affiliates; sales through affiliate links may benefit this site. More...
Original contents copyright 2011 by Michael C. Johnston and/or the bylined author. All Rights Reserved.

mtomkins@imaging-resource.com
For years, pundits and industry leaders alike have predicted the end of the optical viewfinder, but enthusiast and pro photographers have stubbornly clung to them for their benefits over electronic viewfinders. Today, a new microdisplay from French company MicroOLED S.A.S. promises to take a big step towards erasing those advantages. MicroOLED's latest microdisplay model has a resolution of over five million square dots, with a dot pitch of 4.7 microns, and is said to have no gaps between pixels. That's an astoundingly high resolution -- it's more than double that of Sony's XGA (1,024 x...
(read more)
Seth Finkelstein, a long-time crusader against online censorship, made what seemed like a jaundiced comment on my recent post Piracy and Privacy. I had raised the possibility that online activists, fresh from their SOPA fight, might now come to the support of efforts to give people more control over the personal information that companies collect and trade online. Will the activists rise up again? I wondered. To which Seth replied: No. Or maybe, they will rise up AGAINST privacy, because they will be fed a line that this is going to Censor The Net. Turns out Seth wasn't being jaundiced. He was being prescient. Shortly after he made his comment, a Harvard Law School blog posted a lathery rant, under the judicious title "More Crap from the E.U.," by Jane Yakowitz, a professor at the Brooklyn Law School. Yakowitz blasted the European Commission's new proposal to strengthen online privacy protections. Europe, she wrote, has been "flailing around" with internet regulation. It has enacted "miserable" policies. The EC's reasoning is "hogwash." Its actions are "regressive." Its proposed new directive represents "a misguided attack on the information economy." Goodness. I think Professor Yakowitz must have eaten a bad mussel in Brussels once....
apple ios ipad macosx mw2012 news omni productivity projectmanagement softwarechris.foresman@arstechnica.com (Chris Foresman)

Users who rely on the desktop version of OmniPlan but wish to manage complicated projects and workflows on-the-go will soon be able to pinch and swipe their way to Gantt chart nirvana. According to Omni Group CEO Ken Case, project management app OmniPlan will make its way to the iPad, perhaps as soon as the second quarter of this year. Case was on hand at the Macworld|iWorld 2012 conference in San Francisco to give Ars a sneak peak at an early build of the mobile version, as well as fill us in on the company's plans for the next year or so.

When we spoke to Case at the Macworld Expo last year, he told us about the company's plans to update OmniPlan on Mac OS X to version 2.0. That version included a major overhaul to the backend engine, which improved cloud syncing abilities and offered integration with the GTD-based task manager OmniFocus. OmniPlan for the iPad will use the same engine, making it possible for multiple users to update project plans simultaneously. Those changes can be reviewed, approved, or rejected using the iPad just as on the desktop.

iPad users will be able to tap and swipe to organize and plan long term projects when OmniPlan comes to iOS later this year.
iPad users will be able to tap and swipe to organize and plan long term projects when OmniPlan comes to iOS later this year.

The early build we saw this week wasn't complete, but the UI made full use of pinch and swipe gestures to control how much of a project's timeline was in focus at any given time. Multitouch gestures also make it easy to manipulate task times and connect tasks and milestones to others to establish dependencies.

An early look at the interface for the upcoming OmniPlan for iPad.
An early look at the interface for the upcoming OmniPlan for iPad.

Though Case didn't want to commit to a hard deadline, he told Ars that Omni is shooting for a release in the second quarter of 2012. And while he didn't mention a price, he noted that "if you follow our pricing strategy, you can make a very good guess." In other words, expect to pay half of the desktop version's $199.99 price.

When OmniPlan for iPad ships, it will complete Case's original 2010 plan to bring five unique apps to the platform. With that line item crossed off of his OmniFocus task list, full attention will go back to Omni's Mac software. Specifically, Omni developers have been hard at work planning OmniFocus 2 and OmniOutline 4. Updating those apps will become the top priority of the development team once OmniPlan for iPad is launched on the App Store, according to Case.

That doesn't mean the company's iOS apps are getting ignored, however. OmniGraffle, OmniGraphSketcher, and OmniOutliner are expected to get updates shortly which will offer syncing via iCloud or Omni Sync Server. To go along with the update, the apps will get a new document management view to easily access all available documents.

Omni's document-based iPad apps will get a new file management UI when updates enable iCloud syncing.
Omni's document-based iPad apps will get a new file management UI when updates enable iCloud syncing.

The company has considered adding integration with Dropbox and other cloud-based storage systems, but those services have compatibility issues with Omni's bundle-based file formats. "The bundle offers advantages, especially when it comes to writing files to an iOS device or to iCloud," Case told Ars. "But from a user perspective, it can cause some problems."

Bundles are essentially specially formatted folders that are treated as a single file in Mac OS X (and by extension, iOS). It might contain text files, XML-formatted data, images, and other files that are linked together. If one small bit of a bundled file is edited, say in the text portion, only the text file within the bundle is changed, and only that part of the bundle is written to disk or synced to iCloud. The rest of the files within a bundle remain unchanged, so no additional filesystem updates or cloud syncing operations are needed.

Case said the company is considering alternatives to using bundles which may maintain the speed and ease of updating a bundle but storing data in a monolithic file which can work across various cloud-based platforms.

With Omni's initial five-app plan nearly complete, and progress being made on the company's Mac OS X apps, what else does the company have in store? "There are things we would like to do," Case said. "We have some ideas, but we don't want users of our current app to feel abandoned—they are so passionate and loyal."

Read the comments on this post

amazon compromisedaccounts fraud kindle thecloudLaura Northrup

The cloud of invisible information that surrounds is is a wonderful thing, but there are dangers as well. Brandon ordered a Kindle as a gift for his girlfriend, and upgraded to one-day shipping, but the package went astray. Amazon overnighted a new Kindle and things were glorious...until Brandon started receiving purchase confirmations of Kindle apps and content using his credit card and e-mail address. His girlfriend wasn't making the purchases. So who was?

Last week I placed an order on Amazon which I use frequently like most of the universe for various purchases. I was ordering the Kindle as a gift for my girlfriend. Wanting to expedite the process, I decided to pay for the one day shipping for only $15. The next day I watched as the tracking info never updated. This isn't uncommon for quick deliveries because often the package arrives before the systems know what's going on. When it didn't arrive that day, I was disappointed, but understand things happen.

Knowing Amazon usually has exceptional customer service; I called up rather late at night (10:30pm EST) and spoke to a friendly agent. She refunded me my $15 for shipping and explained to me the package seemed to hit a snag at some point and it would arrive within the next two days. Again, I was irritated by the delay but understand things happen and went on with my life.

Two days go by and the package still does not show up. Now frustrated one notch higher than I was before, I called up customer service again. This customer service agent again was very friendly, but explained the Kindle had been lost in shipment. Explaining how I've now waited several days past when I originally wanted the package, they offered to next day me another Kindle as soon as they possibly could. Amazingly the Kindle arrived 14 hours later, and on a Saturday no less!

I gave the gift to my girlfriend and she was very happy with it. I was happy with the response Amazon gave me despite what happened being beyond their control.

Over the past few days I started getting weird emails from Amazon. I would get a dozen plus emails at a time displaying purchases made on a Kindle. I asked my girlfriend if she had used my email to register the Kindle thinking that could explain the purchases showing up in my email. This wasn't the case as she explained to me that she used her own information to register it. I let it go for a day thinking it may be just an error of some sort. All of the purchases up to that point were for free apps.

The next day I got another hoard of emails and that's when it hit me. The first Kindle that was lost in shipment was being used by someone, and better yet, this someone had access to all of my information including the credit card I used to purchase the Kindle!!!

I immediately called Amazon and explained what was going on. This time it took some effort to really get the customer agents to understand what I was saying. At first they thought I was telling them that my girlfriend was making purchases on her kindle using my credit card. After I explained to them that this wasn't the case and she had registered the Kindle with her own card and email, they finally understood what I was saying.

My suspicions were confirmed. Someone else had gotten their hands on the first Kindle lost in shipment and because the Kindle came preloaded with my name, email, address and credit card information, this person(s) was able to make purchases on this Kindle. I spoke with the customer service agents who after understanding did their best to help me. They refunded all the purchases that were made. By the time I discovered this, the fraudulent user had made several big purchases of entire TV seasons. They also deactivated the Kindle so the person could no longer use it.

I was chatting with my girlfriend as this happened while I was at work, and as soon as the deactivation occurred my girlfriend informed me that now her Kindle had been shutdown. The customer service agents had deactivated either the wrong one, or both Kindles. I now had to reinitiate a chat and explain this to another customer service agent. Now that the story had grown more complicated, it required yet even more details and time to get the agent to understand the situation. Better yet, now my girlfriend had to call Amazon and prove to them that she was who she was, and that I was who I was so they could reactivate the Kindle.

Forty-five minutes later, my girlfriend's Kindle was back in action. I have yet to receive any more emails informing me of purchases being made on my account which is a good sign.

I love Amazon. They make my life so much easier. What I do not like is them sending out extremely critical information pre-loaded onto Kindles. No one should be able to simply open a box and begin purchasing items on someone else's credit card. I am lucky the user of the lost Kindle seemed to not really understand this because they didn't begin purchasing anything until they were 3 days into it.

I am usually quite careful with my personal information, and it really bothered me that it was so available to someone else. I just wanted to let the Consumerist know about this so prospective buyers of Kindles or other electronic products can be wary of what is being stored in the device before you ever get your hands on it. I will admit I made the purchase rather quickly, but I didn't see anything about having the Kindle pre-loaded with information. While they are trying to make people's lives convenient, and I'm sure for anyone who got their Kindle it was, it had the opposite effect on mine.

By the way, my girlfriend loves the Kindle. I like it too, but it immediately left a bad taste in my mouth for dealing with all the issues in getting one (and subsequently getting rid of one which I never got in the first place...)

aircraft airfoils d-dalus february 2012 headlines hovercrafts scienceKaitlin Miller

Last year, the Austrian engineering firm IAT21 set out to construct a flying machine that floated like a hummingbird, traveled as fast as a jet, was as quiet as a hot-air balloon, and was simple enough that a car mechanic could repair it. The company's working prototype, called D-Dalus, is roughly five feet by three feet square and can lift about 100 pounds. But the size and lift are not what's most impressive. A flying machine with no airfoil, rotor or jet propulsion can travel where most cannot: in very tight spaces and through terrible weather.

ROTOR ASSEMBLIES

The craft's four rotors spin at 2,200 rpm, and six blades attached to carbon-fiber disks create directional thrust. The blades act as mini airfoils, their angle of attack constantly shifting in relation to rotation. For vertical lift, a blade's leading edge rises away from the center of the disk at the top of its rotation and toward the center of the disk at the bottom [pictured], creating a pressure differential.

FRICTIONLESS BEARINGS

Existing bearings were unable to withstand 1,000 Gs of force between the carbon-fiber disks and their blades and still deliver some degree of maneuverability. Engineers at IAT21 developed their own bearings, shaped like metal barrels, that hold up to the force better than spheres (think: arches) but can still roll enough for the blades to move.

D-Dalus Rotors:  Graham Murdoch

AUTOMATIC STABILIZATION

Servo motors communicate with the rotor assemblies to automatically correct the craft's speed, position and balance by adjusting the blades' angle. If the pilot jerks the radio controls too hard in one direction, the craft will keep itself from pitching or yawing by increasing opposing thrust. The system can adjust for turbulence and heavy winds, too.

ADVANCED NAVIGATION

Radar, GPS and three multispectral cameras (visible, EHF-extremely high frequency- and infrared) act as the D-Dalus's eyes. Visual information is fed into the craft's collision-avoidance algorithm. The system is so sensitive that D-Dalus can fly within inches of power lines, hover above moving platforms (a ship's deck in rough seas, for example), or refuel another D-Dalus in flight.

accessories aesthetics notebook case pebbled leather practicality silver hardware trudeauadmin
RSS Post Image

The infallible combination of practicality and refined elegance is what WANT LES ESSENTIELS DE LA VIE is always good for with this laptop bag being no exception to it. An understated blend of supple pebbled leather and polished silver hardware cares for the considered aesthetics on the conference table while the organized innards provide the right dose of utility to make it a valuable notebook case.


BUY for $1195
amazonec2 azure business cloud distributedfilesystems features googlerelated hadoop newssean.gallagher@arstechnica.com (Sean Gallagher)

Consider the tech it takes to back the search box on Google's home page: behind the algorithms, the cached search terms, and the other features that spring to life as you type in a query sits a data store that essentially contains a full-text snapshot of most of the Web. While you and thousands of other people are simultaneously submitting searches, that snapshot is constantly being updated with a firehose of changes. At the same time, the data is being processed by thousands of individual server processes, each doing everything from figuring out which contextual ads you will be served to determining in what order to cough up search results.

The storage system backing Google's search engine has to be able to serve millions of data reads and writes daily from thousands of individual processes running on thousands of servers, can almost never be down for a backup or maintenance, and has to perpetually grow to accommodate the ever-expanding number of pages added by Google's Web-crawling robots. In total, Google processes over 20 petabytes of data per day.

That's not something that Google could pull off with an off-the-shelf storage architecture. And the same goes for other Web and cloud computing giants running hyper-scale data centers, such as Amazon and Facebook. While most data centers have addressed scaling up storage by adding more disk capacity on a storage area network, more storage servers, and often more database servers, these approaches fail to scale because of performance constraints in a cloud environment. In the cloud, there can be potentially thousands of active users of data at any moment, and the data being read and written at any given moment reaches into the thousands of terabytes.

The problem isn't simply an issue of disk read and write speeds. With data flows at these volumes, the main problem is storage network throughput; even with the best of switches and storage servers, traditional SAN architectures can become a performance bottleneck for data processing. 

Then there's the cost of scaling up storage conventionally. Given the rate that hyper-scale web companies add capacity (Amazon, for example, adds as much capacity to its data centers each day as the whole company ran on in 2001, according to Amazon Vice President James Hamilton), the cost required to properly roll out needed storage in the same way most data centers do would be huge in terms of required management, hardware, and software costs. That cost goes up even higher when relational databases are added to the mix, depending on how an organization approaches segmenting and replicating them.

The need for this kind of perpetually scalable, durable storage has driven the giants of the Web—Google, Amazon, Facebook, Microsoft, and others—to adopt a different sort of storage solution: distributed file systems based on object-based storage. These systems were at least in part inspired by other distributed and clustered filesystems such as Red Hat's Global File System and IBM's General Parallel Filesystem

The architecture of distributed file systems separates the metadata about content from the data itself, allowing for high volumes of parallel reading and writing of data across multiple replicas, and tossing concepts like "file locking" out the window. 

The impact of these distributed file systems extends far beyond the walls of the hyper-scale data centers they were built for— they have a direct impact on how those who use public cloud services such as Amazon's EC2, Google's AppEngine, and Microsoft's Azure develop and deploy applications. And companies, universities, and government agencies looking for a way to rapidly store and provide access to huge volumes of data are increasingly turning to a whole new class of data storage systems inspired by the systems built by cloud giants. So it's worth understanding the history of their development, and the engineering compromises that were made in the process.

Google File System

Google was among the first of the major Web players to face the storage scalability problem head-on. And the answer arrived at by Google's engineers in 2003 was to build a distributed file system custom-fit to Google's data center strategy—Google File System (GFS).

GFS is the basis for nearly all of the company's cloud services. It handles data storage, including the company's BigTable database and the data store for Google's AppEngine platform-as-a-service, and it provides the data feed for Google's search engine and other applications. The design decisions Google made in creating GFS have driven much of the software engineering behind its cloud architecture, and vice-versa. Google tends to store data for applications in enormous files, and it uses files as "producer-consumer queues," where hundreds of machines collecting data may all be writing to the same file. That file might be processed by another application that merges or analyzes the data—perhaps even while the data is still being written.

"Some of those servers are bound to fail—so GFS is designed to be tolerant of that without losing (too much) data"

Google keeps most technical details of GFS to itself, for obvious reasons. But as described by Google research fellow Sanjay Ghemawat, principal engineer Howard Gobioff, and senior staff engineer Shun-Tak Leungin in a paper first published in 2003, GFS was designed with some very specific priorities in mind: Google wanted to turn large numbers of cheap servers and hard drives into a reliable data store for hundreds of terabytes of data that could manage itself around failures and errors. And it needed to be designed for Google's way of gathering and reading data, allowing multiple applications to append data to the system simultaneously in large volumes and to access it at high speeds.

Much in the way that a RAID 5 storage array "stripes" data across multiple disks to gain protection from failures, GFS distributes files in fixed-size chunks which are replicated across a cluster of servers. Because they're cheap computers using cheap hard drives, some of those servers are bound to fail at one point or another—so GFS is designed to be tolerant of that without losing (too much) data.

But the similarities between RAID and GFS end there, because those servers can be distributed across the network—either within a single physical data center or spread over different data centers, depending on the purpose of the data. GFS is designed primarily for bulk processing of lots of data. Reading data at high speed is what's important, not the speed of access to a particular section of a file, or the speed at which data is written to the file system. GFS provides that high output at the expense of more fine-grained reads and writes to files and more rapid writing of data to disk. As Ghemawat and company put it in their paper, "small writes at arbitrary positions in a file are supported, but do not have to be efficient."

This distributed nature, along with the sheer volume of data GFS handles—millions of files, most of them larger than 100 megabytes and generally ranging into gigabytes—requires some trade-offs that make GFS very much unlike the sort of file system you'd normally mount on a single server. Because hundreds of individual processes might be writing to or reading from a file simultaneously, GFS needs to supports "atomicity" of data—rolling back writes that fail without impacting other applications. And it needs to maintain data integrity with a very low synchronization overhead to avoid dragging down performance.

GFS consists of three layers: a GFS client, which handles requests for data from applications; a master, which uses an in-memory index to track the names of data files and the location of their chunks; and the "chunk servers" themselves. Originally, for the sake of simplicity, GFS used a single master for each cluster, so the system was designed to get the master out of the way of data access as much as possible. Google has since developed a distributed master system that can handle hundreds of masters, each of which can handle about 100 million files.

When the GFS client gets a request for a specific data file, it requests the location of the data from the master server. The master server provides the location of one of the replicas, and the client then communicates directly with that chunk server for reads and writes during the rest of that particular session. The master doesn't get involved again unless there's a failure.

To ensure that the data firehose is highly available, GFS trades off some other things—like consistency across replicas. GFS does enforce data's atomicity—it will return an error if a write fails, then rolls the write back in metadata and promotes a replica of the old data, for example. But the master's lack of involvement in data writes means that as data gets written to the system, it doesn't immediately get replicated across the whole GFS cluster. The system follows what Google calls a "relaxed consistency model" out of the necessities of dealing with simultaneous access to data and the limits of the network.

This means that GFS is entirely okay with serving up stale data from an old replica if that's what's the most available at the moment—so long as the data eventually gets updated. The master tracks changes, or "mutations," of data within chunks using version numbers to indicate when the changes happened. As some of the replicas get left behind (or grow "stale"), the GFS master makes sure those chunks aren't served up to clients until they're first brought up-to-date.

But that doesn't necessarily happen with sessions already connected to those chunks. The metadata about changes doesn't become visible until the master has processed changes and reflected them in its metadata. That metadata also needs to be replicated in multiple locations in case the master fails—because otherwise the whole file system is lost. And if there's a failure at the master in the middle of a write, the changes are effectively lost as well. This isn't a big problem because of the way that Google deals with data: the vast majority of data used by its applications rarely changes, and when it does data is usually appended rather than modified in place.

While GFS was designed for the apps Google ran in 2003, it wasn't long before Google started running into scalability issues. Even before the company bought YouTube, GFS was starting to hit the wall—largely because the new applications Google was adding didn't work well with the ideal 64-megabyte file size. To get around that, Google turned to Bigtable, a table-based data store that vaguely resembles a database and sits atop GFS. Like GFS below it, Bigtable is mostly write-once, so changes are stored as appends to the table—which Google uses in applications like Google Docs to handle versioning, for example.

The foregoing is mostly academic if you don't work at Google (though it may help users of AppEngine, Google Cloud Storage and other Google services to understand what's going on under the hood a bit better). While Google Cloud Storage provides a public way to store and access objects stored in GFS through a Web interface, the exact interfaces and tools used to drive GFS within Google haven't been made public. But the paper describing GFS led to the development of a more widely used distributed file system that behaves a lot like it: the Hadoop Distributed File System.

Hadoop DFS

Developed in Java and open-sourced as a project of the Apache Foundation, Hadoop has developed such a following among Web companies and others coping with "big data" problems that it has been described as the "Swiss army knife of the 21st Century." All the hype means that sooner or later, you're more likely to find yourself dealing with Hadoop in some form than with other distributed file systems—especially when Microsoft starts shipping it as an Windows Server add-on.

Named by developer Doug Cutting after his son's stuffed elephant, Hadoop was "inspired" by GFS and Google's MapReduce distributed computing environment. In 2004, as Cutting and others working on the Apache Nutch search engine project sought a way to bring the crawler and indexer up to "Web scale," Cutting read Google's papers on GFS and MapReduce and started to work on his own implementation. While most of the enthusiasm for Hadoop comes from Hadoop's distributed data processing capability, derived from its MapReduce-inspired distributed processing management, the Hadoop Distributed File System is what handles the massive data sets it works with.

Hadoop is developed under the Apache license, and there are a number of commercial and free distributions available. The distribution I worked with was from Cloudera (Doug Cutting's current employer)—the Cloudera Distribution Including Apache Hadoop (CDH), the open-source version of Cloudera's enterprise platform, and Cloudera Service and Configuration Express Edition, which is free for up to 50 nodes.

HortonWorks, the company with which Microsoft has aligned to help move Hadoop to Azure and Windows Server (and home to much of the original Yahoo team that worked on Hadoop), has its own Hadoop-based HortonWorks Data Platform in a limited "technology preview" release. There's also a Debian package of the Apache Core, and a number of other open-source and commercial products that are based on Hadoop in some form.

HDFS can be used to support a wide range of applications where high volumes of cheap hardware and big data collide. But because of its architecture, it's not exactly well-suited to general purpose data storage, and it gives up a certain amount of flexibility. HDFS has to do away with certain things usually associated with file systems in order to make sure it can perform well with massive amounts of data spread out over hundreds, or even thousands, of physical machines—things like interactive access to data.

While Hadoop runs in Java, there are a number of ways to interact with HDFS besides its Java API. There's a C-wrapped version of the API, a command line interface through Hadoop, and files can be browsed through HTTP requests. There's also MountableHDFS, an add-on based on FUSE that allows HDFS to be mounted as a file system by most operating systems. Developers are working on a WebDAV interface as well to allow Web-based writing of data to the system.

HDFS follows the architectural path laid out by Google's GFS fairly closely, following its three-tiered, single master model. Each Hadoop cluster has a master server called the "NameNode" which tracks the metadata about the location and replication state of each 64-megabyte "block" of storage. Data is replicated across the "DataNodes" in the cluster—the slave systems that handle data reads and writes. Each block is replicated three times by default, though the number of replicas can be increased by changing the configuration of the cluster.

As in GFS, HDFS gets the master server out of the read-write loop as quickly as possible to avoid creating a performance bottleneck. When a request is made to access data from HDFS, the NameNode sends back the location information for the block on the DataNode that is closest to where the request originated. The NameNode also tracks the health of each DataNode through a "heartbeat" protocol and stops sending requests to DataNodes that don't respond, marking them "dead."

After the handoff, the NameNode doesn't handle any further interactions. Edits to data on the DataNodes are reported back to the NameNode and recorded in a log, which then guides replication across the other DataNodes with replicas of the changed data. As with GFS, this results in a relatively lazy form of consistency, and while the NameNode will steer new requests to the most recently modified block of data, jobs in progress will still hit stale data on the DataNodes they've been assigned to.

That's not supposed to happen much, however, as HDFS data is supposed to be "write once"—changes are usually appended to the data, rather than overwriting existing data, making for simpler consistency. And because of the nature of Hadoop applications, data tends to get written to HDFS in big batches.

When a client sends data to be written to HDFS, it first gets staged in a temporary local file by the client application until the data written reaches the size of a data block—64 megabytes, by default. Then the client contacts the NameNode and gets back a datanode and block location to write the data to. The process is repeated for each block of data committed, one block at a time. This reduces the amount of network traffic created, and it slows down the write process as well. But HDFS is all about the reads, not the writes.

Another way HDFS can minimize the amount of write traffic over the network is in how it handles replication. By activating an HDFS feature called "rack awareness" to manage distribution of replicas, an administrator can specify a rack ID for each node, designating where it is physically located through a variable in the network configuration script. By default, all nodes are in the same "rack." But when rack awareness is configured, HDFS places one replica of each block on another node within the same data center rack, and another in a different rack to minimize the amount of data-writing traffic across the network—based on the reasoning that the chance of a whole rack failure is less likely than the failure of a single node. In theory, this improves overall write performance to HDFS without sacrificing reliability.

As with the early version of GFS, HDFS's NameNode potentially creates a single point of failure for what's supposed to be a highly available and distributed system. If the metadata in the NameNode is lost, the whole HDFS environment becomes essentially unreadable—like a hard disk that has lost its file allocation table. HDFS supports using a "backup node," which keeps a synchronized version of the NameNode's metadata in-memory, and stores snap-shots of previous states of the system so that it can be rolled back if necessary. Snapshots can also be stored separately on what's called a "checkpoint node." However, according to the HDFS documentation, there's currently no support within HDFS for automatically restarting a crashed NameNode, and the backup node doesn't automatically kick in and replace the master.

HDFS and GFS were both engineered with search-engine style tasks in mind. But for cloud services targeted at more general types of computing, the "write once" approach and other compromises made to ensure big data query performance are less than ideal—which is why Amazon developed its own distributed storage platform, called Dynamo.

Amazon's S3 and Dynamo

As Amazon began to build its Web services platform, the company had much different application issues than Google.

Until recently, like GFS, Dynamo hasn't been directly exposed to customers. As Amazon CTO Werner Vogels explained in his blog in 2007, it is the underpinning of storage services and other parts of Amazon Web Services that are highly exposed to Amazon customers, including Amazon's Simple Storage Service (S3) and SimpleDB. But on January 18 of this year, Amazon launched a database service called DynamoDB, based on the latest improvements to Dynamo. It gave customers a direct interface as a "NoSQL" database.

Dynamo has a few things in common with GFS and HDFS: it's also designed with less concern for consistency of data across the system in exchange for high availability, and to run on Amazon's massive collection of commodity hardware. But that's where the similarities start to fade away, because Amazon's requirements for Dynamo were totally different.

Amazon needed a file system that could deal with much more general purpose data access—things like Amazon's own e-commerce capabilities, including customer shopping carts, and other very transactional systems. And the company needed much more granular and dynamic access to data. Rather than being optimized for big streams of data, the need was for more random access to smaller components, like the sort of access used to serve up webpages.

According to the paper presented by Vogels and his team at the Symposium on Operating Systems Principles conference in October 2007, "Dynamo targets applications that need to store objects that are relatively small (usually less than 1 MB)." And rather than being optimized for reads, Dynamo is designed to be "always writeable," being highly available for data input—precisely the opposite of Google's model.

"For a number of Amazon services," the Amazon Dynamo team wrote in their paper, "rejecting customer updates could result in a poor customer experience. For instance, the shopping cart service must allow customers to add and remove items from their shopping cart even amidst network and server failures." At the same time, the services based on Dynamo can be applied to much larger data sets—in fact, Amazon offers the Hadoop-based Elastic MapReduce service based on S3 atop of Dynamo.

In order to meet those requirements, Dynamo's architecture is almost the polar opposite of GFS—it more closely resembles a peer-to-peer system than the master-slave approach. Dynamo also flips how consistency is handled, moving away from having the system resolve replication after data is written, and instead doing conflict resolution on data when executing reads. That way, Dynamo never rejects a data write, regardless of whether it's new data or a change to existing data, and the replication catches up later.

Because of concerns about the pitfalls of a central master server failure (based on previous experiences with service outages), and the pace at which Amazon adds new infrastructure to its cloud, Vogel's team chose a decentralized approach to replication. It was based on a self-governing data partitioning scheme that used the concept of consistent hashing. The resources within each Dynamo cluster are mapped as a continuous circle of address spaces, and each storage node in the system is given a random value as it is added to the cluster—a value that represents its "position" on the Dynamo ring. Based on the number of storage nodes in the cluster, each node takes responsibility for a chunk of address spaces based on its position. As storage nodes are added to the ring, they take over chunks of address space and the nodes on either side of them in the ring adjust their responsibility. Since Amazon was concerned about unbalanced loads on storage systems as newer, better hardware was added to clusters, Dynamo allows multiple virtual nodes to be assigned to each physical node, giving bigger systems a bigger share of the address space in the cluster.

When data gets written to Dynamo—through a "put" request—the systems assigns a key to the data object being written. That key gets run through a 128-bit MD5 hash; the value of the hash is used as an address within the ring for the data. The data node responsible for that address becomes the "coordinator node" for that data and is responsible for handling requests for it and prompting replication of the data to other nodes in the ring, as shown in the Amazon diagram below:

This spreads requests out across all the nodes in the system. In the event of a failure of one of the nodes, its virtual neighbors on the ring start picking up requests and fill in the vacant space with their replicas.

Then there's Dynamo's consistency-checking scheme. When a "get" request comes in from a client application, Dynamo polls its nodes to see who has a copy of the requested data. Each node with a replica responds, providing information about when its last change was made, based on a vector clock—a versioning system that tracks the dependencies of changes to data. Depending on how the polling is configured, the request handler can wait to get just the first response back and return it (if the application is in a hurry for any data and there's low risk of a conflict—like in a Hadoop application) or it can wait for two, three, or more responses. For multiple responses from the storage nodes, the handler checks to see which is most up-to-date and alerts the nodes that are stale to copy the data from the most current, or it merges versions that have non-conflicting edits. This scheme works well for resiliency under most circumstances—if nodes die, and new ones are brought online, the latest data gets replicated to the new node.

The most recent improvements in Dynamo, and the creation of DynamoDB, were the result of looking at why Amazon's internal developers had not adopted Dynamo itself as the base for their applications, and instead relied on the services built atop it—S3, SimpleDB, and Elastic Block Storage. The problems that Amazon faced in its April 2011 outage were the result of replication set up between clusters higher in the application stack—in Amazon's Elastic Block Storage, where replication overloaded the available additional capacity, rather than because of problems with Dynamo itself.

The overall stability of Dynamo has made it the inspiration for open-source copycats just as GFS did. Facebook relies on Cassandra, now an Apache project, which is based on Dynamo. Basho's Riak "NoSQL" database also is derived from the Dynamo architecture.

Microsoft's Azure DFS

When Microsoft launched the Azure platform-as-a-service, it faced a similar set of requirements to those of Amazon—including massive amounts of general-purpose storage. But because it's a PaaS, Azure doesn't expose as much of the infrastructure to its customers as Amazon does with EC2. And the service has the benefit of being purpose-built as a platform to serve cloud customers instead of being built to serve a specific internal mission first.

So in some respects, Azure's storage architecture resembles Amazon's—it's designed to handle a variety of sizes of "blobs," tables, and other types of data, and to provide quick access at a granular level. But instead of handling the logical and physical mapping of data at the storage nodes themselves, Azure's storage architecture separates the logical and physical partitioning of data into separate layers of the system. While incoming data requests are routed based on a logical address, or "partition," the distributed file system itself is broken into gigabyte-sized chunks, or "extents." The result is a sort of hybrid of Amazon's and Google's approaches, illustrated in this diagram from Microsoft:

As Microsoft's Brad Calder describes in his overview of Azure's storage architecture, Azure uses a key system similar to that used in Dynamo to identify the location of data. But rather than having the application or service contact storage nodes directly, the request is routed through a front-end layer that keeps a map of data partitions in a role similar to that of HDFS's NameNode. Unlike HDFS, Azure uses multiple front-end servers, load balancing requests across them. The front-end server handles all of the requests from the client application authenticating the request, and handles communicating with the next layer down—the partition layer.

Each logical chunk of Azure's storage space is managed by a partition server, which tracks which extents within the underlying DFS hold the data. The partition server handles the reads and writes for its particular set of storage objects. The physical storage of those objects is spread across the DFS' extents, so all partition servers each have access to all of the extents in the DFS. In addition to buffering the DFS from the front-end servers's read and write requests, the partition servers also cache requested data in memory, so repeated requests can be responded to without having to hit the underlying file system. That boosts performance for small, frequent requests like those used to render a webpage.

All of the metadata for each partition is replicated back to a set of "partition master" servers, providing a backup of the information if a partition server fails—if one goes down, its partitions are passed off to other partition servers dynamically. The partition masters also monitor the workload on each partition server in the Azure storage cluster; if a particular partition server is becoming overloaded, the partition master can dynamically re-assign partitions.

Azure is unlike the other big DFS systems in that it more tightly enforces consistency of data writes. Replication of data happens when writes are sent to the DFS, but it's not the lazy sort of replication that is characteristic of GFS and HDFS. Each extent of storage is managed by a primary DFS server and replicated to multiple secondaries; one DFS server may be a primary for a subset of extents and a secondary server for others. When a partition server passes a write request to DFS, it contacts the primary server for the extent the data is being written to, and the primary passes the write to its secondaries. The write is only reported as successful when the data has been replicated successfully to three secondary servers.

As with the partition layer, Azure DFS uses load balancing on the physical layer in an attempt to prevent systems from getting jammed with too much I/O. Each partition server monitors the workload on the primary extent servers it accesses; when a primary DFS server starts to red-line, the partition server starts redirecting read requests to secondary servers, and redirecting writes to extents on other servers.

The next level of "distributed"

Distributed file systems are hardly a guarantee of perpetual uptime. In most cases, DFS's only replicate within the same data center because of the amount of bandwidth required to keep replicas in sync. But replication within the data center, for example doesn't help when the whole data center gets taken offline or a backup network switch fails to kick in when the primary fails. In August, Microsoft and Amazon both had data centers in Dublin taken offline by a transformer explosion—which created a spike that kept backup generators from starting.

Systems that are lazier about replication, such as GFS and Hadoop, can asynchronously handle replication between two data centers; for example, using "rack awareness," Hadoop clusters can be configured to point to a DataNode offsite, and metadata can be passed to a remote checkpoint or backup node (at least in theory). But for more dynamic data, that sort of replication can be difficult to manage.

That's one of the reasons Microsoft released a feature called "geo-replication" in September. Geo-replication is a feature that will sync customers' data between two data center locations hundred of miles apart. Rather than using the tightly coupled replication Microsoft uses within the data center, geo-replication happens asynchronously. Both of the Azure data centers have to be in the same region; for example, data for an application set up through the Azure Portal at the North Central US data center can be replicated to the South Central US.

In Amazon's case, the company does replication across availability zones at a service level rather than down in the Dynamo architecture. While Amazon hasn't published how it handles its own geo-replication, it provides customers with the ability to "snap shot" their EBS storage to a remote S3 data "bucket."

And that's the approach Amazon and Google have generally taken in evolving their distributed file systems: making the fixes in the services based on them, rather than in the underlying architecture. While Google has added a distributed master system to GFS and made other tweaks to accommodate its ever-growing data flows, the fundamental architecture of Google's system is still very much like it was in 2003.

But in the long term, the file systems themselves may become more focused on being an archive of data than something applications touch directly. In an interview with Ars, database pioneer (and founder of VoltDB) Michael Stonebraker said that as data volumes continue to go up for "big data" applications, server memory is becoming "the new disk" and file systems are becoming where the log for application activity gets stored—"the new tape." As the cloud giants push for more power efficiency and performance from their data centers, they have already moved increasingly toward solid-state drives and larger amounts of system memory.

Read the comments on this post

computer speaker system reviewsTyll Hertsens

Gentlemen, start your engines

Show Promo Image

You never get over your first, and the Audioengine A2 was my first computer speaker. Other speakers came and went, but the little A2 was my original desktop "reference."

There was something magical about the little guy, and it wasn't what I would call a high-resolution speaker, but it made most things, including low bit rate streaming audio, sound decent. A year later the Audioengine P4 replaced the A2 on my desktop, and now I'm checking out the A5+. It's bigger than the other 'engines, but what about the sound?

Jan 26 2012
Audioengine
(877) 853 4447
www.audioengineusa.com

related stuff

Timthumb 1 php

We installed flexible LED light strips in our kitchen for under cabinet and within cabinet lighting. These are very low energy consumption, cool to the touch, and rated to last for 50,000 hours.

LEDstrip

The strips are about .5 cm wide and 2 mm thick. The strips come on a spool with a sticky tape side. You press the sticky side to the bottom of the cabinet (or the sides inside) and the strip gives a very diffuse effective and efficient light. They are so thin, you can't really see the light strip itself, only the glow. The strip is a circuit of LEDs in a row. They have marked segments about every 2-3 inches where you can cut them to fit. They typically run off of 12 volts; the transformer can sit i a cabinet, attic, or basement. You can also specific different color temperatures (very warm to very cool). The lights are dimable.

LEDcabinets

LEDlightcabinet

We used them under our cabinets and inside of one cabinet (picture above).

There are tons of manufacturers pedaling flexible LED strips now. You can purchase them in meter strips or on 5 meter reels. Here is one supplier with many products and variations: Superbrightleds.com. I have no experience in using this outfit. It is a new market so quality varies.

We used a local California-based manufacturer, Aion. Their prices are higher than many of the imports (usually from China), but they had a deliverable guarantee of 5 years. Unfortunately they don't deal retail, wholesale only through electricians, who can reliably install it.

If anyone has experience with installing DIY LED strips, please let us know.

And these nifty strips can be used for all kinds of other illumination where flexibility and thinness is desired.

-- KK