Some rambling about eBooks, newspapers and the future

Marco Arment writes that Barnes & Noble will be offering discounts on the Nook to customers who commit to certain periodical subscriptions:

• $119.88 for a year of People magazine gets a $50 discount off the $250 Tablet.
• $239.88 for a year of The New York Times gets a $100 discount off the $100 Simple Touch or $200 Color.

To me, this mainly speaks to how disconnected from reality the pricing is of electronic versions of periodicals.¹ I’m still waiting these publishers to wake up and listen to what Steve Jobs told them (according to Isaacson) about the price point for an electronic newspaper: $5/month. I like the New York Times and I would be happy to subscribe for that much, but no more. Instead, I’ll make do with what little the NYT iPad app doles out for free, or get my news elsewhere.

Subjectively, I feel like this discount scheme also makes the newspaper appear valuable while making the Nook look cheap. Doesn’t seem great for the latter’s brand image. I guess B&N is getting desperate?

It’s a shame. I like the Nook over the Kindle because it uses ePub, which seems at least a little more future-proof than Amazon’s pointlessly proprietary format².

Sometimes there may be reasons a company wants to make a proprietary format instead of using an existing standard; mainly when the standard doesn’t do what the company’s product needs to do. As far as I’m aware, the Kindle formats do nothing that isn’t possible in ePub. Obviously, tech companies also love proprietary formats for the evil reason of customer lock-in. Even Steve Jobs was guilty of using customer lock-in as a direct justification (again, according to Isaacson).

Me, I mainly just want to be able to read my books a few years from now.

As a programmer, I can’t help but look at formats through the lens of how easy would this be to reverse-engineer if everyone else on the planet suddenly stopped supporting it and all existing software ceased to function? (Which, it turns out, isn’t really that far-fetched). ePub readers are relatively straightforward to implement, since ePub is mainly just based on HTML and CSS, and we can be reasonably assured that that will be around decades from now—although, perhaps not renderable in perfect, original fidelity.

On the other hand, almost all eBooks, in any format, are wrapped in some kind of DRM these days, which means in practice I mostly don’t end up buying them.³ I have no problem buying, for example, games from Steam, so I guess my expectations differ depending on the medium. But still, for lack of any really appealing options, I’m inclined to root for the companies that are doing the thing closest to what I want. But I also think we’ll somehow fare better if we are able to keep some good competition going, and don’t let any one company ‘win’. My hope is that this will eventually force the different vendors to sell stuff that’s compatible with each other, like what happened with music.

Things I listened to in 2011

I like music. Do you like music?

This is mostly (but not all) recent music, but what it all has in common is that I listened to and enjoyed it in 2011.

Here’s a list of the songs featured:

10. Basement Jaxx - Raindrops (from Basement Jaxx vs Metropole Orkest; iTunes; Amazon)
9. Bonobo - Animals (from Black Sands; iTunes; Amazon)
8. Red Hot Chili Peppers - Ethiopia (from I’m With You; iTunes; Amazon)
7. Electric President - Feathers (from The Violent Blue; iTunes; Amazon)
6. Owl City - The Yacht Club (from All Things Bright and Beautiful; iTunes; Amazon)
5. The National - Exile Vilify (from the game Portal 2; available as a single; iTunes; Amazon)
4. Ico - You Were There (from the game Ico; soundtrack exists if you can find it)
3. Way Out West - Survival (from We Love Machine; iTunes; Amazon)
2. Big Giant Circles feat. Jeff Ball - Thunderstruck (from 25YEARLEGEND, a remix album based on the game series The Legend of Zelda; available for free)
1. Radiohead - Separator (from The King of Limbs; iTunes; Amazon)

The final background picture in #5 is from here.

I arbitrarily limited myself to 10 songs; otherwise I might have never finished it. Here were some runners-up, which almost made the cut:

• Caia - Remembrance (from The Magic Dragon; iTunes; Amazon)
• Moby - After (from Destroyed; iTunes; Amazon)
• Basement Jaxx - Samba Magic (from Atlantic Jaxx Recordings; Amazon)
• Orbital - Don’t Stop Me (single; iTunes; Amazon)
• Yasunori Mitsuda - The Beginning and the End (from Myth: The Xenogears Orchestral Album; Amazon)
• GaMeBoX - Subsonic Sparkle (from Sonic the Hedgehog: The Sound of Speed, another remix album; available for free)

It was fun to make. I might do it again next year.

Tumblr suspended my blog

Around 5 PM yesterday, I went to post a quote, and discovered that my blog (this) had been suspended by Tumblr. Attempting to post using MarsEdit gave an API error; attempting to get to the blog homepage gave a 404; and attempting to get to any part of the main Tumblr web site got me caught in an infinite redirect loop.

As instructed on the 404 page, I sent an email to support@tumblr.com with everything I knew. And then I waited.

This was a particularly bad time for this to happen to me, because I had intended to pass along my web site URL to some new professional contacts that very evening. There is more to my web site than just the blog, but still: it doesn’t look very professional if following a link on your home page leads to a 404.

To Tumblr’s credit, they replied and fixed my blog within two hours, outside of normal business hours (their main office is in NYC, as far as I know):

Hello David,

Your content has been restored. We are sorry that this problem occurred and will ensure that it does not happen again.

Try deleting your web browser’s cookies for Tumblr.com and see if that helps. Let me know if you still have a problem.

Info on how to delete cookies is at:

http://www.aboutcookies.org/Default.aspx?page=2

Please let me know if there’s anything else I can help you with.

Thanks for using Tumblr!

Quick resolution and an apology, but no explanation. I guess that’s 2 out of 3.

Reading between the lines here, with some relatively unfounded assumptions, here’s my best guess as to what happened:

1. Tumblr has some kind of automated spam detector, and my blog got flagged. Presumably they don’t have a human reviewing these as it would take too long, so they just automatically suspend the blog immediately. Perhaps it’s because of all of my recent writing about certain braaands? Who knows.

2. The support person reverted the ‘spam’ flag on my blog, and set some kind of new flag that prevents me from being automatically flagged as spam again.

The above is entirely speculation - I could be wrong about all of it. But, for lack of an explanation from Tumblr, it’s one possible story that fits what I saw.

But this incident made me realise some other troubling things about my use of Tumblr.

The will to back up

I didn’t have any backups of my Tumblr blog. When they suspended it, four years of writing, as far as I knew, was gone. Granted, this is mainly my fault. Tumblr even provides a Mac app that will back up your Tumblr blog to your computer, which I immediately used after my blog was restored. However, this has some further problems:

• The app has only had a single beta release, which is two years old. This makes sense once you know that the co-founder who wrote it doesn’t work there any more, and in fact doesn’t even use Tumblr any more. But it doesn’t speak well to how long this app will be officially supported by Tumblr. I calculate an 85% probability that it will suddenly and permanently stop working at some point in the future.
• Even now, the app fails cryptically sometimes. Retrying immediately a few times in a row will eventually get it to work. But that’s worrisome.
• Backing up is a manual process. Every sysadmin knows that one the cardinal rules of backing is that it must be automatic, otherwise it simply will not get done.
• It’s Mac-only. This isn’t a problem for me, but it might be for some people. Or it might be a problem for me in the future.

In sum, it would be much better if this app were written in a well-supported, portable scripting language, and open-source, so that there was some reasonable assurance it will continue to work in the future, and so that it could be configured to run automatically. I’m usually a strong proponent of native apps, but in this case, a great UI is not a meaningful advantage and does not outweigh the disadvantages.

Bureaucratic loop

I mentioned before that visiting any Tumblr page while my blog was suspended resulted in a redirect loop. I couldn’t even get to the support site to see if that had any information. I wasn’t even sure to a certainty that I was suspended: my only clue was the URL, which didn’t tell me much.

As a web developer, this is a fairly rookie mistake that I myself have made at least once. It happens when you set up a rule like this to be executed on every page request (in pseudocode):

if (user is suspended) then
  redirect user to "your account is suspended" error page
else
  show the page the user requested
end

See a flaw in this logic? What happens when the user requests the error page?

The same thing that happens when the user tries to visit any page: they get redirected to the error page. Even though that’s the page they requested in the first place. Because there’s nothing in the above logic to handle the case in which the error page was where they were trying to get to. The flow goes like this:

1. System sets user status to suspended
2. User requests home page
3. System checks if user is suspended
4. System redirects user to error page
5. User requests error page
6. Go to line 3

This type of mistake is fairly common, to the point where most web browsers will stop after receiving several redirects in a row from the same site, and display an error message. (Safari’s looks like this.)

The fact that this mistake exists isn’t enough to bother me, because I know how easy a mistake it is to make. But it’s also an easy mistake to fix, and the fact that it’s remained unfixed after at least three years of Tumblr being aware of it suggests something else about Tumblr.

Indeed, the support response above suggests a lack of understanding about why the redirect loop is happening, and the likely conclusion that it must be some kind of ‘cookies thing’ that is probably the user’s fault, probably based on someone’s past experience that cookies can cause mysterious problems that are nobody’s fault and things just start working if you clear them.

Either the bug report about the redirect loop never made it to the bug team, or it was incorrectly bounced back as ‘user error’.

Unfortunately, the advice for clearing cookies is not going to help anyone. To illustrate why, here’s a truth table showing the different states the user can be in (suspended or not, logged in or not) and the result of their actions.

As you can see, telling a logged-in, suspended user (state #4) to clear their cookies will only result in logging them out (state #3). As soon as they log in again, they will be in state #4 again and the problem will occur again, which is not helpful because presumably they want to be logged in. Whereas, if the user is not suspended (state #1 and #2), they will not experience the problem anyway, so there’s no reason to tell them to clear cookies. Clearing cookies does nothing useful in any of these cases.¹

So not only have Tumblr not fixed the redirect loop in three years, but instead of fixing it, they seem to have institutionalised some kind of cargo-cult non-fix in their support responses to this problem, which allows the development team to avoid responsibility for the problem.

Ordinarily, as a user, I would shrug something like this off, since it’s a problem that will only ever affect a small number of users, and only those who needed to contact support anyway. But a post today from Rachel By The Bay made me think more about it, after noticing the similarity to what she describes. I would categorise this as somewhere in between a ‘replay’ and a ‘ramrod’ response, as support in this case may be trying to replay a solution to an unrelated problem that happened in the past, despite lack of reasoning or evidence to suggest that it actually works.

Overall, the redirect loop was a minor problem compared to the larger problem of account suspension that I was having. But it still gives me pause because it’s a bureaucratic problem, rather than a technical problem, and the former are the most annoying kind because they’re so hard to work around.

What now?

I don’t mean to pick on Tumblr too much; it’s still commendable that, for a free service that owes me nothing, I received better support from them than almost every service I pay for, especially those from big corporations.

But it’s clear that I’ve placed too much trust in Tumblr, compared to the importance I place on the data I let them keep for me. My resolution of this problem hinged on the fast and (mostly) accurate response from a support person; but as a technical user, I’m used to fixing problems on my own. I can’t predict what’s going to happen to Tumblr as a company, and I’m not comfortable assuming that I will get an equally good support response in the future, for problems caused through no fault of my own, which I am unable to fix myself.

So it seems like it’s time to move back to a self-hosted blog. This will not be fun.

For one thing, I love Tumblr’s clean and simple UI design; I have never seen any compelling alternatives. On the other hand, I mainly use MarsEdit to post these days, so I rarely see Tumblr’s UI any more anyway. Of the ones I know about:

• Chyrp’s main goal is to replicate Tumblr’s core functionality, including the Tumblr post types. It can even import Tumblr blogs. However, it seems lightly maintained (it was dead and resurrected at one point) and not very compatible with MarsEdit. And there are time costs related to using software that isn’t widely-used, as you’re more likely to run into problems that nobody has encountered before. Like Marco Arment says, you never want to be the biggest user of any software.
• WordPress is the popular behemoth, but it’s famous for its security holes and performance problems. It seems like every time I read about it, there’s a different caching plugin you’re supposed to use just to get decent performance; this alone seems like something that should be a Solved Problem, built-in to the core functionality. The fact that it’s apparently not is very concerning.
• Movable Type is also fairly popular behind WordPress, and (if I recall correctly) uses static HTML pages for ideal performance. I still get the impression that it has way too much in the way of features and clutter than I need, but it’s looking like the best option right now.
• Or I could just write my own, but that’s just so cliché at this point, and there are more interesting things I could be working on.

Migrating data to another system is usually difficult and tedious, especially when you’re as concerned as I am about preserving the original data as closely as possible (including the original URLs, design, etc.) There seem to be very few existing ways to move data from Tumblr to something else, so I will probably have to write my own.

Clearly, it’s not something I’ll be able to do quickly or easily, but having control over my blog should make it worthwhile.

An easier way to search DuckDuckGo from Safari

I previously wrote about a workaround I made to replace the Yahoo search in Safari with DuckDuckGo, my new favourite search engine. (Other browsers don’t need workarounds.)

An unfortunate complexity of the method I wrote about was that you needed to run your own web server. But I emailed Yegg, the founder of DuckDuckGo, explaining what I did, and he was interested in helping on their end to make this easier.

So, good news: you no longer need to run your own web server for this to work! DuckDuckGo now takes care of that part (which makes sense, since I gather they have a few web servers that they were already running).¹

You now just need to add this one line to your /etc/hosts file, restart Safari, and ‘Yahoo’ will be DuckDuckGo from now on:

184.72.115.86 search.yahoo.com

To make this even easier for people who don’t know how to edit their hosts file, I’ve put together a simple Mac app which does this for you. Download it here. Patches welcome via GitHub.

You can also read the support doc on DuckDuckGo, which explains in more detail. Of course, there are still other ways to accomplish this, but I think mine is the best way.

It’s really cool to use a search engine that’s so responsive to user requests, and even willing to collaborate with me. I’m making an effort to do most of my searches with DuckDuckGo now, and only falling back to Google when I can’t find something. (Will you take the DuckDuckGo Challenge?) I hope to see DuckDuckGo give Google some much needed competition.

Script to search DuckDuckGo from the Safari search bar

Update: There’s now an easier way to do this.

DuckDuckGo is a promising new search engine, especially if the way Google is bubbling and tracking you is starting to drive you crazy like it is to me.

Only one problem, if you’re a Safari user: Apple doesn’t provide a way to change the engines used in the search bar, beyond the three provided by default (Google, Yahoo!, and Bing). Boo, Apple.

There are several ways around this, as suggested by DuckDuckGo themselves:

• One of several Safari extensions that provide search functionality. These tend to be ugly and don’t actually integrate with the search bar you’re used to using, because Apple’s extension API doesn’t allow it. DuckDuckGo suggests this one, and here’s a slightly better one.
• InputManager hacks (a.k.a. SIMBL) that can make Safari unstable, un-upgradeable, and are generally a bad idea. I’m not even going to link to these.
• Configurable web proxies that sit between Safari and your internet connection, such as GlimmerBlocker (with a filter). My main reluctance here is that it’s just something else to install at the system level, and it does more than I need it to.
• Hacking the binary. You’d have to be pretty crazy and desperate to try this.

I don’t really like any of those options, so I thought of another way. It comes in two steps:

1. Edit /etc/hosts to forward search.yahoo.com to your local web server (assuming you run one)
2. Drop a simple Ruby script in your web root to intercept queries and redirect them to DuckDuckGo.

You can view or download it here.

I chose Yahoo because that way I can still easily switch back and forth between Google, DuckDuckGo (labelled Yahoo in the UI), and Bing. Since Yahoo gave up on being a real search engine and only shows you Bing’s results anyway, I can’t imagine why I’d ever need to search Yahoo. Still, I made it so that I could still search from Yahoo’s homepage, if I ever wanted to. The script should be easy enough to modify if you’d rather intercept Google or Bing instead. Improvements via GitHub are welcome, of course.

This is still a hack, but to me personally, it strikes the best balance between least intrusion and greatest capability to do what I want. It’s probably not the best solution for everyone, but I’m putting it out there in case it helps someone else.

I might get around to packaging this as a friendlier installer for non-technical users, although I’m not really sure if that’s a good idea; with these sorts of hacks, it’s generally best if the person doing it understands how it works, so they can undo it if it stops working. That’s why I’ve deliberately left some basic concepts (e.g. how to enable the built-in web server, or edit your hosts file) as an exercise for the reader.

Now, if only I had a way to make this work on my iPad…