Recently in Software Category

 

March 3, 2012

Something annoying but also instructive happened during my build of Chromium today. Everything started when I checked out a clean version and went to do a build, only to be greeted with the following exciting error:
/Users/ekr/dev/chromium/src/third_party/WebKit/Source/WebCore/WebCore.gyp
ar: input.a is a fat file (use libtool(1) or lipo(1) and ar(1) on it)
ar: input.a: Inappropriate file type or format
rm: /Users/ekr/dev/chromium/src/out/Debug/obj.target/\
  webkit_system_interface/geni/adjust_visibility/self/cuDbUtils.o: No such file or directory
make: *** [out/Debug/libWebKitSystemInterfaceLeopardPrivateExtern.a] Error 1
make: *** Waiting for unfinished jobs....

Luckily, I've run into this problem before so I know what the problem is. The script third_party/WebKit/Source/WebCore/WebCore.gyp/mac/adjust_visibility.sh, which does some library mangling, uses file to determine what kind of library it's dealing with. Unfortunately, it invokes file with an unqualified name, and since MacPorts wants to put itself at the beginning of PATH this means that you get the file implementation from MacPorts which has a slightly different output than the system file. The result is that adjust_visibility.sh decides that you have a thin version of libWebKit...a and tries to run ar on it. When ar fails, so does the build.

The fix here is to move MacPorts below /usr/bin in your path. I'd already done this—or so I thought— but it turned out that MacPorts had inserted itself twice in .cshrc so I had to edit .cshrc and then run source .cshrc. I did this, and after correcting a typo things looked good and I and went to rerun the build, only to be greeted with:

  CXX(target) out/Debug/obj.target/base/base/sync_socket_posix.o
In file included from base/sync_socket_posix.cc:18:
./base/file_util.h:416:56: error: no type named 'set' in namespace 'std'
                                            const std::set& group_gids);
                                                  ~~~~~^
./base/file_util.h:416:59: error: expected ')'
                                            const std::set& group_gids);
                                                          ^
./base/file_util.h:413:44: note: to match this '('
BASE_EXPORT bool VerifyPathControlledByUser(const FilePath& base,
                                           ^
2 errors generated.
make: *** [out/Debug/obj.target/base/base/sync_socket_posix.o]

I know what you're thinking here—or at least what I thought—someone forgot to #include <set> and for some reason the automated builds didn't catch it, perhaps due to some conditional compilation problem getting triggered on Lion. But checking the source quite clearly showed that set was being included. Moreover, other STL containers like vector work fine. Changing from clang to GCC didn't help here, so eventually I reverted to gcc -E. For those of you who don't know, this runs the preprocessor but not the compiler and so is really useful for diagnosing this kind of include error. Here's the relevant portion of the result:

# 18 "./base/file_util.h" 2





# 1 "./set" 1
# 24 "./base/file_util.h" 2

It's a little hard to read, but if you know what to look for, it's telling you that instead of including set from /Developer, where the system include files live, the compiler is getting it from the local directory. Now, you might ask what the heck a file named set is doing in the local directory, especially as when I looked it was totally empty. Naturally, it was my fault, but it took a minute to realize what. Remember I said that I had to correct a typo in .cshrc but now what the typo was. Well, the problem was that I had written:

>set OSVER=`uname -r`
Instead of
set OSVER=`uname -r`

Of course, when I ran this it create a file called set in the current directory and since the compile flags included the current directory in the include path, the compiler duly included it instead of the system include file. And since the file was empty, there wasn't any definition of std::set and we got a compile error. Time wasted by this error: 11 minutes (not including writing this up).

 

January 23, 2012

You have to have used git to really understand this one, but...
[16] git checkout f4a56
Note: checking out 'f4a56'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at f4a560b... Foo
As you may have gathered from this long warning, you most likely don't want to be in a detached head setting, you probably just meant to create a branch or wanted to rollback a commit but typed the wrong thing. Which is why there are lots of pages about what this means and how to get yourself out. My contribution to this literature can be found below the fold.

 

December 31, 2011

Spent some of today getting my 2011 charitable donations out of the way, so I've been experiencing a lot of different Web forms. Remember, these people want my money, so it would be nice if they didn't make the experience so irritating. On that basis, here are some things not to do:
  • Refuse to accept spaces or dashes in my credit card number, phone number, social security number, etc. Don't force me into your stupid format; parse whatever I send you. Here, let me help. The following JS code strips out spaces and dashes. input = input.replace(/[ \-]/g, "");. For an appropriately huge consulting fee I'll show you how to replace periods and pluses, too.
  • Force me to tell you what kind of credit card I have. This information is encoded in the leading digits of the credit card number. This table may help. I know that things change, but seriously, you could at least try to guess.
  • Force me to select "USA" out of the end of an incredibly long drop-down list of countries. It's true that you can generally determine someone's country by looking at their IP address, but I can certainly understand not wanting to bother with that, but if most of your customers are American, it's silly to force them to scroll all the way to the end out of a misguided notion of national equity. Make my life easy and put the USA as the first item in the list, people.
  • Make me enter my state and my zip code. In nearly all cases, the zip code encodes the state.

Also, not a Web form issue, but I also wish there were some way to tell these organizations not to ask me for donations during the year. I give once a year, at the end of the year. It's just a matter of convenience. Sending me a bunch of physical letters asking for money just wastes your fund raising dollars and my time.

 

September 2, 2011

Note to self, when you see the following error:

Undefined symbols for architecture x86_64:
  "vtable for Foo", referenced from:
      Foo::Foo()   in ccqNJSZ0.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status

It means you have declared virtual functions but you haven't defined any of them non-inline. Here's the code that caused this:

class Foo {
    virtual void f();
};

int main(int argc, char **argv)
{
  Foo foo;

  return(0);
}

Now, it's true that you would expect a linkage error here, but not this linkage error. Rather, you'd appear to get a complaint about function f() being missing, which is what happens when you change the class definition a bit to have f() defined but b() undefined.

class Foo {
    virtual void f();
    virtual void b();
};
void Foo::f() {}

This gives the expected error:

Undefined symbols for architecture x86_64:
  "Foo::b()", referenced from:
      vtable for Fooin ccxuo26H.o
ld: symbol(s) not found for architecture x86_64

What's going on here is described in a GCC FAQ entry:

The ISO C++ Standard specifies that all virtual methods of a class that are not pure-virtual must be defined, but does not require any diagnostic for violations of this rule [class.virtual]/8. Based on this assumption, GCC will only emit the implicitly defined constructors, the assignment operator, the destructor and the virtual table of a class in the translation unit that defines its first such non-inline method.

Therefore, if you fail to define this particular method, the linker may complain about the lack of definitions for apparently unrelated symbols. Unfortunately, in order to improve this error message, it might be necessary to change the linker, and this can't always be done.

Pretty sweet, huh?

 

August 28, 2011

The (arguably counterintuitive) syntax for interpolating templates into other templates using jqtpl and Express is:

  {{partial(<variables>) "<inner-template>" }}

Concretely, if we have template X and we want to interpolate template Y with variable foo = "bar", then template X contains

  {{partial({foo:"bar"}) "Y" }}

This is probably not of interest to you, unless you just spent 10 minutes trying to work this out from the sparse (and I think in this case wrong; bug reported) documentation.

 

August 21, 2011

I recently upgraded to Lion. Probably it would have been wiser to wait until 10.7.1 but I wanted the PDF "signing" feature so I went ahead anyway. Overall, things went really smoothly. Notes below.

  • As everyone now knows, by default Lion is an app store only purchase and doesn't come with any kind of media. Apple charges $50 extra for a USB stick, but as I imagine most everyone knows, if you dig around in the install package, there is a .dmg file and you can burn that to DVD. [*] This seems advisable but takes some time.
  • The actual install was pretty fast, say about 30 minutes or so on my Macbook Air.
  • However, Lion comes with a new version of Filevault which encrypts the whole drive. Turning this on took a while to encrypt the disk but that can happen in the background, so it's not too bad as long as you're happy to have the machine on for a while as it happens, or you're willing to have it happen over a few days. Apple offers to let you store the recovery key with them. I declined this offer.
  • New Filevault works just fine with old Filevault, so if you've been using old Filevault you can transition easily. Every so often you get asked if you'd like to unencrypt your old partition, but Lion doesn't make you.
  • However, if you want Time Machine to work well rather than badly—which is how it has historically worked with Filevault—you need to move to new Filevault, which means encrypted backups. There is a setting in Time Machine to encrypt your backup disk. This takes a very very long time if you have an existing non-encrypted backup disk.
  • Installing Lion blows away your existing—or at least, my existing—copy of Xcode 3. Xcode 4 is now free, but this means that you will be without debugger, compiler, etc. until you download another 3GB worth of Xcode, so something to keep in mind. There's probably some way to patch in the old Xcode but this seemed inadvisable.
  • I'm mixed on the new gesture support. Obviously, I want the old scrolling behavior, not the "natural" scrolling behavior (where the scrolling goes in the direction of your finger like with the iPhone), but that's easily turned off. Mission control seems like it should be really cool, but other than periodically swiping to see it happen, I haven't figured out what it's for. (Incidentally, just this gesture stopped working on my magic trackpad, but not my built-in trackpad, necessitating a call to Apple support. Strangely, changing it from three fingers to two and back again solved the problem.)
  • The UI changes are all pretty subtle. I can take or leave the auto-scrollbars and the rounded dialogs, buttons, etc. seem fine. Probably the most noticeable change is the way that apps keep their state. I'm used to using Command-Q to quit the app, but now this means that if I quit Preview, and then restart it I end up with all the same documents I had before, so I'm not sure this is that great. I guess I just need to learn to use Option-Command-Q or reset the defaults using the command line.

Anyway, this was all pretty smooth for a major OS upgrade (I really appreciate not having to run mergemaster). And the feature where you can take a photo of your signature and embed it into documents really is pretty cool. I may never need to print-scan-sign-scan-email again.

 

August 3, 2011

Recently I had the dubious pleasure of working simultaneously in C++, Python and JavaScript. I'm not saying that there is anything wrong with any of these languages, but if you rapidly switch back and forth between them (as, for instance, when you're developing a JavaScript Web 2.0 application with the server in Django and the front end), things can get pretty confused. The P90x guys claim that muscle confusion leads to increased strength, but in my experience, programming language confusion mostly leads to problems.

The basic problem is that these languages have fairly similar syntaxes and so it's pretty easy to inadvertantly use the syntax of language A with language B. Here's a sampling of some common tasks in these languages:

TaskC++PythonJavaScript
Statement Separation; terminatednew line; semicolon separated (optional)
Length of an array v.size() (for STL vectors) len(v) v.length
Append to an array v.push_back(x) v.append(x) v.push(x)
Iterate through an array for(size_t i; i<v.size(); i++) for(x in v) for(var i=0; i<v.length; i++)
Is an element in an associative array? if (a.count(k)) if k in a if (a[k])
Creating a new object new ClassName() ClassName() new Constructor()

To make matters worse, sometimes what you would do in one language is syntactically valid, but undesirable, in another language and invalid in a third. For instance, in Python you don't use semicolons to terminate statements at all, but it doesn't choke if you use them. In JavaScript, you mostly need them and JS will "insert" them as needed [*]. In C++, semicolons are required and if you don't add them, the compiler will throw an error. So, if you switch back and forth, you're constantly adding spurious semicolons in Python code (which makes Python people sad) and omitting them in C++.

To take a more irritating example, consider asking if an element is an associative array. This is actually a lot more painful: if you do if (a[k]) on a C++ map, as is natural in JavaScript, C++ will automatically instantiate a new copy of whatever's in the array using the default constructor and place it at location k. This can have really undesirable consequences, since you've just modified the structure you mean to be examining. On the other hand, if you dereference a nonexistent location, it throws a KeyError, like so:

>>> a = {}
>>> a['b']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'b'
>>> 

Unfortunately, this is a runtime error, so it's easy not to notice this mistake until you get fairly far into the program, especially if you're just checking for existence as a corner case where the key is usually present.

Iterating through an array is another case where it's easy to mess up. In JavaScript, like C++, you iterate through an array by using an index from 0 to array length - 1. In Python, however, you can iterate through it more easily just by using for x in v where v is the array. Unfortunately, a similar construct is syntactically legal in JavaScript, but the result isn't what you want:

> a = ['a', 'b', 'c']
["a", "b", "c"]
> for (x in a) { console.log(x); }
0
1
2
undefined

Instead of iterating through the array, we're iterating through the array indices. Again, this is syntactically valid code, it just doesn't do what you want (though perhaps you might want to use this as an alternative to the familiar for (i=0; i<a.length; i++) idiom.) Unfortunately, if you don't test your code carefully, it might not be something you noticed. To make matters more confusing, the for (x in a) idiom works fine for another common data structure enumeration task: enumerating the keys in a map in both Python and JavaScript.

None of this is intended to be a criticism of the syntax of any language, the problem is the conflict between the syntax of each language. This is particularly troublesome for Web applications because the client side more or less must be written in JavaScript but the most popular Web frameworks are written in either Python (Django) or Ruby (RoR), so it's common to have to work in two languages at once. I don't even want to think about what happens if you have to work in a framework that uses Java, a language, which, despite it's name, is not really related to JavaScript, though they have confusingly similar syntax. This is one advantage of Node, a server for developing Web (or any other server) application in JavaScript. Since the client side being written in JavaScript is a fixed point, Node allows you to write your entire system in JavaScript. Of course, depending on your opinion of JavaScript, that may seem like a distinctly mixed blessing.

 

June 5, 2011

I've been writing some JavaScript lately and figured unit testing might be a good idea. I'm using JQuery, so QUnit seemed appropriate. By and large it seems pretty solid, but I recently discovered something annoying. I was working on a test suite in which I first created and stored an object and then retrieved it. Everything was going along swimmingly and then I messed something up in the retrieve code. No problem, that's the kind of thing unit testing is supposed to catch, so I fixed the bug, but the retrieve still didn't work.

A little debugging revealed the proximal cause. The store returns a new object identifier (actually a sequence number), which I was using to retrieve the object. But when I went to get the object, the identifier was 0; it had never been set. For a while I thought I was just missing something important about JavaScript variable scoping, but after a bunch of debugging I uncovered the real problem: when you have a set of QUnit tests and one of them fails, QUnit remembers. The next time you run the test suite, it helpfully runs the tests that failed first. So, consider the following code:

       module("Test");
       asyncTest("Test 1", function(){
           d = new Date();
           console.log("Test 1: " + d);
           ok(true);   // report success
           start();
         });
       
       asyncTest("Test 2", function(){
           d = new Date();
           console.log("Test 2: "+d);
           ok(false);  // report failure
           start();
         });

The first time you run the test suite, Test 1 runs first (and succeeds) and then Test 2 runs (and then fails). However, the next time, Test 2 runs first, then Test 1. In a real scenario where Test 2 depends on Test 1, Test 2 will fail again, which means it will run first again, ad infinitum.

This seems like a good idea from some perspective, I guess: why should you have to wade through all the tests that work in order to retest the one that failed? Unfortunately, if the tests need to be run in a specific order then everything goes to hell.

I don't see this feature in the documentation; I found it by source code inspection. I suppose it's probably in there somewhere, though. Anyway, there's a way to force the tests to run in order. You just do: QUnit.config.reorder = false;

Outstanding!

P.S. Mrs. G tells me that unit tests are supposed to be order independent and so I should make any operations that need to run in sequence a single test. That's one way to do things, I guess, but I don't really want my software silently forcing it on me. `

 

April 4, 2011

Dear Twitter people,

It's not good when your site gets wedged so not only do I just get the title bar with no content, but I can't even log out to reset. Had to delete all my Twitter cookies to restore state. Outstanding!

Love,
EKR

 

February 5, 2011

I recently installed a brand-new Windows 7 machine (no fear, readers, it was in a VM for extra safety...) and captured the following screen shots. On the left is the screen you see when Chrome starts up, asking you which search engine you want to use. On the right is IE's search engine selection screen:

This contrast is sort of revealing. First, Chrome offers you choices up front. By contrast, IE just assumes you want to use Bing; to access this menu you need to pull down a dialog next to the search box, which isn't exactly obvious. Second, Chrome features Google's main competitors: Yahoo and Bing (anyone know if it offers Baidu in China?), whereas Bing seems to think you're more likely to want New York Times visual search than Google. If you look at the scroll bar, you'll see that I had to scroll down to even reveal Google. Outstanding!