On the length of standards development

| Comments (5) | Networking
David Coursey complains about how long it took IEEE to develop 802.11n:
802.11n is the poster child for a standards process gone wrong. Seven years after it began and at least two years after 802.11 "draft" devices arrived, the IEEE has finally adopted a final standard for faster, stronger, more secure wireless.

Ideally, standards arrive before the products that implement them. However, the IEEE process moved so slowly that vendors adopted a draft standard and started manufacturing hardware. After a few little glitches, the hardware became compatible and many of us have--for years--been running multivendor 802.11n networks despite the lack of an approved standard.

...

If standards bodies expect to be taken seriously, they need to do their work in reasonable periods. Releasing a "final" standard long after customer adoption has begun is not only anti-climatic but undercuts the value of the standards making process.

In this case, the process failed. The IEEE should either improve its process or get out of the way and left industry leaders just create de facto standards as they see fit. That is not preferable, but if the IEEE process is stuck, it will be what happens.

My experience with IEEE standards making is limited, but I have extensive experience with IETF's process, and I'm a little puzzled as to what Coursey thinks the problem is here. Developing standards is like developing any other technical artifact: you start out with an idea, do some initial prototypes, test those prototypes, modify the design in response to the testing, and iterate till you're satisfied. Now, in the case of a protocol standard, the artifact is the document that defines how implementations are supposed to behave, and the testing phase, at least in part, is implementors building systems that (nominally) conform the the spec and seeing how well they work, whether they interoperate, etc. With any complicated system, this process needs to include building systems which will be used by end-users and seeing how they function in the field. If you don't do this, you end up with systems which only work in the lab.

There's not too much you can do to avoid going through these steps; it's just really hard to build workable systems without a certain level of testing. Of course, that still leaves you with the question of when you call the document done. Roughly speaking, there are two strategies: you can stamp the document "standard" before it's seen any real deployment and then churn out a revision a few years later in response to your deployment experience. Alternately, you can go through a series of drafts, refining them in response to experience, until eventually you just publish a finished standard, but it's based on what people have been using for years. An intermediate possibility is to have different maturity levels. For instance, IETF has "proposed standards", "draft standards", and then "standards". This doesn't work that well in practice: it takes so long to develop each revision that many important protocols never make it past "proposed standard." In all three cases, you go through mostly the same system development process, you just label the documents differently.

With that in mind, it's not clear to me that IEEE has done anything wrong here: if they decided to take the second approach and publish a really polished document and 802.11n is indeed nice and polished and the new document won't need a revision for 5+ years, then this seems like a fairly successful effort. I should hasten to add that I don't know that this is true: 802.11n could be totally broken. However, the facts that Coursey presents sound like pretty normal standards development.

5 Comments

I think that the issue is that as long as the standard protocol isn't officially final, anyone who develops something using the protocol in effect does it for some ad-hoc implementation which may be used in the industry but isn't a standard.

In Theory IEEE could have decided to make a change to 802.11n that would have made all existing devices non-compliant. This can happen with any implementation done before the standard is final.
So in this case all vendors who claimed to have 802.11n compliant devices would have become guilty of false advertising, etc, and anyone getting a next generation 802.11n would have their devices maybe not working with other devices, despite all of them claiming to be using 802.11n .
Alternately, vendors would have said at that point that they don't really care about what IEEE claimed 802.11n is, and as far as they're concerned the standard is the ad-hoc one, while the official one doesn't matter and is ignored.

These two scenarios didn't happen here because the final standard, and the equipment already on the market, are similar enough. But that's not a requirement of the process.

With a short cycle, it's not an issue. But spending year producing equipment using a protocol that you don't know whether you actually support or not, is a problem.

Personally I didn't realize 802.11n wasn't a standard until the news articles a few days ago. The excuse that it 'requires testing' doesn't really hold water (and is rather naive). If you have any experience with any standards process - you would realize that the reason these processes take so long is that people argue and have different agenda's. It has little to do with the technical aspects.

The process is broken. But its unclear what the fix is. But I don't think it really matters. Vendors will move forward with or without it.

As it happens, I have quite a bit of experience with standards processes,
seeing as I'm a listed author or editor of on the order of 15 IETF RFCs, chair of the
IETF TLS WG, etc.

It's certainly true that people do have different agendas and that in some
cases the standards process take a long time to shake out those agendas,
reach compromises, etc. However, this really isn't the only issue. I've worked
on plenty of documents (TLS 1.2, ICE, etc.) where there wasn't any major contention about
what they should contain, but the WG was full of people who kept
finding small issues (because the stuff is complicated) and/or just didn't
feel it was baked enough to sign off. Unfortunately, both of these
processes take a lot of time.

I'm certainly not arguing that standardization isn't too slow: it clearly is,
though 7 years isn't really that slow for a major effort like 802.11n when
compared to other efforts. What I'm arguing is that a 2 year gap between
when the standard is basically baked enough that people feel they
can start deploying prototype gear and the time when the standards
organization finally signs off isn't that unreasonable in the context
of a 5+ year standards process, which, as I've said, is normal.

To Yaron's point, "...because the final standard, and the equipment already on the market, are similar enough. But that's not a requirement of the process."

That's the reality of the process. Look, standards is a repeated game among all the participants. They develop reputations over a long period. Moreover, they have to be responsive to the market over a long period. So call the time between "baked" and "official" "interim". If any individual member defects and tries to gain advantage by changing the standard significantly for gain during "interim", he will be punished on the next round. If there's a major change during "interim" that confuses the market, the market will punish the entire standards body.

This is a relatively stable situation where the only departures will be if the body jointly thinks they discover a real problem.

@Kevin - I do understand the theory, it's just that in practice it may not feel, or be, that way for all the participating vendors.
There will be vendors who may not be the strong pushers in IEEE (or any other standards body), and who need to know if what they develop does, or does not, support the standard.
Having machines on market for a while who are in interim stage is fine. But years... is a long time for this. Customers expect devices that can say "support X" rather than "Support X, maybe, for the current temporary iteration of X, pending..." . It's understandable for developers to feel uncomfortable with not being able to provide that for years.

Leave a comment