The futility of high tech forecasting

In my other blog, I took the authors of a famous business book to task for making bad projections about future technological change. But it's not fair to single them out -- the same sort of problem happens to experts making projections inside technology companies. I've seen a lot of those projections over the years. The usual pattern is that technology predictions with a two year horizon are pretty good, because a technology has to be almost in prototype stage now in order to appear in high-volume products two years from now. Five year predictions are moderately useful, but subject to embarrassing errors. Ten-year predictions are almost useless, and twenty-year predictions are best used as plot outlines for science fiction novels.

This came home to me the other day when I stumbled across an old document from Palm that I had put aside years ago. It was the company's projection, created in 1997, of what future handhelds might look like in five, ten, and twenty years.

Keep in mind that the folks at Palm were, at the time, the world's leading creators of handheld technology. It's humbling to see how many things they missed. Here's what the experts forecast, with some comments from me:


Five years in the future (in other words, predicted for 2002):

-320x320 display. Okay, that came along with the Tungsten T at the end of 2002. And it was in other companies' handhelds before that.

-56 kbps wireless data. Didn't happen in a Palm-branded product until the Tungsten C in 2003.

-Built in cell phone. Early smartphones were appearing by that time, although they were pretty clunky. I'll give this one only a partial correct score because I think the people making the prediction were looking at what would be built into a typical handheld, and smartphones weren't typical by then.

-Multi megabyte secondary storage (hard drive or flash). OK.

-MicroCD player. No, a clear miss. Rotating removable media turned out to be much more delicate, expensive, and power hungry than we'd all like it to be.

-Voice synthesis. Didn't happen with the level of quality that was needed for it to become commonplace.

-High speed serial for desktop connectivity. I think USB 2.0 qualifies here, although the authors thought it would be FireWire. But I don't think any handhelds were shipping with either USB 2.0 or FireWire in 2002 (please correct me if I missed one).


Ten years in the future (in other words, 2007):

-Size and thickness of a sheet of cardboard. If only. Part of the problem is that it's hard to make something that thin also rigid enough to resist breakage.

-Foldable screen; can be used in folded or unfolded mode. Not going to happen next year, alas. Maybe in another five years, although personally I'm even a little skeptical about that.

-24-bit color. Okay.

-Voice recognition and synthesis. Not with the reliability that would let you use it as your primary interface to the device.

-10 mbit/second wireless data. They didn't specify whether they meant local wireless or cellular. We won't have cellular that speed in most places, but some companies have announced 802.11g modules for handhelds, so maybe...

-Built in cell phone. Okay.

-Built in TV/FM receiver. This is possible but there doesn't seem to be huge demand for it, at least not in the US.

-Built in video camera. Okay, but it'll be a pretty low-res one.

-Built in GPS. Okay. But let me point out that no one I'm aware of is doing GPS + TV + FM + video camera + cellphone + 802.11g in a single device, which is what was predicted.

-Body heat powered. Uhhh, no.


Twenty years in the future (ie, 2017).

This prediction still has a long time to run, so maybe we can revisit it if I'm still alive and blogging in 11 years. In the meantime, here's how the prediction looks now:

-Device is surgically implanted into brain. I doubt it, more for cultural reasons than technical ones. Even if we can find a way to do this, I think the current generation will be uncomfortable with implanting anything into their brains. I think people will have to grow up with that idea in order to be comfortable with it. Kind of like nose rings.

-Displays images to optic nerve. There have been some interesting experiments in stimulating the optic nerve to produce synthetic sight for people with retinal damage. The research is still very early, though. I don't doubt that this will happen someday, but I think that in 11 years it won't be something that you'd use with anyone other than a medical patient.

-Talks into auditory nerve. Similar to the vision situation.

-Input through voice, muscle input, or thought. I am completely confident that we won't have handhelds that can read thoughts eleven years from now. Reading gestures is more interesting; there has been a lot of research on reading eye movements and such. The challenge seems to be more on the software side than in hardware – how would a gesture-driven interface work, and how would you train people to use it?

-Powered by blood movement. I guess it would have to be if you're going to put it in someone's brain. You wouldn't want to have to plug your head into the wall to recharge every night.


Twenty years seems to be the magic horizon at which we think anything is possible. Maybe that says something about how we all think. Or maybe it's just a sign of the pending Singularity.

7 comments:

Anonymous said...

"Twenty years seems to be the magic horizon at which we think anything is possible."

I had a good laugh over this one!

Count me among the "Singularity is any day now" skeptics. I sometimes wonder if one reason we repeatedly overestimate what we'll accomplish technically is that the actual pace of technical development--taken as a whole--is slowing. If you compare the achievements of the first and last half of the 20th century for example, it's hard not to conclude that the number and significance of technical breakthroughs was a lot higher in the first half. It's hard to put a metric on it but life in 2000 sure looked a lot more to me like life in 1950 than 1950 looked like 1900.

All us boomers were brought up to believe that if you could put technology on a timescale the graph would look like a hockey stick. I wonder if those of us who grew up on the Apollo missions are having trouble adjusting to not having our minds blown every few years.

Great post, Michael.

Hey, will we be getting another chapter of "Eliminating the Middleman"? I'm eager to hear what you have to say about the guys who own the the wireless pipes.

Michael Mace said...

Good comments, folks. Thanks!


Tom wrote:

It's interesting to see how some of the predictions in voice technology (synthesis, recognition, etc.) have not come true.

Yeah, I've been burned on that one a few times myself. When I was at Apple in the mid-1990s, a guy by the name of Kai-Fu Lee was in charge of speech technology for the Mac.

Yeah, that Kai-Fu Lee.

He and his team came up with a speech interaction system for the Mac that included an on-screen avatar that would listen to your commands, carry them out, and also respond in synthesized speech.

Or at least that was how it was supposed to work.

The team did some amazing things with speech synthesis. I can't remember all of the details, but there was wild stuff like modules that would speak to you in voices that sounded like musical instruments, and APIs to let developers extend all of it. The architecture was incredibly rich and exciting and I was in love with it.

The only problem was that it didn't actually work all that well.

The speech synthesis sounded like someone with a serious learning disability, and the recognition was so inconsistent that you ended up yelling at the computer. A friend of mine who was one of the senior directors in Apple's product marketing team referred to the on-screen avatar, Victoria, as "the bitch" because she wouldn't do anything you asked her to.

The geek side of me is still in love with the idea of speech technology. It can be used for useful things, and if anyone can make it work well I'm sure Kai-Fu can. But what we all really want from it is to have the computer converse with us just like we converse with a human being. To do that you have to basically simulate an entire human mind, and I don't think we'll see that happen before I retire (and I'm not planning to retire for a long, long time).


David wrote:

life in 2000 sure looked a lot more to me like life in 1950 than 1950 looked like 1900

I get the same feeling (although you can make a pretty good case for 1850 to 1900 as a time of fundamental change as well -- society is recognizably modern by 1900, and in many ways it's not in 1850).

Ray Kurzweil makes a pretty good case for the Singularity in his book. I'd like him to be right, but I keep coming back to the feeling that the Jetsons promised me a flying car, I don't have a flying car, and I feel kind of cheated by that.


Hey, will we be getting another chapter of "Eliminating the Middleman"?

There are several more chapters in the works, but it's taking me an embarrassingly long time to write them. I apologize for the delay. Two things have happened -- the consulting business has picked up, and I'm involved in a zoning protest between my neighborhood and a developer who's in love with putting very large houses on very small lots. Much of the time that I had been spending writing blog entries has instead gone into meeting with the neighbors (we're all up in arms) and making presentations for use with the City Council.

It looks like the zoning thing is going on hold for a month, so I hope to have some more time for writing. E-books are the subject I'll hit first.


Tomi wrote:

When they said cardboard thickness, if they were shown today's slimmest phones that approach the thickness of credit cards, I think they would give us that forecast as being very close to the mark.

Fair enough, I was probably a little too harsh there. But keep in mind that they were predicting a full smartphone at that thickness, not just a mobile phone. I think the smartphone of today misses cardboard thickness by a pretty wide margin.


When you say "not with reliability to use as primary interface" - I would argue that the speech-dialling and very simple features work fine in voice recognition

They work okay if you have the patience to train them (many people don't). But again you have to look at the context -- they were predicting a full two-way speech interface for a handheld (in other words, a miniature computer). That's a lot more demanding than voice dialing, and we're nowhere near having a speech command system that flexible in a small mobile device.

You're completely right that speech synthesis on large networked servers has improved a lot. That's probably more useful in Europe where wireless access can be more or less taken for granted. Here in the connectivity-challenged US, an interface technology that depends on a reliable wireless connection is a non-starter.

So maybe the answer to this one depends on which continent you're living in. ;-)


you have clearly not seen the early test data from trials in Berlin Germany, Helsinki Finland, Oxford UK etc.

You're right, I haven't. Please give me a link or two to the results -- I'd like to read about them.

My personal guess (and it's just a guess) is that mobile video will do best in places where people have a lot of transit time (subways and buses). But I'm very open to being persuaded otherwise.


Anyway, good comments and please drop in anytime.

Anonymous said...

I'm with Tomi on this one. I think they nailed a lot of these. A few specific comments...

1) Cardboard-thickness. Where did you say that they were predicting a smartphone vs a handheld? The Visor Edge came out pre-2002 and wasn't much thicker than a piece of corrugated cardboard. Two big problems with this prediction are: a) Battery life hasn't improved at the same degree as other technologies, thus limiting the ability to make devices that are both small and powerful, and b) There's such a thing as "too thin" where a device doesn't feel as good to hold and use as one that's a bit thicker (see also "too small").

2) Built-in video cameras. The problem with quality here, similar to the issue of poor quality built-in digital cameras, is one of choice, not necessity. By that I mean that PDA/smartphone companies that have integrated digital cameras that offer poor quality because they have opted to use the cheapest camera part that they can source. Take a look at some ultra-small dedicated digicams from a couple of years ago. They took great photos. We often hear how these built-in cameras are just "gimmicks" or are just "for fun". That's because they've given us poor quality cameras. Nokia seems to be leading the way in changing this. Hopefully for Palm's sake, they won't be too late to get on board with the idea.

Having said all that, I have to say that this was a pretty conservative and unimaginative list of predictions (except for the foldable screen), so it doesn't surprise me that they were as close as they were. Most of what they predicted were technologies that already existed in 1997 and they were just predicting that they'd be integrated into future handhelds. Well, whoop-de-doo.

Michael Mace said...

Interesting comments. I wanted to follow up on a couple more things...


Tomi wrote

Built-in TV and FM tuners. I think you are being remarkably strict about this.

Actually, after sleeping on this one, I think we both kind of skipped past the actual forecast and talked about video in general. The prediction was about built-in video and FM tuners, not the use of video overall.

If you have an FM or TV tuner in a mobile device, there is no service for anyone to sell -- people just watch whatever's going out over the air. This is technically feasible today (there have been tiny mobile TVs in Japan for about ten years), and there have been FM tuners in some phones in Europe. But in most countries there isn't huge demand for it. I don't think the operators are going to push it either, as they can't charge for it.

Recorded or demand-streamed video on a mobile device is a different and much more complex issue. In the research I've seen, there is an audience for this, principally among young people who want entertainment on the go. The big question, and I don't personally know the answer to it, is how much they're willing to pay for it.

I don't think video on mobile devices will be a ubiquitous horizontal killer app, though.

Getting back to the context of the forecast, I think they were right on in terms of technical feasibility, but not in terms of market dynamics. That's a common problem with tech forecasts -- they tend to assume anything that's feasible will also be adopted by the market.

And before it sounds too much like I'm trying to beat up on the Palm folks, let me acknowledge that I've been in on some of these forecasting exercises in the past and haven't done any better than anyone else.


Scott wrote:

>Where did you say that they were predicting a smartphone vs a handheld?

The forecast was predicting the canonical Palm device for five, ten, and twenty years in the future. The devices in ten and twenty years both had phones built in as part of the feature set, making them by definition smartphones. But nobody I know of was using that actual term at that time.


Battery life hasn't improved at the same degree as other technologies, thus limiting the ability to make devices that are both small and powerful

Exactly. There's one of the things I mis-forecast years ago -- I thought fuel cells would be along a lot sooner. I learned from that not to trust a forecast for any technology that's not in working prototype yet. A technology either works in prototype form, or it's unfinished and very difficult to forecast accurately.


>There's such a thing as "too thin" where a device doesn't feel as good to hold and use as one that's a bit thicker (see also "too small").

I suppose there could be a theoretical point where a device is too thin to be comfortable, but I don't think we're anywhere close to that point yet. (Maybe if it's so thin that it slices the skin...)


By that I mean that PDA/smartphone companies that have integrated digital cameras that offer poor quality because they have opted to use the cheapest camera part that they can source.

I think this is one of those cases in which trying to design for the average customer screws you up. For a lot of people the built-in camera is irrelevant -- they won't pay anything extra for it and therefore you can get away with a very junky camera. For a minority of people the camera matters and they will pay extra for quality.

But I think my biggest mistake on this particular forecast is that I said the video camera would be low resolution. Video cameras are almost by definition low resolution compared to still cameras, since you have to capture so manhy images. I'm not sure what I was thinking when I wrote that.

I think they were just plain right in that particular prediction.


I have to say that this was a pretty conservative and unimaginative list of predictions (except for the foldable screen).

I dunno, I think the "powered by body heat" prediction was the most radical. The foldable screen has been kind of a holy grail among the display guys for years, they just haven't been able to deliver it yet (see my comments on fuel cells, above).

Anonymous said...

An interesting article today from Thomas Wailgum of CIO magazine... "The problem I have with long range IT Forecasts." (http://blogs.cio.com/the-problem-i-have-with-long-range-it-forecasts)

Didn't have time to go through it and the comments yet, but he reiterates your point... it's pretty rediculous to believe in long term tech forecasting.

I made a living doing forecasts for a Fortune 500 company early in my career, and I've even hit an annual product forecast with 7-figure units within about 25! I must be really good, huh? Well, maybe. But mostly I was very lucky, and one year is not the same as seven years or ten years or twenty years.

I can tell you this... long term forecasting is a tough business if you have to be right. But it's really not about being right, it's about opening the eyes of planners to the kinds of environments that they might face down the road so that they can make wise decisions now. And it's about quantifying those environments so they can have something concrete to work with. You don't want to get caught playing catch up because you didn't prepare or ramp up for something that was considered a most likely outcome. If you are running a business, that is downright negligent!

Anyway, just wanted to share the new reference, even if it's an older post.

Michael Mace said...

Bob wrote:

>>it's really not about being right, it's about opening the eyes of planners to the kinds of environments that they might face down the road so that they can make wise decisions now. And it's about quantifying those environments so they can have something concrete to work with.

Exactly! A good forecast should show possibilities, rather than a single outcome, and should describe which factors would lead to each outcome. Unfortunately, almost all of the industry forecasts you'll see show a single set of numbers, with at best a margin of error. They try to make the future look like a static thing that's going to happen, rather than a flexible thing that we can change by our actions.

You wonder if the same people who use these forecasts also believe in predestination. Hmmm, "industry analysts vs. free will." that would make a fun (if strange) blog entry.

By the way, no need to apologize for commenting on an old post. Due to the wonders of Google search, these articles seem to be turning into ongoing discussion items, which is cool.

Anonymous said...

I sure would love to have a converged handheld device with:

+ 802.11
+ InfraRed (IR)
+ Bluetooth
+ 3G or greater Wireless Data
+ USB Host Controller
+ Hi Res Screen > 800x600
+ TV Tuner
+ HDTV (Digital TV)
+ AM & FM w RDS Tuners
+ Digital Radio
+ Other Over the Air Signals
+ GPS
+ Digital Camera > 5MP w Flash
+ Video Cam
+ Optical Zoom > 3x
+ Character Recognition
+ Face Recognition (& more)
+ Lots of Storage
+ Long Battery Life
+ Rugged
+ Video Output like HDMI
+ Surround Sound Out
+ More than 1 built-in Speaker
+ Great Web Browser
+ Cut Copy Paste!!!
+ Plays All Video Formats
+ Records Any Stream
+ PIM Apps
+ Maps built-in
+ 3rd Party Apps
+ Google Earth
+ Reliable
+ Secure
+ Never Crashes
+ VoIP
+ ...
with NO monthly fees or other hidden charges.