Saving Private Information

I just googled “information” and got 5,930,000,000 hits, or nearly 6 billion hits in 0.06 seconds.

Compared to someone sitting at home about 20 years ago, my access to information from within the comfort of my home is a few orders of magnitude higher. Hal Varian and Peter Lyman at UC Berkeley estimated that the rate of production of information was two exabytes, or two billion billion bytes in 2001. That information could take a stack of floppy discs about 2 million miles high. That rate must have gone up and I can reasonably assert that for this year we will need a stack 3 million miles high to contain all our NEW data. Thankfully we don’t use floppy discs and depend on hard-drives.

Talking about the total amount of information available, one would start sounding like Carl Sagan and say “billy-uns, and billy-uns.” Like the good astronomer saw galaxies by the billions, we too see pages by the billions, and like the galaxies, all these pages are in a very specific sense not personally accessible. They exist out there but it does not make any practical difference to us. So I distinguish between public and private information.

All those exabytes of information is what I call public information which is potentially accessible and constitutes the world wide web. What information you actually process is private information. How much private information you have matters to you, and not how much public information exists out there.

Information, like talk, is cheap. More accurately, public information is cheap given that there is so much of it around. The supply far outstrips the demand. But private information is not cheap. I propose this “Atanu’s Law of Information: Public information is a public good, and private information is private good.

The process of converting public information into private information involves costs and that is what makes it valuable. The cost is primarily time. How much time you take to read a book depends on your ability to comprehend the written word of course, but the time to watch a movie or listen to a song is not compressible. Since time is a binding constraint in our case (we are not immortals), every action has an opportunity cost. As the process of converting public information into private information takes time, it has an opportunity cost and therefore there is a limit to how much private information you can have.

What are the operational implications of this distinction? First, it helps us understand that just because exabytes of information is publicly available does not suddenly solve our informational problems. Private information is what matters to a person. Processing this information efficiently requires skills that were not so acutely needed in the past. Our educational system has to fundamentally change in light of this new situation.

Second, we have more private information than ever before. Think of the huge number of pages that you have read and may need to refer to sometime in the future; of the thousands of pictures you have an interest in; of the songs and movies that you wish to keep and be able to locate sometime later; of the people you have met and need to keep track of; the list goes on. So the challenge is to manage that private information. And where there is a challenge, there is an opportunity for developing a solution.

So what do Yahoo, Google, and others of their ilk really do that is relevant to you? They manage some of your private information (media files, email, etc) and they also funnel some public information into your information horizon. This funneled information we can call “pseudo-private” and requires your processing time before it becomes private information.

Managing private information comprehensively is an art that is slowly emerging but a lot needs to be done. Fundamental opportunities exist out there. A glimpse is all that I am allowed to give you here. Think mobile phones. Mobile phones will morph into “private information access devices” and not something that you use to talk to your friends.

More to come.

Author: Atanu Dey


5 thoughts on “Saving Private Information”

  1. Atanu,

    I googled information now and got 5.96 bn hits in 0.04 seconds – 0.03 bn addl. hits in the space of some hours……is this some indication of the the growth of knowledge/ information I wonder.

    The challenge remains making the information comprehensible, relevant and timely to a user. One issue that bothers me at a personal level is the amount of stuff I need to wade through in any book or article to add to my understanding of a subject. I am going a bit beyond the public vs private info divide; seeking to know if there is any initiative that can make learning more efficient. Even with access to “private information”, the processing and sieving of this information remains a challenge. It would be wonderful if there was some way in which humans could learn to improve their thought processes and technology could enable that. I hope this doesn’t sound too much like wool-gathering!- any thoughts?


  2. Mr. Ramchandra,
    While getting more pages on a google search may not necessarily translate into more information being present “out there”, it certainly is true for “all practical purposes”. As of today, if something does not turn up in a Google search result, its pretty much non-existent.
    However, talking about your idea of “improving our though-processes”, I do think that this has indeed been going on over the centuries. We have developed much finer abilities to comprehend information that is thrown at us. The more interesting thing is what technology can do to aid it. Maybe crawlers that can read through blog entires (and comments) and create summaries that are relevant to a particular person. While work such areas is already going on on a fairly large scale, what we need is a mechanism to store (and represent) “knowledge” as opposed to “information”. This would be the situation, for example, where an organisation acquires know-how about certain procedures. As older employees leave, this know-how has to be transferred to newer ones. Standardized mechanisms to do this would add tremendous amounts of productivity to both organisations as well as individuals. Ofcourse, this would require more than just “technology” as we know it – it would require a confluence of engineering, psychology, philosophy, linguistics and a whole bunch of other stuff.


Comments are closed.

%d bloggers like this: