Here’s a new exaflood metric for you — tweets per second.
From the Twitter blog:
Folks were tweeting 5,000 times a day in 2007. By 2008, that number was 300,000, and by 2009 it had grown to 2.5 million per day. Tweets grew 1,400% last year to 35 million per day. Today, we are seeing 50 million tweets per day—that’s an average of 600 tweets per second. (Yes, we have TPS reports.)
A number of interesting new articles and forums deal with our exaflood theme of the past few years.
“Striving to Map the Shape-Shifting Net” – by John Markoff – The New York Times – March 2, 2010
“Data, data, everywhere” – The Economist – Special Report on Managing Information – February 25, 2010
“Managing the Exaflood” – American Association for the Advancement of Science – February 19, 2010
“Professors Find Ways to Keep Heads Above ‘Exaflood’ of Data” – Wired Campus – The Chronicle of Higher Education – February 24, 2010
“The defenders of modern macroeconomics argue that if we just study the economy long enough, we’ll soon be able to model it accurately and design better policy. Soon. That reminds me of the permanent sign in the bar: Free Beer Tomorrow.
“We should face the evidence that we are no better today at predicting tomorrow than we were yesterday. Eighty years after the Great Depression we still argue about what caused it and why it ended.
“If economics is a science, it is more like biology than physics. Biologists try to understand the relationships in a complex system. That’s hard enough. But they can’t tell you what will happen with any precision to the population of a particular species of frog if rainfall goes up this year in a particular rain forest. They might not even be able to count the number of frogs right now with any exactness.
“We have the same problems in economics. The economy is a complex system, our data are imperfect and our models inevitably fail to account for all the interactions.
“The bottom line is that we should expect less of economists. Economics is a powerful tool, a lens for organizing one’s thinking about the complexity of the world around us. That should be enough. We should be honest about what we know, what we don’t know and what we may never know. Admitting that publicly is the first step toward respectability.”
— Russ Roberts, February 27, 2010
That’s the question I ask in this Huffington Post article today.
Among all the books, articles, and academic papers analyzing the financial meltdown, very few have pinpointed and exposed what I think was the accelerant that turned a problem into an all-out panic: namely, the zealous application of mark-to-market accounting beginning in the autumn of 2007. In this video, two of these very few — Brian Wesbury and Steve Forbes — discuss the meltdown, mark-to-market’s crucial role, and the stock market’s short and mid-term prospects. Wesbury and Forbes have also written two great books explaining the Great Panic, why it’s not as bad as you think, and how capitalism will save us.
Holman Jenkins today also picks up the theme of mark-to-market’s central role in the panic.
Wyoming wireless operator Brett Glass has 20 questions for the FCC on Net Neutrality. Some examples:
1. I operate a public Internet kiosk which, to protect its security and integrity, has no way for the user to insert or connect storage devices. The FCC’s policy statement says that a provider of Internet service must allow users to run applications of their choice, which presumably includes uploading and downloading. Will I be penalized if I do not allow file uploads and downloads on that machine?
4. I operate a wireless hotspot in my coffeehouse. I block P2P traffic to prevent one user from ruining the experience for my other customers. Do the FCC rules say that I must stop doing this?
6. I am a cellular carrier who offers Internet services to users of cell phones. Due to spectrum limitations, multimedia streaming by more than a few users would consume all of the bandwidth we have available not only for data but also for voice calls. May we restrict these protocols to avoid running out of bandwidth and to avoid disruption to telephone calls (some of which may be E911 calls or other urgent traffic)?
7. I am a wireless ISP operating on unlicensed spectrum. Because the bands are crowded and spectrum is scarce, I must limit each user’s bandwidth and duty cycle. Rather than imposing hard limits or overage charges, I would like to set an implicit limit by prohibiting P2P, with full disclosure that I am doing so. Is this permitted under the FCC’s rules?
14. I am an ISP that accelerates users’ Web browsing by rerouting requests for Web pages to a Web cache (a device which speeds up Web browsing, conceived by the same people who developed the World Wide Web) and then to special Internet connections which are asymmetrical (that is, they have more downstream bandwidth than upstream bandwidth). The result is faster and more economical Web browsing for our users. Will the FCC say that our network “discriminates” by handling Web traffic in this special way to improve users’ experience?
15. We are an ISP that improves the quality of VoIP by prioritizing VoIP packets and sending them through a different Internet connection than other traffic. This technique prevents users from experiencing problems with their telephone conversations and ensures that emergency calls will get through. Is this a violation of the FCC’s rules?
18. We’re an ISP that serves several large law offices as well as other customers. We are thinking of renting a direct “fast pipe” to a legal research database to shorten the attorneys’ response times when they search the database. Would accelerating just this traffic for the benefit of these customers be considered “discrimination?”
19. We’re a wireless ISP. Most of our customers are connected to us using “point-to-multipoint” radios; that is, the customers’ connection share a single antenna at our end. However, some high volume customers ask to buy dedicated point-to-point connections to get better performance. Do these connections, which are engineered by virtually all wireless ISPs for high bandwidth customers, run afoul of the FCC’s rules against “discrimination?”
See our new commentary at CircleID:
The Internet has two billion global users, and the developing world is just hitting its growth phase. Mobile data traffic is doubling every year, and soon all four billion mobile phones will access the Net. In 2008, according to a new UC-San Diego study, Americans consumed over 3,600 exabytes of information, or an average of 34 gigabytes per person per day. Microsoft researchers argue in a new book, “The Fourth Paradigm,” that an “exaflood” of real-world and experimental data is changing the very nature of science itself. We need completely new strategies, they write, to “capture, curate, and analyze” these unimaginably large waves of information.
As the Internet expands, deepens, and thrives—growing in complexity and importance—managing this dynamic arena becomes an ever bigger challenge. Iran severs access to Twitter and Gmail. China dramatically restricts individual access to new domain names. The U.S. considers new Net Neutrality regulation. Global bureaucrats seek new power to allocate the Internet address space. All the while, dangerous “botnets” roam the Web’s wild west. Before we grab, restrict, and possibly fragment a unified Web, however, we should stop and think. About the Internet’s pace of growth. About our mostly successful existing model. And about the security and stability of this supreme global resource.
Excellent analysis of Google’s plan to build a few experimental fiber networks from my former colleague Barbara Esbin:
NetworkWorld reports that by constructing its own fiber network, Google “is trying to push its vision for how the Internet as a whole should operate.” I wish the company all the success in the world with GoogleNet. Business model experimentation and new entry to the broadband Internet service provider market like this should be encouraged. If this “open access” common carrier network proves to be a viable business model that attracts both customers and followers, it will be a fabulous addition to the domestic Internet ecosystem. But this vision should not be turned into unnecessary government mandates for other Internet network operators who are similarly trying to experiment with their business models in this brave new digital world.
Surprisingly, I also agree with Harold Feld’s analysis:
the telecom world is all abuzz over the news that Google will build a bunch of Gigabit test-beds. I am perfectly happy to see Google want to drop big bucks into fiber test beds. I expect this will have impact on the broadband market in lots of ways, and Google will learn a lot of cool things that will help it make lots of money at its core business — organizing information and selling that service in lots of different ways to people who value it for different reasons. But Google no more wants to be a wireline network operator than it wanted to be a wireless network operator back when it was willing to bid on C Block in the 700 MHz Auction.
So what does Google want? As I noted then: “Google does not want to be a network operator, but it wants to be a network architect.” Oh, it may end up running networks. Google has a history of stepping up to do things that further its core business when no one else wants to step up, as witnessed most recently by their submitting a bid to serve as the database manager for the broadcast white spaces devices. But what it actually wants to do is modify the behavior of the platforms on which it rides to better suit its needs. Happily, since those needs coincide with my needs, I don’t mind a bit.
I do mind.
This tax-and-budget analysis from Bruce Bartlett is wrong on many levels — in both its particulars and its overall sweep.
Bartlett claims the famous supply-side tax-cutters at The Wall Street Journal editorial page have, in a major reversal, opened the door to a Value Added Tax and thus a major expansion of overall taxation and American government. He thinks a new Journal opinion article from Columbia Business School dean Glenn Hubbard represents a big shift in the thinking of economic conservatives. I don’t see it that way at all. (more…)
Cisco’s latest Visual Networking Index, this one focusing mobile data traffic, projects 108% compound growth through 2014.
“I have only one project, one big idea: uncertainty. It crosses many different disciplines — math, political science, psychology, risk management — and I swing in between those, but it is always on what we call the epistemological question. There are two parts to this question: math and computation, and psychology. The second causes us to think we know more than we do. It is an endless topi. Bernanke has six problems: One, his education is in tools that aren’t helpful — and he doesn’t know it. Two, he studied the Great Depression, and he thinks he knows too much — this is nothing like the Great Depression. You can’t compare this and the Depression. Three, 99% of risk is tied to the debt/leverage and the explosion of connectivity. It’s like he did not see a truck coming right at him. Four, he has no notion of nonlinearities, and how monetary policies can be responsive in nonlinear ways. Five, he doesn’t understand fat tails. Six, he doesn’t realize that the biggest risk of failure is signified by the Federal Reserve: He thinks we need more regulation; we actually need smaller institutions. And not one person in Congress had the presence of mind to ask him these questions.”
— Nassim Nicholas Taleb, AI5000, Jan/Feb 2010
We’ve been discussing the dramatic growth of the global Internet and the expansion of physical devices and virtual spaces that come with the mobile revolution, social networking, cloud computing, and the larger move of the Net into every business practice and cultural nook.
Last week ICANN, the organization that administers the Internet’s domain space, announced that fewer than 10% of current-generation Internet addresses (IPv4) remain unallocated. In any network realm, a move above 90% capacity is an alarm bell that needs attention. IPv6 is the next generation address space and is being deployed. But the move needs to accelerate to ensure the unabated growth of the Net.
Developed in the 1990s, IPv6 has been available for allocation to ISPs since 1999. An increasing number of ISPs have been deploying IPv6 over the past decade, as have governments and businesses. The biggest attraction of IPv6 is the enormous address space it provides. Instead of just 4 billion IPv4 addresses – fewer than the number of people on the planet – there are 340,282,366,920,938,463,463,374,607,431,768,211,456 IPv6 addresses. An easier way to think of this number is 340 trillion trillion trillion addresses.
Or, the famous comparison: If IPv4 is a golf ball, IPv6 is the Sun.
See our new analysis of Net Neutrality regulation’s possible impact on the U.S. job market.
“Rationing happens today! The question is who will do it.”
— Rep. Paul Ryan, February 2, 2010, in a terrific interview on health care reform that drills to the center of the debate.
Also see:
The Wall Street Journal‘s Digits blog asks, “Could Verizon Handle Apple Tablet Traffic?”
The tablet’s little brother, the iPhone, has already shown how an explosion in data usage can overload a network, in this case AT&T’s. And the iPhone is hardly the kind of data guzzler the tablet is widely expected to be. After all, it’s one thing to squint at movies on a 3.5-inch screen and quite another to watch them in relatively cinematic 10 inches.
“Clearly this is an issue that needs to be fixed,” says Broadpoint Amtech analyst Brian Marshall. “It can grind the networks to a halt.”
Just two more New York Times articles that point out what’s obvious around here: the Internet’s dramatic and unpredictable disruption of the whole “media” space. Isn’t Washington’s assumption that it can sort all this out and impose particular business models on the media space through prescriptive Net Neutrality regulation, a case of supreme hubris?
“My attitude is this: if you are getting attacked by Krugman, you must be doing something right.”
— Eugene Fama, University of Chicago professor, in The New Yorker
Amazon’s Paul Misener gets all reasonable in his comments on the FCC’s proposed net neutrality rules:
With this win-win-win goal in mind, and consistent with the principle of maintaining an open Internet, Amazon respectfully suggests that the FCC’s proposed rules be extended to allow broadband Internet access service providers to favor some content so long as no harm is done to other content.
Importantly, we note that the Internet has long been interconnected with private networks and edge caches that enhance the performance of some Internet content in comparison with other Internet content, and that these performance improvements are paid for by some but not all providers of content. The reason why these arrangements are acceptable from a public policy perspective is simple: the performance of other content is not disfavored, i.e., other content is not harmed.
For those of you not wishing to sift through 15,000 comments submitted to the FCC for its Net Neutrality proposed rule making, let me recommend what — so far — is the best technical filing I’ve read. It comes from Richard Bennett and Rob Atkinson of the Information Technology Innovation Foundation.
Also very useful is a new post by George Ou on content delivery and paid peering, with important policy implications.
These are among the least discussed — but most important — items in the whole Net Neutrality debate.
Separately, from the FCC’s “Open Internet” meeting at MIT last week, see summaries of each panelist’s remarks: Opening Presentations, Panel 1, Panel 2.
Maximum Entropy