Brewster Khale of Wayback Machine and Internet Archive was quoted by the NY Times today when asked to comment on all of the "junk" he must be archiving along with all of the "worthwhile stuff."

See the article by John Schwartz:
From Unseemly to Lowbrow, the Web's Real Money Is in the Gutter

Brewster Kahle, who has created a large Internet archive he calls the Wayback Machine, which contains several times the amount of information in the Library of Congress, said that the number of questionable sites is beside the point so long as search engines do their job.

"We don't worry about how many pages that I don't care about are in the Internet archive," he said. "What you do care about is, `Does it have the pages that I want?' "

He acknowledged, however, that "we haven't done a very good job of putting the good stuff up there" on the Internet to dilute the bad, and that as a result today's leaders "are shortchanging the next generation."

...To Mr. Kahle, the Internet's diversity, good and bad, means that people will find the information they want, as narrowly and as deeply as they care to explore it. While he, too, would like some method to control the spam that flows into his e-mailbox, he said he preferred a complex ecosystem to a monoculture as bland and regular as a suburban lawn.

"I grew up where almost everybody could sing all three verses of the `Gilligan's Island' theme song," he said. "I don't want my children to grow up like that."

Here's the full text of the article in case the link goes bad:

http://www.nytimes.com/2002/08/26/technology/26CYBE.html

From Unseemly to Lowbrow, the Web's Real Money Is in the Gutter
By JOHN SCHWARTZ

Life is good at the bottom of the cyberspace pit. When the Arizona attorney general's office recently shut down a Scottsdale company, CP Direct, it offered a glimpse into the spoils of the Internet's dark side. The company sold pills via the Web that promised to increase penis length, bust size and body height.
Advertisement

Enough people had spent $60 an order — mostly for the "Longitude" treatments, with ingredients that included pumpkin seed, sarsaparilla and "oyster meat" — that the state was able to seize more than $30 million in luxury real estate and a herd of Mercedes-Benzes, Rolls Royces and a Lamborghini.

The dot-com bust has left the economy littered with the husks of companies that said they would transform the way that people work, live and play. But aside from such notably profitable Internet winners as eBay and edging-into-profitability giants like Amazon, it is not easy to find many pure Internet companies that make healthy profits without appealing to baser interests or using questionable business practices. Even huge companies like AOL Time Warner appear to be struggling to figure out — still — how to come up with online content and services that mainstream consumers will be willing to pay for.

Those who provide some of the sites that many people object to say they have nothing to hide. "Obviously, there's a want for this stuff, because there's billions of dollars spent on it every year," said David Marchlak, who has brought the world sites like Voyeur Dorm, a Web precursor to the "Big Brother" television show that watches a Florida house shared by a cast of young women. "We're not forcing it on anybody."

Yet these days, Internet users complain of a proliferation of Web sites that offer a peek up Anna Kournikova's skirt or that hawk pills to increase the size of their sex organs. The Internet was supposed to make people's brains bigger.

People now talk about wasting too much of their time sifting through e-mail invitations to view photographs of nymphets or unspeakable acts with farm animals, as well as requests to send money to people who say they are relatives of Jonas Savimbi of Angola or Mobutu Sese Seko of Zaire so they can release their stolen African millions.

Cyberspace is "debasing itself in front of our eyes," said Bruce Sterling, a science fiction author. Mr. Sterling, who sees the Internet becoming a pit of spam and swindles, pornography, corporate advertising and government surveillance, warns, "We will lose the Internet if we don't save it."

Why is so much high technology so lowbrow, with abominably written prose, horrific images like the beheading of the journalist Daniel Pearl and the sweaty sensibility of a tromp through the febrile fantasies of adolescent boys and middle-aged men?

One answer is that the Internet businesses that thrive most readily are the kinds that do well in every medium — including those selling titillation and cheating their customers.

"Every industry has its charlatans, and e-commerce is getting its share," said Safa Rashtchy, an analyst with U.S. Bancorp Piper Jaffray. "I don't see evidence that it is more than you would expect, especially in a new industry."

Not everyone who hangs out a shingle in the grimy quarters of the online world becomes rich. Maria, a woman who runs adult Web sites and a telephone sex service, said that dreams of quick riches are quickly dashed, especially for those entrepreneurs who spend thousands of dollars for pornography-site-in-a-box kits that are sold online, which often include stockpiles of images that are already in wide circulation.

"I don't see people getting rich," said Maria, who spoke on the condition that her full name not be used. "I see people getting burned" by their own inexperience and crushed by so-called chargebacks — cancellations of credit card charges by consumers.

Louis Bajjali, who runs a number of pornography sites, said that those who succeed share some characteristics with their counterparts in mainstream business: They are on the scene early to establish brand recognition, work hard and stick with what they know.

"We're businessmen," he said. "It's actually a simple and clean job."

At least the online pornography purveyors deliver what they promise, however unseemly. CP Direct, the penis-pill merchants, sold snake oil to consumers by the thousands, the Arizona attorney general, Jeanne Napolitano, said in her announcement of the civil forfeiture action.

"The product was purchased by CP Direct for $2.45 a bottle, and sold for $59.95 — that's a nice markup, isn't it?" said Lawrence J. Warfield, a certified public accountant who was appointed by the state to administer the sale of assets and distribution of money to victims of the swindle.

John Hannah, a lawyer for Michael Consoli, one of the principal figures in the CP Direct case, noted that the state seizure was not a final disposition of the case. "I have every reason to believe that Mr. Consoli will deny that he did anything wrong and will ask for his property back," he said.

CP Direct is one of many companies that — at least, until law enforcement showed up — proved that the wages of sin can, in fact, be bountiful. Last November, the Federal Trade Commission and the New York attorney general settled charges against a New York company, the Crescent Publishing Group, after proving that Crescent and 64 affiliated companies billed thousands of consumers as much as $90 a month for access to pornography sites that gave the appearance of being free. In a settlement with the company, the government got $30 million for consumer refunds.

Pyramid schemes, too, flourish online: In August 2001, the Federal Trade Commission fined the creators of BigSmart.com $5 million in a settlement over the business practices of its online shopping mall. BigSmart's customers paid $110 for a home page with links to other merchants, and the only way to make money was to recruit newcomers to the scheme, the commission found.

The reasons that people engage in such activity are simple enough, said Eric Wagner, a lawyer in the division of marketing practices at the Federal Trade Commission. "Fraud is profitable," he said. "If it wasn't profitable, people wouldn't be doing it."

The relatively low cost of using the Internet for marketing makes it every bit as attractive to swindlers as it is to legitimate businesses.

The commission has declared war on spam and fraud, and has trained people from 1,700 federal, state and local law enforcement agencies to detect and prosecute online fraud. The F.T.C. has collected 13 million spam messages in its efforts to study the problem, and has initiated dozens of "Netforce" sweeps to drive spammers out of business.

From Unseemly to Lowbrow, the Web's Real Money Is in the Gutter

(Page 2 of 2)

To be sure, the better neighborhoods of the Net — where one can find learned discussion of Kierkegaard or analysis of Gram Parsons's influence on rock music — are flourishing. But critics like Mr. Sterling argue that those leafy digital neighborhoods are increasingly surrounded by wildly expanding zones of slums, bad taste and risk.

He thinks "civil society" has broken down online, and that "it's not just a digital problem, it's the digital reflection of a global problem."
Advertisement

Criticism of the Internet has its antecedents in the fears of every new technology that has come along, said Paul Saffo, an analyst at the Institute for the Future in Menlo Park, Calif. Some of the earliest printing presses were put to use publishing pulp and trash, and even early printed playing cards had salacious images. "A collective appetite for sleaze," he said, "seems to be a real constant."

Despite that, or perhaps because of it, reports of the death spiral for the online medium are premature, said Gary Chapman, director of the 21st Century Project, a nonprofit research and education program on science and technology policy at the University of Texas.

"I am astonished practically every day by something new on the Internet," he said. The medium has grown so extensively that "at this point, it's almost impossible to characterize what the Internet is like."

Spam, for example, "is an annoyance and something that is a regrettable display of the human tendency to go for the lowest common denominator," he said. "But it can't possibly be viewed as representative of the entire Internet."

Brewster Kahle, who has created a large Internet archive he calls the Wayback Machine, which contains several times the amount of information in the Library of Congress, said that the number of questionable sites is beside the point so long as search engines do their job.

"We don't worry about how many pages that I don't care about are in the Internet archive," he said. "What you do care about is, `Does it have the pages that I want?' "

He acknowledged, however, that "we haven't done a very good job of putting the good stuff up there" on the Internet to dilute the bad, and that as a result today's leaders "are shortchanging the next generation."

Other initiatives to dilute the bad by raising the concentration of the good have also begun. Project Gutenberg, an arduous effort conducted largely by volunteers, has put more than 400 books online. The Million Books Project at Carnegie Mellon University is trying to create an online home in every field for major works that are no longer protected by copyright.

And the Digital Promise Project, begun by a former Federal Communications Commission chairman, Newton N. Minow (who famously called 1961-era television a "vast wasteland") and Lawrence Grossman, the former chairman of NBC and PBS, aims to improve education and the Internet by earmarking some $18 billion raised in federal auctions of rights to use portions of the broadcast spectrum.

"There's so much good stuff that is trapped in libraries that should be digitized and made available," Mr. Minow said. "It's just yearning to get free."

To Mr. Kahle, the Internet's diversity, good and bad, means that people will find the information they want, as narrowly and as deeply as they care to explore it. While he, too, would like some method to control the spam that flows into his e-mailbox, he said he preferred a complex ecosystem to a monoculture as bland and regular as a suburban lawn.

"I grew up where almost everybody could sing all three verses of the `Gilligan's Island' theme song," he said. "I don't want my children to grow up like that."