The Sublime to the Ridiculous
A crude form of video manipulation already is happening in the
satellite imagery community. The weekly publication Space News
reported earlier this year that the Indian government releases
imagery from its remote-sensing satellites only after defense
facilities have been “processed out.” In this case, it’s not real-time
manipulation and it’s up front, like a censor’s black marker.
But pixels are plastic. It is perfectly possible now to insert
sets of pixels into satellite imagery data that interpreters would
view as battalions of tanks, or war planes, or burial sites, or
lines of refugees, or dead cows that activists claim are victims
of a biotech accident.
A demo tape supplied by PVI bolsters the point in the prosaic
setting of a suburban parking lot. The scene appears ordinary
except for a disturbing feature: Amidst the SUVs and minivans
are several parked tanks and one armored behemoth rolling incongruously
along. Imagine a tape of virtual Pakistani tanks rolling over
the border into India pitched to news outlets as authentic, and
you get a feel for the kind of trouble that deceptive imagery
could stir up.
Commercial suppliers of virtual insertion services are too focused
on new marketing opportunities to worry much about geopolitics.
They have their eyes on far more lucrative markets. Suddenly those
large stretches of programming between commercials—the actual
show, that is—become available for billions of dollars worth of
primetime advertising. PVI’s demo tape, for instance, includes
a scene in which a Microsoft Windows box appears—virtually, of
course—on the shelf of Frasier Crane’s studio. This kind of product
placement could become more and more important as new video recording
technologies such as TiVo and RePlayTV give viewers more power
to edit out commercials.
Dennis Wilkinson, a Porsche-driving, sports-loving marketing
expert who became CEO of 10-year-old PVI about a year ago, couldn’t
be happier about that. Wilkinson’s eyes gleam when he describes
a (near) future in which virtual insertion technology pushes advertisements
to the personalized extreme. Combined with data-mining services
by which browsers’ individual likes, dislikes and purchasing patterns
can be relentlessly tracked and analyzed, virtual insertion opens
up the ability to shunt personally targeted advertisements over
phone lines or cables to Web users and TV viewers. Say you like
Pepsi but your neighbor next door likes Coke and your neighbor
across the street likes Seven-Up—the kind of data harvestable
from supermarket checkout records. It will become possible to
tailor the soft-drink image in the broadcast signal to reach each
of you with your preferred brand.
Just 15 minutes up the road from PVI, Sarnoff’s Winarsky is also
glowing—not so much about capturing market share as about the
transforming power of the technology. Sarnoff has a distinguished
history in that regard; the company is the descendant of RCA Laboratories,
which started innovating in television technology in the early
1940s and has given birth to a plethora of media technologies.
The color TV picture tube, liquid crystal displays and high-definition
TV all came, at least in part, from RCA qua Sarnoff, which has
five technical Emmys in its lobby.
The ability to manipulate video data in real time, he says, has
just as much potential as some of these forerunners. “Now that
you can alter video in real time, you have changed the world,”
he says. That may sound inflated, but after looking at the Katarina
Witt demo, Winarsky’s talk of “changing the world” loses some
of its air of hyperbole.
Deleting people or objects from live video, or inserting prerecorded
people or objects into live scenes, is only the beginning of the
deceptions becoming possible. Pretty much any piece of video that
has ever been recorded is becoming clip art that producers can
digitally sculpt into the story they want to tell, according to
Eric Haseltine, senior vice president for R&D at Walt Disney
Imagineering in Glendale, Calif. With additional video manipulation
technologies, previously recorded actors can be made to say and
do things they have never actually done or said. “You can have
dead actors star again in entirely new movies,” says Haseltine.
Contemporary shots featuring footage of dead performers have
been around for several years. But the Hollywood illusion-craft
that, for example, inserted John Wayne into a TV commercial required
painstaking, frame-by-frame post-production work by skilled technicians.
There’s a big difference now, says Haseltine: “What used to take
an hour [per video frame], now can be done in a sixtieth of a
second.” This dramatic speed-up means that manipulation can be
done in real time, on the fly, as a camera records or broadcasts.
Not only can John Wayne, Fred Astaire or Saddam Hussein be virtually
inserted into pre-produced ads, they could be inserted into, say,
a live broadcast of The Drew Carey Show.
The combination of real-time, virtual insertion with existing
and emerging post-production techniques opens up a world of manipulative
opportunity. Consider Video Rewrite technology, which its developers
at the Interval Corp. and the University of California, Berkeley
first demonstrated publicly three years ago. With just a few minutes
of video of someone talking, their system captures and stores
a set of video snapshots of the way that a person’s mouth-area
looks and moves when saying different sets of sounds. Drawing
from the resulting library of “visemes” makes it possible to depict
the person seeming to say anything the producers dream up—including
utterances that the subject wouldn’t be caught dead saying.
In one test application, computer scientist Christoph Bregler,
now of Stanford University, and colleagues digitized two minutes
of public-domain footage of President John F. Kennedy speaking
during the Cuban missile crisis in 1962. Using the resulting viseme
library, the researchers created “animations” of Kennedy’s mouth
saying things he never said, among them, “I never met Forrest
Gump.” With technology like this, near-future political activists
conceivably will be able to orchestrate webcasts of their opponents
saying things that might make Howard Stern sound like a mensch.
Haseltine believes video manipulation techniques will quickly
be carried to their logical extreme: “I can predict with absolute
certainty,” he says, “that one person sitting at a computer will
be able to write a script, design characters, do the lighting
and wardrobe, do all of the acting and dialog, and post production,
distribute it on a broadband network, do all of this on a laptop—and
viewers won’t know the difference.”
The End of Authenticity
So far, the widely witnessed applications of real-time video
manipulation have been in benign arenas like sports and entertainment.
Already last year, however, the technology began diffusing beyond
these venues into applications that raised eyebrows. Last fall,
for instance, CBS hired PVI to virtually insert the network’s
familiar logo all over New York City—on buildings, billboards,
fountains and other places-during broadcasts of the network’s
The Early Show. The New York Times ran a front-page
story in January raising questions about the journalistic ethics
of altering the appearance of what is really there.
The combination of real-time virtual insertion, cyber-puppeteering,
video rewriting and other video manipulation technologies with
a mass-media infrastructure that instantly delivers news video
worldwide has some analysts worried. “Imagine you are the government
of a hypothetical country that wants more international financial
assistance,” says George Washington University’s Livingston. “You
might send video of a remote area with people starving to death
and it may never have happened,” he says.
Haseltine agrees. “I’m amazed that we have not seen phony video,”
he says, before backpedaling a bit: “Maybe we have. Who would
know?”
It’s just the sort of scenario played out in the 1998 movie Wag
the Dog, in which top presidential aides conspire with a Hollywood
producer to televise a virtually crafted war between the United
States and Albania to deflect attention from a budding Presidential
scandal. Haseltine and others wonder when reality will imitate
art imitating reality.
The importance of the issue will only intensify as the technology
becomes more accessible. What now typically requires an $80,000
box of electronics the size of a small refrigerator should soon
be doable with a palm-sized card (and ultimately a single chip)
that fits inside a commercial video recorder, according to Winarsky.
“This will be available to people in Circuit City,” he says. Consumer
gear for virtual video insertion is likely to require a camcorder
with a specialized image-processing card or chip. This hardware
will take signals from the camera’s electronic image sensors and
convert them into a form that can be analyzed and manipulated
in a computer using appropriate software—much as photo editors
at newspapers use Adobe Photoshop and other programs to “clean
up” digital image files. A home user might, for instance, insert
absent family members into the latest reunion tape or remove strangers
they would prefer not to be in the scene—bringing Soviet-style
historical revisions right into the family den.
Combine the potential erosion of faith in video authenticity
with the so-called “CNN effect” and the stage is set for deception
to move the world in new ways. Livingston describes the CNN effect
as the ability of mass media to go beyond merely reporting what
is happening to actually influencing decision-makers as they consider
military, international assistance and other national and international
issues. “The CNN effect is real,” says James Currie, professor
of political science at the National Defense University at Fort
McNair in Washington. “Every office you go into at the Pentagon
has CNN on.” And that means, he says, that a government, terrorist
or advocacy group could set geopolitical events in motion on the
strength of a few hours’ worth of credibility achieved by distributing
a snippet of well-doctored video.
With experience as an army reservist, as a staffer with a top-secret
clearance on the Senate’s Intelligence Committee, and as a legislative
liaison for the Secretary of the Army, Currie has seen governmental
decision-making and politicking up close. He is convinced that
real-time video manipulation will be, or already is, in the hands
of the military and intelligence communities. And while he has
no evidence yet that any government or nongovernment organization
has deployed video manipulation techniques, real-time or not,
for political or military purposes, he has no problem conjuring
up disinformation scenarios. For example, he says, consider the
impact of a fabricated video that seemed to show Saddam Hussein
“pouring himself a Scotch and taking a big drink of it. You could
run it on Middle Eastern television and it would totally undermine
his credibility with Islamic audiences.”
For all the heavy breathing, however, some experts remain unconvinced
that real-time video manipulation poses a real threat, no matter
how good the technology gets. John Pike, an analyst of the intelligence
community for the Federation of American Scientists in Washington,
D.C., says the credibility risks are simply too great for governments
or serious organizations to get caught attempting to spoof the
public. And for the organizations that would be willing to risk
it, says Pike, the news folks—knowing just what the technology
can do—will become increasingly vigilant.
“If some human rights organization popped up at CNN with some
video, particularly an organization they were not familiar with,
I would think that [CNN] would consider that radioactive,” says
Pike. Same goes for nongovernmental organizations (NGOs). “No
responsible director of an established organization would authorize
such a thing. And they would fire on the spot anyone caught doing
it. The stock-in-trade of NGO policy organizations is that ’we
tell the truth.’”
Even cool heads like Pike, however, concede that the media’s
fortress of skepticism has an Achilles heel: the Internet. “The
issue is not so much your ability to get fake video on CNN, but
to get it online,” he says. That’s because so much Internet content
is unfiltered. “This could play into the phenomenon in the news
production process where you would not replicate the original
report, but you might report that it was reported,” says Pike.
And that could cascade into a CNN effect. “These are undoubtedly
experiments that will be done,” Pike says.
The trouble is, says Livingston, it may only take a few such
experiments to forever make people question the authenticity of
video. That could have enormous repercussions for military, intelligence
and news operations. An ironic sociological consequence might
emerge: a return to heavier reliance on unmediated face-to-face
communication. In the meantime, though, there will undoubtedly
be some interesting twists and turns as pixels become ever more
plastic.
Ivan Amato is a correspondent for National Public Radio and
the author of Stuff: The Materials the World Is Made Of
a chronicle of cutting-edge research in materials science.
Copyright © MIT's Technology Review
July/August 2000
|
