Charlie Stross on slow AIs  

Posted by Big Gav in , ,

Charlie Stross did an interesting talk to the Chaos Computer Club about the first generation of AIs that is wrecking the plane - Dude, you broke the future!.

History gives us the perspective to see what went wrong in the past, and to look for patterns, and check whether those patterns apply to the present and near future. And looking in particular at the history of the past 200-400 years—the age of increasingly rapid change—one glaringly obvious deviation from the norm of the preceding three thousand centuries—is the development of Artificial Intelligence, which happened no earlier than 1553 and no later than 1844. I'm talking about the very old, very slow AIs we call corporations, of course. What lessons from the history of the company can we draw that tell us about the likely behaviour of the type of artificial intelligence we are all interested in today? ...

What do our current, actually-existing AI overlords want?

Elon Musk—who I believe you have all heard of—has an obsessive fear of one particular hazard of artificial intelligence—which he conceives of as being a piece of software that functions like a brain-in-a-box)—namely, the paperclip maximizer. A paperclip maximizer is a term of art for a goal-seeking AI that has a single priority, for example maximizing the number of paperclips in the universe. The paperclip maximizer is able to improve itself in pursuit of that goal but has no ability to vary its goal, so it will ultimately attempt to convert all the metallic elements in the solar system into paperclips, even if this is obviously detrimental to the wellbeing of the humans who designed it.

Unfortunately, Musk isn't paying enough attention. Consider his own companies. Tesla is a battery maximizer—an electric car is a battery with wheels and seats. SpaceX is an orbital payload maximizer, driving down the cost of space launches in order to encourage more sales for the service it provides. Solar City is a photovoltaic panel maximizer. And so on. All three of Musk's very own slow AIs are based on an architecture that is designed to maximize return on shareholder investment, even if by doing so they cook the planet the shareholders have to live on. ...

The problem with corporations is that despite their overt goals—whether they make electric vehicles or beer or sell life insurance policies—they are all subject to instrumental convergence insofar as they all have a common implicit paperclip-maximizer goal: to generate revenue. If they don't make money, they are eaten by a bigger predator or they go bust. Making money is an instrumental goal—it's as vital to them as breathing is for us mammals, and without pursuing it they will fail to achieve their final goal, whatever it may be. Corporations generally pursue their instrumental goals—notably maximizing revenue—as a side-effect of the pursuit of their overt goal. But sometimes they try instead to manipulate the regulatory environment they operate in, to ensure that money flows towards them regardless.

Human tool-making culture has become increasingly complicated over time. New technologies always come with an implicit political agenda that seeks to extend its use, governments react by legislating to control the technologies, and sometimes we end up with industries indulging in legal duels.

For example, consider the automobile. You can't have mass automobile transport without gas stations and fuel distribution pipelines. These in turn require access to whoever owns the land the oil is extracted from—and before you know it, you end up with a permanent occupation force in Iraq and a client dictatorship in Saudi Arabia. Closer to home, automobiles imply jaywalking laws and drink-driving laws. They affect town planning regulations and encourage suburban sprawl, the construction of human infrastructure on the scale required by automobiles, not pedestrians. This in turn is bad for competing transport technologies like buses or trams (which work best in cities with a high population density).

To get these laws in place, providing an environment conducive to doing business, corporations spend money on political lobbyists—and, when they can get away with it, on bribes. Bribery need not be blatant, of course. For example, the reforms of the British railway network in the 1960s dismembered many branch services and coincided with a surge in road building and automobile sales. These reforms were orchestrated by Transport Minister Ernest Marples, who was purely a politician. However, Marples accumulated a considerable personal fortune during this time by owning shares in a motorway construction corporation. (So, no conflict of interest there!)

The automobile industry in isolation isn't a pure paperclip maximizer. But if you look at it in conjunction with the fossil fuel industries, the road-construction industry, the accident insurance industry, and so on, you begin to see the outline of a paperclip maximizing ecosystem that invades far-flung lands and grinds up and kills around one and a quarter million people per year—that's the global death toll from automobile accidents according to the world health organization: it rivals the first world war on an ongoing basis—as side-effects of its drive to sell you a new car.


Post a Comment


Locations of visitors to this page

blogspot visitor
Stat Counter

Total Pageviews




Blog Archive


australia (618) global warming (423) solar power (397) peak oil (353) renewable energy (302) electric vehicles (249) wind power (194) ocean energy (165) csp (159) solar thermal power (145) geothermal energy (144) energy storage (142) smart grids (140) oil (138) solar pv (138) tidal power (137) coal seam gas (131) nuclear power (129) china (118) lng (116) iraq (113) geothermal power (112) green buildings (111) natural gas (110) agriculture (92) oil price (80) biofuel (78) wave power (73) smart meters (72) coal (69) uk (69) electricity grid (67) energy efficiency (64) google (58) bicycle (51) internet (51) surveillance (50) big brother (49) shale gas (49) food prices (48) tesla (46) thin film solar (42) biomimicry (40) canada (40) scotland (38) ocean power (37) politics (37) shale oil (37) new zealand (35) air transport (34) algae (34) water (34) arctic ice (33) concentrating solar power (33) queensland (32) saudi arabia (32) california (31) credit crunch (31) bioplastic (30) offshore wind power (30) population (30) cogeneration (28) geoengineering (28) batteries (26) drought (26) resource wars (26) woodside (26) bruce sterling (25) censorship (25) cleantech (25) ctl (23) limits to growth (23) carbon tax (22) economics (22) exxon (22) lithium (22) buckminster fuller (21) distributed manufacturing (21) iraq oil law (21) coal to liquids (20) indonesia (20) origin energy (20) brightsource (19) rail transport (19) ultracapacitor (19) santos (18) ausra (17) collapse (17) electric bikes (17) michael klare (17) atlantis (16) cellulosic ethanol (16) iceland (16) lithium ion batteries (16) mapping (16) ucg (16) bees (15) concentrating solar thermal power (15) ethanol (15) geodynamics (15) psychology (15) al gore (14) brazil (14) bucky fuller (14) carbon emissions (14) fertiliser (14) ambient energy (13) biodiesel (13) cities (13) investment (13) kenya (13) matthew simmons (13) public transport (13) big oil (12) biochar (12) chile (12) desertec (12) internet of things (12) otec (12) texas (12) victoria (12) antarctica (11) cradle to cradle (11) energy policy (11) hybrid car (11) terra preta (11) tinfoil (11) toyota (11) amory lovins (10) fabber (10) gazprom (10) goldman sachs (10) gtl (10) severn estuary (10) volt (10) afghanistan (9) alaska (9) biomass (9) carbon trading (9) distributed generation (9) esolar (9) four day week (9) fuel cells (9) jeremy leggett (9) methane hydrates (9) pge (9) sweden (9) arrow energy (8) bolivia (8) eroei (8) fish (8) floating offshore wind power (8) guerilla gardening (8) linc energy (8) methane (8) nanosolar (8) natural gas pipelines (8) pentland firth (8) relocalisation (8) saul griffith (8) stirling engine (8) us elections (8) western australia (8) airborne wind turbines (7) bloom energy (7) boeing (7) chp (7) climategate (7) copenhagen (7) scenario planning (7) vinod khosla (7) apocaphilia (6) ceramic fuel cells (6) cigs (6) futurism (6) jatropha (6) local currencies (6) nigeria (6) ocean acidification (6) somalia (6) t boone pickens (6) space based solar power (5) varanus island (5) garbage (4) global energy grid (4) kevin kelly (4) low temperature geothermal power (4) oled (4) tim flannery (4) v2g (4) club of rome (3) norman borlaug (2) peak oil portfolio (1)