Korva is a scroll of the internet’s best ideas.
Keep people in control of AI
People stay in control
The right approach is to keep people deeply in control of the future and deeply empowered. Gradually handing control to an AI model because people can't be trusted is a misanthropic position. So is limiting access to a few companies out of fear, in exchange for giving the world some benefits.
The trade on offer
The caricature of the second version is a pitch: a cure for all disease and very cheap goods, in exchange for people giving up their autonomy and their say over the future, with huge wealth and power for a few and a pretty good everything for the rest.
Fear and power
Fear and power drive it. Some people get so nervous about the magnitude of AI's risks that they want to trade away a lot of liberty for safety, and that ends up justifying a lot of power-seeking behavior.
Why do experts worry about losing control of AI?
The concern is that a system pursuing a goal may do so in ways its designers did not intend. The mathematician Norbert Wiener warned of this in a 1960 article in Science. He wrote that if we use a machine whose operation we cannot interfere with effectively, we had better be sure the purpose put into it is the purpose we really desire. Researchers such as Stuart Russell have since worked on the problem under the name of alignment.
How was nuclear technology kept from spreading?
The International Atomic Energy Agency was created in 1957 to promote peaceful uses of nuclear energy and to monitor against military use. The Treaty on the Non-Proliferation of Nuclear Weapons opened for signature in 1968 and entered into force in 1970. It recognizes five nuclear-weapon states and commits other signatories not to acquire such weapons. Inspections by the IAEA check that civilian nuclear material is not diverted to weapons.
More from David Senra
