Serendipity35

There were some alerts this past summer that made it sound like an artificial intelligence (AI) system being developed at Facebook was taking over the world. At least that is what some anti-AI folks seemed to be saying. 

The story seemed to be that the AI had started its own conversation between two AI agents developed inside Facebook. They were speaking to each other in plain English. The revelation to the researchers was that because of a mistake in programming the AI had created its own language. It developed a system of code words to make communication more efficient. The researchers shut the system down when they realized it was no longer using English - and they didn't understand what the two agents were saying.

The "singularity" (at least the tech one, not the mathematical or gravitational versions) is the point hypothesized when an upgradeable artificial intelligence will enter a "runaway reaction" of self-improvement cycles. It improves itself to the point of being a superintelligence that surpasses human intelligence. It's when the machines are smarter than us. John von Neumann first used the term "singularity" back in the 1950s i talking about technological progress causing accelerating change.

Why is Facebook messing around with this? For one thing, they want to build chatbots that can have conversations and negotiate with humans in a way that mimics human responses so that they can then make decisions on their own.

Does that scare you?

Facebook was trying to get the chatbots working with a "partner" to divide up several objects that had different numerical points value. That requires negotiation to work out the best way to divide the objects and accumulate the highest possible number of points.

The event is not the first example of AI diverging from its training in English to develop its own language. The new language is nonsense to humans but has semantic meaning when interpreted by AI agents.

A chatbot (like the ones shown conversing above) repeating "to me" five times might mean to run a routine five times. It's shorthand. A + B = C is the kind of unsophisticated math we can easily understand, but to the computer the “A” could mean thousands of line of code and that is when we are lost.

It's not that the Facebook chatbots gave up on using English in order to hide from the human observers, it was just more efficient to use another language.

The scary factor is that when Bob the chatbot says "I can can I I everything else” and chatbot Alice replies “Balls have zero to me to me to me to me to me to me to me to me to” we really don't know what they are saying. 

At the OpenAI artificial intelligence lab founded by Elon Musk, they experimented with letting AI bots learn their own languages and it worked. This strikes fear in the hearts of many people, but there's not enough evidence to determine whether AI presents a real threat that could enable machines to overrule their operators.

The team that works on Google Translate believes that the AI created the most efficient solution to some problems.

The singularity is not here yet, but it is coming.