Jason Shun Wong

Committee of Infrastructure Part 1

Committee of Infrastructure proposes that AI systems can represent different types of physical infrastructure. Three settings are proposed: a bridge, traffic light, and a light rail system. Giving personality to these infrastructures is an attempt to build a system that interacts with people at a human level. Instead of a series of data points, the bridge can describe problems that are communicated in the same manner humans talk. The bridge jokes and argues facilitating a communication between itself, operators, and engineers.

As AI systems become super optimized with more data and advances in neural net algorithms they become extremely sensitive to areas that it has been calibrated for (e.g. corrosion level of a steel support system). This type of finely tuned optimization can create an excessive and irrational anxiety for every slight disturbance an AI detects, these systems will be able to self-diagnose every plausible problem it encounters. Because these AI systems are constantly checking and outputting data they are in effect systems of constant rumination. Human intervention is needed to address the concerns of the system; for example, a bridge engineer must act if a bridge experiences too much stress during a storm. In order to match the personality of the AI systems, the output of these systems has learned how to communicate from the films of Woody Allen. Woody Allen films have long explored psychoanalysis, displaying emotional rumination and self-diagnosis through a comedic tone. By matching the personality of the AI systems and characters in Woody Allen films we can see fallibility of technologies over optimization.

Year
2017

Discipline & Research Area
Design Research, Speculative Design, Civics, Industrial IoT, Artificial Intelligence, and Machine Learning

Responsibilities
Research, Concept Development, World-building, and Data Collection

Ubiquitous Urban Sensing

Internet of things (IoT) have become ubiquitous objects in the home. Devices such as the Nest thermostat and the Philips Hue bulbs have become increasingly common. Not only are they limited to households, but major corporations (GE, Cisco, AT&T, etc.) are also creating Industrial IoT devices that are created for the urban environment.  A single IoT such as a sensor that monitors noise can be trained to listen to different noises (ambulance, car crash, gunshots, etc.) and categorize these sounds in real time¹. Hardware neural nets are being built into these sensors so that the sensors are performing machine learning tasks right at the source and diagnosing the situation. No longer is the data captured being sent to the cloud; it is processed locally, creating autonomous IoT’s that act on the information that they collect. 

Research Questions

In order to inform the direction of the project, a few questions guided my work throughout the process.

• When does over optimization become non-functional?
• What kind of personality does an AI take when it interacts with humans?
• Self-diagnosis of edge devices (sensors with hardware neural nets) is already occurring, what happens when AI systems are too good at self-diagnosis?
• Are the machine learning neural nets “hallucinating” patterns?

AI Personality

The vast quantities of data produced by the sensors needed to be distilled into a sensible and meaningful format for humans to act on. Some estimates foresee that an autonomous car will generate 25 gigabytes of data every hour². Therefore, there is a need to process and communicate concise, timely, and meaningful information.

Committee of Infrastructure proposes that an AI can adopt the human communication system with a distinct personality in order to facilitate interactions between engineers or system operators who needed to intervene and act. In order to match the personality of the AI systems the output of the AI systems I trained the communication patterns through the films of Woody Allen. Woody Allen films have long explored psychoanalysis, displaying emotional rumination and self-diagnosis through a comedic tone. Specifically, I used Andrei Karpathy’s char-rnn model³ to create a Conversational User Interface (CUI) that notifies the engineer through speech.

Films / Source Data
I trained the char-rnn model on 10 Woody Allen films. 

• Annie Hall
• Bananas
• Crimes And Misdemeanors
• Hannah And Her Sisters
• Love And Death
• Manhattan
• Manhattan Murder Mystery
• The Purple Rose Of Cairo
• Sleeper
• Zelig

Prime Text Examples
After training the neural network, I gave a piece of prime text that would start the conversation from which the neural network would create a response. 

• Notify traffic engineers to reroute all traffic routes.
• There is a car accident involving a grey 2016 Tesla Model 3 and a blue 2017 Toyota Prius.
• The 2017 Toyota Prius D34567 is speeding.
• Passenger still trying to board the train after the 3rd attempt.

Prime Text to Output
Click here to see a repository of responses.

Scenario
Bridge,
Traffic Light,
and Light Rail

I chose three scenarios (bridge, traffic light, and light rail car) to demonstrate how this concept could work. In the bridge scenario, a tinyTile sensor⁴ acts as a vibration and noise sensor affixed to a steel beam on the bridge. By detecting different types of sounds it communicates the status of the bridge to engineers and operators.

The bridge detects a car crash between two vehicles. It uses the vibration, noise, and camera vision sensor to give information about the accident. In addition to giving detailed information about the accident, it ruminates with extended talking mimicking the characters in Woody Allen films. Human intervention is needed to act on the information accordingly. In this case, ambulances, police, and bridge safety operators can act to resolve the car crash.

At the AAAI Symposium

I had the privilege of presenting Committee of Infrastructure at the AAAI symposium in Palo Alto for the UX of AI series in March 2018.

Reflection & Development

Committee of Infrastructure proposes different interactions between AI systems and humans that aren't based on efficiency and optimization. It demonstrates the importance of humor and personality as valuable assets to communicate. The project also proposes how a bridge could communicate in languages relevant to the different members of the system. For example, a paramedic, engineer, and car crash victim, all need information relevant to their situations and needs. The bridge could change its personality to engage with these different stakeholders. 

I believe that the project is only a starting point for alternative means of communication with AI. As a final thought, a few questions to consider:

• Can psychological baggage give more context to inform the conversation between an AI and a human? 
• What is the hierarchy of representation for AI entities?
• Can the way AI’s communicate or talk reveal the health of the system they are monitoring? 
• If there are over 100 sensors, does one AI system represent the interests of the collective? 
• What is the negotiation process like between humans and AIs? 
• How is AI embodied? 

The work completed in Committee of Infrastructure Part 1 has served as a foundation for Driverless Government.