There was the wish to create a convincing human-like interaction with a machine to personify Gerkzeuk. The (pseudo) holographic display via “Pepper’s ghost effect” is one aspect of that, one crystal is also equipped with camera, microphone, speaker and micro computer.
My thoughts about simulating aspects of human appearance and interaction while exploring artificial intelligence since 2015 had their biggest concrete intensity realizing this part of the project.
Head of a virtual humanoid as persona of Gerkzeuk, as seen on the exhibition ALLEZ ALLEZ.
On the software side of things i experimented with:
OpenFace – Face detection. Deep-Learning based method for the computer to recognice faces, differentiate them, associate names and remember them.
Affdex – Sentient detection. Interprets facial expressions based on 7 base emotions.
SpeechRecognition – Speech recognition. Experiments with grammatic analysis and keyword detection using “Natural Language Toolkit”.
ChatterBot – Machine-Learning based dialogue system enabling the computer to repeat the recipients choice of words in a logical manner.
Watir Webdriver – Automatically use a web browser
Unity – Game Engine, in which the modules came together
Further designs and work process:
pfeffergeist_17_01