Projects 2013-1995

  Multimedia Library and Museum Platform 2012 - 2014

Multimedia Library and Exhibit Platform
 Even venerable cultural institutions such as libraries and museums have to and want to keep up with the times. Their treasures gathered over centuries are being digitized, their most of their services and publications are digital anyway nowadays. We are used to search for books or to inform ourselves about an exhibition via the internet. The usual procedure is refining lists generated by search engines or "scanning" web pages in search of interesting bits.

What if the visit of a digital library or a digital exhibition would be more like talking to a librarian or a museum guide? Actually an enlightening experience inspiring us with new ideas?

In this research project,  we are trying to achieve exactly this experience. Users interact with mobile-friendly, highly adaptive and responsive interfaces. Behind the scenes, a cloud-based service continuously analyzes search keywords, interaction behavior and the digital databases. It asks meaningful questions, presents relevant media and helps with navigation not only for finding what you are looking for, but suggests media and information you did not know they where interesting.

Partners in this project are: Universitäts- und Landesbibliothek Darmstadt, Städel Museum Frankfurt, Software AG, mtG, nterra, House of IT.

The project is funded by grants by the LOEWE program of the state of Hesse, Germany (LOEWE projects HA-Projekt- Nr. 320/12-10, 321/12-11, 322/12-12).


Virtual Planetarium 2008-2011


The Virtual Planetarium is an interactive installation in our 3D-Lab at h_da. Controlled by iPhone or iPad it allows visitors to travel through our solar system visiting planets and moons. Our 3D projection renders the experience stereoscopic and immersive. The project was developed as a student playground. I have provided the celestial mechanics, basic graphical rendering and camera control. The student contributions incorporate until now user interface, music, star background and real-time earth cloud coverage.

The project has been presented in public at our annual mediale* exhibition and at Ars Electronica festival 2010 in Linz, Austria as part of our exhibition "Expanded Interface".


Presenccia, 2006-2007


During my work in the Presenccia project at Universitat Politècnica de Catalunya (UPC) in Barcelona, our group constructed a test scenario for measuring the feeling of presence in a virtual environment. The experiment took place in UPC's CAVE placing the subject in a simulated Jazz bar surrounded by virtual musicans and guests. Suddenly a fire breaks out. After a while, the virtual characters notice the fire and start to panic and flee. During the experiment the subject's bodily reactions like galvanic skin response, respiration and heart rate are measured. The recorded data was later evaluated to find out, if the subjects reacted as if the scenario was real and to find out how to improve the scenario to provoke an even stronger reaction.


Fraport 2015, 2005


Fraport 2015 is a model of the Frankfurt Airport including the extensions planned until 2015. A physical model shows the airport, runways, train station and nearby Frankfurt motorway interchange. When observing the model though stereoscopic binoculars, visitors see all the dynamic features of the airport overlaid precisely to the physical model. Planes are all over in the air, starting and landing on the runways taxiing from and to the gates. Cloud layers are and flight data are visualized. The plane motions are a result of airport simulations performed by Fraport and rendered in real-time.


Space Debris, 2004-2005


Space Debris is a VR installation developed at Cybernarium by order of European Space Agency (ESA). It shows drastically the growing problem of space debris and explains possible solutions. Actual data of more than 9000 objects orbiting earth where provided by ESA. Using a large 3D screen as a display and a touch sensitive monitor as input device, visitors can navigate though the animated clouds of active and defective satellites, as well as mission related objects (rocket stages, lost tools, and other pieces of debris). Visitors can travel into the past and understand the developments beginning in 1957. Time travel is also possible to the future, where different simulation scenarios show how the situation will evolve until the year 2100, depending on measurements of mitigation.


Energized – The Interactive Flow of Electric Energy, 2004


This installation was the first exhibit we developed at Cybernarium and at the same time our first multi-user installation. It educates visitors how electric energy is produced, transformed and delivered to households and factories. Each of up to eight visitors plays the role of an element of the energy network, such as power station, transformer, factory or apartment building. Players are tracked by an optical tracking system. When players move, ‘their’ elements move simultaneously on a large screen in the front. Electric lines snap in when two matching elements are close and tear apart if they loose contact. The goal of the game is to cooperatively form a functioning network.


Cybernarium, 2002-2005


During the year 2001, while I was still working as a senior researcher at Frauhofer Institute for Computer Graphics, an idea emerged: So many interesting Virtual and Augmented Reality exhibits had been developed and successfully presented in the recent years. Why not exhibit them permanently and publicly? The idea became a serious project aiming to found a spin-off company, the Cybernarium. In order to boost the foundation by a unique event showing off the potential of VR/AR technology and proofing the applicability of the technology a first public exhibition was organized. In January 2002, Cybernarium Days presented 15 VR/AR exhibits at Darmstadt’s Centralstation; during 6 days more than 10000 citizens visited the exhibition. Although one had to stand in line up to four hours, a questionnaire showed that more than 92% of the visitors where satisfied. The Cybernarium Days created a tremendous press echo including all major German TV networks and helped the successful foundation of Cybernarium GmbH. I led the teams for software development and exhibition design as well as the exhibition organisation. The exhibition was repeated two times in 2003 with similar success. In 2004, the Cybernarium became a permanent institution in Darmstadt.


Augmented Man, 2000


'Augmented Man' is an artistic Augmented Reality installation realized on the occasion of 600th birthday of Johannes Gutenberg. It consists of a stereoscopic projection fed by a stereo camera, acting as a virtual mirror. Visitors see their reflection time-shifted by a few seconds. Moreover the 'mirror' shows an additional virtual person not present in the real auditorium, the Augmented Man. He approaches the visitors and asks questions of fundamental meaning answered simultaneously by citations of famous scientists speaking from the off. Augmented Man is controlled and rendered by software processing the images of the stereo camera and an additional camera mounted to the ceiling. By this means, augmented man knows where the visitors are standing and can be rendered correctly occluded by visitors closer to the mirror.


Virtual Cathedral of Siena, 1999-2000


The Virtual Cathedral of Siena has been developed as an exhibit for the World Exposition Expo 2000 in Hanover, Germany. The exhibit opens up new vistas of one of the most beautiful gothic cathedrals. Guided by an avatar, called Luigi, visitors roam through the interior of the cathedral and learn about famous artwork of the church. By touching a virtual book they select places of interest and ask Luigi questions about art, history and architecture of the current location. Luigi is rendered as a real-time animated human figure in traditional costume. He features a lively facial expression, lip-sync speech and holds eye contact.


IDEAL, 1998-2000


IDEAL (Interaction Device Abstraction Layer) is a client- server based system connecting VR/AR applications to multidimensional interaction devices such as tracking systems, data glove, spacemouse etc. It provides a hardware-independent object oriented API. Novel features are: Hot swap capabilities, optimal low-latency communication protocols, concurrent access by multiple applications and advanced calibration and testing procedures. Until today IDEAL is a core part of IGD’s VR system 'Virtual Design II' and in daily operational use in German automotive industry.


Virtual Oceanarium, 1996-1998


The Virtual Oceanarium is a 3D visiual simulation of Europe’s largest aquarium, the Oceánario in Lisbon, Portugal. I headed the research and development activities for this project, performed by a joined team between Portugal and Germany. The virtual aquarium we developed consisted of five fish tanks populated with more than 1000 fish with simulated behaviour plus much more static or pre-animated creatures. The Virtual Oceanarium has been presented to the global audience of the world exposition EXPO '98. My article in Communications of the ACM can be found here, a video by my Portuguese colleguages is available on YouTube.


Astronaut Training Simulator, 1995


The astronaut training simulator was developed during my internship at NASA’s Johnson Space Center (JSC). Two VR systems, one running in Houston, USA, one in Darmstadt at Fraunhofer-IGD, Germany where linked by an ISDN line for communication. The participants shared a common training environment depicting the Space Shuttle docked to the Hubble Space Telescope. Each astronaut was rendered by a space suit with a movable arm and hand. Two astronauts situated at JSC and IGD simultaneously controlled their avatars by HMD, data glove and navigated with a spacemouse. Together they trained the replacement of defective power supply by a spare part. The experiment has been successfully replayed on SIGGRAPH’s Digital Bayou in 1996. Our publication in German Science Magazine 'Spektrum der Wissenschaft' can be found here.