Jttrgen Suhnel.Virtual Reality Modeling for Structural Biology Institut

 

New Trends in Bio-Nanorobotics using Virtual Reality Technologies

Antoine Ferreira ¹⁾, Gaurav Sharma²⁾, Constantinos Mavroidis ²⁾

¹⁾ Laboratoire Vision et Robotique, University of Orleans, 18000, Bourges, France.

²⁾ Bio-Nano Robotics Laboratory, Northeastern University, Boston

[e-mail], [e-mail]

Abstract—The scientific community has been working in the field of VR for some years now, having recognized it as a very powerful human computer interface. Although many of the described technologies have been developed into more or less mature products for robots acting in the macro-world, the nano-size of the robots poses extreme challenges and requires a complete rethinking of the visual and haptic perception of nano-world. Using simulated nanoenvironments in VR, the operator can design and characterize through physical simulation the behavior of nano and bionanorobots. Adding haptic interaction, the operator can explore and prevent the problems of nano and bionano robots in their native environment. This article describes some of the emerging applications of VR currently underway in the field of bio-nanorobotics.

I. Introduction

Recent advances in the field of nanotechnology have enabled widespread opportunities to investigate and manipulate matter on the nanoscale. This exciting young field is bound to shed new light on the world around us, as well as provide new opportunities for the design and development of devices exhibiting unprecedented capabilities. Eventually, these devices will be evolved into robotic machines that are able to autonomously perform complex tasks at the nanoscale. The development of bio-nanorobotic systems and their components present difficult fabrication and control challenges. Such devices will operate in biological environments whose physical properties differ from those encountered by conventional parts. The idea is to use molecular motors as the actuators of such nanorobots [1-2], where the structural elements are carbon nanotubes, while the joints are formed by appropriately designed protein and/or DNA elements [3-4]. Development of a complete database of different biomolecular machine components, the ability to interface or assemble different machine components and the development of accurate models are some of the challenges to be faced in a near future. The application of advanced virtual reality techniques (VR) in bio-nanotechnology allowed to study, simulate, visualize and interact with molecular biology, genomics, proteomics, structural and computational biology. As far we know, only basic molecular elements have been studied until now. As a perspective of these developments, VR-based technology could be used for the biomolecular robot design. VR refers to computer-generated, interactive, 3-D environments into which people are immersed. It provides a way for people to visualize, manipulate, and interact with simulated environments through the use of computers and extremely complex data. Emerging VR applications such as

manipulation of molecules for development of molecular machines will enhance the operator's perception (vision and haptics) by approaching more or less a state of 'full immersion' or 'telepresence'. They will enable the designer of molecular machines to extend his / her eyes and hands into the molecular-world and also enable new types of exploration and whole new classes of experiments in the biological and physical sciences. Although many of the described technologies have been developed into more or less mature products for robots acting in the macro-world, the nano-size of the molecular robots poses extreme challenges and requires a complete rethinking of the visual and haptic perception of nanoworld. Using simulated biological nanoenviroments in VR, the operator can design and characterize through physical simulation the behavior of bio-nanorobots. Adding haptic interaction, the operator can explore and prevent the problems of bio-nanorobots in their native environment. These immersive VR techniques allows the 3D visualization of scientific data and for modeling and animating complex nanoengineering systems. This article review some of the emerging applications of VR currently underway in the field of bio-nanotechnology with envisioned applications to molecular robotics.

II. VR-Based Perception at the Bio-Nano Scale

1) VR for Structural Biology

Structural information on biological macromolecules is an essential requirement for our understanding of biological functions. From the very beginning of structural biology, visualization was essential for determining and understanding structures. The initial work in this area was the use of Visualization and Virtual Reality for Bio-informatics, especially for the 3D structural analysis of bio­molecular system. Surface and volume based visualization provide three-dimensional concepts of bio-molecular structure. Virtual Reality offers a channel to reach-into the molecular space in an immersive and interactive environment. Progress in the X-ray and nuclear magnetic resonance (NMR) instrumentation and computer and software technology has led to an increasing rate of accumulation of new biological structures such as proteins and DNA. The Protein Data Bank [5] has a database of coordinate entries of 21,248 proteins (Fig.1). Proteins are complex biological structures made up of several hundred atoms. The usual approach to visualize such a molecule is to retrieve the coordinate files from the database and then use one of the molecular graphics software packages. These

0-7803-9315-5/05/$20.00 ©2005 IEEE

packages largely provide mono representations of molecular images without any interactivity. Use of Virtual Reality Modeling Language (VRML) can provide a 3D and immersive environment with provision for stereo representations of the molecules [6]. This not only helps in better understanding the molecular structure but also provides insight into the chemical and biochemical properties of the molecule such as number of hydrogen bonds.

2) Molecular Structure Modelling for Design and Analysis

The bio-systems are made up of atoms and since atoms cannot be observed directly, we can see them only in our imaginations. Commonly available physical models provide an intuitive representation of structural molecular biology. When applied to large molecules, such as proteins and the molecular assemblies found in cells, computer graphics simulation can be used to accurately portray various molecular computational models, with their varying complexities. Electrostatic field data around the molecules of interest, interatomic forces, laws of quantum mechanics, and so on represents the variety of bio-molecular nano-interactions [7]. However, these methods

Fig.1: Secondary structure representation of the Bacillus circulans xylanase (PDB code - 1bcx) complexed with sulfate and cyclic xylose in the VRML viewer WebSpace (helix - tube, sheet - ribbon). (Courtesy of: Dr. Jttrgen Suhnel, Institut fur Molekulare Biotechnologie, Germany).

lack the beneficial tactile and kinaesthetic attributes of real physical models. By coupling an accurate molecular dynamics simulation code to an immersive virtual reality display with interactive capabilities and manual force feedback, "immersive" visualization of molecular atoms could be improved [8]. In order to create multimodality enhancements of such tangible models, VR-based technology is an interesting tool. By superimposing additional graphical information on top of fabricated models, e.g. augmented reality, by incorporating support for voice commands and by providing haptic interface for sensing the electrostatic charges and interatomic collisions, the user would be able to interact with these virtual enhancements haptically while manipulating the physical model.

3) Unification Problem of Enhanced Vision and Force Display for Automatic Bionanomanipulation

In order to precisely control and manipulate biomolecules, we need tools that can interact with these objects at the nano scale in their native environments [9,10].

Existing bio-nanomanipulation techniques can be classified as non-contact manipulation including laser trapping [11,12] and electro-rotation [13], and contact manipulation referred to as mechanical stylus-, AFM- or STM-based nanomanipulation [14]. The rapid expansion of AFM studies in biology/ biotechnology results from the fact that AFM techniques offer several unique advantages: first, they require little sample preparation, with native biomolecules usually being imaged directly; second, they can provide a three-dimensional reconstruction of the sample surface in real space at ultra-high resolution; third, they are less destructive than other techniques (e.g. electron microscopy) commonly employed in biology; and fourth, they can operate in several environments, including air, liquid, and vacuum. Rather than drying the sample, one can image quite successfully with AFM in fluid. The operation of AFM in aqueous solution offers an unprecedented opportunity for imaging biological molecules and cells in their

Fig.2:Visual tracking of tip pattern (on the left) and real-time construction of the virtual biologic environment (on the right) with a continuously updated object model. (Courtesy of: Dr. Toshio Fukuda, University of Nagoya, Japan)

physiological environments and for studying biologically important dynamic processes in real time [15]. Currently, these bio-nanomanipulations are conducted manually; however, long training, disappointingly low success rates from poor reproducibility in manual operations, and contamination call for the elimination of direct human involvement. Furthermore, there are many sources of spatial uncertainty in AFM manipulation, e.g., tip effects, thermal drift, slow creeping motion, and hysteresis. To improve the bio-nanomanipulation techniques, automatic manipulation must be addressed. Visual tracking of patterns from multiple views is a promising approach, which is currently investigated in autonomous embryo pronuclei DNA injection [16]. Interactive nanomanipulation can be improved by imaging 3D viewpoint in a virtual environment. Many of these systems have been implemented in research prototypes and demonstrations that may lead to useful applications of nanoassembly are beginning to appear. The nanoManipulator (www.nanomanipulator.com), developed by a group at University of North Carolina, Chapel Hill, was the first being commercialized. Different commercial products are starting to be available to study nanoscale science such as the nanomanipulator NanoFeel^TM300 (www.nanofeel.com), software products such as NanoMove manipulation software commercially available from Veeco Instruments Inc. (www.veeco.com) or 3-D displays of AFM data from Asylum Research Inc. (www.asylumresearch.com). Construction of a VR space in an offline operation mode for trajectory

planning combined with a real time operation mode for vision tracking of environmental change ensures a complete 'immersed' visual display. Fig.2 shows an example of 3D bio-micro/nano manipulation system with a 3D VR model of the environment including the bio cell, and carrying out the user viewpoint change in the virtual space [17]. Since its discovery by Binnig, the AFM has been the main contact mode tool and is used to manipulate macromolecular biological assemblies as well as individual bio-molecules [18]. However, contact mode can also be used where higher force is desired such as in cutting or dissection. As application, elastic properties of DNA and identification of the different force-extension regimes in DNA are currently controlled with a haptic interface. However, contact manipulation can damage specimens due to uncontrolled forces at the contact interface between the tip and the surface. Introduction of physically-based models of contact nanoforces (friction, rugosity, elasticity of cell membrane), liquid manipulation forces (viscosity, density) and continuum models (electrostatic, adhesive) into 3D virtual objects will generate realistic virtual n-environments for guided simulations. Exemplary works made at Michigan State university [19,20] and Carnegie Mellon university [21] introduces 3-D models of cantilever-tip interaction with the nano-environment in order to provide real-time force feedback to the operator through Augmented Reality interfaces. In [22], force models of tip-substrate-object interactions are integrated in an augmented reality environment to provide 3-D visualization and realism of surface nano-deformations. In [23], a bio-membrane mechanical model has been developed for vision-based bio-membrane force sensing, assimilating vision and force sensing modalities. It affords the development of real-time 3D virtual simulator with force display and controlled tip/surface interactions [24].

4) Molecular Dynamics Simulations in Virtual Environment. Molecular dynamics (MD) is a computer simulation technique where the time evolution of a set of interacting atoms is followed by integrating their equations of motion. In MD we follow the laws of classical mechanics, and most notably Newton's law:

F. = т.л.

i

(1)

for each atom i in a system constituted by N atoms. Here, m_i is the atom mass, a_i =d²ri dt² its acceleration, and F_i

the force acting upon it, due to the interactions with other atoms. MD simulations are based on the calculation of the free energy that is released during the transition from native to fusogenic state and compute the atomic trajectories and coordinates by numerically solving the equations of motions using an empirical force field that mimics the actual inter­atomic forces in the molecular system. MD simulations of complex molecular systems require enormous computational power and produces large amount of data in each step. The resulting data includes number of atoms per unit volume, atomic positions, velocity of each atom, force applied on

each atom and the energy contents. These results of MD simulations need to be visualized to give the user a more intuitive feel of what is happening. Haptic interaction used in conjunction with VR visualization helps the scientist to control/monitor the simulation progress and to get feedback from the simulation process as well [25]. Fig.3 shows the virtual reality visualization of molecular dynamics simulation.

Fig.3:  A block diagram showing virtual environment for Molecular dynamics simulations (Reprinted from: Z. Ai, T. Frohlich).

5) Computational steering and visualization of complex molecular systems.

Computational steering is the ability of the user to design or modify a simulation interactively in a virtual environment during run-time, which gives a tremendous advantage over post-simulation visualization and analysis of results. In run­time steering the user does not have to wait till the end of the simulation to see the results of his modifications, instead he can immediately see the result of the interactively changed parameters giving him an opportunity to detect and modify them to steer the simulation to a desired output. In computational steering the user can steer molecular dynamics simulation by applying external forces into the computations. These external forces can help a complex molecular system overcome a potential energy barrier and can even steer the system to a new geometric conformation for further analysis. Thus it provides a great advantage over Targeted Molecular Dynamics [26] which also targets a molecular dynamics simulation to a desired output but in which the user has no control over the simulations once it has started. Another advantage of Steered Molecular Dynamics over conventional molecular dynamics is the possibility of inducing relatively large conformational changes in molecules on nanosecond time scales [27]. Computational steering of molecular dynamics simulation with the aid of virtual reality visualization can thus help molecular scientists and researchers to explore new models, their structural behaviour and also to study the elastic/mechanical behaviour of bio-nanotechnology relevant complex molecular systems.

III. Current Experimental VR Systems for

BIONANOTECHNOLOGY

The Virtual Molecular studio at Imperial College London [28] and the Technische Hochschule Darmstadt [30] have

done some pioneering research in this field. Also researchers at the Institut fur Molekulare Biotechnologie, Germany have set up an internet-based Image Library of Biological Macromolecules [31]. This was one of the first VRML applications in biology. The VRML division of this library contains more than 650 VRML representations of biopolymer structures. One of the challenges in computational structural biology is to enable the efficient use and inter-operation of a diverse set of techniques to simulate, analyze, model, and visualize the complex architecture and interactions of macromolecular systems. Scientists at The Scripps Research Institute, California in collaboration with the Central Institute for Applied Mathematics, Germany are developing a 3D graphics package termed SenSitus [32] that is capable of supporting VR devices such as stereo glasses, 3D trackers, and force-feedback (haptic) devices. This development effort will permit scientists to build models, combine atomic and volumetric data, and perform morphing and warping (flexible docking) interactively within a single computational environment. A force feedback device measures a user's hand position and exerts a precisely controlled force on the hand. The software supports this by calculating forces according to the correlation coefficient of density maps and crystallographic data. The high sampling frequency required for force feedback (refresh rate >1kHz) is achieved by means of the vector quantization algorithm developed by the group that reduces the complexity of the data representation to manageable levels. In addition to automated fitting, microscopists have a need to evaluate and manipulate docking models interactively "by eye". 3D capabilities and the "physics of touch" offer tangible benefits for modelers who wish to explore a variety of docking situations in a VR environment. In the Visualization Studio of the Delaware Biotechnology Institute at the University of Delaware (Fig.4), research is focused on molecular biology, genomics, proteomics, structural and computational biology, and biomedical imaging. The Institute's interactive, immersive Visualization Studio consists of a SGI Reality Center system linked to a 15x7-foot FakeSpace [33] rear-projection screen. A six-processor Onyx 3200 visualization supercomputer with two graphics pipes drives a pair of Mirage 2000 projectors to deliver an edge- blended image with a total resolution of approximately 2560x1024 pixels. The Studio is currently being outfitted with a haptic feedback system to further improve the immersive interface with the visualized environment. The Institute of Technical Biochemistry at the University of Stuttgart (Fig. 5) has an ongoing project regarding the application of virtual reality technology in the area of molecular modeling [34]. The Highly Immersive Molecular Modeling (HIMM) [35] project is aimed on the integration of computer aided molecular modeling tools and virtual reality systems like CO VISE [36]. Recently, however, another way of visualizing atoms has become available. Researchers in the NAS data analysis group have developed an application called Virtual Mechanosynthesis or VMS [37]. The user of this application sees various collections of atoms floating in the space above

 

Fig.4: Interacting with a room-sized display of a three-dimensional protein in CAVE. (Courtesy of: Dr. Karl V. Steiner, Delaware Biotechnology Institute, University of Delaware).

the NAS Visualization Laboratory's Immersive Workbench, made by Fakespace Inc. The VMS utility allows the user to see, move, and even "feel" simulated molecular structures in 3-D.

Fig.5: Geometrical representation model for the molecule data. (Courtesy of: Dr. Juergen Pleiss, HIMM project, University of Stuttgart, Germany)

A haptic, or force-feedback device can also be used in VMS to interact with the simulation. Once users "attach" to an atom using the haptic interface, they can feel the attractive and repulsive forces as this atom pushes and pulls on its neighbors. Another kind of VR interface for molecular dynamics simulation has been developed at the Fraunhofer Institute for Computer Graphics, Germany. This system, called RealMol [38] is implemented to run on CAVE or any other computer system with head mounted display or a stereo projection screen.The MD simulation program for RealMol is NAMD and the communication between the two to exchange molecular data is achieved by RAPP [39] developed at the National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign. The IDEAL Interaction DEvice Abstraction Layer system [40] is used to interface RealMol with the CAVE rendering system. IDEAL is independent of the choice of rendering system, provides an easy to use interface and handles devices for interaction in a virtual environment. With the use of IDEAL a cyberglove can be displayed in the virtual environment as a mapping of hand. (Fig.6). Modeling proteins as biological motors as well as designing proteins not found in nature is a rapidly developing field. Keeping this in mind researchers at the University of North Carolina at Chapel Hill have built SMD [41], a system for interactively steering molecular dynamics calculations by adding user-specified external forces into the computation on the fly. Steering implies that the user is able to 'tug' an atom or a group of atoms towards a desired target position. SMD

(a)                                           (b)

Fig.6:Molecules can be grasped with a cyberglove and moved to a desired position. (b) 3-D menu can be activated and selected by a gesture of the cyberglove. (Reprinted from: Z. Ai, T. Frohlich,)

consists of two software components. First one is the software component for performing the molecular dynamics simulations on a molecular system and to study its response to an externally applied steering force.The display component of SMD is VMD (Fig.7).

(a)                        (b)                           (c)

Fig.7: (a) Specifying a "tug": selected atom is highlighted as the red sphere at upper right. (b) Moving the tug: target position follows pointer. (c) System response after 300 fs of simulation. (Courtesy of: Jon Leech, Jan F. Prins, Jan Hermans, University of North Carolina at Chapel Hill)

An example from the University of Washington is given in Fig.8, where the superimposition of multi-modal interaction can greatly complement and accelerate the learning process of the operator. For initial demonstration a superoxide dismutase (SOD), an essential enzyme for cellular functioning which exhibits a strong electrostatic funneling effect is modeled. In this scenario the user holds the superoxide radical with the haptic device probe and, as it nears the charge field of SOD, strong forces pull the superoxide free radical toward the Cu and Zn ions at the active site of SOD. At the same time the user sees the secondary structure of the SOD enzyme as an augmented reality overlay on top of the physical model. They have also developed a Virtual Reality Peripheral Network (VRPN) [42]. The VRPN is a set of classes within a library and a set of servers that are designed to implement a network-transparent interface between application programs and the set of physical devices (tracker, etc.) used in a VR system. The idea is to have a PC or other host at each VR station that controls the peripherals (tracker, button device, haptic device, analog inputs, sound, etc). VRPN provides connections between the application and all of the devices using the appropriate class-of-service for each type of device sharing this link. STALK [43] is a VR based system developed at the University of Illinois at Urbana Champagne and Argonne National Lab for studying the docking of a

ligand molecule into a protein binding site. STALK, shown in Fig.9, uses a parallel genetic algorithm library to search for a low-energy conformation. An interface to the CAVE virtual reality system allows a scientist to (1) visualize the genetic algorithm's progress and, (2) interact with the algorithm by, for example, changing the position or structure of the drug molecule and restarting the algorithm so that the new parameters are incorporated [44].

(a)                                        (b)

Fig.8:Display of force vector field around active site of superoxide dismutase (SOD) and (b) user interacting with SOD models using HMD and PHANToM. (Courtesy of: Ganesh Shankarnarayan, Human Interface Technology Lab, University of Washington)

Fig.9:A user in STALK (Courtesy: MCS Division, Argonne National Laboratory).

The Virtual Biomolecular Environment (VIBE) [45] is a VR based system proposed at Argonne National Lab to provide an environment for Drug Design. It consists of (1) a massively parallel computing to simulate the physical and chemical properties of a molecular system, (2) the Cave Automatic Virtual Environment (CAVE) for immersive display and interaction with the molecular system, and (3) a high-speed network interface to exchange data between the simulation and the CAVE. VIBE enables molecular scientists to have a visual, auditory, and haptic experience with a chemical system, while simultaneously manipulating its physical properties by steering, in real-time, a simulation executed on a supercomputer.

IV. Conclusion

The review presented in this paper has focused on the applications and utility of VR in computational biology and bionanotechnology where it is starting to be applied, in some cases with spectacular results. Although, there are many projects out there that deal with molecular simulation using VRML and desktop-based systems, even with PHANToM or

some other haptic device, very few of them can be classified as truly Virtual Reality design environments since they lack some of the most important aspects of such environments -full immersion and a highly evolved interaction method-. It opens new fields of research on macro-micro-nano interactivity and multimodality perception such as haptic, visual, kinesthetic, gesture, motion and intuitive interfaces.

References

[I] T.Yamamoto, O.Kurosawa, H.Kabata, N. Shimamoto, M. Washizu,
Molecular surgery of DNA Based on Electrostatic micromanipulation, IEEE
Trans. on Industry Applications, Vol.36, No.4, pp.1010-1017 (2000).

[2] S.Thalhammaer, R. Stark, S.Muller, J.Winberg, W.M., Heckl, Atomic

Force Microscope as a New Microdissecting Tool for the Generation of

Genetic Probes, Jour. of Structural Biology, 119, pp.232-237 (1997).

[3] T.Funatsu, Y.Harada, H.Higuchi, et al., "Imaging and nano-manipulation

of single biomolecules", Biophysical Chemistry, vol.68, pp.63-77 (1997).

[4] K.Svoboda, C.Schmidt, B.Schnapp, S.Block, Direct Observation of

Kinesin Stepping by Optical Trapping Interferometry, Nature, vol.365,

pp.721-727 (1993).

[5] H.M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T.N. Bhat, H.

Weissig, I.N. Shindyalov, P.E. Bourne The Protein Data Bank, Nucleic

Acids Research, 28 pp. 235-242 (2000).

fur Molekulare Biotechnologie, Postfach 100813, D-07708 Jena / Germany.

[7] Ouh-Yong, G.H., Pique M., Hughes J., Srinivasan N., Brooks F.P., Using

a Manipulator for Force Display in Molecular Docking, IEEE Robotics and

Automation Conf., Philadelphia, April 1988, pp.1824-1829.

[8] G. Sankaranarayanan, S. Weghorst, M. Sanner, A.Gillet, A.Olson, Role

of Haptics in Teaching Structural Molecular Biology, 11th Symposium on

Haptic Interfaces for Virtual Environment and Teleoperator Systems, March

22 - 23, Los Angeles, California pp.365 (2003).

[9] K. Castelino,Biological Object Nanomanipulation, Review Report,

University of California, Berkeley (2002).

[10] T.Fukuda, F.Arai, L.Dong,Nano Robotic World —From Micro to Nano,

Int'l Workshop on Nano- \ Micro-Robotics in Thailand (2002).

[II] A.Ashkin, Acceleration and Trapping of Particles byRadiation
Pressure, Physical Review Letters, Vol.24, No.4, pp.156-156 (1970).

[12] T.N. Bruican, M.J.Smyth, H.A.Crissman, G.C.Salzman, C.C.Stewart,

J.C.Martin, Automated Single-Cell Manipulation and sorting by Light

Trapping, Applied Optics, Vol.26, No.24, pp.5311-5316 (1987)

[13] M.Nishioka, S.katsura, K.Hirano, A.Mizuno, Evaluation of Cell

Characteristics by Step-Wise Orientational Rotation using Optoelectrostatic

Micromanipulation, IEEE Transactions on Industry Applications, Vol.33,

No.5, pp.1381-1388 (1997).

[14] A. A .G. Riquicha, et al.,Manipulation of Nanoscale Components with

the AFM: Principles and Applications, IEEE International Conference on

Nanotechnology, Maui, HI, October 28-30, 2001.

[15] Y. Sun, K. T. Wan, K.P. Roberts, J.C. Bischof, and B.J. Nelson,

Mechanical Property Characterization of the Mouse Zona Pellucida, IEEE

Trans. on NanoBioScience, Vol. 2, No. 4, Dec. 2003

[16] F.Arai, T.Fukuda, 3D Bio Micromanipulation , International Workshop

on Microfactoryies IWMF'98, December 7-9, pp.143-148 (1998)

[17] Y.Ishii, A.Ishijima, T. Yanagida, Single Molecule Nanomanipulation of

Biomolecules, Trends in Biotechnology, Vol.19, No.6 (2001)

[18] D.Fotiadis, S.Scheuring, S.Muller, and et al., Imaging and manipulation

of biological structures with the AFM, Micron, vol.33, pp.385-397 (2002).

[19] G.Li, N.Xi, M.Yu,W-K.Fung,"Development of Augmented Reality

System for AFM-based Nanomanipulation", IEEE/ASME Transactions on

Mechatronics, Vol.9, N°.2, June 2004, pp.358-365.

[20] G.Li, N.Xi, H.Chen, A.Saeed, M.Yu, "Assembbly of Nano structure

using AFM based Nanomanipulation System", IEEE Int. Conferrence on

Robotics and Automation, New Orleans, April 2004, pp.428-433.

[21] M. Sitti, S. Horiguchi, H. Hashimoto, Nano Tele-Manipulation using

Virtual Reality Interface, Proc of IEEE Int. Symp. On Industrial Electronics,

pp 171-176, South Africa, 1998.

[22] W.Vogl, M.Sitti, M.F.Zah,“Nanomanipulation with 3D Visual and Force Feedback using Atomic Force Microscopes", IEEE Int. Conferrenceon Nanotechnology, Munich, Germany, August 2004.

[23] Y. Sun, K.-T. Wan, K.P. Roberts, J.C. Bischof, and B.J. Nelson, Mechanical Property Characterization of the Mouse Zona Pellucida, IEEETrans. on NanoBioScience, Vol. 2, No. 4, Dec. 2003.

[24] M. Ammi, A. Ferreira, ‘^’Realistic Visual and Haptic Rendering for Biological-Cell Injection^’’IEEE International Conference on Robotics and Automation (ICRA'05), April 18-22, Barcelona (Spain), pp.930-935, 2005. [25] Disz, M. Papka, M. Pellegrino, R. Stevens, andV. Taylor, Virtual reality visualization of parallel molecular dynamics simulation, in Proc. 1995Simulation Multi-conference Symp., (Phoenix, Arizona), pp. 483-487, Society for Computer Simulation, (April 1995).

[26] Schlitter J.,Targeted Molecular Dynamics: A New Approach for Searching Pathways of Conformational Transitions, Journal of MolecularGraphics, 12, 84-89, 1994.

[27] Sergei Izrailev, Sergey Stepaniants, Barry Isralewitz, Dorina Kosztin, Hui Lu, Ferenc Molnar, Willy Wriggers, and Klaus Schulten, Steered molecular dynamics. In P. Deuflhard, J. Hermans, B. Leimkuhler, A. E. Mark, S. Reich, R. D. Skeel, ed.,Computational Molecular Dynamics: Challenges, Methods, Ideas, volume 4 of Lecture Notes in Computational Science and Engineering, pages 39-65. Springer-Verlag, Berlin, 1998. [28] Casher, O., Leach, C., Page, С S., Rzepa. H. S. Advanced VRML based chemistry applications: A 3D molecular hyperglossary. J. Mol. Struct.(THEOCHEM), 368, 49, 1996.

[29] O. Casher, С Leach, С S. Page, H. S. Rzepa, Virtual Reality Modelling Language in Chemistry, Chemistry in Britain, 34, 26, 1998. [30] Casher, O., Rzepa, H.  S.A chemical collaboratory using explorer eyechem and the common client interface. Comp. Graphics, 29, 52, 1995.[31] Willy Wriggers and Stefan Birmanns. Using Situs for Flexible and Rigid-Body Fitting of Multi-Resolution Single Molecule Data. J. Struct.Biol., Vol. 133, pp. 193-202, 2001.

[32] ImmersaDesk, http://www.fakespace.com/workdesk1.shtml, © 2003 Fakespace Systems Inc.

[33] H. Haase, J. Strassner, and F. Dai, VR techniques for the investigation of molecule data, Computers & Graphics, Special Issue on Virtual Reality, 20(2), pp. 207-217 (1996). Elsevier Science Ltd.

[34] R. С Drees, J. Pleiss, D. Roller, and R. D. Schmid, Highly Immersive Molecular Modeling (HIMM): an architecture for the integration of molecular modeling and virtual reality, in Computer Science and Biology,Proceedings of the German Conference on Bioinformatics (GCB'96), (Leipzig, Germany), pp. 190-192, Sep-Oct 1996.

[35]Collaborative Visualization and Simulation Environment (COVISE), http://www.hlrs.de/organization/vis/covise/ , last updated Mar 30, 2001 [36] C. Levit, S.T. Bryson, C.E. Henze, Virtual Mechanosynthesis, Fifth Foresight Conference on Molecular Nanotechnology, California, Nov 1997. [37] J.E. Stone, J. Gullingsrud, K. Schulten, P. Grayson, A System for Interactive Molecular Dynamics Simulation, In2001 ACM Symposium on Interactive 3D Graphics, pp 191-194, New York, 2001, ACM SIGGRAPH. [38] W.F. Humphrey, A. Dalke, and K. Schulten. VMD - Visual Molecular Dynamics. J. Mol. Graphics, 14: pp 33-38, 1996.

[39] M. Roth, T. Frohlich,IDEAL Interaction DEvice Interaction Layer User's Manual,. Fraunhofer-IGD, internal report, 1997. [40] J. Leech, J.F. Prins, J. Hermans, SMD: Visual Steering of Molecular Dynamics for Protein Design, IEEE Com. Sci. and Eng., pp. 38-45, 1996. [41] Russell M. Taylor II, Thomas C. Hudson, Adam Seeger, Hans Weber, Jeffrey Juliano, Aron T. Helser,VRPN: A Device-Independent, Network-Transparent VR Peripheral System, In ACM VRST 2001 Nov. 2001.[42] D. Levine, M. Facello, P. Hallstrom, G. Reeder, B. Walenz, and F. Stevens. STALK: An Interactive Virtual Molecular Docking System, In IEEE Computational Science & Engineering, 1996.

[43] С Cruz-Neira, R. Langley, P.A.Bash, VIBE: a Virtual Biomolecular environment for interactive molecular modeling, Computers Chem, 20(4), p. 469, 1996.

[44] K. Arthur, T. Preston, R. M. Taylor II, F. P. Brooks, Jr., M. С Whitton, and W. V. Wright, Designing and Building the PIT: a Head-Tracked Stereo Workspace for Two Users,2nd International Immersive Projection Technology Workshop, Iowa, May 11-12, 1998.

[45] Jan F. Prins, Jan Hermans, Geoffrey Mann, Lars S. Nyland, and Martin Simons, A Virtual Environment for Steered Molecular Dynamics, Future Generation of Computer Systems, Vol. 15, No 4, Jul. 1999, pp. 485-495.