The 4Sight project: using VR in the drug discovery space
C4X Discovery, an AIM-listed company founded as a spin-out from the University of Manchester, is developing and combining cutting-edge immersive technologies to create best-in-class drug candidates, with the aim of becoming a self-sustainable drug discovery engine company.
The company’s 4Sight project pioneers the creation of a specialised visualiser to allow its research scientists to view, understand and interrogate the complex, multidimensional molecular shape data of drug molecules. Having measured these shapes using the company’s proprietary Conformetrix technology, this “4D molecular data” can then be visualised and manipulated to inspire the design of drug molecules in new and innovative ways. Working from both a desktop environment and within a VR space, the visualiser also facilitates simultaneous collaboration with multiple stakeholders across various sites.
“Scientists are constantly working with very complex data but the tools previously didn’t exist for 4D visualisation so it is an ongoing challenge to realise these molecules. By bringing them alive in a virtual 3D environment, brain space is freed up to focus on other issues.”
— Dave Sage, Software and Technology Development Strategist, C4X
4Sight will also act as a hub to visualise drug molecules using the results from experimental data sources including nuclear magnetic resonance (NMR), molecular dynamics, and other 3D molecular data sources. It will become the central tool for conformational drug design within the company, enabling designers to make fast progress in developing new and more effective drugs for the medical industry’s needs.
Work on the project started with a feasibility study in mid 2016 and initial tests took place using Google Cardboard and an android phone. Research into games engines demonstrated a synergy between the scientists’ requirements and the gaming technology available, and the team investigated the suitability of Epic Games’ Unreal Engine for the real-time rendering of their 4D molecular data. Plus, its VR capability offered users the opportunity to actually step inside those drug molecules. A games developer was employed to build the 4Sight system and the first major iteration of the application has just been launched internally. 4Sight has now replaced C4X’s existing visualiser Envision.
Why choose Unreal Engine?
Built for the high-end gaming industry, Unreal Engine was used as the technological foundation for 4Sight, chosen to provide a cutting-edge render. This has provided the base framework to create a visualiser, while its C++ code foundation gives the flexibility to build the science side of the project. The team has partnered with Epic’s enterprise division, which provides support and streamed updates to the engine, allowing their render to stay current while they focus on creating its bespoke features.
“The gaming industry has been a pioneer in the burgeoning VR and AR space and Epic is at the forefront,” says Dave. “Using their engine allows us to stay at the cutting edge of technology and quickly implement new features as they are released to the market, priming us to be able to take advantage of improvements in the VR, AR and mixed reality development. VR is important to us now to really see into the complexities of our 4D data but we see AR becoming more important for usability within a normal working environment.”
Using Oculus Rift and HTC Vive provides the users with a fully immersive room environment. This gives the scientists the same tactile feeling that is experienced using traditional physical ball and stick models, but with the benefits of virtual overlays bringing rich information into play. The system has the added advantage of being able to quickly display graphs and query databases, and its predictive engine automates many of the scientists’ tasks.
“4Sight allows users to see the results of an experiment and a view of drug molecules in ways that have never before been possible”, as C4X’s Chief Scientific Officer Craig Fox explains. For example, working with the Lisinopril molecule in a 2D view, below …
… is a very different experience to working with a 4Sight render of the data.
“Also, in the near future, rather than watching the results of a molecular dynamic simulation of a drug and its biological target on a screen, scientists could step right inside the protein pocket and see the simulation take place around them,” says Craig. “They could then compare the results with different molecules to understand how to optimise them further.”
Working with an agile development system, the design philosophy is user-driven to allow the project to quickly adapt to changes in requirements. Saving time is a crucial part of the project, success for the scientists is the speed at which they can design better drugs more quickly.
“You can tell that this system is leveraging the best of both game technology and cutting-edge science”, explains Dave. “It has the necessary scientific features and workflows but they are implemented with all the techniques that the games industry takes for granted in terms of usability and playability. Taking complicated chemistry data and vast amounts of information into this visualisation space has been an interesting journey for our teams, and now we are all seeing the benefits of what is possible when you bring the two disciplines together and form this synergy.”
Coping with the challenges
Being pioneers in this space brings its own challenges of course, as Dave confirms. “This is the first time that game engine technology has been applied to this sector in this way. And merging gaming knowledge with science brought its own particular issues – not least of which was finding the right people to work on the project.”
This involved enticing specialists from the games industry, which can be a difficult task. Few enterprise companies are currently using gaming technology, so the process of recruitment was challenging, as was trying to get recruitment agencies to understand and communicate the cutting edge nature of what C4X is doing, says Dave.
However, the programmer recruited to lead the software development project, Phil Muwanga, has been inspired by the possibilities of merging games technology with traditional science. “What excites me is making an environment that scientists can inhabit, creating a system that allows them to do their jobs more efficiently as well as pushing the boundaries of the technology and stimulating additional insights from C4X’s drug shape information.”
Working with Unreal Engine technology since his time as an undergraduate, he says there were a few hurdles to overcome in preparing the games engine for use in the enterprise space. “We used the engine’s code as a framework and it does handle a lot of the heavy lifting. Plus, I have full access to the Unreal developers for support. They are keen to develop this as an enterprise tool, they can see its potential in this space.”
And a lot of the gamers’ requirements automatically transfer to this new audience of scientific specialists. “Playing a game involves doing a series of complex tasks over and over again,” says Phil. “Scientists have the same experience. So, the process needs to be enjoyable for the users to get the most benefit.”
Other challenges the team encountered were mainly around visualisation issues, interface design and maintaining scientific accuracy. The technical challenge was to create a custom visualiser that could cope with the 4D nature of C4X’s dynamic 3D-shape data, while retaining clarity and an intuitive user interface. Traditional visualisation tools provide a static snapshot of a molecule that loses many of its dynamic properties and this has been overcome by the development of 4Sight.
When working with complex data and “4D ensembles”, multiple 3D representations of molecules, users need to be able to easily access the required data. Efficient UX design was required to improve user experience and productivity. Creating VR and multiplayer functionality with seamless interaction created challenges and here the games industry has also provided the working model. Games’ design practices and methodologies were used to bring a new perspective to the more traditional design structures of existing scientific software.
Where do they go from here?
With the first version launched, 4Sight will continue to grow with new features pulling in results from selected development streams in the company, building on user feedback. Its VR and collaborative capabilities will be enhanced to allow scientists to continue to explore the results of their experiments, visualising at the atomic level and improving their ability to use conformational design in their drug discovery work.
Speaking of the impact of their work on the sector to date, Craig Fox reveals that other players in this space are currently looking at Artificial Intelligence as a way of identifying and optimising drug molecules beyond conventional methods, and AI is gaining a lot of momentum in the sector. C4X acquired several AI-based drug discovery technologies from Molplex Ltd last year to combine with its Conformetrix platform, and considers the combination of AI and human guided drug design to be optimal.
The process of introducing new and ground-breaking ways of working into a traditional industry needs to be carefully managed, explains Dave Sage. “It’s important to know when is the right time to take people into VR, when to show them the prototype.”
And in this world, where interaction between 2D and 3D elements is key, the gaming industry is again providing a useful template reveals Phil. “When working with different interfaces, it’s important to be able to move easily between the two and this has to be intuitive. You can’t frustrate your audience. The gaming industry learned this quickly, because otherwise audiences wouldn’t buy their products. By the end of a game, the players are doing incredibly complex actions and understanding how they build up to this point is important. How do you teach them to move around the technology, how can they reach a molecule that is in a different location for example?”
He is considering using other gaming devices to promote interaction and provide support for users, such as the game “leaderboards” that show top system users who can be accessed to provide information for others. It’s all about getting people to work in a different way he believes.
What is the future for VR in the drug discovery space?
Often people think of VR as only a visual experience but sound is equally as important. The 4sight project team is already considering molecular sonification as an additional way of imparting the data to users to further differentiate molecules. In the VR world, the user can identify an errant sound in space, locate where it is coming from and move focus to it for further inspection to speed up the design process further. The team is planning to collaborate with academics to develop this field of interest.
Phil is also very interested in the possibility of using colour to encode data into multiple dimensions to see correlations in VR. “That is why I was willing to be pulled out of the games industry to work in this industry. I’m interested in the application of ground-breaking technology in a totally different sector to the one it was developed for, using it in a way it has never been used before and with an end goal to therapeutically help people in future through the drugs that are designed using this application.”
Words by Bernadette Fallon