Immersive Education in Mining

Introduction

The key challenge addressed by this project is the limited opportunity for mining engineering students to interact with mining equipment in real-world settings. Real mine site visits are often limited due to logistical constraints, remote locations, and safety concerns.

This project creates an interactive VR learning environment where students can explore a virtual mineral processing facility, engage directly with machinery, and understand their roles within the processing workflow. By providing a safe and immersive alternative, the tool enhances both learning outcomes and understanding of the mineral processing systems.

Background

For mining engineering students, understanding how raw ores are extracted, crushed, separated, and refined requires exposure to large-scale industrial machinery such as crushers, mills, and flotation cells. This equipment is often located in remote and hazardous environments. Safety concerns and limited access to operational sites make it difficult for students to engage consistently with these systems during their studies.

Primary Features

Realistic mineral processing plant model: Developed a highly detailed virtual model of an open-pit mine and processing plant to give students a realistic sense of scale, environment, and operation flow. This immersive setting allows students to explore the entire mining lifecycle in a way traditional diagrams and textbooks cannot.
Topological model: Teleportation via location flags on topological model.
Machine interaction: Interact with machinery like SAG mill, ball mills, and cone crusher. Users can adjust the transparency of machinery to reveal internal components and observe how systems operate, such as Passthrough functions, Rubble Spawning, and Speed dynamics.
An interactive flowchart: Students can see the whole processing workflow and teleport to specific stages.
A quiz mode: Students can reinforce their understanding with location specific questions.

FUNCTIONALITIES

No.	Task	Priority	Status
F1	Interactable SAG Mill	Must have	Complete
F2	Interactable Ball Mill	Must have	Complete
F3	User can select activity with a topological map.	Must have	Complete
F4	User can learn info about different machines around map	Must have	Complete
F5	Interactable Cone Crusher	Nice to have	Complete
F6	User can be quizzed about the area	Nice to have	Complete
F7	User can teleport to different areas with a mini map/ flow chart	Nice to have	Complete
F8	User can observe a completely realistic ball mill rubble flow	Nice to have	Partially Complete
F9	User can enter a museum of all the machines	Nice to have	Incomplete
F10	Augmented Reality Topological Map	Nice to have	Complete
F11	Second Activity	Nice to have	Incomplete

Methods

Tech Stack and Development Overview

Technical Components


Tool	Description
Unreal Engine 5.5	Game engine used to develop the project
Perforce	Version control system used to manage and track changes
Quixel Bridge / FAB	Source for environment assets, realistic animal models and animation
Figma	Interface design tool for creating user interfaces and prototypes
Trello	Project management software used for project and team organization
Blender	Used for 3D modelling
Adobe Illustrator 2024	Vector graphics editor used to create scalable graphics and logos

Topological Map Model

The map is one of the tools that users use to transport to different sections of the mining site. Unlike the flowchart, which is rendered inside a widget component, this is a full model made for visualization purposes. It can be breakdown into two processes: model creation and system.
The models that were used were both generated from heightmaps via modelling mode and imported *.stl files. Since these can come into different shapes and sizes, we used a displacement shader that moves vertices that are outside an acceptance radius based on the object’s center, downwards to create a lateral surface and mask out ones that doesn’t contribute to it.

Map Selection System

The map is structured so users can magnify different sections of the site. They are greeted with a view of the whole mining site until they zoom closer and transport them to a specific area they chose.
Flags were created as an interactive zoom mechanic for this. Each time you click a flag, you go one level deeper inside the map. The map has two main events that represent this: Zoom and Open Level. This can be found in the topology map’s blueprint: Game/Blueprints/Gameplay/BP_TopoMap.
To represent this, we structured the system with a list of Topology Section Datasets F_TopoSectionData that acts like a tree. Each section data comprises the current section, the parent section, the topology mesh, and a list of child sections named SubFlags.
With this tree-like structure, the Zoom event acts like a tree traversal to the selected child node. Once the current selection is a leaf, or an Activity in this context, we use Open Level event instead of Zoom.

Mining Site Flowchart

To teleport around the map as well as to see where you are, we use a flowchart of the mineral processing plant, which can be accessed from the wrist menu. This flowchart is found at BP_Flowchart.

The flowchart highlights the user's current location with pulsing buttons that correspond to predefined activity areas. Each area is defined by a bounding box, and the system continuously checks which area the user is in to update the highlight in real time.
Flowchart buttons also serve as teleport triggers. When selected, they call a teleport function that moves the player to a predefined location, with checks to ensure the user is on the correct map and not already teleporting. A smooth camera fade is used to transition the payer to the new area.

Onboarding Tutorial:

The tutorial system, implemented in BP_TutorialSystem with the Level Selection Map, is activated when the user interacts with the floating question mark widget.
This is mainly a system that processes two types of components: Widgets and Tutorial States. Each state would be comprised with a set of widgets that the user would have to interact with to proceed to the next state or finish the whole tutorial.
States are defined in E_TutorialState and managed by RunCurrentState(), which handles switching and widget creation.
For creating components, a helper CreateWidget(WidgetClass, DrawSize, bIsRelativeTransform, Transform, GeometryMode) is there for the developers to easily setup their widgets inside the system once they created widget blueprints. This can be used anywhere inside the system and should appear in the correct state as long as you plugged it in the proper switch branch in RunCurrentState().

Equipment Interaction:

Except for the hydrocyclones, each machine is implemented as its own blueprint class with actor components that simulate and visualize equipment behavior.
Passthrough and rotation speed: Rotation controls the machine's movement around a set axis each tick, while passthrough adjusts mesh transparency levels to show or hide parts of the equipment. To add visual clarity, component tags were added for static mesh components that needed to be invisible when in passthrough mode, which can be helpful when passthrough features are needed on new imported equipment.
Machine information: It is presented through the WBP_MachineController widget, which includes a PageSwitcher component for Control, Info, and Video screens. When attached to a machine blueprint, the Control screen is active and linked to the parent machine; when used in a standalone info panel (BP_InfoScreen), the Control screen is disabled. Users navigate pages by incrementing the active widget index, looping between available screens. Info text is set based on the machine’s activity, either directly in the info panel or via the owner’s blueprint for controlled machines. Video playback requires a media player, media source, and media texture, which are set up in BP_InfoScreen or the owner blueprint for controlled machines. Videos are prepared by creating a media player and source, linking them to a media texture material with a UI domain, and connecting it to the widget for display.

Wrist Menu:

The wrist menu is constructed by enabling its input mapping context to track joystick movement. priority is set to override the teleportation controls. If it had the same priority as another IMC, both inputs would fire simultaneously.
We then bind two event dispatchers, one for the quiz spawn and one for the flowchart spawn
During updates, the menu is positioned on the user’s wrist and billboarded, while joystick input is monitored to simulate directional button presses when movements exceed the activation threshold.

Quiz system:

The quiz system presents multiple-choice questions randomly selected from a predefined question pool. Each question has four options, and users can select one or more answers.
Correct answers remove the question from the pool, while incorrect answers leave it for future attempts.
Questions are displayed through WBP_QuizQuestion, with BP_Quiz managing the logic and tracking user selections.
Answer correctness is determined by comparing the user’s choices to the correct options.
Feedback is provided via WBP_QuizCorrect and WBP_QuizWrong, and the quiz ends with WBP_QuizComplete, which allows users to reset or exit.
Widget visibility is managed by a Reveal() function, ensuring only the relevant widget is shown at a time.

Design Overview

Design Assets

The UI prototypes and user flow were designed on Figma, and widgets on UE. This includes onboarding tutorial flow, wrist menu, quiz widgets, control panel.

Missing parts of the cone crusher was modelled on blender.

Note that in the original proof of the concept phase, we used assets which are within the EULA Standard License. These assets would not be part of the open-source distributable project but will be visible and used in the build.

“Factory Environment Collection” (Factory Environment Collection | Fab) by Denys Rutkovskyi is licensed under EULA Standard License. Used for structures in the mining plant.
“Rocks” (Rocks | Fab) by Helindu.Art is licensed under Creative Commons Attribution (CC BY 4.0), used for rocks in the mills.

User Roles


User	Description
First-Year Mining Engineering Students	These students are beginning their mining education and use the VR application to familiarize themselves with the layout, function, and operation of common mineral processing equipment in a safe and engaging environment.
Second-Year Mining Engineering Students	With a foundational understanding of mining concepts, these students use the VR tool to reinforce classroom knowledge, visualize process workflows, and prepare for future site visits or internships.
Undeclared or Interested Students	These are students who are considering mining engineering as a major. They use the VR environment as an exploratory tool to gain exposure to mining operations and machinery without needing to visit a physical site.
Mining Engineering Instructors or Teaching Assistants	Instructors can use the VR application to supplement lectures, assign interactive lab tasks, and guide students through complex system processes virtually, ensuring safer learning. They can save time by showcasing equipment and workflows without travelling to mining sites.
Lab or Course Coordinators	These users manage access, content updates, and troubleshooting within the VR system to ensure it runs smoothly and stays aligned with course learning outcomes.

User Stories

1. Start Homepage: As a Mining Engineering student who has not been to a mineral processing plant I will be welcomed to the app and can look at a topographical model of a mining site to see what a mining site looks like.

2. Zoomed in model: As a Mining Engineering student who has not been to a mine I can zoom into certain parts of the topographical model to get a closer look at particular region of the mine.

More info: The topo model transitions to a zoomed in model when zooming in and goes back to the previous model when zooming out.

3. Tutorial: As a user who has not used a VR headset or this app before I can go through the onboarding tutorial to get familiar with how to use the headset and the app.

4. Laser point at flag: As a Mining Engineering student who has not been to a mine, I can point at a flag to see what is at that region.

5. Teleport from topo map: As a Mining Engineering student who has not been to a mine, I can click on the flag to go see the machine and the process at that region.

6. Flowchart: As a Mining Engineering student who has not been to a mine, I can look at and click on the flowchart to see the entire mining process and teleport to a particular section.

7. Exit: As a user I can exit the mineral processing plant from the wrist menu by selecting level selection to return to the topographical model.

Challenges

When both the controllers are aimed at a button, then one of the controllers leaves the button, it will be unselected even when the other controller is hovering it. Further research should be done to find a solution to this issue.

Code

The project file was developed using Unreal Engine Version 5.5.4

GitHub Repo: https://github.com/ubcemergingmedialab/25-3002-mining

Libraries

Plugins

Make sure the Procedural Content Generation Framework (PCG) and MetaXR are installed in the Plugins section.

First Time Setup Guide

Getting Started with Unreal Engine

Visit Download Unreal Engine - Unreal Engine and install the Epic Games Launcher
After following the steps to install the launcher, select Unreal Engine.
Go to the Library Tab and press the “+” Icon. Choose Unreal Engine 5.5.4
In Folder, choose where to install Unreal Engine. It is recommended not to change this if you have the space on your primary drive.
In path, click on options, and unselect the Android, IOS, and Linux platforms. This will save you about 22 GB on installation.
Let the installation happen and then run Unreal Engine. The first run may take a while, so please be patient!

Plugin Installation

Make sure the Procedural Content Generation Framework (PCG) and MetaXR are installed in the Plugins section. To enable XR, the MetaQuest Link application should be logged in with a developer account. You can verify this in Settings>Beta. If you can’t see Developer Runtime Features, create an account to be a Meta Horizon Developer and enable Passthrough over Meta Quest Link. If your Unreal Engine version does not have the MetaXR plugin yet, you can integrate it with these instructions: https://developers.meta.com/horizon/downloads/package/unreal-engine-5-integration

Using Oculus

Visit the Meta Quest website and scroll down to Quest 3 and click on download software.
After pressing download software, follow through with the instructions to install it.
On your mobile phone as well, install the Oculus App.

Future Considerations

To further increase the visual fidelity and realism of Immersive Education in Mining, a realistic cascade of particles and steel balls for the ball mills is a good start. Our current Niagara system is plausible for the slightly larger sediment of the SAG mill but is inaccurate for the ball mill. Ball mill particles are much smaller and behave as more of a viscous liquid when mixed with steel balls then discrete particles. If one would use the same Niagara system as the current one but replace the rubble with larger steel ball meshes, they would likely fall through the surface of the ball mill, due to the inaccuracy of particle collisions (although this needs testing to confirm). Fluid simulation is tempting but would most likely be too expensive (estimated additional 10ms to frame time per cascade). As mentioned before, chaos physics for each ball is accurate but far too expensive. The best solution we would recommend is to simulate and bake the animation in blender and import to UE as something like a geometry cache. This way, we don't have to care about any performance constraints. There may need to be multiple versions of the animation corresponding to different rotation speeds of the ball mill. In future development, a second activity will likely be implemented. The area of this activity is up to the designers and PIs, but a suggestion would be for a primary crushing area. The reason for this is we would be able to reuse a large amount of assets as well as systems designed for our current machines. Another area could be directly in the pit, allowing users to analyze markers for different ores, etc. Many systems are recommended to be refactored for extensibility.

A museum area can be implemented for convenience of the user. The user will be able to enter an area where all models can be found which are scaled down to fit in the room to analyze. Since we already have info data for the machines, we can just reuse those info panels next to the scaled down models. There will be no new machines, just the ones from existing areas in the processing plant.

Poster

Development Team