IMLE (Immersive Materials Learning Environment)

Emerging Media Lab
About EML
A collaborative space for UBC faculty, students and staff for exploration of emerging technologies and development of innovative tools and solutions. This wiki is for public-facing documentation for our projects and procedures.
Subpages
	Main EML wiki page; Public EML website; EML Projects;

Introduction

The Immersive Materials Learning Environment is a browser-based, interactive learning tool for APSC 278/279 and MECH 221, core second year materials science and engineering courses that serve nearly 700 students from different engineering disciplines annually. By providing experiential access to concepts that are otherwise difficult or impossible to observe or practice, the IMLE promotes deep learning and higher-order cognitive engagement, enabling students to move beyond memorization toward application of concepts to real-world engineering problems.

Context

Before development, several barriers to student learning were identified, revealing a common risk pathway that guided IMLE’s design and intended role.

Project Details

The current IMLE project includes 5 interactive modules aligned with selected course topics:

Atomic Bonding Module:

Students select elements from an interactive periodic table to explore which elements they are likely to bond with and the resulting bond type.

Unit Cell (Crystal) Module:

Students interact with unit cells to visualize atomic arrangements in different crystal structures and determine key characteristics such as coordination number and atomic packing factor.

Lattice Defects Module:

Students explore different lattice defects, visualize their structures, and examine how they move and distort the surrounding lattice.

Tensile Testing (Lab) Module:

Students independently perform virtual tensile tests on a wide range of materials without the equipment, safety, or specimen limitations of a physical laboratory.

Graph Analysis Module:

Students explore an interactive graph to understand how different material’s physical properties, and atomic structure change and evolve during a tensile test

App deployment

Currently the app is hosted using Github Pages, from this repo.

There are two versions deployed, the full domain names for both are:

Production: Production Build

Development: Developement Build

The two deployments are made automatically whenever code is submitted to the main branch for production, and development branch for the development deployment.

This is done through the Github Workflows found here: Github Workflow

Feature Status

The trello board can be accessed here.

DONE: Major feature that is complete
TODO: Major feature that is incomplete
BUG: Bug that needs to be fixed
TWEAK: Minor feature or change that is incomplete
STRETCH: Stetch goal that is incomplete

Feature Status

General

Status	Task	Description
Done	Page navigation	Users can navigate to different modules using a top navigation menu
Done	Help button	Users can access instructional information for each module
Done	Color Blind Assessment	Adjust all colors in the app to be suitable for use for color blind users
Todo	Light/Dark Mode Toggle	Add a toggle to switch between light and dark modes
Todo	Tooltips	Users can highlight over different components and get useful information about what they are/do (e.g. APF = atomic packing factor)
Todo	Native cell shader rendering	Currently textures are baked onto 3d models, switching to native rendering will reduce file sizes (no textures) and gives more programmatic control over materials (e.g. dynamically changing colors)
Stretch	Mobile support	Make the app work with touch controls, vertical displays and small displays
Stretch	Sound Effects	Add sound effects for different parts of the app
Stretch	UI Animations / Effects	Add additional animations to the UI across the app

Atomic Module

Status	Task	Description
Done	Atom rendering	atoms are rendered with accurate atomic radii and electron counts
Done	Atom dragging	users can drag atoms around a bounded 2d world to move them
Done	Atom snapping	when atoms move close to a bonding position, they automatically snap into ideal bonding locations
Done	Camera controls	users can move the camera around the bounded 2d world by dragging and zooming
Done	Ionic bonding	ionic bonds form between atoms and electrons are transferred from one atom to another
Done	Covalent bonding	covalent bonds form between atoms with electrons being shared between atoms
Done	Metallic bonding	metallic bonds form between atoms with electrons animating in a "sea of electrons"
Done	Atom info panel	users can click on atoms to learn useful information such as their atomic mass
Done	Bond info panel	users can click on bonds to learn useful information such as their ΔEN
Done	Periodic table for adding atoms	users can add atoms to the world by dragging them in from a periodic table
Done	Button for deleting atoms	users can delete atoms (found on the atom info panel)
Todo	Hydrogen bonding	hydrogen bonds form between atoms and are distinctively rendered
Todo	Van der walls bonding	van der walls bonds form between atoms are distinctively rendered
Todo	Camera controls ui	users can use arrow UI to move the camera around the scene
Todo	Ionic lattices	users can form lattices out of ionic bonds (e.g. NaCl crystal)
Todo	Preset loading	users can use a drop down menu to load preset atom configurations (e.g. aluminum lattice), so they don't have to manually construct them
Bug	Electron Bug	Electrons can fly off when deleting atoms from metallic bonds
Tweak	Metallic bond interaction	Make metallic bonds clickable over the entire sea of electrons
Tweak	Optimization	Optimize the rendering of electrons so that they are not reloaded whenever an atom is added/removed
Tweak	Metallic bond visual	Render a backdrop for the atomic sea of metallic bonds

Unit Cell Module

Status	Task	Description
Done	Unit cell rendering	Unit cells can be rendered in Reduced, Normal and Cutoff modes
Done	Crystal info panel	Users can see useful info about the current crystal type including APF
Done	Crystal type control	Users can change which crystal type is currently being shown
Done	Unit cell expansion control	Users can expand unit cells away from each other and hide all but 1 unit cell
Done	Unit cell transparency control	Users can change the transparency of atoms in a unit cell
Done	Dimensions calculation	Users can calculate the dimensions of a unit cell
Done	Dimensions axes	Axes representing the different dimensions are rendered while calucation dimensions. Users can click on dimension variables in equations to see which axes it represents
Done	Atom count calculation	Users can calculate the atom count in a unit cell
Done	Atom count highlighting	Users can see which atoms align to which parts of the calculation
Done	Apf calculation	Users can calculate the APF of a unit cell
Done	Coordination number toy	Users can see a special view of the crystal to count the coordination number
Bug		Fix highlight toys for APF, $V_a$ should highlight all the atoms and $V_{uc}$ should highlight the entire cube taken up by the unit cell
Bug		No other calculations should be able to be enabled while Coord. Number calculation is active.
Tweak		Find a way to synchonize the flashing animations between html/css components (e.g. calculation symbols) and r3f components (e.g. unit cells, axis)

Lattice Defect Module

Status	Task	Description
Done	Point defect rendering	Four different types of point defect are randomly generated and rendered within a 2D lattice
Done	Point defect UI	Changing bond colors depending on their length can be toggled
Done	Edge dislocation rendering	Multiple edge dislocations are generated within a 3D lattice, alongisde their Burger's vectors
Done	Edge dislocation movement	Edge dislocations can be translated left and right by shearing the bottom half of the lattice
Done	Edge dislocation UI	Background layer opacity and the presence of Burger's vectors can be configured
Done	Screw dislocation rendering	A screw dislocation is rendered within a 3D lattice, alongside its dislocation line
Done	Screw dislocation movement	The screw dislocation can be translated left and right by shearing the two halves up and down
Done	Screw dislocation UI	Unit cell face opacity, the presence of the dislocation line, and the highlighting of the screw dislocation bonds can be configured

Lab Testing Module

Status	Task	Description
Done	Sample storage containers	Users can spawn samples of different materials onto the measuring mat by clicking on the storage containers
Done	Measuring mat / Draggable samples	Students can drag samples around a measuring mat. Samples are rendered based on how much they have been elongated.
Done	Caliper	Students can use a virtual caliper to measure the samples
Done	Sample holder	Students can move samples over to a sample holder to take to the experiment page
Done	Machine assembly	Students can assemble a tensile test machine using parts found in an actual tensile testing machine
Done	Elongation controls	Users can elongate samples using the elongation controls, and also reset the elongation with a reset button
Done	Machine animation	The machine animates as users elongate the sample
Done	Graph	A Force(N) / Elongation (mm) graph shows the current and historical state of a sample
Bug		Sometimes samples do not render with the proper elongation.
Tweak		Make sub page navigation button not rely solely on an image (only the arrow should be an image).
Tweak		Add dot tracker to tensile graph to show current point.

Data Analysis Module

Status	Task	Description
Done	Macroscopic view	A zoomed in version of a physical tensile test from the Lab module
Done	Material selector	Menu for selecting which material to analyze
Done	Graph	A Stress/Strain graph, with key points marked and descriptors for the key points when they are hovered
Done	Elongation controls	Users can elongate samples using the elongation controls, and also reset the elongation with a reset button
Done	Graph information panel	A panel showing the current stress, strain and deformation type of the sample
Todo	Microscopic view	A bubble raft model of the material, showing the movement of point dislocations across the sample
Tweak		Fix the rendering of graph sections for various samples, as the sharp increase in stress and the beginning of most tests causes the section labels to be rendered under the graph line
Tweak		Add Stress/Strain information to key points info panels.

Developers Guide

This is a React project that renders 3d scenes using React Three Fiber. It is built using vite and deployed via GitHub Pages. Source code is stored on GitHub.

Setup

Install node
Run npm install (in the root project directory)
Install vscode
Install the ESLint extension

Building / Deployment

Create build ( npm run build )
Push/merge changes to main or develop
Check the GitHub Pages page for any errors in deploying

Commands Cheat Sheet

npm run dev - run a local development build with hot reload
npm run build - create a production build
npm run preview - preview the production build
npm run lint - run the linter
npm run docs - regenerate the typedoc documentation

Git Branches

main - production branch, deployed here

develop - development branch, deployed here

Do not merge directly to main , all commits must go through develop. Ideally branch off of develop before pushing.

develop before pushing

NPM Packages Overview

Non Dev-only

Package	Description
@react-spring/three	used for animating state changes in 3d scenes
@react-three/csg	used for creating csg meshes for 3d scenes
@react-three/drei	helpers for rendering 3d scenes; premade components
@react-three/fiber	used for rendering 3d scenes
@react-three/postprocessing	used for post processing of 3d scenes
@visx/visx	react friendly visual primitives - currently used for the graphs in the lab module
clsx	conditionally merge css module classes for react
react-moveable	draggable HTML components
papaparse	csv parsing
jotai	global state management
lightningcss	css parser used for builds
mitt	event system
priority-queue-typescript	priority queue data structure implementation
react	it's react.
react-dom	it's react (but again)
three	used for rendering 3d scenes
usehooks-ts	additional hooks for react

Dev-only

Package	Description
@emnapi/core	WebAssembly support
@emnapi/runtime	WebAssembly support
@eslint/js	JavaScript linter
@stylistic/eslint-plugin	linter rules
@types/node	types for node (see above)
@types/papaparse	types for papaparse (see above)
@types/react	types for react (see above)
@types/react-dom	types for react (see above)
@vitejs/plugin-react	react support for vite
eslint	linter
eslint-plugin-react-hooks	linter for react hooks
eslint-plugin-react-refresh	vite compatible linter
globals	collection of global identifier reference environments for ESLint
typedoc	documentation generator
typedoc-plugin-markdown	render TypeDoc as Markdown files
typedoc-plugin-rename-defaults	customize TypeDoc default module and page names
typescript	it's typescript
typescript-eslint	ESLint support for typescript
vite	frontend build tool and dev server

Project Folders

Folder	Description
src	all the code
src/components	all react components
src/data	class, interface, and type definitions
src/events	mitt event defintions
src/scripts	additional typescript helper functions and data storage
src/state	jotai global state definitions
src/main.tsx	entry point
public/assets	all assets for the project
public/assets/data	csv data files for tensile tests in module 4 and 5
public/assets/fonts	font files
public/assets/models	glb models
public/assets/textures	2d images: svg, png, webp
public/assets/typefaces	special json files for rendering 3d text

Project Documentation

Documentation for the codebase can be found at docs/modules.md, as well as on this wiki page.

Resources

Designers Guide

General Workflow

The following design tools were used in the design and development of the project:

Design Files

Relevant design files can be found at:

ATOMIC FIGMA PAGE - Contains all atomic design work created on figma (you must have access to the "Emerging Media Lab" on figma to access)
MAIN FIGMA DESIGN FILE - References all current designs and concepts (you must have access to the "Emerging Media Lab" on figma to access)

26_design_files folder on GitHub. This includes Blender Files (Contain the bulk of UI development work, containing all the 2D mockups for the overall UI and some visual tests) and Videos + Images (Rendered or exported visuals used as demos or as content within a UI design)
- 26_design_files/Active - Files currently used in design development or containing files that are references/models of the ones used in the current build.
- 26_design_files/Inactive - Files that were used and iterated upon in the design process and have been kept for documentation purposes.

Design Framework and Methodology

The IMLE was designed referencing existing scientific tools and visualizations.

Reference Work	Description
William D. Callister, “Materials Science and Engineering”, Wiley, 10th edition	Textbook for APSC 278/279 containing various visuals of scientific concepts
Chemical Bonding Simulation	Simple atomic bonding sandbox simulation
AtomicLab Sandbox	Individual atom builder and analysis toolkit
Unit Cell Visualizer	Simple interactive unit cell visualizer
Crystallographic Visualizer	Visualizer for crystallographic planes and directions
Flash Tensile Test	Flashy Science virtual tensile testing experience
UTM Virtual Lab	Analysis oriented tensile testing and graphing tool

The IMLE was also designed with consideration of UDL guidelines, key guidelines that influence design work for the project include:

Support opportunities to customize the display of information (1.1)
Support multiple ways to perceive information (1.2)
Clarify Vocabulary, symbols, and language structures (2.1)
Support decoding of text, mathematical notation, and symbols (2.2)
Connect prior knowledge to new learning (3.1)
Cultivate multiple ways of knowing and making meaning (3.3)
Maximize transfer and generalization (3.4)
Optimize choice and autonomy (7.1)
Nurture joy and play (7.3)

Persistent design elements

Help tooltip
LM/DM button
Header module selection
General UI layout and organization
Asset style and aesthetic
Colour consistency
- Blue Tone - Interactive tools/features
- Red Tone - Navigation elements
- Yellow Tone - Help or tutorial elements

A light/dark mode toggle has been designed into the project but has not been implemented yet. A dedicated high contrast mode could also be added in future

Design Process

In the creation and development of the designs:

Top layer 2D UI design was completed in Figma
3D mock ups of interactive content were made in Blender and rendered for use in UI mock-ups
2D UI and rendered 3D content were layered in Figma

3D assets

The project uses a toon, cel shaded aesthetic shown in the attached files and reference designs. Textures were created in Blender/EEVEE, before being reformatted for cycles and baked for exporting. This method does not allow for the shaders to react to lighting within the application.

Project Poster

Project Team

Principal Investigators (PI):

Dr. Farzaneh Farhang Mehr
Dr. Amir Dehkhoda

Subject Matter Expert (SME):

Chihiro Ohara

EML Developers:

Ahren Spadinger-Fengler
Alex Day
Dante Cerron

EML Designers:

Johannes Soderstrom

Other:

Maryann Kempthorne
Phoebe Titus
Avni Kant
Saeed Dyanatkar

EML Documentation License

Permission is granted to copy, distribute and/or modify this document according to the terms in Creative Commons License, Attribution-ShareAlike 4.0. The full text of this license may be found here: CC by-sa 4.0