Photogrammetric Memories

Duration

2021.03 - 2021.05

Tools

HTML, CSS, Javascript,
Blender, Meshroom, glitch

about the project

What is Photogrammetric Memories?

Photogrammetric Memories is an interactive web application composed of 3D scans of real-life objects and environments using photogrammetry. Completed near the end of my undergraduate studies at RISD, this project documents the places and things around the RISD campus that are meaningful to me.

Project research

Why photogrammetry?

As one of my last projects during my undergraduate studies, I had wished to use this project to document my life at RISD. I went into research on creative 3D imaging technologies that would generate scans or point clouds. I compared the costs, technical complexity, and feasibility of different scanning techniques before choosing photogrammetry. Photogrammetry allows me to recreate real-life 3D environments with a series of photos taken from different angles.

Software & Techniques

After deciding to use photogrammetry, I researched further into methods to present the photogrammetric scans. I discovered Meshroom, a photogrammetry software that composites realistic 3D environments from photos. And to make the 3D scans accessible to everyone, I used Three.js, a 3D javascript library, to load the files into web pages. As a proof of concept, I started by scanning and compositing a wooden sculpture I had made to practice and perfect the workflow before starting the scanning process.

capturing

Scanning the environment

The first step is to identify the environments and objects I wish to scan. I reflected on my undergraduate life and created a list of places and things that were meaningful to me during my time at RISD. I went to these locations and took pictures from different angles, heights, and distances with my phone to capture as many details as possible. I then loaded the images into Meshroom to composite them into 3D models.

A yellow bandsaw in a woodshed photographed from the left

Raw Photo from Angle 1

A yellow bandsaw in a woodshed photographed from the centre

Raw Photo from Angle 2

A yellow bandsaw in a woodshed photographed from the right

Raw Photo from Angle 3

Point cloud of details that were used to composite the bandsaw in the woodshed as shown in Meshroom

Meshroom 3D Composite from 322 images

Meshroom is a node-based software that allowed me to create the most suitable workflow to streamline the image compositing process. I added "mesh decimation" to limit the size of the 3D files for web formatting and used "texturing" to retexture the decimated 3D models without using any other software.

Steps being connected in sequence of processing in Meshroom that shows Meshroom's node-based workflow

Node-based Workflow in Meshroom

formatting the scans

From 3D models to presentable files

After researching the optimal 3D file format for the web, I had decided to use GLTF/GLB. I used blender to convert the obj and texture files exported by Meshroom and adjust the 3D models into the correct orientation for proper web presentation. With the newest version of blender, I was able to export 3D files directly to the GLB format that would work perfectly on the web.

Scanned 3D Model for the bandsaw being cleaned up in blender

Coding the Pages

Bringing scenes to life in web

To put the GLB files on the web, I decided to use Three.js, a javascript library that enables a range of 3D web applications. For each 3D model, I coded the web pages in HTML and CSS to present them in an interactive web format. The 3D models will automatically rotate in the browser, and users can drag around or zoom in the models if they wish to. For some models, I have also included audio recordings from the location that further enhance the experience.

Scanned 3D Model for the bandsaw presented in a web browser

Hosting the website

Making the memories available to everyone

Loading the 3D scans into web pages with Three.js and p5.js requires a server. During the coding process, I used PHP localhost to view the website on my local machine. To make the website accessible for everyone, I decided to use glitch.com and its Express.js web application to host my website.

final design

Photogrammetric Memories

For the landing page, I wanted to mimic the way I randomly move through my memories. I used p5.js to make moving bubbles that represent floating memories. When you click on any bubble, it will randomly take you to one of the pages with the 3D scans.

Experience the website and 3D scans here.

Screenshot of Photogrammetric Memories. White bubbles that represent memories are free flowing on dark grey background

Access my code on Github by clicking here.

next steps

A Unity 3D experience

At the beginning of the project, I explored different ways to recreate 3D environments and present 3D models. As the next step for this project, I would like to research further how I can present these "photogrammetric memories." I had experimented with using Unity to design and develop games for another project. It would be interesting to repurpose the 3D models I have scanned and turn them into VR/AR experiences.

Takeaways

Experimenting with code and photogrammetry

Learning new tools and technologies can be challenging and intimidating. Before this project, I had never experimented with photogrammetry and had to keep learning new skills along the way. I conducted extensive online research and watched countless tutorials to learn how to use Three.js and glitch.com to develop and host web applications. It's common to run into problems when working with new technologies, and the only thing you can do is research and problem solve.