Lunch & Learn: Reassembling the wall paintings of Thera with Szymon Rusinkiewicz

Imagine trying to assemble a jigsaw puzzle without knowing the number of pieces or even what the final image might look like.
The archaeological site of Akrotiri on the small, volcanic island of
Thera (modern-day Santorini, Greece) has yielded an unparalleled trove of artifacts and information from the prehistoric Aegean. The ancient trading civilization was destroyed by a volcanic eruption, which buried the remains of a flourishing Late Bronze Age (c. 1630 B.C.) settlement in ash. Among the most significant finds are numerous wall paintings, ranging from every day scenes and coming-of-age rituals to abstract motifs. However, these paintings are recovered as thousands of plaster fragments, and reassembling them consumes a substantial portion of the effort expended at Akrotiri.

At the September 30 Lunch ‘n Learn seminar, Szymon Rusinkiewicz
summarized a project that has been on-going in Computer Science at Princeton for nearly three years in collaboration with archaeologists and conservators in Greece and researchers at University College London, KU Leuven, and elsewhere. A team has developed a system that uses 3-D and 2-D digitization hardware, together with computer-based matching techniques, to assist archaeologists and conservators in documenting and reassembling the wall paintings.
Apart from the obvious fact that the frescoes were shattered and then buried by the volcanic eruption, they are very well preserved. Most were created while the plaster was still wet. More flaking of the pigment would have occurred had the frescoes had been painted after the plaster had dried.
The difficulty of reassembling the frescoes varies greatly. The
conservators have had a relatively easy time reassembling frescoes that contain designs, but many involve solid colored backgrounds. Hence, there are fragments from as far back as the 1970s that still not reconstructed. The storeroom on site contains crates almost without end, with tens of thousands of fragments.
The pieces are organized by date and location. To assemble a fresco, the conservators usually gather together the crates whose pieces might contribute to that fresco. Nonetheless, pieces from one wall of one room may today be spread among many different crates. Once the easy pattern matches are found, the conservators devote significant time to finding additional matches. Hence, the scale of the problem is very large.
The researchers are scanning in each piece, including detail about shape and contextual cues, with the aim of proposing possible matches. Although mature technologies exist for acquiring images, geometry, and surface normals of small objects, they remain cumbersome and time-consuming for non-experts to employ on a large scale. The newly developed system addresses the scalability, usability, and quality challenges of large-scale 3-D and 2-D digitization, by incorporating new algorithms to align 3-D scans automatically, to register 2-D scans to 3-D geometry, and to compute surface normals from 2-D scans.

To test the system, the conservators constructed a fresco with same characteristics as the ones being excavated, and then professionally destroyed it. The Princeton team then attempted to reconstruct it without any clue about what it ought to look like. They learned that among many variables, the thickness of the fragments and the level of erosion were especially useful cues. And, as it turns out, computers are very good at combining multiple sources of information. Other important features? The front of the frescoes was generally flat, but the back surface had variations and contextual clues that proved useful in finding candidate matches. Door frame pieces were easy to locate because such fragments had thicker portions fitting into the frame. These represent the edges of the puzzle. Strings were originally used to make indentations that served as a guide for painting. They left very faint impressions, a strong cue for matching.
At the talk, Rusinkiewicz explained how the team used the scanners to record the geometries of the fragments. A novel 3-D matching algorithm efficiently searches for matching fragments using the scanned geometric models. In essence, computers can use brute force searching to look for candidate matches by looking for complementary geometries.
Is the effort useful for other applications? Yes, suggests Rusinkiewicz, if you are attempting to reassemble objects that have been broken.
Szymon Rusinkiewicz is an associate professor of Computer Science at Princeton University. His work focuses on acquisition and analysis of the 3D shape and appearance of real-world objects, including the design of capture devices, data structures for efficient representation, and applications (most notably to cultural heritage objects and human skin). He also investigates algorithms for processing complex datasets of shape and reflectance, including registration, matching, completion, symmetry analysis, and sampling. His research interests also include illustrative
depiction through line-drawings and non-photorealistic shading models.
A podcast of the talk is available.

This entry was posted in Lunch & Learn, Training and Outreach and tagged , , , , , , , , . Bookmark the permalink.