Synthetic 2-chlorobenzalmalononitrile

Forensic Architecture are a collective of researchers led Eyal Weizman based at Goldsmiths University in London. The team of architects, scientists and artists work as forensic investigators, bringing attention to human rights abuses worldwide. Founded in 2010, Forensic Architecture were motivated by a rise in the urbanisation of war, open source media and mobile phone footage. They move in university research centres, the art world and international criminal courts and their research uses experimental technologies, such as in project Model Zoo.

Model Zoo investigates ways for better identification of 37mm-40mm tear gas canisters against civilians. Tear gas is prohibited in war under the Geneva Convention, but is commonly employed by police as a riot control agent in protests such as those in Hong Kong and Chile. With the urbanisation of warfare, the line of legality for the use of tear gas becomes increasingly blurred, and its status as a ‘non-lethal’ weapon has led to a lack of transparency regarding its sales by manufacturers. Forensic Architecture identified this gap in knowledge and are using the rise of footage of protests to create an identification system for the use of tear gas. They began by isolating single frames from footage inputting these into the open source tool mtriage. Using machine learning with this real footage, mtriage was able to identify images of gas canisters with a moderate success rate. 

In order to increase this success rate, Forensic Architecture then made use of synthetic images of gas canisters to push mtriage to the upper limit of machine perception. These synthetic images were made from renders of 3D models of gas canisters with complex patterns and artworks imposed on their surfaces, creating ‘extreme’ versions of the gas canister form. 

The more complex and better rendered the synthetic image, the better the computer will be able to recognise the gas canister, as it will have a larger set of data to analyse. Often, scientists try to improve the realism of the synthetic images to improve test accuracy. Forensic Architecture, however, recognised that the image of a tear gas canister is not a complex form in itself, and by imposing complex patterns on the surface of the canister to form the synthetic images, they could make more complex images for their computers to learn from. These extreme training examples at ‘the threshold of recognisability’ improve the computer’s ability to distinguish between gas canisters and objects of a similar shape. Forensic Architecture use both techniques of synthetic images and of isolated real images in conjunction in their machine learning algorithm, as the two complement and make up for each other’s faults.

Combining seemingly outlandish, but still recognisable images with more normative ones, the classifier can detect instances of its object with more nuance.

Forensic Architecture, Detecting Tear Gas: Vision and Sound, 20.02.2020

Look closer at some of the patterns chosen for the renders and you may recognise artworks reminiscent of Mondrian and Sonia Delaunay. There is a parallel, in Model Zoo, between how Forensic Architecture have used these famous artworks and how they operate more largely in art spaces, on which they have a politicising effect. At the 2019 Whitney Biennial, Forensic Architecture presented Triple Chaser, a video about the machine learning algorithm that identifies tear gas canisters. The video focused on the identification of tear gas produced by Safariland, a company whose CEO, Warren Kanders, worked as the Whitney board’s vice chair, until protests against tear gas caused him to quit the board immediately after the 2019 Biennial. This is a more subtle example of Forensic Architecture’s influence and I highly recommend looking through their vast archive of research for more novel and multidisciplinary approaches to investigation.

Tags: artBiennalecrimeInvestigationprotest