Session notes from the Investigative Practice Series
Chair: James Harkin
Director, The Centre for Investigative Journalism
Founding Director, Forensic Architecture
Research Fellow, Forensic Architecture
Founder of Bellingcat
From open-source digital mapping to forensic architectural methods to the kind of data journalism which can aggregate and divine patterns in the ether, what might be called forensic journalism – the application of methods drawn from science, technology, new media and professional expertise – is rapidly gaining ground.
In this seminar, Investigative Practice discussed the issues which arise for investigation when gifted open-source “citizen journalists” work with professionals, and when both work with traditional newsrooms and NGOs.
"The first big open-source story was probably the Boston Marathon bombing, but for all the wrong reasons, because of the Reddit community’s investigation that identified the wrong people and basically led to innocent people being harassed."
The developmental stages of open source investigation
Investigation techniques using open-source information have developed over the last decade or so in roughly 4 stages:
The rise of smartphone use and social media apps lead to huge wealth of reference material becoming available. After the Boston Marathon case, there was a clear need for verification tools and techniques to be developed.
Arab Spring begins and there was lots of social media being shared from that and questions arose as to what was authentic and what wasn’t. It also saw the rise of fake bloggers, for example ‘Gay Girl in Damascus’ who turned out to be an American man, certainly not in Damascus. Eliot Higgins started the Brown Moses blog, doing basic munitions identification in the Syrian conflict using open sources. At the same time a community was starting to build around the practice, including organisations like Storyful who’d already been working in the field for some years and others interested in arms control. The biggest story for Higgins personally came from using rebel Youtube videos to prove that weapons were being smuggled into the area by Saudi Arabia. Because of the limited access on the ground for journalists, these methods became increasingly important.
Governments started making statements that could be checked against open source information, and criminal investigations, for example the Joint Investigation Team, using open source material. Also growing interest in the practice from many different bodies.
We are now seeing large international bodies like the International Criminal Court (ICC) and the International Impartial and Independent Mechanism (IIIM) which was set up by the UN General Assembly to gather evidence and build cases around international crimes in the Syrian conflict. These bodies are now taking open source investigation extremely seriously in their work. In August 2017, the ICC issued an arrest warrant for a commander in Libya based on Facebook videos.In terms of how the process has developed, there’s a much wider network now and much more collaboration between those recording the material on the ground and those analysing it, so there have been initiatives to train activists in how to collect such evidence in ways that will be more useful to analysts.
The main challenge the practice is currently facing is mostly one of data management. How do you archive huge amounts of material such as the 1 million videos collated by the Syrian Archive in ways that are searchable and accessible, but that are also still forensically sound and can regulate who has access to the material? The other question is how to ensure it’s preserved, not just for five years, but for 25, 30 or 50 years? These are the questions that the ICC and the IIIM, helped by a wide coalition of organisations, are now really focused on finding solutions to.
There’s obviously a great need to enable this kind of archive building work, but there are also several potential problems around it, from witness protection issues to organisations that are reliant on the collection and collation of this source material for their funding so are unable to provide it to anyone for free. Then there are problems with practicality, especially when you get to the point of having 20 or more separate archives.
Bellingcat are working with the Syrian Archive and others to build the Archive for Conflict Investigation, a searchable system in which hundreds of thousands of pieces of source material can be indexed using unique identifiers but without making copies of the material itself. It will allow organisations to choose how much detail on their material they wish to share, from just date/time and geolocation data up to much more detailed information, and then other organisations such as the ICC can search the index and send out automated requests for the actual material that is relevant to their work.
Automated open-source investigation
With the exponential increase in source material we are getting beyond the situation where the work of analysis can be effectively done by a team of dedicated human researchers. There’s also a need to develop automation and machine learning for open-source investigation. Forensic Architecture are working on projects to train algorithms to search for and recognise particular munitions or other objects or events of interest to an investigation.
The Syrian Archive are currently developing this capacity for identifying cluster munitions and they have a huge amount of source material with which to train such an algorithm. It’s important for these future systems to be modular, so that multiple organisations with different budgetary constraints can all get practical use out of them, while at the same time ensuring the creation of a community that allows people to help each other and support and advance the work of multiple organisations. Higgins’ initial work was of course facilitated by a huge number of free tools and platforms that made it possible so it’s important to build this in to further projects as far as possible.
This is absolutely essential for justice and accountability in the future, since Syria is by no means a unique case; Bellingcat are already working with a range of organisations on a similar project focused on the conflict in Yemen.
Examples of use cases:
53rd Air Defense Brigade Investigation
Work on this involved trawling through huge amounts of images from social media, but a lot of that work could easily be automated by, for instance, training an algorithm to recognise uniforms in images or something as simple as identifying every single social media account which mentions the 53rd Air Defense Brigade. You could then use that to automatically map that network and flag up the most interesting and relevant parts of it, or scrape all the images available and dump them into an archive for analysis.
Military movements through the Bosphorus
Bellingcat are looking into using web cameras trained on the Bosphorus, and deploying machine learning taught to recognise vessels in the first place, and then specific military vessels as the algorithm becomes more sophisticated, in order to monitor military movements in the region. There are ethical issues that are thrown up by automation of this type since it could be used for a range of different purposes, some less ethical than others. Other problems relate to the transparency of the methodology when using automated or algorithmic processes of open-source investigation. One of the important things to do in these investigations is to show the methodology, especially if you’re using the process to establish facts or some kind of ‘truth’. So there is a challenge to ensure that where AI is used for this, we find ways of ensuring that the explanation of the methodology is still communicated clearly and in an accessible way.
"What we do insist on when we publish with organisations like the New York Times, is to have our own story."
Precise architectural spatial models are a way of creating relations between images that are otherwise very hard for people to see the relations between. It’s about navigational viewing, you have a model in which you locate dozens, hundreds or sometimes more sources in space and time and the model becomes a way of navigating between sources. This can include image sources, audio and video sources, sometimes material evidence that is found on the ground of a particular incident.
In terms of the history of the practice, Eyal Weizman’s work prior to Forensic Architecture was looking at war crimes committed by architects by examining the way Israeli architects were using their own tools of the trade to design settlements, borders, fences, checkpoints and other pieces of infrastructure in the West Bank in order to effectively squeeze people out as part of a policy of ethnic cleansing. Following that he began an alternative practice of contemporary archaeology, analysing the ruins of buildings to determine facts about the course of events.
"We can understand the link between certain things and how events have unfolded, but also pinpoint areas that were blind spots."
So Forensic Architecture like to work with other organisations but often have a slightly different emphasis and always reserve the right to publish separately. In the Douma case, for instance, they published something significantly separately from The New York Times, their media partner.
There are other ways in which these techniques can be used, beyond what is normally thought of as ‘open-source investigation’. Forensic Architecture are working with forensic psychologists to develop a practice of ‘Situated Testimony’ which uses techniques such as 3D modelling and building a representation of an event to bring back certain memories which have eluded a witness or enhance the clarity and remove distortion from existing memories. Sometimes a basic and potentially crude distinction is made between testimony and evidence and this practice provides a way of breaking down this distinction and weaving together both elements in ways that reveal a great deal.
We also need to remember that testimony can be gleaned from recorded footage and imagery. A camera records from both its ends and it’s not only what is in the frame. The work of joining together these sources is one of linking all these details and capturing the full story from what is both inside and outside the frame.
Forensic Architecture are working on frameworks which bring together large bodies of evidence into a 3D model to locate them in time and space, but also ensure that the analysis is streamlined so that you can pinpoint events or identify links very quickly and easily. But the interesting thing is how this process also highlights the gaps in the story and brings into focus the information that we couldn’t see or that we don’t have access to. And this allows us to identify areas for further investigation.
Q What sort of verification and fact-checking processes do you use?
A At Bellingcat there’s an awareness that many people are very ready to attack the organisation if they publish something that’s not verified or turns out to be incorrect, so the editing process is long and painstaking. Investigators there are constantly critiquing each other’s work. There are often long periods where members on a project will debate the precise language used to describe certain objects or ideas.
Working a lot with other organisations, as in the recent Anatomy of a Killing project with the BBC and Amnesty International, means there are many different backgrounds and perspectives being brought in to this process. The best perspectives are often those of the enthusiastic amateurs, as they’re not working for any organisation and just want to find out what’s true so they will challenge people and put theories to the test. But generally it’s possible to have a sensible discussion and come to a conclusion. When Bellingcat and Forensic Architecture collaborated on the M2 Hospital investigation both organisations ran fact-checking processes on the source material to test each other’s theories and produce reliable accurate work.
There’s a need to distinguish between the top-down hierarchical process of fact-checking and verification which comes from the collision and entanglement of different expertise and perspectives. Open-source investigation as a practice generally leans more towards the almost wiki-style process where many people are correcting and checking each other.
Q Does locating this work in an arts space, as in an exhibition, make the data more difficult to refute by anchoring it in a 3D space?
Conversely, in the art world as an institution, fact and fiction are in an unstable relation to each other, so it’s not really articulated as a ‘truth-producing’ institute like a court, scientific lab, university, etc. This makes it difficult to explain to people watching something in an art gallery that what they are seeing should be taken as fact rather than a fictionalisation and this is being used continually against us.
So while it’s very important both in producing and funding Forensic Architecture’s work and in seeking new audiences for their investigations, they often have to defend themselves for having worked in galleries. In fact, they have lost commissions due to the recent Turner Prize nomination, with people saying that the optics on the organisation is now that they are not serious investigators.
On fact vs. fiction though, having access to gallery space and an arts-based forum allows for a different sensibility in understanding events and incidents. Different kinds of relationships can be brought to the fore, so the understanding can be less about identifying specific people as criminals or events as crimes, but more about understanding systematic crimes that are happening in the contextual space around such events.
Q How did you find the source material for the Skripal Poisoning Investigation?
A Bellingcat managed this partly by having people on the team who already knew those databases existed. For the second identification it was helped by the Montenegro GRU suspect whose real and fake ID had been published. The team saw that a few specific details were shared by both identities such as first name, residence and date of birth. So began a process of elimination having found people within databases who share these details with the potentially fake identity of the other suspect, giving a pretty long list that was then whittled down by finding entries on that list on social media and verifying that they were genuinely a different person or cross-referencing them with other databases. This gave us an ever-decreasing list of potential identities until we eventually got to the passport document which we could then verify against other images and footage of the suspect.
"Ears never tell a lie"
The verification process involved looking at specific features on several images, one of which was a distinctive lump on the ear. Carlo Ginzberg actually wrote about the use of ears for verification purposes in the 19th century when investigating art fraud, since often forging the earlobes in a portrait is where fake painters lose their concentration for a second and it can be a more telling clue for identifying forgeries than analysing other features.
Q Isn’t there a danger that these techniques can be used to propagate conspiracy theories?
A Part of the reason Higgins started pursuing this practice was because this was already happening. One example is that of Kevin Dawes, an Asian American who travelled to Libya to become a journalist, but became ‘evidence’ that US soldiers were fighting alongside rebel troops there, despite Dawes not being and never having been in the US Army. So he got involved in open-source, in part, to interrogate conspiracy theories.
A further example can be seen in a live stream from the capture of Tripoli, which was claimed to be fake since markings on a landmark were not visible on a screenshot from the video. It was claimed that this was evidence that the video was a fake, filmed on a set made up to look like Tripoli. But the real reason the markings couldn’t be seen was the very low quality of the footage; it wasn’t a conspiracy theory at all.
One of the problems with this claim was that there was no link to the original footage or source, so at least part of the answer to this danger is ensuring good practice in using the techniques such as including references to original source material.
Q Can open-source investigation be used effectively in local contexts?
A Bellingcat are working on projects to apply these techniques to local issues in both Amsterdam and Utrecht in the Netherlands. Another example came from a training event in London when Bellingcat trainer Christiaan Triebert was the victim of an attempted mugging by a moped gang. Making the best of a bad situation, Triebert turned this into a case study and the trainers and participants were able to find a huge amount of information by using court records and analysing social media profiles of the would-be perpetrators.
Forensic Architecture are currently using these techniques in an ongoing investigation of the Grenfell Tower fire, working with the G20, which is a group of lawyers representing the bereaved relatives and survivors. The problem they have is that the amount of information that exists is in hundreds of thousands of data points. This includes all the people who were in and around the tower at the time of the fire including first responders as well as residents, their interactions and movements, phone calls, social media posts, etc. To investigate such a dense event in which 500 or more people are involved and each needs to have their own account necessitates a huge amount of analysis. So Forensic Architecture are building a spatial database system with a model that can navigate this in space and time. This includes mapping thousands of bits of footage showing the path of the fire into a continuous 3D representation, and a record of the movements and interactions which occurred. It’s intended to be an evidential archive that can be used for any case, whether insurance or criminal.
The second part of the project is looking at the systemic history of the tower, mapping the changes that were made from the original building to identify if the incident was part of a longer-term systemic process. This work includes bringing together historical public documents, which detail the changes made over time, with other evidence such as images published in pamphlets by the company in charge of the more recent development.
Q Is there a danger that in showing the methodology behind open-source investigation, you are making future investigations more difficult?
A To an extent this is true, an example would be the recent Russian legislation banning soldiers from sharing images on social media. However, there’s also constant innovation going on in the field, as well as plenty of legitimate investigative targets, whether individual people or nation states, who remain not as expert or careful as they would like to think.
Changes to social media platforms work both ways in regard to this. There have been many changes to Facebook in the wake of the Cambridge Analytica revelations, and all social media platforms are constantly developing and evolving anyway, but in most cases where methods are closed down other means of investigation open up, so there are always new avenues to explore.
Q What about the potential for trauma in conducting this kind of work?
A Bellingcat has recently published an article on vicarious trauma which provides a lot of best-practice and advice for minimising this risk and addressing the issue where it arises.
- Get to know yourself and your own sensitivities and triggers which will differ from person to person, especially where there is some personal connection between events being investigated or similar incidents.
- Prepare yourself mentally for viewing a potential trauma-inducing content and warn others appropriately before sharing such content.
- If you are working on content in foreign languages, learn to recognise some words that are likely to be associated with trauma-inducing content so you know what to expect.
- Be aware of and set limits on the environment in which you conduct the work. Keep it contained to an office. Alternatively, if you work from home keep it out of your bedroom, so that you have a safe space to return to when you need to rest. Set time limits and minimise work at night.
- There are also several simple technological steps you can take:
- Mute your volume unless you absolutely need to listen; audio can often be more traumatic for people than imagery.
- Use a thumbnail preview in the progress bar to give yourself warning of potentially traumatic content.
- Consider using a sticky note or your hand to cover graphic images.
- Always have autoplay turned off.
- Ensure a community and workplace culture that recognises the risks of secondary trauma, takes them seriously and allows enough flexibility, support and time for investigators to manage and address any symptoms of such trauma.
This discussion was part of the Investigative Practice series at the Logan Symposium, Oct 2018