Who killed the Kiev protesters?

How a trove of mobile phone footage, photos, and surveillance footage, combined with sophisticated 3D modelling might help bring justice for protesters murdered in Ukraine. Mattathias Schwartz reports

ON February 20, 2014, Evelyn Nefertari, a graduate student living in western Ukraine, watched as the most violent day in her country’s recent history unfolded.

On that morning, paramilitary police forces loyal to President Viktor Yanukovych clashed with protesters in Kiev, who were demonstrating against the government’s tilt towards Russia and away from economic integration with western Europe. When the tear gas and smoke finally cleared, four police officers and 48 protesters were dead.

“They were highly educated intellectuals,” Nefertari told me recently over the phone.

“The nation paid a very high price for freedom.”

In the aftermath of the confrontation, she decided to assemble the definitive record of what happened. “The whole country was in mourning,” she remembers. “I knew that I should do it.”

Most of the deaths occurred within half an hour along a few hundred feet of streetscape. The scene unfolded before dozens of cameramen, smartphones, and security cameras.

But these recorded fragments from the day were overshadowed by a fight over what they really showed: The claims of grief-stricken activists, who blamed the Ukrainian paramilitary for shooting the protesters, collided with denials from Yanukovych, who would later testify that the killings were part of a “planned provocation” and “pseudo-operation” carried out by the protesters themselves, a US-backed plot to remove him from power.

Pro-Russia sources went even further, pushing the notion that the February 20 killings were a “false flag” operation carried out by snipers associated with the protesters, or mercenaries from Georgia.

The killings took place within a few blocks of Kiev’s Maidan Nezalezhnosti, or “independence square,” the centre of the nationwide protests. Nefertari began collecting and synchronising snippets of video from the internet and from news broadcasts.

The task seemed impossible given that the videos were shot in different places from different angles and filled with irrelevant noise. Only with great patience, by picking out sounds and landmarks, could she begin to assign them time stamps and co-ordinates and figure out how each related to the others.

The most crucial videos showed civilian protesters in helmets and winter jackets facing off against masked riflemen who had taken up barricaded shooting positions.

The protesters cowered in groups of five or six behind makeshift shields; one would suddenly tumble to the ground and be carried off by comrades on a stretcher. On the anniversary of the killings, Nefertari released a 164-minute real-time video on YouTube showing the standoff from up to nine simultaneous points of view. Within the first month of being posted, it was watched more than 270,000 times.

Three years later, Nefertari’s video trove has turned into the seed of an even more complicated piece of analysis: A sophisticated multimedia presentation that tries to recreate the deaths of three protesters using 3D laser scans of the streetscape, ballistics analysis, and autopsy reports.

The combination of so many disparate data sources into a single 3D model has little precedent. The arduous work of timing and placing individual videos was assisted by artificial intelligence, which helped organise and synchronise the enormous quantity of footage. Now assembled on a mini-PC and received as evidence by a Ukrainian criminal court, the reconstruction project could prove crucial in the trial of five police officers who, Ukrainian prosecutors say, are responsible for the killings.

The Maidan demonstrations are one of many cases in which mass protests, fuelled by popular discontent and social media, have threatened to topple, or at least embarrass, governments around the world.

The response from the international community often hinges on whether the party in power can effectively make the case that its use of force was justified. To avoid being cast as authoritarian, governments must do more than control the crowd; they must control the narrative.

Forensic tools like those used in the Ukraine inquiry can make the difference in pinning down the truth of what happened. Similar investigations, using detailed analyses of open-source data, have been conducted into the use of chemical weapons by the Syrian government and the killing of a Venezuelan activist.

In Ukraine, the Maidan protests began in a similar way to the Tahrir Square demonstrations in Egypt and the Occupy Wall Street movement in the US — a few thousand determined protesters battling the police for a small but highly visible piece of urban terrain.

The anti-Yanukovych crowds toppled a statue of Lenin and cut off the wide Soviet-era boulevards with piles of debris, pulling up paving stones and heaping them up into barricades.

Russian-backed media quickly set about framing the protesters as “fascists” who posed a threat to Russian ethnic minorities in eastern Ukraine and Crimea. By December, protesters were occupying and setting fire to government buildings.

By the third week of February, protesters had spent months living in a tent city in the heart of Kiev. Their purpose was to challenge Yanukovych’s rule and to demand reforms: New elections, more freedom to protest, and closer ties with the EU.

The violence came to a head on February 20. Police officers massed around the protesters, who set their barricades on fire and tried to march on Ukraine’s Parliament.

Protesters threw bricks and Molotov cocktails; the police responded with tear gas and rubber bullets. A wave of people pushed out of the square and down Instytutska St. Defending themselves with helmets and homemade shields, the protesters were met by barricaded riflemen from the Berkut, an elite police force loyal to Yanukovych. By the end of the day, dozens of bodies lay in the streets.

Two days after the shootings, Yanukovych fled to Russia. Over the next few weeks, Ukraine’s parliament held new elections. Pro-western parties won at the polls and would later enter into a trade agreement with the EU.

President Vladimir Putin of Russia claimed that what had just taken place amounted to a coup. He seized Crimea and made inroads into Ukraine’s eastern provinces. In April, prosecutors in Kiev opened an investigation into the February 20 killings, searching for someone to hold responsible for the protesters’ deaths.

Prosecutors hoped to bring charges against the unit of the Berkut that appeared to have taken up shooting positions behind a barricade on Instytutska St. Most of the unit’s members had fled to Russia. Ukrainian prosecutors tried to extradite them; the Russian government ignored the request.

The remaining five members of the Berkut unit awaited trial in a Kiev prison. Their Kalashnikov assault rifles and pump-action shotguns were discovered at the bottom of a nearby lake, sawed into pieces.

A few months after the protests ended, lawyers representing the families of the dead protesters started working with the prosecution to try and put together a reconstruction of the shootings that the court could accept as evidence.

They became aware of Nefertari’s project and reached out to her. “Evelyn’s work was the keystone,” said Pavlo Dykan, one of the lawyers. “It allowed this case to proceed.”

In the summer of 2015, Dykan and his colleague Aleksandra Yatsenko presented Nefertari’s video and discussed the challenges of the Maidan case at a small conference at the Centre for Human Rights Science at Carnegie Mellon University in Pittsburgh.

Among the other presenters was Brad Samuels, a principal of SITU, an architecture firm and research lab in Brooklyn. The firm combined traditional architecture work with grant-funded deep dives at the outer limits of human rights research.

Samuels had come to the conference to present a project that SITU did with Eyal Weizman’s forensic-architecture research group on the death of Bassem Abu Rahmah, a Palestinian protester in the West Bank who was struck in the chest by a tear-gas canister.

“The problem was how to take multiple vantage points and put them together into a coherent analysis,” he said.

“Maidan was the same problem set, but on steroids.”

THE quantity of raw footage amassed by Nefertari was overwhelming, running into the thousands of hours. The problem was how to fuse it into a whole that would persuade the judges and stand up to cross-examination.

Prosecutors would have to do more than cherry-pick a few convincing moments. Their theory of the case would have to be strong enough to survive every alternative scenario, from the mysterious rooftop snipers to the possibility that protesters were killed by friendly fire.

Multiple cameras recorded the deaths of three protesters whose families Dykan and Yatsenko were representing, who were selected for the video presentation. One of the dead was Ihor Dmytriv, a 30-year-old lawyer who arrived at the protest on February 19.

He was crouching behind a makeshift shield when a bullet pierced it and entered his chest. He fell to the ground, rolling on his back and clutching his knee. A tall construction worker in camouflage fatigues rushed forward to help Dmytriv. He was Andriy Dyhdalovych, a 40-year-old long-time protester who had marched with Ukrainian veterans of the Afghanistan campaign.

He was shot in the shoulder and died that day in the street. The third victim was Yuriy Parashchuk, who was 47. He was shot about 15 minutes later on the same street, in the front of the head. None of the three men were armed.

THE Maidan reconstruction is a product of its time, an age when high-quality video can be recorded from any street corner or citizen’s hand, and when gigabytes of data can easily circulate among experts in Pittsburgh, Brooklyn, and Kiev.

The archive the lawyers handed over was huge — a folder of more than 400 videos with different naming conventions and file types.

“This was as robust a data set as we’ve ever had the opportunity to work with,” says Samuels.

Nefertari had spent months going through the footage on her computer, trying to synchronise the videos and wrangle them together. The Centre for Human Rights Science subsequently in Pittsburgh subsequently tried to automate this process, using an AI algorithm that could quickly analyse each file’s audio component and propose possible matches.

“Machine learning is not a magic bullet,” says Jay Aronson, the centre’s director.

“It’s just a tool. You still need a lot of human judgment.”

Once the videos were assembled into a database, SITU narrowed them down to a smaller number — fewer than 20 — that were relevant to the cases. Then, working with collaborators on the ground in Ukraine, they built a virtual model of Instytutska Street. The first version, created using existing site surveys, wasn’t sufficiently detailed, so Nefertari organised surveyors with laser scanners who could capture details at what Samuels calls the “sub-centimetre level”.

The scan was so fine that it documented paving-stone patterns and individual leaves of foliage. The granularity was necessary to get a fix on not only the victims and the supposed shooters but also on the people holding the cameras. The 70-gigabyte master layout, known as a point cloud, was stitched together from 40 individual scans of the street and its environs. When a bullet breaks the sound barrier, it produces a small sonic boom that registers as an audible crack. For people positioned downrange, the crack arrives a fraction of a second before the thump or blast of the weapon actually firing.

In the Maidan case, SITU enlisted a ballistics expert to measure the time that elapsed between the cracks and the thumps. The time difference yielded a maximum and minimum distance between the shooter and the camera, which SITU rendered as a doughnut-shaped “area of interest”.

Using the video archive, SITU positioned the victims’ bodies within the virtual space of the point cloud. Autopsy reports noted the locations of entry and exit wounds, which were joined by thin red lines and extended, with a five-degree margin of error, forward into space.

Viewed from overhead, these five-degree cones trim the doughnut-shaped area of interest down into a narrow segment. In the first two cases, the segments overlapped with a position that the Berkut were defending behind a barricade; in the third, the segment overlapped with a position behind a line of supply trucks.

A crucial piece of additional footage obtained by Nefertari arrived in SITU’s office more than a year into the case. Taken from a surveillance camera at the Ukrainian National Bank, it clearly shows the Berkut positioned behind their front line. In all three cases, individual officers can be seen aiming and firing their rifles during the moments leading to the victims’ deaths.

The Ukrainian court could accept the findings only if they were entered into evidence as a physical object. So SITU loaded its multimedia presentation onto an Intel minicomputer and shipped it to Kiev.

AT THE centre of the exhibit are the three cases, with SITU’s analysis presented in three silent videos, each about five minutes in length. Key snippets of live video are laid on top of the virtual space as footnotes.

The analysis arrives in a clean and linear presentation, and yet SITU manages to show its work, letting viewers see the crack-thump audio files and the frames of video used to establish the positions of the bodies.

“It’s pretty banal,” Samuels says.

“What we’re trying to do is take something that’s already pretty obvious and make it abundantly clear, by putting it into space.

“The criminal court has never admitted evidence of such technological complexity,” Yatsenko told me.

“It’s a milestone.”

There are more than 100 witnesses left to interview in the February 20 case. It could be more than a year before the court reaches a verdict.

Nefertari, meanwhile, continues to work on her own project, an even more ambitious reconstruction of the entire Maidan uprising, with all victims included. “I want people in the future to know the truth. I don’t want rumour turned into facts.”

Mattathias Schwartz is a contributing writer for the magazine who lives in Washington. © 2018 The New York Times