DARPA combines human brains and 120-megapixel cameras to create the ultimate military threat detection system
Sebastian Anthony| Extremetech
After more than four years of research, DARPA has created a system that successfully combines soldiers, EEG brainwave scanners, 120-megapixel cameras, and multiple computers running cognitive visual processing algorithms into a cybernetic hivemind. Called the Cognitive Technology Threat Warning System (CT2WS), it will be used in a combat setting to significantly improve the US Army’s threat detection capabilities.
There are two discrete parts to the system: The 120-megapixel camera, which is tripod-mounted and looks over the battlefield (pictured below); and the computer system, where a soldier sits in front of a computer monitor with an EEG strapped to his head (pictured above). Images from the camera are fed into the computer system, which runs cognitive visual processing algorithms to detect possible threats (enemy combatants, sniper nests, IEDs). These possible threats are then shown to a soldier whose brain then works out if they’re real threats — or a false alarm (a tree branch, a shadow thrown by an overheard bird).
The soldier is linked into the computer system via an EEG (electroencephalogram) brain-computer interface that continually scans his brains for P300 responses. As we’ve discussed previously (see: Hackers backdoor the human brain), a P300 response is triggered when your brain recognizes something important. This might be a face of someone you know or the glint of a sniper scope — it doesn’t matter. P300 responses are very reliable and can even be triggered subconsciously.
In short, CT2WS taps the human brain’s unsurpassed ability to recognize objects. In testing, the 120-megapixel camera, combined with the computer vision algorithms, generated 810 false alarms per hour; with a human operator strapped into the EEG, that drops down to just five false alarms per hour. The human brain is surprisingly fast, too: According to DARPA, CT2WS display 10 images per second to the human operator — and yet that doesn’t seem to affect accuracy. The total overall accuracy of the system is 91% — but that will improve as DARPA moves beyond the prototype stage.
Moving forward, once our computers are suitably power efficient (or there’s a breakthrough in battery efficiency), the ultimate goal is to create binoculars or head-up displays (HUD) with threat detection technology built in. It’s very tiring for a soldier to be constantly on the lookout for threats — but such a system could monitor the surroundings, and then flash up images of potential threats for the soldier to act upon, significantly lowering his workload. With a large enough sensor and the right lenses, such a system could allow the soldier to see for miles in every direction.