Turn a fly into an FPV drone. How a Ukrainian developer taught an insect's brain to control a UAV instead of an internet meme

Digitizing the fly brain

The scientific world recently witnessed a monumental breakthrough in neurobiology. A team at Google Research, in collaboration with HHMI’s Janelia Research Campus, accomplished what once seemed like science fiction: they completely recreated the brain and central nervous system of an adult male fruit fly (Drosophila melanogaster).

This complete wiring diagram, or connectome, contains more than 166,000 neurons and nearly 125 million synaptic connections. To create such a map, the scientists had to slice the insect's brain into 134,000 slices just 8 nanometers thick, scan each one with an electron microscope, and then use sophisticated artificial intelligence algorithms to combine millions of two-dimensional images into a detailed 3D model.

The published dataset contains information on 11,691 types of neural cells, allowing us to trace the pathways from visual receptors to motor neurons, which are responsible for movement. Suddenly, anyone with access to the Internet had the ability to download a complete model of the biological brain and use it for their own purposes. Rather than being limited to purely academic research, the Internet community turned this scientific milestone into a platform for digital experiments, placing the insect's "mind" in a wide variety of virtual environments.

The birth of fly slop

After the map was published, the network was really flooded with the so-called "fly slop". Enthusiasts began to connect connectomes to simulators, forcing a virtual fruit fly to play Minecraft, race in Track Mania, and even complete tracks in Beat Saber. One developer connected brain activity to controls in the game Doom, where each frame stimulated sensory neurons, and receiving damage activated dopamine cells as reinforcement. Others went even further: they taught connectomes to successfully trade bitcoin on the exchange with a starting capital of $ 100, or to sort emails into six categories with an accuracy of about 80%.

"Memes with flies flooded the feed, I probably didn't even see most of them, because they appear in the hundreds every day, primarily because anyone can formulate a task for AI and in a few minutes generate a visualization in which a fly drives a forklift or chases a banana. Probably the first example that came to my mind was "A fly plays Beat Saber", it was the most memorable.

The case with DOOM can be considered a favorite not only because I love watching the imagination of people who launch the game on electric toothbrushes or electronic pregnancy tests, but also because, if you believe the developer, he followed approximately the same path as me,” says Dmytro, a database developer from Kharkiv with over ten years of experience and a specialized education in Artificial Intelligence Systems.

However, behind these viral cases lies an important technical nuance. As Dmitry explained, in most such entertainment projects, the biological essence of the connectome is completely leveled, since the developers use the map only as a static framework for an artificial neural network, where all the weights of the connections are set to the same, and then retrained by an algorithm to perform specific tasks (for example, pressing mouse buttons). In such cases, the fly technically does not "control" anything consciously, and its visual system and internal experience are not involved at all - it is just a mathematical abstraction stretched over a biological scheme.

"There is currently an unannounced trending online competition on the Internet: who will teach not just any neural network, but a network based on a fly's connectome to do the most absurd and ridiculous things. In these projects, there is usually absolutely nothing left of the fly itself ; the connectome is used as a mock-up of static connections of a neural network, which is completely retrained by the user (the weights of the connections are changed beyond recognition), due to which the biological meaning of specific neural pathways is erased and rewritten. In the same cryptotrading, there is nothing that the fly's brain could "understand". Sometimes the fly's connectome even serves as a bandwidth layer for another specialized neural network that does all the work... Any trained synthetic network without a biological basis would give a similar result," explains Dmytro.

Turn a fly into an FPV drone

Observing this trend, Dmitry decided to take a radically different, much more complex path. Instead of retraining a neural network and turning it into a classic machine learning algorithm, he set a goal to use the insect's biological reflexes, honed over millions of years of evolution, to control an FPV drone. In nature, a fly is ideally adapted to flight: its visual systems instantly react to threats and obstacles, transmitting signals to motor neurons. Dmitry decided to place the connectome in the virtual space of the simulator and bring its vision stimulation closer to natural conditions.

"Probably when I "crawled into the comments" to explain how all the aforementioned projects actually worked, I looked at visualizations of neural activations, and curiosity and long-forgotten education took their toll. What if we didn't retrain the connectome at all or distort it, but just placed it in a virtual space that would stimulate the fly's vision in a way that approximates how it happens in nature, and adapt motor reflexes to controlling a drone?", shared Dmytro.

The key element of the system was the binding of control to specific neurons of the visual tract - LPLC2 and LC4. These cells play the role of proximity detectors: they are sharply activated when an object appears in the field of view that is rapidly increasing in size (for example, a tree in the path of the drone). By reading signals from these neurons, the system modulates the control axes, automatically forcing the "fly" to perform roll and yaw maneuvers, bypassing obstacles in the simulation. When the crown of a tree covers the right field of view, the neural circuit natively reproduces the escape reflex, generating a sharp impulse to dodge to the left.

"The fly has neural connections that automatically respond to contrasting objects that rapidly increase in the field of view (LPLC2, LC4), and then, in the sum of their activations, "trigger" the following neurons in the ventral nerve chain, leading to motor responses in the form of take-off or evasion of threats in flight.

Other brain structures force Drosophila to stabilize its field of vision and flight. Track moving objects no more than one eye facet wide (courtship behaviors, mate search).

"If there is anything the connectome can do "natively", it is exactly what a fly does. It seemed to me that using it specifically to control a flying drone would be at least...elegant. After publishing the first results, I remembered one comment I received in Japanese, which contained a laconic approval of "in this example, the attitude towards the fly is the least unworthy", against the backdrop of some discussions about the morality of using a model of the brain of a living being for unnatural things," said Dmytro.

The technical implementation of this concept required a deep understanding of the anatomy of the fruit fly. The developer modeled the insect's complex eye, consisting of 1,769 facets, based on a map of the eye columns from the Google database. The video stream from the drone simulator is decomposed into left and right parts, pixelated and fed directly to virtual photoreceptors, after which the signal independently propagates through the optic lobes of the brain without any artificial intervention. The drone is controlled exclusively by descending motor neurons, which transmit commands to the wings and neck in response to the optic flow.

To stabilize the flight, Dmitry also simulated ocelli — three simple eyes on the crown of a fly that respond to the level of illumination of the horizon. When the drone crashes, these sensors activate ocellar neurons, which transmit an impulse to the cervical motor neurons (CvN5–7), triggering the body-alignment reflex. A separate simple neural network acts as a “reader,” summing these impulses into four channels understandable to the UAV: ​​roll, pitch, yaw, and throttle. Impressively, this entire massive brain simulation of 166,000 neurons works without expensive servers and runs on a regular home computer with an Intel Core i7-3770K processor and 16 GB of RAM, and to speed up the code writing, the developer partially used the Claude Code AI assistant.

“So far, I am additionally stimulating the value of the required height to keep the fly at the level of trees and buildings and observe navigation with obstacles in the simulator at this height. This is the most experimental part, which is constantly changing and relearning by me by adding new neurons to the simulation, into which I still plan to integrate signals from the accelerometer/lidar/altimeter and stimulate the fly's antennae and other organs that measure angular accelerations. It will be very interesting to see what happens if the fly also receives synthetic sensations of space, and to observe whether I can decode anything from the signals received from sensory neurons in the connectome,” Dmitry noted.

Despite the successful first flights, working with a pure connectome hides serious challenges, the main one being the complexity of digitally simulating subtle biological and chemical processes. For example, LC11 neurons in nature are responsible for responding to small moving objects, ignoring large static stripes, such as tree trunks. However, in a simplified computer model, achieving such correct optical filtering is extremely difficult due to the lack of a correct inhibition mechanism.

Without physically modeling the spatial arrangement of dendrites and synapses, and without taking into account chemical delays in the transmission of neurotransmitters, some cells do not behave as in a real fly. As Dmitry noted, this fundamental problem proves that a complete simulation of the brain on ordinary equipment without specialized acceleration remains an incredibly difficult task. Nevertheless, the fact that even a "truncated" biological circuit is capable of natively generating commands to avoid obstacles is a huge achievement of modern science.

"What impressed me most was that even in an extremely simplified model, the basic flight reflexes were preserved. The neurons that respond to the approach of an object fire on the correct side, and the path from the ocelli to the descending neurons of the neck really "codes" the tilt of the horizon. It's not an autopilot out of the box yet, but the foundation is laid by nature even in the very circuit of connections, and it doesn't need to be invented. This is the merit of numerous researchers who have spent years reconstructing every neuron and every synapse in this model down to the smallest detail. Without their titanic work, such experiments would not have been possible," Dmitry noted.

The future of digital brains

In addition to their purely research interest, such simulations open up completely new perspectives for the development of autonomous systems, which is especially relevant for the field of Defense Tech. Insects navigate in an extremely complex space without using energy-consuming lidar or GPS navigation, relying exclusively on optical flow analysis. Transferring these evolutionary algorithms to the world of unmanned aviation could revolutionize the development of UAVs.

As Dmitry said, several years ago, scientists from Taiwan began work on creating a microchip inspired by the visual system of Drosophila , which would calculate evasive reflexes directly in the camera's RAM (compute-in-memory). This approach does not require the deployment of "heavy" machine vision neural networks and is incredibly energy-efficient. In the context of creating Ukrainian autonomous interceptor drones or UAVs resistant to electronic warfare, such biological "microcontrollers" implemented on the basis of FPGAs can provide a lightning-fast reflex response to threats, relying solely on specifications that nature has been testing for millions of years.

"For me, this is more of an interesting weekend project, but I believe that some biological principles of navigation or defensive behaviors can be useful for drones in general in certain niches: avoiding obstacles without active sensors, maintaining the flight direction on a pure optical flow without a map. The answer to drone interceptor attacks? Perhaps such systems can exist on FPGAs or as "compute-in-memory" and work autonomously from more complex machine vision tools, at a low, reflex level. The value of insect research here lies in the specifications of their "controllers", calibrated by millions of years of evolution. From them, we can extract the architecture of the reflex we need and transfer it to a small specialized circuit," the Ukrainian developer concluded.

"Without technical specialists, the army begins to crumble." The head of unmanned systems of the Third Army Corps - why 3AK IT workers are needed and how to get there

On the topic

"Without technical specialists, the army begins to crumble." The head of unmanned systems of the Third Army Corps - why 3AK IT workers are needed and how to get there

"I take a real drone and just burn it. It's things like this that form the boundary between a game and a simulation." The story of creating the most hardcore UAV simulator for the front in an interview with the co-founders of Simtech Solutions

On the topic

"I take a real drone and just burn it. It's things like this that form the boundary between a game and a simulation." The story of creating the most hardcore UAV simulator for the front in an interview with the co-founders of Simtech Solutions

"The game gave an understanding of what you will have to work with in reality." Interview with an artilleryman of the Armed Forces of Ukraine about gamification in the Defense Forces

On the topic

"The game gave an understanding of what you will have to work with in reality." Interview with an artilleryman of the Armed Forces of Ukraine about gamification in the Defense Forces

Read the country's main IT news in our Telegram

On the topic

Read the country's main IT news in our Telegram