Two brains, one fighter
For the third Space-ML experiment, I trained AI fighter pilots to navigate an asteroid arena, create repeatable attack runs and aim a limited turret at moving opponents. The difficult part was not teaching one enormous brain everything at once. It was dividing the problem into two policies that could be trained independently and still cooperate inside the same ship.
Why local sensing was no longer enough
The earlier navigation and racing experiments could rely heavily on nearby sensing. Combat made that approach much less convincing. A fighter does not only need to avoid the asteroid directly in front of it; it needs a usable route through the whole field while chasing a target that continues to move.
I added a three-dimensional voxel navigation layer to give the agent a coarse map of navigable space. The field describes which regions are available and which direction leads around the obstacles. It does not fly the ship. The policy still decides how to turn, thrust, brake and recover while combining that route with its local sensing.
Close to the target, the objective changes. The pilot is guided toward an attack shell around the opponent instead of being rewarded for ramming its centre. After the attack pass, the route guidance hands control to an egress objective that asks the ship to create distance before turning back.
Training the pilot: Combat Engage
Combat Engage is the movement policy and the foundation of the setup. Its job is to build an actual attack run: approach the moving target through the asteroid field, arrive with useful speed and nose alignment, cross the attack window, break away and return for another pass.
I trained that behavior as a sequence rather than one vague objective. Route progress matters during the approach. Aim quality and speed matter near the attack window. Radial progress matters during egress. Completing the pass and returning closes one cycle, and the later curriculum requires the pilot to repeat it.
The graph only shows the runs that survived long enough to become useful. Behind it are dozens of experiments, discarded reward profiles and policies that discovered shortcuts, collisions or stable-looking behavior that did not complete the real task. A brief peak was not enough; I needed a checkpoint whose success, pass quality, speed and safety measurements agreed with what I could see in the arena.
Training the gunner: Combat Shooting
Once the attack-run policy was usable, I kept it fixed and trained a second brain that controls only the turret and trigger. The pilot was frozen as a policy, not as a ship: it continued to fly real attack cycles while the new brain learned when and where to fire.
The gunner observes the ballistic lead solution, turret state, range, line of sight, ammunition and nearby friendly traffic. It receives three continuous controls: horizontal aim, vertical aim and fire intent. The curriculum gradually tightens the accepted target area and increases the number of hits needed for success.
A normal miss has no direct punishment because that can teach a cautious policy to stop shooting altogether. Bad decisions are treated differently. Firing outside the useful envelope, during the pilot's egress, without a clear shot or through a friendly ship is explicitly discouraged. Limited ammunition and the success quota still make wasteful firing expensive without turning every miss into fear of the trigger.
The two brains never need to know how the other one was trained. The pilot creates the attack window; the gunner learns to recognise and use it.
Putting both brains into one ship
In the final demo, every AI fighter runs both exported policies. Combat Engage controls the engines, rotation and brake. Combat Shooting controls the turret and trigger. They share the same physical situation and target, but each owns a deliberately smaller action problem.
Observer mode launches two teams from opposing motherships into the same asteroid arena. New fighters enter in waves, acquire targets, navigate the voxel field and keep fighting while the scoreboard follows both sides. The default setup can grow to twenty-five trained fighters per team.
Challenge mode replaces one side with a player-controlled fighter. The opponent count can be set from one to twenty-five, which turns the same experiment into a direct test: the trained agents are no longer clips on a graph, but opponents you can fly against.