18 of 20
benchmark missions landed
Camera lands it
the boat's radio can help find it, never lands it
0 images
of the test world used in training

The problem

A boat does not wait. It moves, turns, and rises and falls with every wave. The drone has to find it, catch up, and set down at the one moment the deck is safe.

  • A small, moving target. The landing pad is 60 cm wide, on a boat that keeps driving.
  • Wind and waves. Crosswind pushes the drone while the deck pitches, rolls and heaves.
  • Radio to find, vision to land. The boat's radio gets the drone to the target; nothing from the boat steers the landing itself — the camera and the pad geometry do.

How it works

Two neural networks look at the camera image and report what they see. Control code and the PX4 autopilot do the flying. An earlier network that flew the drone directly landed in none of its test flights.

The drone moves, the camera sees a new image — 25 times per second Camera + drone sensors See neural network Measure pad geometry Decide landing controller Fly PX4 autopilot

One flight, six steps

  1. SearchSweep the camera until a boat appears.
  2. IdentifyFly closer and confirm it is the right boat.
  3. ApproachClose the distance and keep the boat in view.
  4. AlignMatch the deck's speed and heading above the pad.
  5. DescendCome down only while the pad is measured.
  6. Touch downLand, disarm and stay on the pad.

Two vehicles, designed in CAD

Both were modelled part by part in FreeCAD and brought into the physics simulation with their real mass, joints and collision shapes.

The drone

A Holybro X500 V2 quadrotor with a pan-tilt camera, flown by the unmodified PX4 autopilot.

Mass
2.07 kg
Camera
640 × 360, 25 Hz
Autopilot
PX4 1.16
Parts
about €1,760
Drone simulation
AERODOCK boat, exterior render AERODOCK boat with the hull hidden: landing bay, lift, batteries and electronics

The boat — AERODOCK

A drone-carrying boat. Two lids slide open and a platform lifts the landing pad above the deck.

Landing pad
60 × 60 cm
Drive
two electric motors
Physics
buoyancy, drag, waves
Cameras
route and landing bay
Boat simulation

Trained in one world, tested in another

All training happened in MuJoCo, a physics simulator with simple graphics. The realistic simulator is an open-source ocean game we extended. The policy never saw it during training, and lands there anyway.

Training

MuJoCo

Accurate physics, plain rendering. Five kinds of coast, random routes, wind, waves and other boats. PX4 runs in step with the physics.

Test only

Realistic simulator

A realistic ocean rendered in the browser. The same physics, autopilot and policy run locally — no server. You can fly it yourself and hand over to the policy with one key.

1000 Hzphysics
250 Hzautopilot sensors and motors
25 Hzcamera and policy

Fly it yourself

The realistic simulator runs in your browser. Fly the drone, then press L and let the policy find the boat and land. It needs a computer with Chrome or Edge, a capable graphics card and about 150 MB of download.

Watch the video
A recorded flight in the realistic simulator: launch, handover, landing.

Results

Every policy flies the same 20 missions on five kinds of coast: boat speeds up to 1.5 m/s, 4 m/s crosswind, waves, a search from 100 m away and other boats in the scene. A landing counts only if the drone ends up disarmed and stable on the pad.

What the final version does well

It landed 18 of 20: every landing, wind and search mission, with boat speeds up to 1.5 m/s, 4 m/s crosswind and the search from 100 m with other boats in the scene. The boat's radio only helps it find the boat; the camera lands it.

Where it still fails

Two missions with waves. The drone reaches the pad, then hovers just above it waiting for a clean measurement of the pad until its time runs out. Everything so far is simulation; there has been no flight with real hardware.

The teacher uses simulator information a real drone would not have. It produces the training examples and marks the upper limit; it is not a deployable policy. Benchmark data and every flight

How it got here

  1. Step 1

    Build the world

    Drone and boat from CAD, water and wind physics, and the real PX4 autopilot in the loop.

  2. Step 2

    Let a network fly

    A network trained to copy the teacher's commands landed in none of its 12 test flights. Every small mistake changed what it saw next.

  3. Step 3

    Split seeing from flying

    From here on the networks only estimate where the boat and the pad are, and control code flies. The first version leaned on the boat's radio: 7 of 20, and none of the six missions without it.

  4. Step 4

    Camera first

    Trust the camera for position. 11 of 20, but it landed only half of the missions that start with a search.

  5. Step 5

    Add a search head

    A small network that marks every boat and picks out the target, trained while the main network stayed frozen. 18 of 20, camera only.

  6. Step 6

    One final version

    The drone flies to the boat's radio position first and lets the camera confirm it, and the small pad finder from earlier versions is gone. 18 of 20 again, and the same build flies in the realistic simulator.