How does it work ?
Each device (DRBD provides more than one of these devices) has a state, which can be ‘primary’ or ‘secondary’. On the node with the primary device the application is supposed to run and to access the device (/dev/drbdX). Every write is sent to the local ‘lower level block device’ and to the node with the device in ‘secondary’ state. The secondary device simply writes the data to its lower level block device. Reads are always carried out locally.
If the primary node fails, heartbeat is switching the secondary device into primary state and starts the application there. (If you are using it with a non-journaling FS this involves running fsck)
If the failed node comes up again, it is a new secondary node and has to synchronise its content to the primary. This, of course, will happen whithout interruption of service in the background.
And, of course, we only will resynchronize those parts of the device that actually have been changed. DRBD has always done intelligent resynchronization when possible. Starting with the DBRD-0.7 series, you can define an “active set” of a certain size. This makes it possible to have a total resync time of 1–3 min, regardless of device size (currently up to 4TB), even after a hard crash of an active node.