Decoherence of ensembles of nitrogen-vacancy centers in diamond