Yep, saw some Alliance warrior taking my stone-blocks yesterday and that was the initiator of a nice OpenPvP battle since there were a lot of hordes and allys around that got involved. (I'm a Dotlock, Corruption heals better when cast on many enemys

)
Definately not a effective way to get it done fast, but if you like to play like an *******, do so.
Also don't do the Flower-Thing at rather mid-populated servers... If you are unlucky you may wait quite some time...
But if you got a friend with the same quest (There are 3 possible quests) let him get the flowers while you fly ahead or farm some stuff.