Unable to cancel Deliver Remainder on a Sales order and Purchase Order : The quantity cannot be reduced

I am trying to cancel Back Orders on a Sales Orders and Purchase Orders but i get an error message that says : 

The quantity cannot be reduced. The number of stock transactions on order is too low because the quantity or part of it is referenced by an output order or a works order or is marked against other transactions.

How do i cancel the Deliver remainder?

Thank you in advance:-)