That is absolutely true.
Typically, the wholesaler or retailer that is claiming credit from the publisher for the return of unsold books returns the covers to the publisher, in lieu of returning the entire copies (since otherwise the cost of return postage would be a very substantial percentage of the return credit they would earn).
The publisher then refunds the customer's purchase price. Those customers who have the privilege of returning covers in lieu of whole copies are expected to destroy the actual books, since if they sell the actual books they have defrauded the publisher by obtaining the credits for the "return" of the books.
Sometimes the retailer doesn't care about fraud; sometimes the retailer brings the books to a facility that promises to destroy them, but instead sells them to a jobber who then resells them... that is why you sometimes see coverless books on sidewalks or, sadly, at Restoration Hardware.