Is there any way to do sec dep CC hold on direct bookings but do a refundable damage deposit on VRBO API only?

Yes, to do that you'd uncheck the "schedule a security deposit using OwnerRez property rules" option on the HomeAway channel settings, so it doesn't follow the normal rules.

Then create a surcharge with the word "Refundable" in it (so it triggers a refundable deposit on VRBO) and set the listing site applicability so it's only applicable to HomeAway and VRBO.