When you convert a Quote to a Project new entity is created. It means that the Quote and Project both exist in the system. I'd suggest canceling the Project, the Quote will remain as approved. After conversion (approval), it's not possible to reject the quote. You can leave an internal note on it that it was approved by mistake.
Option number 2 is deleting the newly converted project, but in a situation where the Quote is approved system will have to delete both, Project and the associated Quote.
I think it'd be helpful to be able to delete just the project and not the quote (and revert it to "not-approved") in situations like this. Perhaps something to be only available to Admins or the super-user.
Customer support service by UserEcho