Shichibukai is a title given to a pirate that owns his allegiance to the World Government. There are only seven Shichibukai seats in One Piece. These pirates can do anything they like but if their action cause any trouble to the World Government they can be stripped down from their position. There are some people at certain positions who can revoke the position of a Shichibukai. So, I have made a list of people that can revoke the title of a Shichibukai’s position.
Gorosei are the head of the World Government and control everything in One Piece. These five people can chose and revoke the position of Shichibukai as they like.
If they find any them not up to their hopes, the Gorosei can take them out. Moreover, the Goresei don’t like to keep weak pirates as Shichibukai. As we saw how Doflamingo tormented Moria and said that they were orders from above. So, it’s pretty clear who would have done that.