Rayleigh is a former member of the Roger Pirates and the trainer of Monkey D Luffy. He was Roger’s right-hand man. Also, he is one of the few characters who can use all three types of Haki superbly well. He also has excellent swordsmanship skills as well. In my opinion, even though he’s old now but still could be on the same level as Yonko. Rayleigh was class apart, and being able to hold Kizaru easily and wound him makes him very powerful.
5. Gol D Roger
Roger is the former king of the pirates. He is the one who has found Raftel and started a new age of pirates by telling everyone that he’s left a treasure behind just before he was executed. He is remembered as the strongest ever pirate to live so he’s easily well above the level of all 4 current Yonko. The one person to be able to match Gol D Roger in terms of strength was Whitebeard, Edward Newgate. Roger was Whitebeard’s rival, and one of the three rulers of the seas before the Yonko era.