A Fusion Drive is the marriage of a HDD and a SSD.
So at this point has your SSD part is failed or the HDD?
If the SSD failed it would make sense to replace it but I wouldn’t replace it with another small SSD and then redo the Fusion Drive. Instead I would setup standalone dual drive config where you use the SSD as your boot drive and leave your HDD as an independent data drive. While its a bit of work its worth it! iMac Intel 27" EMC 2639 Blade SSD Replacement It will give you the performance you really want. Here’s the SSD you’ll need Blade SSD.
If your HDD died then replacing it with a SSD while doable, can’t be joined to your blade SSD that was part of your Fusion Drive. The Faster PCIe interface of the blade SSD is ideal for your boot drive and holding your cache & paging files setup as independent dual drives.