Fusion drives are made up of two physical drives: a HD and a SSD. The idea here is by 'fusing' the two you gain the benefits of both the SSD's speed and the deep storage of a traditional disk drive.
I'm surprised you feel its too slow. It's true it does take a few accesses for the OS to learn what files need to be held in SSD Vs HD. But on the most part it does well. Some people think the SSD is too small to be as effective. Frankly, I think its more on what you're trying to work with.
As an example large video, CAD or DB files won't get placed on the SSD so they will be slow in loading as they will stay on the HD. Some smaller files will get placed on the SSD but these will need to be access multiple times to gain the speed benefit. So if you are accessing different files every few minutes other than the OS files the most will stay on the HD.
So. what to do here to gain some speed?
First altering your systems internals is limited as the SSD is mounted on the back side of the logic board (M.2 blade drive) so you can't easily remove it or upgrade it. Presently, there are no 3rd party SSD's for the iMac's. So the next issue is altering the HD. Sure you could swap it out for a SSD but I don't think thats your best option here.
First you may want to check your Fusion drive to see if the drive(s) are in good shape. You'll need to use an external bootable disk to boot up your system. Then use Disk Utility on the external drive to fix the permissions and disk of the Fusion drive. See if that fixes things (half of the time it does!).
If that doesn't improve things then you'll need to get an external RAID setup. This could be disk or SSD's. Connecting it via the Thunderbolt port. Now the tough part. re-thinking how you use and store your OS, apps & data files. You want to have your OS and Apps on the Fusion drive and all of your data on the external RAID drive set. So now the OS and the App files most accessed are held on SSD on the Fusion, but the larger data files are on the RAID'ed drive set.
This is how we setup our video editing stations and CAD DB server (both handle large files).