One thing I should clear up up-front: While sometimes classified together, Sharks (and the related Rays) are not really fish. They in fact are less related to fish than we humans are. For instance, they don't have bones like we and fish do. There are numerous other physical differences too, but more importantly for your question, their skin is also very different. It is a material similar to their teeth in fact (having skin teeth doesn't seem so weird, once you realize that sharks shed their teeth). This is why the rough skin of a Chondrichth might be useful as an abrasive, where the scales of fish typically are not.
I found a lot of references to Polynesians using it as sandpaper. However, they were not a literate people prior to the colonial era, so I can't find anything prior to colonial times documenting this. It seems possible it was an old practice though. Their seafaring culture dates back roughly 2 millennium, and they certainly had plentiful access to sharks.
Another interesting case is the Chumash of California. A huge amount of their diet was sharks and rays, and they report a cultural practice of using the skins to smooth their seacraft and other woodwork. They certainly would have had plenty of it lying around, as archeologists found it to be their #2 source of protean (sardines were #1). This dietary practice appears to have gone back at least 1000 years.