Perusing this gallery of candid photos from 1890s Oslo, Norway, by Carl Størmer (1872-1957), again I found such a curious image, yet was unable to verify the existence of any portable music player back then.
According to Wikipedia,
Over several years starting in 1894 the Italian inventor Guglielmo Marconi built the first complete, commercially successful wireless telegraphy system based on airborne Hertzian waves (radio transmission).
That device looks awfully small, though; could it be a brush? Was brushing your hair in public without a mirror common then?
I was only able to find others with the same question: What are those things?