Many different rail lines were build in the western United States from the 1850s onwards. Was construction ever delayed/canceled due to difficulty in finding workers, or due to workers leaving the job?
I was surprised to find out that, yes there were troubles with labor shortages. From Public Broadcasting Station "American Experience" article on the "Workers of the Central Pacific Railroad",
I'd recommend either purchasing the DVD the series has on the transcontinental railroad, finding it at your local library, or at least reading the article. I imagine that given your interest in the subject you would really appreciate it.