Past studies of the empirical relationship between immigration and crime during the first major wave of immigration have focused on violent crime in cities and have relied on data with serious limitations regarding nativity information. We analyze administrative data from Pennsylvania prisons, with high-quality information on nativity and demographic characteristics. The latter allow us to construct incarceration rates for detailed population groups using U.S. Census data. The raw gap in incarceration rates for the foreign and native born is large, in accord with the extremely high concern at the time about immigrant criminality. But adjusting for age and gender greatly narrows that observed gap. Particularly striking are the urban/rural differences. Immigrants were concentrated in large cities where reported crime rates were higher. However, within rural counties, the foreign born had much higher incarceration rates than the native born. The interaction of nativity with urban residence explains much of the observed aggregate differentials in incarceration rates. Finally, we find that the foreign born, especially the Irish, consistently have higher incarceration rates for violent crimes, but from 1850 to 1860 the natives largely closed the gap with the foreign born for property offenses.