Leslie was born in 1917 to Isidore and Bella Starr. Isidore was originally from Poland. He worked as a cigar and cigarette maker based in the Corn Exchange in Manchester city centre. Leslie lost both his parents when he was a teenager. Unlike his father, Leslie worked as a cloth cutter in the raincoat industry and was an active member of the Garment Workers Union. He married Mary Flanagan, who also worked in the clothing industry, as a machinist, in 1939.