The oldest son of Fredrick Billings, Parmly Billings was the only member of the Billings' family to actually live in the city of Billings. Parmly moved to Billings in 1885 along with his cousin Edward Bailey to manage his father's business interests. Along with managing Minnesota and Montana Land Improvement affairs, the two started the city's second bank, the Bailey and Billings Bank.

Both father and son had hoped that the same opportunities that had propelled the elder Billings in California would present themselves to Parmly in Billings. But Parmly seemed to lack the business sense that had helped his father gain fame and fortune in San Francisco. Parmly did not enjoy the same success that his father had and sadly developed kidney failure and died in 1888. By the time that Fredrick died in 1890, he had sold off all of the family's holdings in Billings, which included the land and development company and bank. Parmly's brother, Fredrick, Jr., donated almost $30,000 to the city of Billings to build a public library in memory to Parmly. When Fredrick, Jr., died in 1913, another $10,000 was donated to the Parmly Billings library. An additional $20,000 was donated by Elizabeth Billings to build a library wing in memory of brother Fredrick, Jr.

In the 1960's, the library had outgrown the building and moved to a more modern building. The original building now houses the Billings area's premier history museum, the Western Heritage Center.