He is best known for his TV roles such as Steven Kiley on Marcus Welby, M.D.(1969–1976), Peter McDermott on Hotel (1983–1988), John Short on Life in Pieces (2015–2019), and the Narrator on Sweet Tooth and his film roles such as Sgt. Jerome K. Weber in Skyjacked (1972), John Blane in Westworld (1973), General Ralph Landry in Traffic (2000),[2] Jack Barnes in Catch Me If You Can (2002), and Emperor Zurg in the 2022 Toy Story spin-off film Lightyear.

In 1966, he married Jane Cameron Agee, a wildlife activist and aspiring actress at Twentieth Century Fox, 12 days after they first met. The couple had two children, actor Josh Brolin (b. 1968), and Jess (b. 1973). They were divorced in 1984. In 1985, he met actress Jan Smithers on the set of Hotel, and they married in 1986. The couple had a daughter, Molly Elizabeth (b. 1987). Smithers filed for divorce from Brolin in 1995.

In 1996, he met singer and actress Barbra Streisand through a friend, and they married on July 1, 1998. He is stepfather of Streisand's only child, Jason Gould.