Vigoda was born in Brooklyn, New York, to Lena (Moses) and Samuel Vigoda, both Russian Jewish immigrants. His father was a tailor on the Lower East Side. He made his first stage appearance at the age of 17 and plodded away in small theater shows for over 20 years. For the majority of film-goers, Vigoda first came to prominence in The Godfather (1972) as the double-crossing Tessio, pleading with Robert Duvall to get him off the hook "for old times' sake." He also appeared in its sequel.

Vigoda had roles in a few nondescript TV films before landing the plum part of Sgt. Phil Fish on the brilliant sitcom Barney Miller (1974). Perhaps his best known role, Sgt. Fish proved popular enough to be spun off to his own (short-lived) series Fish (1977).

With his long face and unusual looks, Vigoda remained in high demand in mafioso-type roles, and for a while in the mid-1980s, he was mistakenly believed to have been dead, leading producers to remark, "I need an Abe Vigoda type actor," not realizing Vigoda was still alive and well. The 1990s and beyond became busy again for him, with appearances in North (1994), The Misery Brothers (1995), A Brooklyn State of Mind (1998), and Crime Spree (2003). He continued acting into his 90s, surprising audiences with his entertaining style.

Abe Vigoda died in his sleep on January 26, 2016 in Woodland Park, New Jersey.