Born in Staffordshire, David Jackson (as he was then known) grew up in Portsmouth in the Paulsgrove area and he attended the City of Portsmouth Boys School. His parents later moved to Gosport when he was a teenager. He initially learned to play the violin but soon switched to piano and prevailed on his dad to install one in the hall of their Paulsgrove council house. From the age of 16, he played in bars, and won a scholarship to study musical composition at London's Royal Academy of Music. Jackson's first band, in Gosport, was Edward Bear (not to be confused with the 1970s Canadian band fronted by Larry Evoy). The band was later renamed Edwin Bear and later Arms and Legs, but dissolved in 1976 after two unsuccessful singles.  Although he was still known as David Jackson while in Arms & Legs, it was around this time that Jackson picked up the nickname "Joe", based on his perceived resemblance to the puppet character Joe 90.  He then spent some time in the cabaret circuit to make money to record his own demos.  However, whilst at school he had been known as 'Joe' to his schoolmates and teachers from 1972 onwards. In 1978, a record producer heard his tape, and got him signed to A&M Records.  The album Look Sharp! was recorded straight away, and was released in 1979, quickly followed by I'm the Man (also 1979) and Beat Crazy in 1980. He also collaborated with Lincoln Thompson in reggae crossover.

In 1981, Jackson produced an album for the British power pop group The Keys. The Keys Album was the group's only LP.

The Joe Jackson Band toured extensively. After the break-up of the band, Jackson took a break and recorded an album of old-style swing and blues tunes, Jumpin' Jive, featuring songs of Cab Calloway, Lester Young, Glenn Miller, and most prominently, Louis Jordan. The album, and associated single release, was credited to Joe Jackson's Jumpin' Jive.

Jackson's 1982 album Night and Day paid tribute to the wit and style of Cole Porter (and indirectly to New York City).  Night and Day was Jackson's only studio album to reach either the United States or UK Top 10, peaking at #4 (US) and at #3 (UK), and the cuts "Steppin' Out" and "Breaking Us In Two" were chart hits. The tracks "Real Men" and "A Slow Song" have pointed obliquely to the city's early 1980s gay culture. Jackson lived in New York for the next 20 years, incorporating the sound of the city into his music throughout the 1980s and beyond. Almost two years later, Jackson recorded the UK #14 album Body and Soul, also heavily influenced by pop and jazz standards and salsa, showcasing the US #15 hit single "You Can't Get What You Want (Till You Know What You Want)".

In 1986, he collaborated with Suzanne Vega on the single "Left of Center" from Pretty in Pink's soundtrack (with Vega singing and Jackson playing piano).

Jackson followed with the live album Big World, a three-sided double record – the fourth side consisted of a single centring groove and a label stating "there is no music on this side". The instrumental "Will Power" set the stage for things to come later, but before he left pop behind, he put out two more albums, Blaze of Glory and Laughter & Lust.

For some years he drifted away from the pop style, going on to be signed by Sony Classical in 1997.  They released his Symphony No. 1 in 1999, for which he received a Grammy Award for Best Pop Instrumental Album in 2001.

In 1995, Joe Jackson contributed his version of "Statue of Liberty" on a tribute album to the English band XTC called "Testimonial Dinner" (released 1998).

In 2003, he reunited his original quartet[2] for the album Volume 4, and a lengthy tour. As before, the quartet consisted of Jackson, Graham Maby, Dave Houghton and Gary Sanford. In 2004, Jackson performed a cover of Pulp's "Common People", with William Shatner for Shatner's album Has Been.

Jackson toured 45 US and European cities in 2005 with Todd Rundgren and the string quartet Ethel,  appearing on Late Night with Conan O'Brien performing their collaborative cover version of "While My Guitar Gently Weeps".

Thereafter, he embarked on a short tour in a piano-bass-drums trio format. He toured Europe in Spring 2007, again in a trio format.  Jackson's album, Rain was released by Rykodisc on 28 January 2008 in the UK and one day later in the US.[6] The album included a CD and a bonus DVD containing over 40 minutes of material, including concert and behind-the-scenes footage and interviews. Jackson performed two UK shows in spring 2008, followed by a full UK tour