Upon his return home, Marvin rejoined his childhood friend Reese Palmer in a group called "The Moonglows," which also featured future music producer Harvey Fuqua. The group sang background on records for Chuck Berry and Etta James, before breaking up in 1960, when Fuqua took Marvin to Detroit and introduced him to the Motown family. Upon completion of his solo record deal, Marvin changed his last name to "Gaye," rather than the original spelling of "Gay." Marvin believed that it sounded more professional, although it is widely believed that he wanted to separate himself from his father, and the homosexual connotations associated with that spelling, which is probably true, given his hatred for his father's obsession with women's lingerie. Many believe it was a tribute to Sam Cooke, who had also added an "e" to his last name, a claim that Marvin denied. Marvin's stubbornness did not help him blend in with the Motown label's artist development nor its creator, Berry Gordy. Marvin's romantic relationship with Gordy's daughter, Anna, did not help his situation out either. After Marvin's first solo album "The Many Moods of Marvin Gaye," a compilation of Broadway standards and jazz tunes, failed, he wrote about himself in his breakout hit "Stubborn Kind of Fellow." Marvin charted 3 more top 40 singles in 1963, and "How Sweet It Is (To Be Loved By You)" was released in 1964, becoming Gaye's most popular record. His clean cut vocals and boyish good looks were a hit with the female audiences, and Marvin became a favorite performer among his exceptional Motown counterparts.

A successful pairing with young singer Tammi Terrell proved to launch Marvin Gaye into the stratosphere, as the pair made memorable songs like "Ain't No Mountain High Enough," and "Your Precious Love." The chemistry between the two was undeniable, and rumors of a romantic involvement between them began to circulate, infuriating Anna Gordy, whom Marvin had married in 1961. These proved to be just rumors, however, as Marvin was married and Tammi was dating Temptations lead singer David Ruffin. The two singers were performing at a concert at Hampden-Sydney College in Virginia, when Tammi suddenly collapsed during the show, falling into Marvin's arms. Marvin was devastated as Tammi was diagnosed with a malignant brain tumor and was not expected to recover. Motown released another album featuring the two despite Tammi's condition, and this featured the Ashford and Simpson-penned hits "You're All I Need to Get By," and "Ain't Nothing Like The Real Thing Baby."He also issued his biggest solo single, a Norman Whitfield tune called "I Heard It Through The Grapevine," in 1968, and it became Motown's biggest selling record to date. Distraught by Tammi's failing health, Marvin did not revel in the song's success; in fact he downplayed it, even though it had become #1 on the Billboard R&B Charts. "Too Busy Thinkin 'Bout My Baby" also went to #1 in 1970, but in March of that year, Tammi lost her battle with the brain tumor, and passed away at the tender age of 24. Marvin was crushed, and fell into a deep depression and seclusion for the next two years, and vowed to never replace her as a partner, a promise which he kept throughout his life.