Born as Harold Rubin in New York City, he later claimed to be a Jewish orphan who had been raised in a Catholic boys home. In reality he was the son of well-educated Russian and Polish immigrants. He was reared by his pharmacist father and stepmother in Brooklyn.

His first book, Never Love a Stranger (1948), caused controversy with its graphic sexuality. Publisher Pat Knopf reportedly bought Never Love a Stranger because "it was the first time he had ever read a book where on one page you'd have tears and on the next page you'd have a hard-on".

His 1952 novel, A Stone for Danny Fisher, was adapted into a 1958 motion picture King Creole, which starred Elvis Presley.

He would become arguably the world's bestselling author, publishing over 20 books which were translated into 32 languages and sold over 750 million copies. Among his best-known books is The Carpetbaggers, loosely based on the life of Howard Hughes, taking the reader from New York to California, from the prosperity of the aeronautical industry to the glamour of Hollywood.