The film portrays two conflicts that take place around Flagstone, a fictional town in the American Old West: a land battle related to construction of a railroad, and a mission of vengeance against a cold-blooded killer. The main storyline revolves around a struggle for Sweetwater, a piece of land near Flagstone containing the region's only water source. The land was bought by Brett McBain, who foresaw that the railroad would have to pass through that area to provide water for the steam locomotives. When railroad tycoon Morton learns of this, he sends his hired gun Frank to simply intimidate McBain to move off the land, but Frank instead kills McBain and his three children, planting evidence on the scene to frame the bandit Cheyenne and his gang. By the time McBain's new bride, Jill, arrives from New Orleans, the family is dead and she is the owner of the land.
Henry Fonda as Frank
Meanwhile, a mysterious harmonica-playing gunman, whom Cheyenne later dubs "Harmonica", pursues Frank. In the film's opening scene, Harmonica kills three men sent by Frank to kill him, and, in a roadhouse on the way to Sweetwater, he informs Cheyenne that the three gunfighters he killed appeared to be posing as Cheyenne's men. Sometime later, Harmonica kills two men sent by Frank to kill Jill.
Back at Sweetwater, construction materials are delivered to build a railroad station and a small town. Harmonica explains to Cheyenne that Jill will lose Sweetwater unless the station is built by the time the track's construction crews reach that point, and Cheyenne puts his men to work building it.
Claudia Cardinale as Jill McBain
Meanwhile, Frank turns against Morton, who wanted to make a deal with Jill. Frank's betrayal is made easy by the fact that Morton is crippled. After having his way with her, Frank forces Jill to sell the property in an auction. He tries to buy the farm cheaply by intimidating the other bidders, but Harmonica arrives, holding Cheyenne at gunpoint, and makes a much higher bid based on his reward money for delivering Cheyenne to the authorities. After rebuffing another intimidation attempt by Frank, Harmonica sells the farm back to Jill. At this point, some of Frank's men try to kill Frank, having been paid by Morton to turn against him, but Harmonica helps Frank kill them in order to save that privilege for himself.
After Morton and the rest of Frank's men are killed in a battle with Cheyenne's gang, Frank goes to Sweetwater to confront Harmonica. On two occasions, Frank has asked Harmonica who he is, but both times Harmonica refused to answer him. Instead, he mysteriously quoted names of men Frank has murdered. The two men position themselves for a duel, at which point Harmonica's motive for revenge is revealed in one last, clear flashback: When Harmonica was a boy, Frank forced the boy to kill his own brother by tying a noose to the top of an arch, placing it around the brother's neck, and forcing the struggling Harmonica to support his brother on his shoulders with a harmonica in his mouth while playing the song that's heard throughout the film.
Harmonica draws first and shoots Frank, and when Frank again asks who he is, he puts the harmonica in Frank's mouth. Frank nods weakly in recognition and dies. With Frank dead, Harmonica and Cheyenne say goodbye to Jill, who is supervising construction of the train station as the track-laying crews reach Sweetwater. Cheyenne collapses almost immediately, revealing that he was shot by Morton while he and his men were fighting Frank's gang. The work train arrives, signaling the arrival of 'civilization' and the end of the Epic age of cowboys and gunfighters; thus the film ends with Jill carrying water to the rail workers and Harmonica riding off with Cheyenne's body towards the horizon. A creature of the frontier, Harmonica can't "settle down" and decides to ride in search of a place where a man like him can still have a role and a reason to exist.