Colin Sullivan (Damon) is introduced to organized crime by Irish-American mobster Frank Costello (Nicholson) in the Irish neighborhood of South Boston. Costello trains him in becoming a mole inside the Massachusetts State Police. Sullivan is accepted into the Special Investigations Unit, which focuses on organized crime.
Before he graduates from the police academy, Billy Costigan (DiCaprio) is recruited by Captain Queenan (Sheen) and the unpleasant Staff Sergeant Dignam (Wahlberg) to go undercover, as his family ties to organized crime make him a perfect infiltrator. He drops out of the academy and does time in prison on a fake assault charge to increase his credibility.
Both men infiltrate their respective organizations and Sullivan begins a romance with psychiatrist Madolyn Madden (Farmiga). Costigan is seeing her under the terms of his probation and they start a relationship too.
After Costello escapes a sting operation, both moles become aware of each other's existence. Sullivan is told to find the "rat" and asks Costello for information to determine who is the informer within his crew.
Costigan follows Costello into a porn theater, where Costello gives Sullivan an envelope containing personal information on his crew members. Costigan chases Sullivan through Chinatown. When it is over, neither man knows the other's identity. Sullivan has Queenan tailed to a meeting with Costigan on the roof of a building. Queenan orders Costigan to flee while he confronts Costello's men alone. The men then throw Queenan off the building to his death. When they exit, Costigan pretends he has come to join them. Television news reveals that crew member Delahunt (Rolston) has been an undercover cop, working for the Boston Police Department. Dignam resigns rather than work under Capt. Ellerby (Baldwin).
Using Queenan's phone, Sullivan reaches Costigan, who refuses to abort his mission. Sullivan learns from Queenan's diary of Costello's role as an informant for the Federal Bureau of Investigation, causing him to worry about his own identity being revealed. With Costigan's help, Costello is traced to a cocaine drop-off, where a gunfight erupts between his crew and police, resulting in most of the crew being killed. Costello, confronted by Sullivan, admits he is an occasional FBI informant. Costello tries to shoot Sullivan but Sullivan shoots him multiple times. With Costello dead, Sullivan is applauded the next day by everyone on the force. In good faith, Costigan comes to him for restoration of his true identity and to be paid for his work, but notices the envelope from Costello on Sullivan's desk and flees, finally realizing Sullivan is the enemy. Knowing he has been found out, Sullivan erases all records of Costigan from the police computer system.
Madolyn tells Sullivan she is pregnant, but does not reveal who the father is. Later, Sullivan finds her listening to a CD from Costigan with recorded conversations between Costello and Sullivan. He unsuccessfully attempts to assuage her suspicions. He contacts Costigan, who reveals that Costello recorded every one of their conversations, Costello's attorney left Costigan in possession of the recording and the latter intends to implicate Sullivan. They agree to meet at the building where Queenan died.
On the roof, Costigan catches Sullivan off-guard and handcuffs him. As Costigan had secretly arranged, Trooper Brown (Anderson) appears on the roof as well. Shocked, Brown draws his gun on Costigan, who attempts to justify his actions by exposing Sullivan as the rat. Costigan asks Brown why Dignam did not accompany him, but Brown does not answer. Costigan leads Sullivan to the elevator. When it reaches the ground floor, Costigan is shot in the head by Trooper Barrigan (Dale), who shoots Brown in the head and reveals to Sullivan that Costello had more than one mole in the police. When Barrigan looks away, Sullivan shoots him in the head. At state police headquarters, Sullivan identifies Barrigan as the mole and has Costigan posthumously given the Medal of Merit.
At Costigan's funeral, Sullivan and Madolyn stand at the grave. Sullivan attempts to talk to her, but she ignores him. When Sullivan returns to his apartment, he is ambushed by Dignam, who shoots and kills him as he is entering.