Detectives said Wagner sat with the boat’s captain Dennis Davern that night in 1981, even though Natalie Wood could not swim and the seas were rough.
When Mr Davern suggested a call for help on the two-way radio, Wagner allegedly said: “No, we don’t want to call anybody.”
Ninety minutes later, Wagner allegedly asked people in the nearby town to look for her on land – instead of calling the Coast Guard to search the water.
In the US TV documentary Los Angeles County detectives spoke for the first time since they reopened the investigation into Miss Wood’s death in 2011.
They said Wagner was jealous of Miss Wood’s closeness to actor Christopher Walken, who was with the couple that evening on their 60-foot yacht Splendour.
Wagner is now a “person of interest” in the drowning off Catalina Island, California.
West Side Story star Wood was “America’s sweetheart” and just 43 when she died.
The documentary 48 Hours said Wagner had been angry at Ms Wood.
He had smashed a bottle of wine on board the yacht and asked Walken: “What are you trying to do, **** my wife?”
Mr Davern said he then heard sounds of a physical fight coming from below deck – then nothing.
Ten minutes later he went down to the stateroom and found Wagner, now 87, crying: “Natalie’s gone, she’s missing”.
Lieutenant John Corina said Wagner told Mr Davern to search for her.
Mr Davern did so and when he came back Wagner said: “The dinghy’s now missing.”
Mr Davern wanted to turn on the searchlight but Wagner said: “Don’t.”
Lt Corina said that Mr Davern said: “We should get on the radio and call somebody” but that Wagner replied: “No, we don’t want to call anybody. Let’s just wait and see if she comes back.”
Wagner then opened the Scotch and the two drank until 1.30am.
Wagner and Miss Wood were married from 1957 to 1962.
They remarried in 1972 and were together until her death.
Lt Corina said Wagner refused to speak to officers many times even though they travelled to his home in Aspen, Colorado.
Walken, 74, has spoken to the detectives.
Wagner’s representatives declined to comment.