Olympic silver medallist Nathan Baggaley and his younger brother Dru Baggaley are accused of being involved in a plan to get 650kg of cocaine – valued at up to $200 million – from a foreign vessel hundreds of kilometres off the NSW coast.

Proceedings were adjourned early on Friday afternoon after jurors raised concerns about three “young men” staring at them for extended periods earlier in the trial.

Three people had been identified and would not be allowed back in the building during the trial, Brisbane Supreme Court Justice Ann Lyons said on Monday.

Further investigations were “ongoing”.

“We don’t know why the persons who were thought to be staring were here,” she added.

“If they do come to the court house again they will not be allowed to enter the court building.”

Strategies were also put in place to ensure the safety of jurors for the rest of the trial after one juror also believed they were followed last Wednesday evening while walking to the train station, despite changing course.

“Most of us have a sense of when we are being followed and clearly this juror felt that,” Justice Lyons said.

Dru Baggaley, 39, has told the court another man, Anthony Draper, asked him to buy a boat and get it ready to meet a ship from Indonesia that would bring tobacco to be sold in Australia.

Dru Baggaley said Draper provided $100,000 cash for the rigid-hulled inflatable boat (RHIB), which Baggaley could keep to start a whale-watching business.

But when the pair were at the boat ramp at Brunswick Heads about 9.30pm on July 30, 2018, Draper was “panicking” and said: “Mate, you’re f***ing coming with me whether you like it or not.”

Baggaley said he told Draper that was “never the agreement”, he had work in the morning and gets seasick, but the other man insisted.

Draper warned he would tell “dangerous people” involved in the importation of the tobacco worth more than $1 million where his family lived, he said.

“(Draper’s) basically said … I’m going to be held responsible for all this s*** and people are going to come after my family and me if I don’t come on board,” he added.

Draper, 56, earlier told the court he flew from Sydney to Coolangatta at Dru Baggaley’s request.

Draper said he thought the pair were going a couple of kilometres offshore to pick up “smoko” which he thought was marijuana.

They met up with a “big red boat” with “South American people”, some holding guns, on board, Draper testified.

He and Baggaley loaded a “substantial” number of black packages, thrown from the larger vessel, onto the RHIB, he said.

While Draper yelled “no more pot” because there might be “too much” for the small boat, the foreign men referred to “cacao”.

Footage shows the pair throwing packages into the sea when approached by a navy patrol boat. They were arrested by police before reaching the mainland.

The court heard Draper testified as part of an undertaking made when he received a reduced sentence during earlier court proceedings.

The prosecution alleges Nathan Baggaley also had a “key role” in the importation, saying he bought the RHIB for $100,000, got it ready and covered its registration number in heavy duty black tape before the trip out to sea.

But Dru Baggaley told the court on Friday he roped in his brother to help with the RHIB purchase because he “didn’t have enough time”.

Nathan Baggaley’s barrister Anthony Kimmins told the court the prosecution’s case against his client was based on circumstantial evidence that did not prove he was involved in the alleged attempted importation of the cocaine.

The men both pleaded not guilty to a single count of attempting to import a commercial quantity of cocaine into Coolangatta on the Gold Coast between December 2017 and August 2018.

The trial continues.