Ex-minister Crispin Blunt has confirmed he was arrested on Wednesday on suspicion of rape and the possession of controlled substances.
In a statement, the Reigate MP said he had twice been interviewed by police in connection with this incident.
Mr Blunt said he originally raised the incident with police three weeks ago, concerned about "extortion".
He said he was cooperating with Surrey Police's investigation and was confident he would not be charged.
The police were unable to say if the controlled substances were drugs, as tests are being carried out, according to PA Media.
The Conservative Party withdrew the whip, meaning Mr Blunt will sit as an independent MP, after he confirmed he had been arrested. He has been asked to stay away from Parliament.
Mr Blunt was elected as Conservative MP for Reigate, in Surrey, in the 1997 general election, defeating the incumbent Sir George Gardiner, who had switched from the Tories to the Referendum Party.
He served as a justice minister from 2010 to 2012, in the whips' office when the Conservatives were in opposition and chaired the influential Foreign Affairs Committee.
In April, he apologised for defending former MP Imran Ahmad Khan, following his conviction for sexually assaulting a teenage boy. Mr Blunt had called the verdict a "dreadful miscarriage of justice" in a statement, but retracted his comments.
A month later Mr Blunt announced he will be standing down as an MP at the next election.
'Co-operate fully'
In a statement, Mr Blunt said: "It has been reported that an MP was arrested yesterday [25 October] in connection with an allegation of rape. I am confirming that MP was me.
"The fact of the arrest requires a formal notification of the Speaker and then my chief whip.
"I have now been interviewed twice in connection with this incident, the first time three weeks ago, when I initially reported my concern over extortion.
"The second time was earlier this morning under caution following arrest. The arrest was unnecessary as I remain ready to cooperate fully with the investigation that I am confident will end without charge.
"I do not intend to say anything further on this matter until the police have completed their inquiries."
A police spokesman said: "We can confirm a man in his 60s was arrested yesterday morning (25 October) in Horley, on suspicion of rape and possession of controlled substances.
"He has been released on conditional police bail pending further inquiries."
The Conservative Party has declined to comment.
This is the latest sexual misconduct allegation to hit the party.
A separate Conservative MP, in his 50s, was arrested in May 2022 on suspicion of rape, sexual assault, abuse of position of trust and misconduct in a public office.
The MP is currently on bail until mid-February 2024.
