Polling booths opened at 8:00 am (2200 GMT Friday) on a warm spring day with 14.7 million electors taking part in a mandatory ballot across the huge country where last-minute opinion polls showed Abbott on track for a landslide win.
A Newspoll published in The Australian on Saturday put his Liberal/National coalition ahead 54 to 46 percent on a two-party basis, the same as a Nielsen poll in the Sydney Morning Herald.
That represents a four percentage point swing since the last election in 2010.
The latest predictions have Labor losing anything from 14 to 32 seats in the 150-member lower House of Representatives, with the conservatives set to claim a comfortable majority of more than 90 seats.
Rudd has struggled for traction after toppling Julia Gillard, Australia's first female prime minister, as Labor leader just weeks before calling the election and his party looks set for a period in opposition.
He nevertheless remained upbeat.
"It is days like this that we celebrate a democracy, people get to chose their government," Rudd told the Seven Network from Brisbane, where he kept a low profile with no early public appearances, unlike Abbott, who hit the hustings across Sydney.
"I believe we have put our best foot forward. I'm very confident in people's judgement because they will assess what is best for our country's future, their community's future and their family's future."
Asked if he would step down if he lost, Rudd said: "This is politics. You take things one step at a time."
As well as Labor and the conservatives, more than 50 other parties are registered, including the Greens and the Palmer United Party, run by controversial billionaire Clive Palmer, and they are also expected to be beneficiaries of the ruling party's fragmenting base.
A third poll by Morgan showed support for the Greens at 10.5 percent while for independents and others it was 14 percent, reflecting how disaffected some voters are with both Abbott and Rudd.
Abbott, 55, running as opposition leader in his second election, admitted the minor party vote could be important.
"This election could still go either way," he said before casting his vote at the Freshwater surf club in his Sydney constituency, surrounded by his wife and three daughters.
"There are a lot of people who are toying with voting for independents or the minor parties. My strong, strong request to them is don't, because we need a strong and stable government."
Abbott has made a paid parental leave scheme his "signature" policy, while pledging to scrap the carbon tax and make billions of dollars of savings to bring debt down.
Rudd, also 55, has campaigned on his administration's success in keeping Australia out of a recession during the global financial crisis.
He has also promised to scrap the carbon tax brought in by Labor after the 2010 election and move to a carbon emissions trading scheme by July 2014.
Other key policies include a plan to introduce a bill in parliament to legalise gay marriage and the adoption of tough measures to halt asylum-seeker boats.
Voters turned out in force early and many admitted they had tuned out of the campaign and made up their minds some time ago, with the toppling of Gillard by Rudd and instability at the top of Labor a common theme.
"I would have voted for Julia Gillard," one woman said while voting at the Bondi Beach public school of Australia's first female leader. "Labor lost me forever with that."
Most polls close at 6:00 pm Sydney time and the remainder two hours later on the West coast, with the elections deciding the make-up of the lower house and half the 76-seat Senate.
Despite the logistical difficulties in such a large country, Australians overwhelmingly abide by their obligation to vote, turnout never falling below 90 percent since it became compulsory in 1924.
AFP