In the 18 years since compulsory superannuation came into effect it has achieved a great deal but in one area it seems to have been a significant failure.

The challenge of narrowing the gap between people’s expectations and the reality of how long their retirement savings will last remains an elusive goal for the super industry and policy makers alike.

A major longitudinal research study into Australia’s household, income and labor dynamics – the HILDA study – has provided the latest reminder of the challenge that the superannuation industry faces.

The HILDA research effort  is a major program of work - originally funded by the federal government back in 2000 it has just published its fifth statistical report and in the latest survey it has looked at the issues around planning for retirement – in particular perceptions versus reality. The aim of this research is help provide insights into the dynamics of the labor market and provide better understanding for policy makers around household dynamics. It is a massive undertaking covering about 7,600 households and 19,000 individuals.

The latest data collection was done in 2007 – so prior to the global financial crisis and the subsequent hit to super balances - and in the retirement section additional questions were added to gauge the attitudes and expectations of people 45 and over about the amount of money needed in retirement, the savings needed to meet those needs and steps they have taken to prepare for retirement.

The research looked firstly at when people expected to retire compared to when they would prefer to retire. This is perhaps where the first gap appears between preference and reality. It may not be surprising that people want to retire earlier – four years for men, 3.6 years for women – than when they actually expect to retire, indeed it points to a sense of awareness of the fact that they may not be able to retire early and afford the lifestyle they desire.

However, the more concerning indicator is the proportion of non-retired people who believe the main source of funding for their retirement will be private (presumably mostly from superannuation) versus some form of government support like the age pension.

The Hilda data indicates that 83% o men and 73% o women expect the main funding source to be from their private resources. That is in stark contrast to the reality that around 70% o retirees receive some form of pension. Federal government projections in the Intergenerational Report 2010 show that super is having a positive impact in terms of the number of people eligible for a full pension is projected to decline but the number receiving part pension is expected to increase significantly.

Some of the modeling done in the HILDA research report highlights the “disconnect” between the plans and expectations of people approaching retirement and reality. The report concludes “…most people expect to retire before 65 and expect the main source of income in retirement to be sources other than government pensions.”

At the end of this month the federal government will receive a weighty report from the Cooper review into the superannuation system. It has already flagged the need to address a lot of inefficiencies within the system that need to be addressed to deliver better outcomes for members of super funds – in particular a strong focus on lower costs for the majority of members in default options.

That will promote a healthy debate about changes to make the super system more efficient and effective – not least of all reducing the number of lost accounts.

But the challenge to narrow the gap between people’s expectations and the reality of how much they have in their super today, how much they are saving and critically how long that will fund their lifestyle seems set to remain an elusive challenge for both policy makers and the industry for many years to come.