This is a guest post by Peter Green of the Better Time Management blog. It’s a great resource, and he’s a terrific thinker who deserves our gratitude for the research he’s done.
A trawl of the research on the effectiveness of time management training finds a dearth of empirical evidence. Research that does exist is contradictory and inconclusive. On the other hand, research I co-authored with Denise Skinner published in 2005 based on 19 courses found an overall 20% median improvement in time management skills. To see why the findings of our research contrast with earlier studies, I believe we need to look at training content and research methods.
Covey and colleagues have usefully categorised four levels of time management training. They conclude that earlier approaches with their emphasis on efficiency and organisation have failed to help people match what they consider to be important with how they actually spend their time. They describe a fourth generation of time management which includes the best features of previous thinking but also focuses on helping people spend more time on what is really important at both work and home. So if some of the earlier research was based on ‘old school’ time management training, this could have contributed to their mainly negative conclusions. The training in our courses on the other hand was based on the first edition of my book Managing Time: Loving Every Minute and fits Covey’s description of ‘fourth generation’ content.
Major studies published between 1982 and 1996 raise questions on sample size, evaluation, measurement tools and data collection. This helped to shape the design of our own project.
For instance on sample size, some studies were as low as single figures and some purely involved students who arguably are not representative of typical time management training attendees. By contrast ours covered 134 respondents or 53% of people who attended the 19 courses, which were for ten organisations plus others represented on open courses. They covered a wide range of industry sectors and job functions thereby giving a more representative sample.
Evaluation of time management training is universally accepted as being difficult in terms of what to measure, when and how. Readers may be familiar with Kirkpatrick’s four level evaluation model where 1 is reaction to the training, 2 is what has been learnt, 3 is about behaviour changes due to the training and 4 is about organisational benefits directly attributable to the training. Level 4 is notoriously difficult to measure simply and with certainty so we opted for a level 3 study; what changes in time management behaviours were evident in the workplace sometime after the training?
Measurement of time management has proved challenging. A variety of questionnaires have been used in previous research but all were prescriptive and unable to accommodate varying training needs. Furthermore, we felt that their measurements were too detailed. For example Macan, whose work has been mentioned on this site, used a questionnaire with constructs such as ‘I set short-term goals for what I want to accomplish in a few days or weeks’ and ‘I make a list of things to do each day and check off each task as it is accomplished’. The problem with this approach is that in order to cover all the aspects of the subject it is often desirable to have multiple definitions of the same concept. Applying this to time management would result in a huge number of operational definitions, making the instrument unwieldy and impractical. Macan attempted to overcome this by simplifying the subject into three broad areas, raising concerns on oversimplification.
So rather than measure at the operational level, we opted for the concept level. Using a Key Skills Questionnaire, participants were asked to rate themselves against the key principles covered by the time management training. These included the ability to plan, prioritise and schedule each day, spend more time on the important rather than the urgent, manage interruptions, reduce forgetfulness, to say ‘no’ in a non-career-threatening way, procrastinate less, decide what is personally important, improve the work/life balance and influence the organisation’s time culture. The instrument was designed to be quick and easy to complete, comprehensive in its coverage and able to be adapted to slightly varying training needs. For instance managing interruptions was not relevant to five sales teams but achieving monthly objectives was. Seven to fifteen measures were therefore used according to the specific needs and duration of each course.
On data collection, most surveys offered either quantitative or qualitative data but few combined the two. We sought to do both by supporting our quantitative scores with qualitative feedback from participants’ managers.
At the end of each course, participants were invited, on a voluntary basis, to complete their scores on a 1 (low) to 7 (high) rating and again some months after the training. Of the 134 self-reported ‘before and after’ scores, 75 were triangulated through feedback with participants’ managers. 46% agreed and 49% tended to agree with the scores. The remaining 5% who disagreed were equally split between concerns on under and over scoring. Managers were also given the opportunity to provide examples of behaviour changes to support the quantitative findings. This was further supported by two in-depth face-to-face interviews with the managers of two especially high and low scoring courses.
With this methodology, all 19 courses showed a percentage performance gain. The lowest-scoring course averaged an 11% improvement, the highest 48%. Individual improvements varied widely with 89% recording some improvement, 7% no change and 4% showing worse scores. We believe that this is consistent with other research on non-training factors affecting training outcomes. These include personal, psychological and organisational issues, some of which have also been discussed on this website.
In summary, the quantitative and qualitative evidence collected from this survey does suggests that in contrast to many earlier studies, whilst time management training is affected by organisational context and personal motivation, ‘fourth generation training’ does have a positive impact for the majority of participants.
Green, P. (2004). Managing time: Loving every minute (2nd ed.). Cookham, UK: Chartered Institute of Marketing.
Green, P., & Skinner, D. (2005). Does time management training work? An evaluation. International Journal of Training and Development, 9(2), 124–139.
Peter Green: firstname.lastname@example.org and www.bettertimemanagement.co.uk
Denise Skinner: email@example.com and http://wwwm.coventry.ac.uk/researchnet/cucv/Pages/Profile.aspx?profileID=527